Tackling Big Data with Grid Computing

When not working with customers on their IT security needs, I volunteer as coach and administrator for a youth football club. In both environments, I continue to learn new techniques and capabilities that enable me to help others be successful.  What I love about team sports and athletic competition is how analogies can carry over into our professional lives. How many times has one heard the terms “Hail Mary” or “Home Run” used in business? My focus here is how using MetaGrid™ (the grid computing platform from Red Lambda) is similar to the role of an offensive coordinator in football.

In football, the offensive coordinator is charged with conceiving and implementing an offensive scheme for the team. Their goal is to produce the best results with the tools at hand.  He designs the specific offensive plays and then calls the right plays at the right time during the game.

As the offensive coordinator on MetaGrid, you assign player roles like packet capture, Neural Foam (our artificial intelligence engine), log processing, data storage, etc. to the computer systems that are best suited for each application. You are then able to leverage those distributed role-based systems to accomplish a job bigger than any individual system could accomplish by themselves. Grid computing used for these tasks makes up your “team” for the security professional.

When you are ready to run a particular offensive play in football, it starts with the snap of the ball.  In MetaGrid, the offensive play is the “job” and instead of the snap, it starts with data ingestion.

Once your plays have been run, you have the outcome or results.  In football, you have the yardage gained or lost and now prepare for the next play.  In MetaGrid, you have the data results of your jobs that contain valuable security information. Between the start of each play and the next is where the magic happens. Much of the work in the middle goes unnoticed by the casual observer. In football, who really watches the left tackle during the play?  You watch the quarterback and where the ball is. This is where knowing the data you are working with—or in our analogy, where knowing the offensive scheme—comes into play. Just like the offensive coordinator knows double team blocking, QB bootleg action, and fullback kick-out are needed to make the power off-tackle play successful, the data and security analyst will know what data to ingest, what additional context would be useful, what data should be correlated, stored, or have artificial intelligence applied, to get to the best results.

MetaGrid puts the data analyst in the coordinator position, high up in the press box, taking in all the data they need, of any type, giving them the power to distribute key tasks to those best suited, and use a simple graphical workflow to design and execute plays (jobs) on the grid. While you are watching or coaching football this season, you might think about how you could leverage grid computing to tackle big data and other tasks.




Leave a Reply

Your email address will not be published. Required fields are marked *

You may use these HTML tags and attributes: <a href="" title=""> <abbr title=""> <acronym title=""> <b> <blockquote cite=""> <cite> <code> <del datetime=""> <em> <i> <q cite=""> <strike> <strong>