Thoughts on my very first Datathon.
Two days of intense analysis, coding, profiling, mining, developing, stressing, And stressing.
Why did I volunteer to provide free consulting and endure 48 hours of anxiety?
Intellectual curiosity: to see others' approach, challenges, and solutions; to meet other talent who enjoy data; to get in the weeds with our colleagues and showcase data with weapons (softwares) of our choice; to be inspired. Our only enemies were self-doubt and time.
This Datathon was designed to take a few years of Job Postings across the Commonwealth of Virginia, along with any other data we could find and incorporate, and turn it into useful information that could be used to ensure that Virginia was prepared for its economic future.
Datathon facilities were wonderful - the Library of Virginia was a great environment for collaboration, synergy, googling, and heads down development. There were not a lot of privacy or huddle rooms. The nervous energy in the building was great - full of competition and creativity. Food and snack were generously provided to help us stay focused. Subject matter expert volunteers kept us informed, on schedule, and mentored.
We met the Governor himself, 'Cyber Governor' Terry McAuliffe. He was a gracious host. Though I'd love to think our group was special, he took time to greet and take a photograph with every team.
In my opinion, the real challenge was the data source. It was dirty! Messy! And there was a lot of it. A million job postings with paragaphical (I did that on purpose) instead of categorical fields. Also, each team had only five minutes to demo their final product. Brilliant ideas and features were lost in this time limit. But I think that is the challenge of a Datathon - to creatively solve, narrow scope, and be masterful with time management.
In two days, 16 teams used the same data set and presented 16 different solutions! Our team developed a Web Insights portal. Our goal aimed at education influencers, was to reduce unemployment while using current job postings to forecast hard skills that would be demand. It was created from a time-deprived combination of html, Tableau, statistical modeling, predictive analytics, data mining, and Azure Machine Learning.
For me it was a great experience - stressful but worth every metric discovered and wink of sleep lost.
Did our team make the finals… ? We did. Will I participate again? It doesn't take a data scientist to predict yes.
Datathon Website: VA Datathon
About the Author
Wendy Greene is a senior consultant in CapTech's Data & Analytics Practice Area. She is a Data Visualization generalist who has also authored Standards and Guidelines for Tableau, and Spotfire. She also teaches Spotfire flipped course and enjoys creating and demoing dashboards.