Kaggle & Kaggle Tricks

Golden Rules of ML

Golden Rules: Evaluation

Observation of raw data

EDA from aggregations is good.

EDA by looking at thousands of samples is the best.

EDA is for becoming one with your data, with all of its bumps and sharp edges. This is how to do it.

Measure this in hours, not minutes.

Get the data

I've never worked with a company that had a “dataset” waiting for me.

Most companies don't even have any data ready for a data scientist, and some don't even know what useful data looks like.

You'll spend most of your time dealing with these:

• Understanding the problem you want to solve • Defining the data you need • Procuring access to some of that data • Planning to collect the data you don't have • Organizing, slicing, and analyzing the data • Labeling the data

If you are lucky, 5% of your time will be writing code and building models. The other 95% is obsessing about data.