machine learning algorithms don't always work so well on raw data part of our
jobs as engineers and scientists is to transform the raw data to make the
behavior of the system more obvious to the machine learning algorithm this is
called feature engineering feature engineering starts with your best guess
about what features might influence the thing you're trying to predict after
that it's an iterative process where you create new features add them to your
model and see if the result improved let's take a simple example where we
want to predict whether a flight is going to be delayed or not in the raw
data we have information such as the month of the flight the destination and
the day of the week if I fit a decision tree just to this data I'll get an
accuracy of 70% what else could we calculate from this data that might help
improve our predictions well how about the number of flights per day there are
more flights on some days than others which may mean they're more likely to be
delayed I already have this feature from my data set in the app so let's add it
and retrain the model you can see the model accuracy improved to 74 percent
not bad for just adding a feature feature engineering is often referred to
as a creative process more of an art than a science there's no correct way to
do it but if you have domain expertise and a solid understanding of the data
you'll be in a good position to perform feature engineering as you'll see later
techniques used for feature engineering are things you may already be familiar
with but you might not have thought about them in this context before let's
see another example that's a bit more interesting here we're trying to predict
whether a heart is behaving normally or abnormally by classifying the sounds it
makes the sounds come in the form of audio signals rather than training on
the raw signals we can engineer features and then use those values to train a
model recently deep learning approaches are becoming popular as they require
less manual feature engineering instead the features are learned as part of the
training process while this has often shown very promising results deep
learning models require more take longer to train and the resulting
model is typically less interpretable than if you were to manually engineer
the features the features we use to classify heart sounds come from the
signal processing field we calculated things such as skewness kurtosis and
dominant frequencies these calculations extract characteristics that make it
easier for the model to distinguish between an abnormal heart sound and a
normal one so what other features to people use many use traditional
statistical techniques like mean median and mode as well as basic things like
counting the number of times something happens lots of data has a timestamp
associated with it there are a number of features you can extract from a
timestamp that might improve model performance what was the month or day of
week or hour of the day was it a weekend or holiday such features play a big role
in determining human behavior for example if you were trying to predict
how much electricity people use another class of feature engineering has to do
with text data counting the number of times certain words occur in in a text
is one technique which is often combined with normalization techniques like term
frequency inverse document frequency word to Veck in which words are
converted to a high dimensional vector representation is another popular
feature engineering technique for text the last class of techniques I'll talk
about has to do with images images contains lots of information so you
often need to extract the important parts traditional techniques calculate
the histogram of colors or apply transforms such as the haar wavelet more
recently researchers have started using convolutional neural networks to extract
features from images depending on the type of data you're working with it may
make sense to use a variety of the techniques we've discussed feature
engineering is a trial and error process the only way to know if a feature is any
good is to add it to a model and check if it improves the results to wrap up
that was a brief explanation of feature engineering
we have many more examples on our site so check them out