Zeu Park

A curious developer documenting my journey through code, from Python basics to machine learning and beyond

1 minute read

Introduction

Whether I’m working on a Kaggle competition or my stock prediction project, one thing has become increasingly clear:

Models are strong enough — it’s the features that matter most.

We now have a mature set of machine learning tools: LightGBM, XGBoost, CatBoost, ensemble stacking, even deep learning APIs. Most models today reach a baseline of solid performance if your data is clean.

The Real Leverage: Feature Engineering

Here’s what I’ve learned through hands-on experiments:

  • The same model with different features can give totally different results.
  • A model with too many noisy features actually performs worse than one with fewer but relevant ones.
  • The features I assume are important often turn out to be noise.

This is especially true in my stock project, where real-time volume, foreign trades, and price momentum all compete for attention.

Feature ≠ Signal

Sometimes I include a feature because I believe it’s meaningful — but the model sees it as noise. That’s when I realized:

The model doesn’t care what I think. It cares what improves performance.

So I started letting data drive decisions:

  • Using permutation importance
  • Testing SHAP values
  • Iteratively pruning features that didn’t help

Domain Knowledge is a Superpower

Still, good features don’t come from thin air. They come from understanding the problem deeply.

In stock prediction:

  • If I want to predict price spikes, I must understand what causes volatility
  • Is it earnings? Is it volume? Is it foreign investor behavior?

In Kaggle competitions:

  • Top solutions often reflect deep problem-specific insights, not just clever code

That’s why I now spend more time learning how the system works before building any model.

What I Learned

  • Don’t rely solely on model complexity — it won’t fix poor features
  • Trust the data over your intuition, but use intuition to guide data collection
  • Strong domain knowledge leads to better feature creation, faster convergence, and more reliable models

Takeaway

The secret to outperforming with machine learning is not in having the most complex model — it’s in having the most meaningful representation of the data.

That means learning the domain, curating the data, and iteratively refining features until the model speaks the language of the problem.

Next Steps

  • Spend more time researching economic indicators in financial datasets
  • Design new composite features for my stock volume ML project
  • Apply this mindset more rigorously in future Kaggle competitions

You May Also Enjoy