Too Much Amsterdam

After roughly two months of paper trading, something about my own BUY signals started bothering me. There was too much Amsterdam in them.

Not one odd outlier. A ratio. My universe holds around five hundred American stocks and fifty Dutch ones. The American side is where the big growers sit, the names you would expect a system like this to latch onto. And yet that small Dutch corner kept pushing itself to the top of the ranking.

What made the feeling stronger: they weren't even names I got excited about myself. Solid companies, fine, but not the kind you would expect a momentum system to jump on first.

I couldn't point at anything. No error message, no contradiction, not a single number screaming that something was wrong. Just a distribution that felt off.

The question I actually had

What made me curious was something other than "where is the bug". I wanted to know how a factor score behaves over time.

My system gives every stock a score each day, built from four components. That score decides whether something gets bought. But I had never seen how such a score develops across months. Does it move with the price? Does it move ahead of it? Or does it wander around with no relationship at all?

If your system buys based on a rising score, you would want to know whether a rising score means anything.

So I had a page built. Per stock, the price alongside the total score. The four factors separately. Three years of history in one view.

Not to find a bug. To look.

Half a second

The first chart I opened answered immediately. Just not my question.

The score line tracked along nicely, then suddenly jumped up, and a day later it was back down. Nothing in the price explained it. A sawtooth where a line was supposed to run.

This is where it gets interesting. I calculated nothing. Ran no query, did no analysis. My eyes saw in under a second that a line was doing something a line shouldn't do, and the rest of my head was still booting up.

That is not a remarkable skill. That is standard equipment. Humans are bad at adding up columns and phenomenal at noticing when a pattern breaks. It is why you recognise someone you know in a crowd before you consciously register that you are looking at a person.

And it is exactly the sense you don't use when you stare at tables.

Why nothing caught this

The sawtooth turned out to be a real bug. For momentum, the system looked back a fixed number of trading days, but it used the combined calendar of the American and Dutch exchanges. On a day when Wall Street was closed and Amsterdam was open, or the other way round, that lookback landed on a day when the stock's own exchange had no price. No price means no momentum. And then the system calmly computed the total score without that factor.

No error message. No empty value in the table. Just a score that looked entirely normal and had been produced the wrong way. On some days that happened for an entire exchange at once.

Two reasons nothing caught it. The first is mundane: there were zero tests on that component. The second runs deeper. Even with tests nobody would have caught this, because you have to know that two exchanges observe different holidays and that a 147-day lookback can therefore miss. That is an assumption buried so deep in the code that you only see it when something makes it visible.

And then the actual explanation

The next day I looked at the same kind of chart for a French media company. Something odd again: the valuation score dropped from high to low in December and stayed there. The price did nothing.

This turned out to explain my original hunch.

The system works out how expensive a company is using its last four reports. For most companies that means four quarters, so a year. But twenty-six companies in my universe only report twice a year, and a striking number of those are listed in Amsterdam. For them, four reports meant two years of figures divided by today's market value.

They looked roughly twice as cheap as they actually were.

Ten of those twenty-six sat in the top five percent of my entire universe. Four of them were in my portfolio. That was my too much Amsterdam.

After the correction that group settled back to where it belongs. Where twenty-two Dutch names used to sit in the top five percent, there are now four.

What it cost

The bugs weren't only in today's scores, they were in every historical score. So six years of data has been recomputed, 1,713 trading days.

And the simulation I built my confidence on is now invalid. It was calculated on scores I now know were wrong.

So that simulation is running again. As I write this it is grinding through six years of trading days, and tomorrow morning there will be a result. That is genuinely exciting to me. It could well turn out that half my return evaporates and I am looking at a mediocre system. It could also hold up and mean I have something real.

I don't know. Which is rather the point of an experiment.

What I do know: whatever number comes out, it will be the first number that is actually correct. And a lower figure you can build on beats a higher figure resting on sand.

My original question is still open

And here is the honest part. I went looking for a relationship between the factor score and the price development. I did not find one.

For one stock the score climbed for half a year while the price fell. That is precisely the opposite of what you hope for. With the bugs removed the picture looks better, but looking better is not knowing.

So that research still has to start. Not stock by stock, because if you work through five hundred charts you will always find ten that look convincing, and then you have selected noise instead of signal. It has to run across the whole universe at once, with the method fixed in advance so I cannot bend it afterwards toward what I hoped to find.

That is the next step.

What I take from this

There is a temptation, when building with AI, to think verification can be automated too. More tests, more checks, more controls. That helps, and I have hundreds of them by now.

But a test checks whether something does what you expect. It cannot check whether your expectation was right. For that second part you need something we happen to be very good at and machines are not: looking at an image and sensing that something is off.

It started with a distribution that felt skewed. I wasn't hunting for a bug, I wanted to understand how my own system behaves. The bugs surfaced by themselves once I made things visible.

So when something about your system doesn't sit right and you can't pin it down: draw it. Not one chart, ten. Put side by side the things that should move together. And then just look.

Your brain does the rest.