AlgoPy

01

The work is collection, not analysis

Equity research in India is predominantly data collection. The figures are distributed across four sources that do not agree with one another, each company requires ten to twenty fields gathered by hand, and the market has moved by the time thirty have been completed. Reviewing 8,100 is not feasible manually.

The platform performs the collection nightly, so the reader opens only companies that qualified for something. It screens; it does not advise and it does not place orders. That boundary is a design decision rather than an omission.

02

Data as a build artifact rather than a query

End-of-day market data does not change. Yesterday’s prices are final, and anything immutable can be built in rather than looked up — which is what makes a 512MB host sufficient.

A nightly GitHub Actions job fetches from every source, runs each of the 16 screens as an independent step, writes each result as an atomic JSON snapshot, and commits it. The commit triggers a deploy, so the image ships with the data already inside it. Serving a page is then a dictionary lookup: nothing is fetched while a reader waits, and the site cannot be slow because an upstream provider is slow.

• Under 0.3s per page across 8,111 companies, on 512MB.
• No external call in the read path, so there are no rate limits to accommodate.
• Rolling back incorrect data is a redeploy of the previous build.
• A provider outage presents as slightly stale data rather than a broken page.

03

Four figures that required measurement

None of the performance work began as an assumption. Each started with an unacceptable measurement and concluded with the same measurement retaken.

The most severe case was a screen that built its default window at request time, fetching approximately 850 days of candles per company. It took over 200 seconds, beyond the server’s 120-second timeout, so the worker was terminated before it could complete and cache the result, and every visitor restarted the same work. As a committed snapshot it responds in 0.7 seconds.

• Serve time 200s → 0.7s, by moving the build out of the request path.
• Time to first byte 0.95s → 0.30s, by encoding each feed once per data version. 0.30s is the India-to-Singapore network floor.
• Memory 62.3MB → 27.2MB per worker, after identifying four modules each building a separate index of the same instrument list. A two-worker configuration measured 675MB and was OOM-killed, which is why production runs one.
• Upstream requests ~13,000 → ~3,000 per night, via a build-time shared cache guarded so it cannot alter any screen’s output.

04

The screens that found nothing, stated plainly

Most trading projects imply an edge and never test for one. This platform runs the backtest and publishes the result on the page regardless of whether the result is favourable.

Several screens measured as no edge and say so publicly. Stage 2 breakouts returned +0.24% over their own 15-day window with a t-statistic of 0.12 — the in-sample half negative and the holdout positive, which indicates regime rather than skill. The F&O momentum screen wins 29.8% against a 33.3% breakeven at a profit factor of 0.83, and no cost assumption recovers it. A study of a 2013 astrology trading method found 0 of 1,589 rules surviving a multiple-testing correction.

The architecture supports that position: there is no combined score and no average anywhere. The screens disagree — two differed by 44 points on the same company on the same day — and they are of different ages, so every block carries its own date, and a company’s absence from a screen means it did not qualify, never that it should be sold.

05

A pipeline that starved itself of data

The nightly job ran its test suite before building the data. Poor upstream data failed a test, the failed gate skipped all thirty builds, and the only process capable of repairing the file was the build that had just been skipped. The site served stale data on six separate occasions before I identified the pattern.

Data-reading tests are now tagged out of that gate, and the gate itself reports without blocking: a failing test publishes the data and still marks the run red. "Partial publish, loud failure" was stated in the job’s own header throughout — the gate was the single step not honouring it.

06

A figure that was wrong on 1,005 companies

One provider returns NSE history adjusted for splits and bonuses, and BSE history unadjusted. Taking the higher of the two as the all-time high therefore adopted the unadjusted BSE figure and produced drawdowns that had never occurred, across 1,005 companies.

Nothing failed and nothing appeared incorrect, which is what makes this class of defect worth recording. Every BSE all-time high is now labelled unverified on the page rather than silently trusted.