Re: Issue with Angel One Scrip Master Download
I would like to share a few things I have observed while working with SmartAPI, along with some suggestions that might help other developers facing similar issues.
1. Implement a dedicated API request-routing layer:
In my opinion, one of the best ways to handle rate limits is to build a dedicated API request-routing layer between your bot and SmartAPI. This layer should centrally manage rate limits, cooldown periods, and other relevant constraints.
Instead of allowing different modules to send API requests independently, route them through a single layer that controls when requests are sent, handles rate-limit errors, and applies cooldowns where necessary. This can help prevent repeated violations and make the overall system more reliable.
2. Complete historical data fetching before the market opens:
One thing I have noticed is that historical data requests generally work more reliably outside market hours. During market hours, rate-limit errors seem to occur more frequently.
One possible reason could be the increased load on SmartAPI's servers during trading hours, which may affect request processing, especially when new connections are established. I cannot confirm that this is the actual cause, but it is worth looking into.
My suggestion is to fetch all the historical data you need before the market opens. Use that data to initialise your indicators, warm up your calculations, and prepare the rest of your system. By market open, your bot should already be ready to process live data without needing to fetch historical data at the same time.
3. Avoid combining historical and live data requests during market hours:
If possible, avoid sending historical data requests immediately before or alongside live data requests during market hours. Making multiple requests in quick succession can put additional pressure on your API request limits, particularly when the servers are already under heavy load.
Prepare everything in advance and keep market-hour requests focused on operations that genuinely need to happen in real time.
4. A suggestion for the SmartAPI team:*
I also think it would be worth investigating how the API performs under peak market-hour load. If server load or connection handling contributes to the errors developers experience, improvements in request processing and infrastructure scalability could make a real difference.
Of course, developers need to manage their own API usage properly, but the consistency of the API under heavy load is equally important for anyone building a real-time trading system.
I am sharing these observations in the hope that they help other developers avoid similar issues and that the SmartAPI team can investigate any underlying infrastructure bottlenecks if necessary.
The idea is simple: prepare everything you can before market open, manage API requests centrally, and keep your live trading system focused on real-time operations.
