1. Home
  2. Categories
  3. Python SDK
  4. Issue fetching Historical data

Issue fetching Historical data

Python SDK
Log in to reply
  • V

    I am facing a peculiar issue in fetching data. This wasn't a problem earlier when I fetched data using the same code. The code below loops through a few different date ranges to fetch data for that date range. I transform the fetched data into data frames of each date range and then concat it to return one single data frame.

    But when I see the excel sheet that I store the data in the dates that I have fetched the data for are arbitrary, it skips a lot of days in between, sometimes skips years. Starts not from the start date I have set and instead from some arbitrary date, I have given a snippet of the result below. Date starts from 2013 instead of requested 2011, it jumps from 2015 to 2018 and then 2019 without retrieving the complete data. Can anyone help here.

    def hist_data():

    dfs = []
    frm = ['2015-07-12 09:15', '2020-01-01 09:15']
    until = ['2020-12-31 15:30', '2022-11-17 15:30']

    for x, y in zip(frm, until):

        try:
            historicParam = {
                "exchange": "NSE",
                "symboltoken": tokens,
                "interval": 'ONE_DAY',
                "fromdate": x,
                "todate": y
            }
            dfs.append(pd.DataFrame(obj.getCandleData(historicParam)['data'],
                                    columns=['Date', 'Open', 'High', 'Low', 'Close', 'Volume']))

        except Exception as e:
            print(f'Historic Api failed: {e}')


    df = pd.concat(x for x in dfs)
    df['Date'] = df['Date'].str.split('T').str[0]

    return df

for tokens, symbols in zip(nifty_50['token'], nifty_50['symbol']):

    stock_data = hist_data()
    stock_data.to_csv(f'/Users/varadjoshi/Documents/Markets Data/Price Volume Data/Nifty 500 Stocks Data/{symbols}.csv', index=False)
    time.sleep(1)
    print(symbols)

    Output in the excel sheet below

    ![f9cd5502-b1d8-4f4d-9944-cffb397e39a3-image.png](/assets/uploads/files/1668693905626-f9cd5502-b1d8-4f4d-9944-cffb397e39a3-image.png) code_text

    ![dd9bcd9f-cbb0-4960-a0da-abac7e2a1955-image.png](/assets/uploads/files/1668693836453-dd9bcd9f-cbb0-4960-a0da-abac7e2a1955-image.png) code_text
```26/06/15	169.99	170.44	163.28	167.57	2150489
29/06/15	166.51	166.84	160.28	165.81	1140046
30/06/15	164.73	170.95	164.12	169.44	1197561
01/07/15	168.08	170.29	166.03	166.81	792876
02/07/15	164.42	174.04	164.42	172.46	1866024
03/07/15	172.89	173.34	168.05	170.86	1611144
06/07/15	168.05	173.46	166	172.8	2376410
07/07/15	173.49	177.9	171.5	177.15	1906853
08/07/15	175.64	175.91	167.57	168.87	3483626
09/07/15	168.96	169.26	166.24	166.96	1521201
10/07/15	167.66	169.2	165.33	167.36	655030
26/12/18	283.81	289.77	277.8	286.35	3413076
27/12/18	288.95	292.58	277.65	280.03	5553470
28/12/18	280.55	289.8	280.4	287.83	1932203
31/12/18	291.04	291.25	285.38	286.32	1383559
01/01/19	288.23	288.35	282.79	285.2	986138
02/01/19	285.08	286.77	278.92	281.76	1550368
03/01/19	281.15	286.41	274.44	277.04	1653258
04/01/19	277.95	283.99	276.38	280.61	1745111
07/01/19	284.42	285.29	278.67	282.15	1675020
08/01/19	279.34	282.3	275.95	279.16	1713143
09/01/19	280.49	281.7	273.2	277.77	1790489
10/01/19	277.77	278.95	274.87	277.77	1632163
11/01/19	277.1	278.16	270.27	273.87	2006360
14/01/19	270.21	273.23	262.96	265.44	3997561
15/01/19	265.98	269.91	265.04	268.43	3103563
16/01/19	267.52	272.51	266.8	270.15	2311584
17/01/19	271.96	276.35	268.07	270.66	2779574
18/01/19	272.33	273.14	265.13	266.92	1582590
    1 V A 2 Replies
  • V

    @admin Thanks admin, I tried this and it works, however earlier I could fetch 2000 records for daily candles. Even this page says I can fetch 2000 days for 1 Day candles

    https://smartapi.angelbroking.com/docs/Historical

    0
  • A

    Hi @Vypy1

    Historical data API is only able to fetch 500 records at a time.

    Meaning if you are trying to fetch the data using below parameters it will fetch data from 13th Oct 2020 to 18th Oct 2022. Because the Gap between fromdate and todate is > 500 days, it is designed to return latest 500 values as follows = todate-500 days.

    {
    "exchange":"NSE",
    "symboltoken":"11821",
    "interval":"ONE_DAY",
    "fromdate":"2020-08-09 11:00",
    "todate":"2022-10-18 11:40"
    }

    Hope this helps to clarify.

    1 V 1 Reply
  • V

    @admin what's the update on this?

    0
  • V

    @Vypy1 @admin what's the update here?

    0
  • V

    @Vypy1 @admin I have been following up for a response on this and you guys don't care to even respond. What is the issue?

    0
  • V

    @Vypy1 @admin waiting for your response

    0
  • V

    @Vypy1 @admin what is the update on this?

    0
  • V

    @Vypy1 @admin why is there no response on this? You make the API, if your users have a question its better to respond to them than be completely silent about it.

    0
  • V

    @Vypy1 @admin any update here?

    0 V 1 Reply
  • V

    @Vypy1 I did some digging and it seems the request itself is not receiving the correct data.

    I used the below code to check if your _request function is receiving the right data, but there seems to be some problem here.

    My date requested from is 2019-1-1 and I start getting data from 2020-1-1

    params = historicParam = {
            "exchange": "NSE",
            "symboltoken": 11821,
            "interval": 'ONE_DAY',
            "fromdate": '2019-01-01 09:15',
            "todate": '2022-11-17 15:30'
        }
route = 'api.candle.data'
method = 'POST'

print(obj._request(route=route, method=method, parameters=params)['data'])

    [['2020-11-10T00:00:00+05:30', 104.5, 105.4, 101.15, 101.8, 662663], ['2020-11-11T00:00:00+05:30', 102.55, 103.0, 100.5, 101.85, 653453], ['2020-11-12T00:00:00+05:30', 101.9, 103.2, 101.0, 102.0, 594231], ['2020-11-13T00:00:00+05:30', 102.0, 103.9, 100.0, 101.45, 1197144], ['2020-11-17T00:00:00+05:30', 102.2, 110.85, 101.5, 107.0, 2480276], ['2020-11-18T00:00:00+05:30', 107.0, 110.9, 106.75, 109.45, 2004444], ['2020-11-19T00:00:00+05:30', 109.0, 112.0, 105.7, 107.9, 1102268], ['2020-11-20T00:00:00+05:30', 108.75, 114.0, 108.2, 112.7, 2950348], ['2020-11-23T00:00:00+05:30', 113.8, 113.8, 111.1, 111.6, 1840552], ['2020-11-24T00:00:00+05:30', 111.95, 113.6, 111.0, 111.4, 2382269], ['2020-11-25T00:00:00+05:30', 112.25, 116.0, 111.3, 115.2, 3936048], ['2020-11-26T00:00:00+05:30', 116.05, 118.0, 114.05, 115.65, 1452319], ['2020-11-27T00:00:00+05:30', 116.0, 120.0, 114.95, 118.1, 2300417], ['2020-12-01T00:00:00+05:30', 118.2, 127.35, 116.55, 125.65, 3557627]
    0
  • V

    @Vypy1 @admin any update on this?

    0 V 6 Replies