ERA5 dataset is three years outdated?

[iuryt]

Hi,

I am trying to access ERA5 dataset from Planetary Computer Data Catalog using Python.

https://planetarycomputer.microsoft.com/dataset/era5-pds

On the page, it says that the temporal extent for this dataset is 01/01/1979 – Present, but if I follow the tutorial on the Example Notebook and change the code to

search = catalog.search(
    collections=["era5-pds"], datetime="2021-01", query={"era5:kind": {"eq": "an"}}
)

for example, the search returns no items.

In fact, in https://planetarycomputer.microsoft.com/api/stac/v1/collections/era5-pds/items it is clear that the "end_datetime":"2020-12-31T23:00:00Z".

Is this correct?
I am very frustrated with that because the service worked very well for my research, but I need data up to the present.

Thanks for the service, it accesses the data faster than I was expecting.

Best,

[TomAugspurger]

Thanks for reaching out.

You're correct that that collection's data is current not updating. The provider we were getting it through let us know they weren't making it available anymore, but that's perhaps not the case now.

We have a work-in-progress replacement pipeline getting things directly from the ECMWF, but that's not yet ready. You can follow our GitHub discussions page for announcements for when that new pipeline is available.

[iuryt]

I see, so in the case of my project, I should then add a workaround to access >2020 data from somewhere else. Should I leave this issue open?

[iuryt]

In that case, I suggest that the website should be updated from "01/01/1979 – Present" to "01/01/1979 – 31/12/2020", right?

[lukasValentin]

Hi @TomAugspurger, is there any news on this issue? It'll be awesome to have access to the latest ERA5 on MSPC (we are running crop growth models that require up-to-date reanalysis data ...)

[TomAugspurger]

No news, but I'll post here if / when we get a new pipeline in place.

…

________________________________ From: Lukas Valentin Graf ***@***.***> Sent: Monday, January 22, 2024 5:48 AM To: microsoft/PlanetaryComputerDataCatalog ***@***.***> Cc: Mention ***@***.***>; Comment ***@***.***>; Subscribed ***@***.***> Subject: Re: [microsoft/PlanetaryComputerDataCatalog] ERA5 dataset is three years outdated? (Issue #426) Hi @TomAugspurger<https://github.com/TomAugspurger>, is there any news on this issue? It'll be awesome to have access to the latest ERA5 on MSPC (we are running crop growth models that require up-to-date reanalysis data ...) — Reply to this email directly, view it on GitHub<#426 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAKAOISCMLZWSA6NRFYD4C3YPZGYVBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVEZTCNZUGAZDAMRYQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRRGY2TEMRXGY3DQNVHORZGSZ3HMVZKMY3SMVQXIZI>. You are receiving this email because you were mentioned. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.

[Real-Slin-Shady]

Might it be possible to just add the data from 2021 to 2023 in the meantime with the old pipelie? I mean just to get the data somewhat up to date.

[TomAugspurger]

Apologies, but not easily 😕

…

________________________________ From: Real-Slin-Shady ***@***.***> Sent: Tuesday, February 20, 2024 4:56 AM To: microsoft/PlanetaryComputerDataCatalog ***@***.***> Cc: Comment ***@***.***>; Subscribed ***@***.***> Subject: Re: [microsoft/PlanetaryComputerDataCatalog] ERA5 dataset is three years outdated? (Issue #426) Might it be possible to just add the data from 2021 to 2023 in the meantime with the old pipelie? I mean just to get the data somewhat up to date. — Reply to this email directly, view it on GitHub<#426 (comment)> or unsubscribe<https://github.com/notifications/unsubscribe-auth/AAKAOISE23HZRA46HWW7VQDYUR6OVBFKMF2HI4TJMJ2XIZLTSOBKK5TBNR2WLJDUOJ2WLJDOMFWWLO3UNBZGKYLEL5YGC4TUNFRWS4DBNZ2F6YLDORUXM2LUPGBKK5TBNR2WLJDUOJ2WLJDOMFWWLLTXMF2GG2C7MFRXI2LWNF2HTAVFOZQWY5LFUVUXG43VMWSG4YLNMWVXI2DSMVQWIX3UPFYGLLDTOVRGUZLDORPXI6LQMWWES43TOVSUG33NNVSW45FGORXXA2LDOOJIFJDUPFYGLKTSMVYG643JORXXE6NFOZQWY5LFVEZTCNZUGAZDAMRYQKSHI6LQMWSWS43TOVS2K5TBNR2WLKRRGY2TEMRXGY3DQNVHORZGSZ3HMVZKMY3SMVQXIZI>. You are receiving this email because you commented on the thread. Triage notifications on the go with GitHub Mobile for iOS<https://apps.apple.com/app/apple-store/id1477376905?ct=notification-email&mt=8&pt=524675> or Android<https://play.google.com/store/apps/details?id=com.github.android&referrer=utm_campaign%3Dnotification-email%26utm_medium%3Demail%26utm_source%3Dgithub>.