esa logo

AutoICE

Challenge closed

OverviewRulesPrizesDataTeamsLeaderboardForum
Forum

AUTOICE - New reference dataset

Annekatrien

1 year ago

Dear participants,

After the competition kickoff, we identified a couple of issues with the dataset. To address this, we are providing the following update to the datasets: New test dataset: Unfortunately, the ice charts were by mistake included in the Ready-to-train (RTT) test set. To avoid any misuse, a new version of the test set has now been made available, without the ice chart label data. Updated training datasets: The previous test set is now included in both the raw and RTT version of the training data set, i.e. 513 scenes are now available for training. The RTT dataset has been processed again to accommodate the updated mean and STandard Deviation (std) due to including new training files in the training dataset. The mean and std are used during the standard scaling of the dataset. SOD and FLOE variables have been slightly altered, as the threshold was incorrectly set to 70% and 50%, respectively, instead of the 65%, which was otherwise specified in the dataset manual. The github repository has been updated with new training and testing lists in ‘datalists’. ‘misc’ (contains class bins for each scene) and ‘utils.py’ has also been updated. We do not expect this update to have an impact on the functionality of the get-started toolkit. Four corrupt files in the raw training dataset have been identified. These have been updated. The files in question are:

  • S1A_EW_GRDM_1SDH_20190507T101218_20190507T101251_027120_030E8D_E795_icechart_cis_SGRDINFLD_20190507T1011Z_pl_a.nc
  • S1A_EW_GRDM_1SDH_20190802T183536_20190802T183640_028394_033568_3716_icechart_dmi_201908021835_CentralEast_RIC.nc
  • S1B_EW_GRDM_1SDH_20200610T214040_20200610T214145_021977_029B62_C439_icechart_cis_SGRDINFLD_20200610T2127Z_pl_a.nc
  • S1B_EW_GRDM_1SDH_20200620T104216_20200620T104259_022116_029F8D_8890_icechart_cis_SGRDIEA_20200620T1041Z_pl_a.nc

These are located in the following two zip files: S1A_20190419T203541_20190823T114541.zip S1B_20191028T132359_20200714T184241.zip

How does this impact me? If you are working with files downloaded from the data.dtu.dk website:

  • Everyone: download the new test set with the respective dataset version you want to use.
  • RTT users: download the new, updated version of the dataset
  • Raw users: download the two new zip files mentioned earlier and the previous test set added as “added_v2.zip”
  • Get-started toolkit users: update the repository (most notably the datalists, utils.py and misc folder)

PolarTEP / EOX platform users (NOTE 23/12: this has not yet been updated!):

  • The datasets have been updated directly on the cluster.
  • The datalists, the ‘misc’ folder and ‘utils.py’ files have been updated on the cluster side, however, you need to update these in your client.

We apologise for the inconvenience of the update but we are certain that it will improve the overall quality of the competition and the final results. The AI4Arctic team wishes you all a pleasant holiday season.

Comments (1)

stokholm commented:

1 year ago

In addition, we have reset the leaderboard as the submission there was on the previous test dataset.

ct replied:

1 year ago

Hmmm… can't reach this pageIt looks like the webpage at https://data.dtu.dk/ndownloader/files/38619422 might be having issues or it may have moved permanently to a new web address. ERR_INVALID_RESPONSE

https://data.dtu.dk/ndownloader/articles/21762830/versions/1 502 Bad Gateway nginx

stokholm replied:

1 year ago

I can confirm that I also have issues with the download, I suspect it may be a temporary issue. I will reach out to the platform for a clarification/fix.

Thank you for pointing this out.

ct replied:

1 year ago

It doesn't appear to be temporary

stokholm replied:

1 year ago

I very much agree with that assessment, and I regret not updating the thread earlier. I have been informed by the data hosting provider that there is an issue with the servers on which the data is stored. They assured us that a solution would be implemented yesterday but it appears to be dragging out.

We very much apologize for the inconvenience this is causing. I will update the thread when more information is available.

Niya Shao replied:

1 year ago

Is the data website working for anyone? Currently I still cannot download anything from the website.

stokholm replied:

1 year ago

I am happy to announce that it is working now (as within the past hour). I will make a general post on the matter. Thank you for the patience and we sincere apologies for the inconvenience of the dataset unavailability.

Niya Shao replied:

1 year ago

Thank you! It works for me now

stokholm replied:

1 year ago

Happy to hear. :)

Forum
Privacy PolicyTerms & ConditionsCookies

AI4EO is carried out under a programme of, and funded by the European Space Agency (ESA).

Disclaimer: The views expressed on this site shall not be construed to reflect the official opinion of ESA.

Contact Us

Contact

Follow Us