You are here

The e-IDI – real time environmental data in one spot

The e-IDI proof of concept website and live data feeds were decommissioned on 30 June 2018. Technical and evaluation reports of the proof of concept can be accessed at the bottom of this page. For more information please contact info@mfe.govt.nz.

Background to the e-ID

The Environmental Integrated Data Infrastructure or “e-IDI” made environmental data more discoverable, shareable and accessible. The e-IDI was developed by the Ministry, Natural Resource Sector, regional government sector and research organisations. The project was funded by the Better Public Services Seed Fund, administered by the Treasury. The service builds on the National Environmental Monitoring Standards data exchange protocol (Environmental Observation Data Protocol) developed between NIWA, Landcare, GNS, and Horizons Regional Council.

Although work is underway to standardise data and processes, central and local government agencies and crown research institutes currently generally use manual, one-to-one processes to send, search and access data. The result is that each time data is shared amongst agencies, the data must be collated, cleaned and reformatted before even basic use or analysis of the data may begin. That takes a lot of time!

The e-IDI was created by applying a linked data approach which allows data from multiple sources and different systems to be brought together using a common set of standards and vocabularies.

What the e-IDI can be used for

There are many uses for e-IDI.

  • Nationally accessible real-time linked data. This data could be used across NZ for disaster management (ie: flooding)
  • Ability to filter, group and compare sites across regions and time series. This data can inform multiple monitoring and analysis decisions.
  • Machine readable data to support use across various machine-to-machine platforms. The streamlining of accrual and manual processing has a number of financial and efficiency gains.
  • Standardised definitions and vocabularies for ease of interpretation. This allows for greater interpretation and use of data in a standard way.
  • It’s a project that’s at a proof of concept stage.

Water quantity data only at this stage

Water quantity data was piloted because the data is operational and non-personal. Additionally, water quantity data was already standardised in real-time for the Land, Air, Water Aotearoa (LAWA) website in 2015, making it easier than other domains to test in a shared infrastructure.

What happened next

The e-IDI was funded just for a proof of concept, not for a pilot and for rolling out an operational system across the sector.

The Ministry for the Environment and Horizon's Regional Council ran workshops from early November 2017 to share what was learnt from the proof of concept.

Reports and other outputs

Reports are available in both PDF and markdown formats, in keeping with linked data’s human and machine readable requirements.

All the reports and other outputs are openly licensed, as the PoC’s main aim was learning and sharing the results. We have also used persistent identifiers for the reports.

  1. (Decommissioned) The e-IDI Proof of Concept Beta site, a brokering solution that transforms different data from disparate systems into a harmonised form for integration is here:  http://envdatapoc.co.nz
    Watch the walk-through video for the e-IDI Proof of Concept Beta site.
  2. ​(Decommissioned) The  interactive river flow data explorer for the e-IDI was built with RShiny, view the interactive flow data
    Watch the 
    walk-through video for the RShiny application
    The code for the RShiny application is here
  3. The summary report is a short, plain summary of the project and recommendations. 
    eIDI Summary Report [PDF, 710 KB]
  4. The evaluation report has more details about the project and is shorter and easier to read than the technical report.
    eIDI Evaluation Report [PDF, 1.1 MB]
  5. A long form technical report of the e-IDI PoC, which includes data models,  recommendations and appendices 
    e-IDI technical report [PDF, 1.8 MB]
    Markdown versions of the two summary reports are included as appendices in the technical report from here http://purl.org/net/e-idi/doc/poc-technical-report
  6. The application for Better Public Services Seed Funding: Seed Fund Application 
  7. The supporting documentation for the application, which includes the analysis and background documents: BPS seed fund application - supporting documentation
  8. A visual narrative of the application: A4 Visual Narrative 
     

    Disclaimer

    This web page was established for the Environmental Integrated Data Infrastructure Proof of Concept (e-IDI POC) project and was for use during the e-IDI POC project. The information and data provided on this web page are for the purposes of the e-IDI POC project only and are not recommended for operational use.

Reviewed:
25/07/18