Soil data sharing

Key Points

We implemented a functional and useful soil data federation system accessible via a soil data portal that includes public sector soil data.
Farmer groups and catchment managers can access a web-based spatial soil data management system that suits their purposes and is relevant to their location.
For many of the project participants, data and metadata management and stewardship practices are often immature and ad hoc.
Farmers view researchers and research organisations not sharing their data as a barrier to sharing their own data.
Encouraging universities and research agencies to serve data to the soil data federation is considered equally important in the project.
Displaying the data for the best end-user experience remains a challenge.

Our Research

Our primary goal was to design and implement a shared soils data base that fulfilled the requirementsof researchers, farmer groups, catchment managers and other stakeholders.

To achieve this, we visited each of the project participants to elicit their views on the value proposition for the Visualising Australasia’s Soils (VAS) project. Interviews were designed to explore soil issues and needs, expectations, motivations, and concerns about the project.

It became clear that the preferred model for the VAS system is one based on a federated data supply model, in which the data remains in the control and management of the custodians. However, as not all the participants have the technical capability to interoperably serve their data according to the required standards, we constructed a soil data aggregator as an interim solution.

The aggregator allows the participants to load their data to the federation via secure cloud-based data infrastructure. While the open data is visible to the public, participants’ data is made private by default. It is visible to nominated individuals via authentication in a standard internet browser, which was the method preferred by all project participants.

Project partners have access to basic tools to filter and graph their data. Filters can be used to select data by parameter, date, and value. For sites that have test values recorded over different dates, the trends over time can be viewed. Similarly, for multiple samples taken at a single location, their values can be graphed by depth.

We have developed seven educational videos, uploaded to YouTube, to demonstrate key functionality of the VAS system. Each video, varying from 30 seconds to 2 minutes, focusses on a different piece of functionality and how it can be accessed.

Research Findings

The value proposition of the VAS for farmer groups and their members is clear, especially if it includes an intuitive toolkit of functions that makes their data relevant to their geographic patch and supports their stakeholder communities.

An often requested improvement is the inclusion of contextual data alongside the soil data, such as paddock histories, treatments, yields, or climate records, so that decision support tools can be auto populated with their latest data. However, this would require data custodians to share their data with users outside of their trusted networks, a value proposition that is yet to be explored.

A clear lesson from the first phase of the VAS project is the generally poor level of data literacy among soil data custodians. Most participants recognise the unfortunate reality that high-quality soil data may be lost, forgotten or ignored because of inadequate data management systems in the office, and due to staff turnover.

However, for those data that have been mapped into the system, they now conform to international data exchange standards and are, with consent, interoperably available for any future purpose, including as data for other tools and applications, artificial intelligence engines, or decision support systems.

The soil data are more incongruent than we expected and require harmonisation to ensure that value comparisons are valid.

We are still investigating how best to display the massive volume of open public soil data that exists for Australasia in a user-friendly manner on the VAS data portal.

In the current public view, only a limited number of soil data sets are visible, as we are uncertain whether adding all the possible (1,628+) datasets would be of value.

Significance of Findings

We now have a functional and useful soil data federation accessible via a soil data portal. Australasian soils data from disparate sources can now be federated into a system that makes them findable, accessible, interoperable and reusable (FAIR), and shared, subject to the access rules set by the data custodians.

Bringing together the data has been unexpectedly challenging because of the lack of metadata, immature data stewardship practices, poor data literacy of the participants, and the incongruency of the data. Harmonising and modelling the federated data into reliable products has posed a significant challenge.

Next Steps

We plan a demonstrable value case that encourages soil data custodians to share their data.

We want to demonstrate the value of harmonising disparate soil data to create aggregated deidentified views. These could be used to benchmark areas, show trends over time, propose fit-for-purpose soil performance indicators or identify significant research gaps. The aim is to encourage collaborative new projects based on the evidence drawn from data analytics supplied by the VAS.

In Phase 2, we will introduce existing tools that value-add to the data, develop automatic data entry from sensor systems, and soil data from legacy projects. By expanding the range of tools and visualisation capability of the data, we hope to encourage new discoveries in the data for both soil practitioners and researchers.

Download Fact Sheet