Info: This article will discuss the last step in the publishing process, known as the Production Publish phase, publishing your data and making it available for use. Most of the work in this step is taken care of by us (the system administrators), but it’s important to be aware of what’s involved, know what’s required of you, and understand how long this step will take.

What’s involved

There are quite a few steps between when you sign-off your data, and when it’s made available on data.wa.gov.au as APIs and dataset snapshots. While most of the work is undertaken by us, it’s still important that you understand the process and timeline.

• Preview Sign-off: This is the previous step in the process where you checked your preview services and provided sign-off to release your data.
• Added to change queue: As soon as you sign-off your dataset, it is added to the queue of datasets for release.
• Scheduled for publishing: This is the start of the release process. Your data will be examined to decide how it will be released (as APIs, as data snapshots), and which services it will be added to. The big picture will be considered to see if there are other changes on the horizon to enable them to be optimally scheduled
• Communicated: Once a date is scheduled for release, this will be communicated to the data community through the data.wa.gov.au status page 
• Published: On the date selected to release your dataset, it will be published and added to any required existing services.
• Available on data.wa.gov.au: Within a day of the APIs and data snapshots being published they’ll automatically appear on your dataset in data.wa.gov.au’s data catalogue. Your data is now released!

There are only two things you need to do as part of this step:

1. Check that your DataWA metadata record is ready. Back in the second article in this series you created and published your metadata record to data.wa.gov.au’s data catalogue. Before your data is released, it pays to double check that it’s ready for release. Read below for what you should be checking for.
2. One last set of testing. Once your services are published, a few standard checks will be conducted to make sure the data is available, can be retrieved, and can be queried. You know your data very well, so you will need to complete one last set of testing on your published services to make sure everything is in order.

Time: The whole process of moving your data from the preview service into your final published services and communicating will take some time. For an average dataset that is shared publicly, the end-to-end process from when you accept and sign-off to when it is published and available on data.wa.gov.au will usually take between 10 and 15 working days.

Check your metadata record

Some time may have elapsed since you first created your metadata record, so it’s useful at this point to make sure that:

Checklist

1. the metadata record is complete, has a comprehensive description, and all relevant fields have been filled out
2. “Visibility” set to “Public”,
3. Data Dictionary has been provided
4. Update frequency is correct

Added to change queue

As soon as you sign-off, your data is added to the queue of datasets that need to be released. The number of datasets in the queue varies and depends on how many other new datasets are being published at the same time and how many changes to existing datasets are being made.

Time: For the average dataset during an average week, it may take between two and five (2 and 5) working days to schedule the change.

The queue is continually monitored and data is triaged as efficiently as possible. Similar datasets are grouped together into a single change window in order to minimise the impact to users. For example, if both the Department of Transport and Department of Primary Industries have registered new datasets to be included in the Marine public data service then those two requests are grouped together so they happen at the same time.

Info: In most cases, the change queue works on a first in, first out basis. The datasets that have been in the queue the longest are processed ahead of newly added datasets. That said, if you have a need to release your data immediately, please get in touch to discuss prioritisation.

Scheduled for publishing

This is the start of the release process. Once your data reaches the top of the change queue it will be examined to decide how it will be released (as APIs, data snapshots), and which data services it should be added to. The data will also be assessed for its impact on users. Lastly, it will be assigned a change level, and the change scheduled to occur during a regular change window.

There are three change levels, which roughly correspond to the degree of impact your data may have on users.

• Simple change – 5 working days or less. These changes have no, or very minimal impact on end users of data services. These can often be applied very quickly and without the need to formally schedule or announce the change. For example, a simple symbology change to an existing dataset.
• Standard change – 5 to 10 working days. These changes will cause some disruption for users, but can be guaranteed not to break any applications that are connected to data services. For example, releasing a new dataset and appending it to the end of an existing data service.
• Major change – 10 to 15 working days. These changes will cause disruption for users, and run the risk of breaking automated processes or applications using data services. These changes need to be scheduled and announced. This gives the community of data users time to assess, test and plan changes to applications they have that are reliant on the data being changed. For example, a change to the structure of an existing dataset, or the retirement and removal of a dataset.

Communicate

Once a date is set for release of your data, the release date will be communicated to the data community through data.wa.gov.au, including:

• the data.wa.gov.au status page
• directly to you and the business owner contact nominated when you registered your dataset.

Scheduling and communicating changes allows time for the users of data – or a whole data service – to assess the impact of a change, and test and implement any changes required on their end. For example, if a change is made to the structure of a dataset that users rely on, this change window gives them time to:

• see if the change impacts them
• understand the new structure of the data,
• schedule their own application or software changes for the same day as the data changes.

In rare cases, users of data sometimes raise an issue with a planned change. This can result in it being pushed forward, or a request for more details about the reasons for the change.

For new datasets, five (5) working days’ notice is given. For structural changes to existing datasets, it is 10 working days.

Tip: If you want to provide additional contextual information about the release of your datasets, then the metadata record is usually the place to do that. Do this either in the description, or as a link to a page on your corporate website that has details of the dataset and any related initiatives and projects.

Publish

On the selected release date, your dataset will be published. What is involved in publishing a dataset varies depending on its access level (public, restricted, etc.), and which types of service you decided to make available:

• All datasets. Will have their data snapshots process configured for them (if you opted in when you registered your dataset).
• Public datasets. All public datasets will be:
  • added to the public data services available on data.wa.gov.au’s homepage (e.g. Transport, Planning, Marine, etc.). The “Geospatial Theme” that you chose when you published your metadata record to data.wa.gov.au’s data catalogue is used
  • included in Locate, the public map viewer,
  • added to the relevant group on data.wa.gov.au’s data catalogue. Again, this is based on the “Geospatial Theme” that you selected.
• Restricted datasets. All restricted datasets will be added to your agency’s restricted service.

Available on data.wa.gov.au

The day after your services are published they will begin to appear on data.wa.gov.au, including:

• Automatically attached to your dataset on the data catalogue. One attachment for each type of service published (e.g. WMS, Esri REST Services, et cetera), and one for each type of dataset snapshot created (e.g. Shapefile, MapPackage).
• On the @datagovwa Twitter account, and in regular newsletters like SLIPStream.

Important: Let us know immediately if your dataset doesn’t have the appropriate services attached to it on data.wa.gov.au’s data catalogue. This is often caused by a mismatch in the title provided for the dataset in the data catalogue, and the title provided in DUT. They must match exactly.

Testing

Now that your dataset is published, it pays to run one last set of testing. When it is published, a few checks are done to make sure the data is available and can be retrieved and queried. You may want to run a few final tests yourself.

It is recommended that you test:

• That you can connect to and use each of the types of services published for your dataset (e.g. WMS, Esri REST Services, WFS).
• That the dataset snapshots that were created and attached to your dataset can be downloaded, unzipped, and loaded into your preferred GIS software.

If you find a problem

Help: If you encounter any issues during testing or have something you need to clarify, please let us know immediately. If it’s a significant issue, an incident will be raised and communicated via status.data.wa.gov.au to let the community know that there was a problem with the release.

When you report an issue, please provide as much information as you can. This includes where you encountered the issue, any error messages, screenshots of the issue, and the specific service URLs you were testing.