Why public utility companies and government-owned companies must publish open data.
The government of Oman has put a significant amount of effort into promoting the release of open data by government entities. This effort should not be focused only on the government itself but must expand to reach public utility companies and government-owned companies, especially as the number of these companies, and the amount of data they control, continues to grow.
The government of Oman operates through a culture of secrecy, where everything is assumed to be confidential unless proven otherwise. This culture restricts the ability of the government to be truly open, and substantially limits the flow of information between and within government entities. This secrecy is a result of a number of factors, such as the lack of proper governance structures for decision-making within most government entities, the unjustified fear around any security-related issue, and the frequently misunderstood Law on the Classification of State Records and Governance of Protected Places (the “Classification Law”).
For data to be truly considered open, the data must be complete in regard to the content of the data itself and in regard to the metadata describing the information that an open data publisher is releasing. Creating a comprehensive open data catalogue is an effective way for ensuring that published open data is accompanied by the minimum required amount of metadata.
Even though many government entities in Oman have published open data in the past, the majority of these government entities do not include a sufficient description of the data they publish, and instead upload a collection of Excel files that do not include any details other than the file name.
This webpage shows an example of a government entity publishing a collection of files without any metadata or other useful information to provide context for the open data it publishes.
It is highly recommended that an entity creates an open data catalogue to ensure that the data is properly identified and easily accessible. Such a catalogue should reasonably be made published on the internet and should be the primary method for enabling users to locate and download the data.
In particular, this open data catalogue should include separate entities for each dataset covering the following elements:
Title
The catalogue should include a short title describing the dataset in question.
Description
The catalogue should include a brief explanation of the contents of the dataset, a description of why this data is significant, and the manner in which it was collected.
Period Covered
If applicable, the catalogue should provide details of the time period covered by the data. For example, if the data relates to COVID-19 deaths, the data should indicate that the data covers the period from March 2020 to January 2021 (as an illustration).
Update Frequency
If the data is expected to be updated, the data catalogue should indicate to the user how often this data would be updated. For example, daily, monthly, yearly, or any other frequency. If the data is supplied in real-time, the catalogue should also indicate this. Certain datasets, such as maps or other reference datasets, might not be updated on a regular basis or at all, and this information should also be indicated to the user.
File Format
In addition to the importance of publishing open data in a machine-readable format, it is equally important to indicate to the end-user what format exactly is available for download. Examples include CSV, XML, and JSON.
Name and Contact Details of Person or Department
For big organisations, it is important for the end-user to have the contact details of the person or department responsible for the dataset in question. This is useful from a data validity point of view, but can also assist end-user that seek clarifications and those who would like to report errors found in the data.
Metadata
In addition to the general description details of the dataset, the catalogue should include a description of each field in the dataset explaining the content of that field and the format of the data provided in that field. For example, if the dataset has a name field, the catalogue must indicate if this will be provided in the Arabic or English language, and if the dataset has a date field, the catalogue must indicate the date format in which this will be provided (e.g. YYYY-MM-DD).
Example
The points explained on page can be used to transform the example shown above to become a more useful listing as follows:
Dataset Description
Title
Number of hospitals and beds in the Governorate of Dakhiliya
Description
This dataset provides the details of the hospitals in the Governorate of Dakhilya with the details of the number of beds in each hospital.
Period Covered
This dataset captures the state of hospitals in this region in December 2020.
Update Frequency
Yearly at the end of every year.
File Format
XLSX
Contact
Directorate General of Health in the Governorate of Dakhiliya. Name: XXXX Email: [email protected]
Metadata
Field
Description
Wilayat
The name of the Wiyalat in English.
Hospital Name – AR
The name of the hospital in Arabic.
Hospital Name – EN
The name of the hospital in English.
Beds
The number of beds in the hospital.
Each published dataset requires an individual description that provides the general information on the dataset, as well as an individual metadata table that explains what each field in the dataset refers to.
GCC countries have invested significant resources in developing national data portals, but most of these portals have yet to publish a single dataset relating to the biggest global pandemic of our lifetimes, COVID-19. Instead of utilising their familiar and tested platforms for releasing data, GCC countries launched independent dashboards for making COVID-19 data available to the public. The issue with this approach is that these new dashboards fail to comply with the most basic principles of open data. This has limited the ability of members of public to use this data in ways that would otherwise have contributed to managing the crisis.
The Omani government has reacted swiftly to the COVID-19 pandemic and has shared with the public substantial amounts of data in regard to the cases, recoveries, deaths, and vaccinations. However, regrettably, the COVID data published by the Omani government does not comply with the majority of principles of open data, and this limits the benefits and utility of this data.
Oman has signed a significant number of international agreements with countries and international organisations on a variety of matters ranging from taxation and investment, to human rights and the environment. This dataset provides a list of the international agreements that Oman has ratified by royal decree from the year 1974 onwards. The dataset includes the title of the agreement, details of the ratification royal decree, the date of signature, the classification, the counter-party, and the subject.
Dataset Description
Source
This dataset is based on the open data published on Duwaly.gov.om, and later independently updated from the data published on Qanoon.om.
Time period covered
1974 to 2021.
Frequency of update
Yearly
Metadata
Field
Description
Code
Serial code to identify the treaty.
Title – AR
The title of the treaty in Arabic.
Title – EN
The title of the treaty in English.
Royal Decree No
The royal decree number in the format of YYYY-NNNN where YYYY is the year in a four-digit format and NNNN is the sequence number in a four-digit format.
Royal Decree Date
The date on which the royal decree was issued in the format YYYY-MM-DD.
Signing Date
The date on which the treaty was signed in the format YYYY-MM-DD. This usually is only available for bilateral treaties.
Classification
The classification of a treaty as bilateral, closed multilateral, or open multilateral. Bilateral treaties are signed between two states. Closed multilateral treaties are treaties signed by more than two states and open only to a select group of states. Open multilateral treaties are treaties that are open for any state to join.
Counter-party
The counter-party to a treaty. This applies only to bilateral treaties.
Subject
The subject of a treaty. This applies only to bilateral treaties.
Today, the Ministry of Transport, Communications, and Information Technology issued a new policy on government use of AI technology. Similar to other policies issued by the MTCIT, this document is not legally binding it is essentially a form of instructions and recommendations for government entities.
The policy appears to have been finalised last June and is addressed to government entities only. It sets out six very high level principles for the use of AI: inclusiveness, human-centrism, accountability, fairness, transparency, and safety.
The key concrete policy provisions include:
A requirement to undertake a risk assessment in regard to the use of AI (but no details as to when such a risk assessment should be done, if it is a one-off requirement or a periodic requirement).
A requirement to ensure that no AI bias takes place and disclose to those affected by the AI system of the existence of such a system and the impact it may have on them.
A requirement to obtain prior approval of those affected by the AI system.
The policy also briefly mentions a requirement to comply with open data policies.
You can read the Arabic version of the policy on this link.
This dataset covers the key COVID-19 corona virus data, such as cases, recoveries, deaths, and hospitalisation, as published by the official Omani government COVID-19 twitter account.
The date on which the COVID-19 data was reported in the format YYYY-MM-DD.
New Cases
The number of newly confirmed individuals infected with COVID-19.
New Cases -Omani
The number of newly confirmed individuals of Omani nationality infected with COVID-19.
New Cases – non-Omani
The number of newly confirmed individuals of non-Omani nationality infected with COVID-19.
Total Reported Cases
The number of confirmed COVID-19 cases since reporting started.
New Tests
The total number of new COVID-19 tests that have been carried out.
Total Tests
The total number of COVID-19 tests since the reporting of tests started.
New Deaths
The total number of newly confirmed COVID-19 deaths.
Total Deaths
The total number of COVID-19 deaths since reporting started.
New Recoveries
The number of newly recovered individuals infected with COVID-19.
Total Recoveries
The total number of total recoveries of individuals infected with COVID-19 since reporting started.
New Admissions
The number of COVID-19 patients newly admitted to a hospital within the last 24 hours.
Current Admission
The total number of COVID-19 patients currently in admission at a hospital.
Current ICU Admission
The number of COVID-19 patients admitted in an intensive care unit at a hospital.
Source
A link to the tweet publishing the data for the relevant date.
Simplified Dataset
wpDataTable with provided ID not found!
Visualisation
wpDataChart with provided ID not found!
Full Dataset
wpDataTable with provided ID not found!
Notes
The dataset has gaps due to the government’s decision at a later stage to reduce the amount of information released and to suspend reporting on weekends and national holidays. You can read our blog post on this issue here.
Journalism has seen an increasingly growing trend of using data to produce stories, enhance reporting, and enable cross-checking. The trend is designed to increase the accuracy of reporting and ultimately win over the public’s trust. Coined as “data journalism”, this newer form of news reporting involves broad practices from using big data or open data in order to uncover stories, to using analytics and coding in order to identify relationships and predict future trends.
Supported by the government’s open data efforts, Oman is riding the data journalism wave and witnessing a paradigm shift in its local journalism. This is seen from young independent journalist’s shift away from conservative reporting that relies on publishing official statements towards investigative reporting. With this new approach, independent journalists go on fact-finding missions relying on open data sources to explain government decisions and to provide its readership with greater context in a simple and digestible manner.
Atheer and WAF are two independent Omani news outlets that have written investigative pieces using what are effectively open datasets. In one piece, Atheer examined the relationship between the newly established Private Office and the Diwan of Royal Court by analysing all royal decrees issued from the 1970s onwards that either determined or modified the mandate of the Diwan of Royal Court. The research for this piece was conducted using Qanoon.om, an independent website for publishing royal decrees in open format. Qanoon.om relies on data from the official website of the Ministry of Justice and Legal Affairs which, although does not publish its data in technically open format, does make them available free of charge. The piece by Atheer would not have been possible without the government data published by the Ministry of Justice and Legal Affairs, and the government data proved to be useful despite not satisfying the technical standards for open data. In another piece, WAF examined the expansion in government-owned companies in the former five-year plan by analysing company data from the website of the Ministry of Commerce, Industry, and Investment Promotion and the website of Muscat Clearing and Depository and then commenting on this government trend.
Although we have yet to see sophisticated forms of data journalism in Oman that rely on analytics or other advanced data methods, open data journalism has proved to be a significant enabler of the Omani government’s primary vision – the development of an empowered knowledge-based society. Wider adoption of open data through the release of more datasets that are free of charge and in easily processable formats would no doubt continue to directly contribute to a richer journalism industry.
Legislation might not be an obvious topic to explore from an open data perspective, but it is regularly considered by international open data indicators as a key dataset that governments should prioritize publishing on the internet in a manner that complies with open data principles. Legislation is one of the most important pieces of data that the government generates and publishes as having access to the law is one of the most fundamental rights in all societies and legal systems.
The government of Oman has made significant efforts in publishing its legislation on the internet and laws can be downloaded free of charge from the website of the Ministry of Justice of Legal Affairs. In fact, the government has recently stopped publishing the Official Gazette (the publication in which laws are made available to the public) in hardcopy format and makes it available exclusively as a free digital download.
From an open data perspective, legislation in Oman is uniquely automatically deemed legally open because the Omani copyright law excludes legislation from copyright protection and makes it part of the public domain. This allows anyone to copy it, re-utilise it, and sell it without the need to seek the permission of the government or pay them any fees.
Legislation is also published systematically and in a timely manner on the internet, making this vital government dataset very accessible and useful to society. However, legislation published by the government still does not comply with certain technical aspects of the principles of open data especially in regard to machine-readability and structure. In particular, legislation is usually published as PDFs that do not easily allow users to copy the content from them, that are difficult to search, and that are unavailable for download in bulk, all of which make this data inaccessible and undiscoverable.
Notwithstanding the technical shortcomings of the legislation data that the government publishes, the fact that the government releases this data on the internet without copyright restrictions has enabled both civil society and the private sector to utilise the data and fill the usability gaps left by the government. This can be seen on civil society projects such as Qanoon.om (a project I co-founded) and Oman Legal Network, as well as private sector projects such as Mohamoon, all of which build on the data openly made available by the government to create more usable solutions for accessing the law.
The openness of legislation data in Oman and the efforts of the government in making the contents of the Official Gazette available in a timely manner on the internet set an example for publishers of other types of government data to release their data openly. This should be celebrated as a success story in releasing government data in Oman.