Open Data versus the Coronavirus

Unfortunately due to the uncertainty surrounding the Wuhan novel coronavirus outbreak in Hong Kong, it has been decided to call off the International Open Data Day 2020 event and hackathon being planned in Hong Kong. This is a particular pity because Open Data is needed more than ever to get the outbreak under control, fill the information vacuum leading to conspiracy theories and shops running out of food. Life-and-death situations are particularly susceptible to wild speculation and panic filling the vacuum, and closed risks lives.

While it’s frustrating that most public meetups and gatherings for the next few months have been cancelled in Hong Kong, for people that want to do something to help or at least get themselves more informed, with modern digital technologies we fortunately don’t have to leave the house. To the rescue there are some virtual events specifically focused on understanding and tackling the outbreak using open data and data science being organised.

Coronavirus open dataAll this makes the coronavirus crisis a perfect opportunity to harness the benefits and showcase the utility of open, data driven approaches. Firstly, our friends at Open Source Hong Kong are running a project collecting data on the outbreak from the news, press release and social networks, and releasing it under an open license on GitHub. Even the Hong Kong Government seems to be following more data literate approaches, setting up a real-time Hong Kong Coronavirus dashboard very similar to the global Johns Hopkins University CSSE one (and likely leveraging their connections with ESRI).

Finally, ODHK are pleased to participate in an interactive, online event organised by DSHK: the data science community of Hong Kong, on Sunday 9th February from noon to 1.30pm Hong Kong time. Following in the tradition of the “Hack the Zika Virus” hackathon we participated in back in 2016, we hope this can empower citizens with the knowledge and tools to better equip themselves to understand and join the fight against the outbreak. Dr. Guy Freeman from DSHK, Dr. Scott Edmunds from ODHK and GigaScience, and Professor Ben Cowling from HKU will present a 90-minute workshop on how to understand and analyse the Wuhan coronavirus, from gathering relevant data to analysing and visualizing its spread. Sign up and subscribe to the youtube channel to watch and ask questions online, or catch up and watch the archived live stream later.

Rundown on Sunday 9th February from 12 noon HKT (UTC +8):
12:00 HKT: Dr. Guy Freeman from DSHK will kick things off by giving a data science perspective on gathering online data for one’s own purposes, e.g. the number of cases over time, or search volumes of certain terms.

From 12:30pm ODHK’s Scott Edmunds will talk about the open sequencing data, what is available and how to analyse and understand it, e.g., via open and interactive tools like nextstrain.

From 1pm Professor Ben Cowling will talk about the data and calculations behind measuring the transmission dynamics of 2019-nCoV that were used to write his NEJM paper.

As an online hangout hopefully there will be an opportunity to ask questions online and via social media.

Follow along on youtube here:

Celebrating the Success of Hong Kong’s Citizen Scientists

Open Data Versus The Dengue
With at least 23 locally transmitted cases of Dengue in Hong Kong this year – this is the cases reported since records began in 1994. This summer has also seen a sharp spike in the mosquito carried West Nile virus infections in Europe, following soaring temperatures. At least 400 cases of the disease have been recorded, with 22 deaths, according to the European Centre for Disease Prevention and Control (ECDC). The increase in these mosquito carried disease is thought to be linked to global warming, with an extremely hot and wet summer in Hong Kong, and record heatwaves in Europe.

Hong Kong's Citizen ScientistsThe rapid and deadly spread of these diseases requires an equally rapid response, and intelligent data driven approaches in tackling them could give us a key advantage in this fight. Singapore provides excellent KML mosquito breeding site data, but unfortunately in Hong Kong the authorities limited Oviposition Trap detection system only covers <2% of our territory, and other than very archaic PDF forms. Unable to understand the benefits of open data that Singapore is seeing, the FEHD will not share the supporting data (see our attempts at making FOI requests), with their representative stating on the Pearl Report that “The more information, the more unnecessary misunderstandings”.

School Children 1: Professionals 0

Hong Kong's Citizen ScientistsWe’ve previously written about Hong Kong’s participation in the “Global Mosquito Alert” alliance of citizen-science organisations bringing together thousands of volunteers from around the world to track and control mosquito borne viruses. These efforts also lead to kick-off of a network of its own: CitizenScience.Asia – bringing together Citizen Science projects and practitioners in Hong Kong and across Asia. Coming out of an ODHK-co-promoted “Zika-hackathon” a Cantonese version of the Mosquito Alert app was developed and promoted. Working with schools, the Chinese Foundation Secondary School did an amazing job testing the app with their students, presenting their efforts at the HK SciFest 2017 at the Hong Kong Science Museum. These efforts have now paid off, with the Hong Kong (mostly school children collected) datapoints being ingested alongside another 4,000 observations into the GBIF—the Global Biodiversity Information Facility. This is an international network and research infrastructure funded by the world’s governments, and is the main home for open access to data about all types of life on Earth. Zooming into East Asia you can see a small but prominent cluster of observations in Hong Kong. Proof that even school children can contribute to the global sum of research knowledge.

This is even more impressive as the Hong Kong research community is particularly conservative, with very few researchers depositing their data in global databases such as GBIF. Despite being a global norm and mandated by a growing number of research funders in Europe, North America and Australasia, and with even the Ministry of Science Technology in Mainland China announcing new policies ‘promoting open access to, and sharing of, science data’. Hong Kong in contrast has no policies, and the chair of the main research funded has publicly stated that he believes sharing data would reduce Hong Kong’s competitiveness. It is heartening that at least our school children are putting their elders to shame in contributing to the global sum of knowledge, and Hong Kong’s Citizen Scientists will not make the same mistakes the supposed “Professional” scientists will.

This is one of many examples of the potential of Citizen Science to capture the interest of the Hong Kong public, and be a leader and hub for these activities in the region. Another being in the recent City Nature Challenge, where utilizing the iNaturalist App Hong Kong ranked 8th in “most observations” (20,268), 4th in “most species” (2,934), and 7th in “most observers” (755) among 68 cities in the world. As a first time participant Hong Kong really put itself on the map, contributing the most new species: 57 out of 124 global new species (46.0%) in the iNaturalist database. The consequences of this are starting to be felt, with the Tai Tam Tuk Foundation organised Hong Kong Inter School City Nature Challenge kicking off this November, and visitors coming to see how we did so well.

Next week Professor Hiromi Kobori from Tokyo City University is coming to Hong Kong to fact find with the Tai Tam Tuk Foundation on City Nature Challenge and Citizen Science opportunities across the region. Prof Kobori has been a citizen science advocate in Japan for a long time, and is also the main organizer of the Tokyo City Nature Challenge. Our colleagues at CitizenScience.Asia and DIYBIO Hong Kong have organized a meetup and talk from Prof Kibori on the “Characteristics and Challenges of Citizen Science in Japan”. On Thursday 13th September head out the all new MakerBay in Tsuen Wan from 7pm, and there are plans to live-stream and film the talk, so sign up on Facebook and watch out on social media for links for that.

ODHK lobbying open data policy via the UN UPR

DaY5rPjU0AAaIG-.jpg largeHong Kong governance and transparency is under the spotlight in a landmark submission to the United Nations Human Rights Council (UNHRC) for the third Universal Periodic Review (UPR) on China. It has been presented to the UNHRC for a review that will take place in November. Open Data Hong Kong is among 45 civil society organisations contributing feedback to the Hong Kong UPR Coalition, representing the collaborative efforts of civil society to hold the Hong Kong SAR government accountable to its human rights commitments. Today a press conference (pictured) presented the submission and introduced the many signatories who were involved in the process.

The submission details 109 recommendations, and Open Data Hong Kong has specifically provided feedback on its data and archiving policy. The lack of an archives law or robust and binding access to information legislation is one of the many barriers that has held back open data in the Hong Kong SAR. The specific sections relating to Freedom of Information and Archives law are recommendations 26 and 27.
Screen Shot 2018-04-10 at 14.23.44While a subcommittee of the government has been working on this issue for over half a decade, and mentions of the issue were included in Hong Kong Chief Executive Carrie Lam’s election manifesto, we applaud efforts to keep the topic on the agenda and (unlike our archives) not gather further dust. More specific data related feedback on top of the points listed above that we would also promote is the need for all public/statutory bodies to included in the ordinance, as they such a huge and ever growing part of Hong Kong governance and are not subject to even the weak access to information code we have. As well as removal of the fuzzy and non-interoperable (non-creative commons) licensing restrictions of the data we currently have in Hong Kong.

Simon Henderson, the spokesperson for the Coalition and Senior Policy Advisor at Justice Centre Hong Kong says of why this coalition was put together: “The submission provides a roadmap of specific, measurable and achievable recommendations for Hong Kong to abide by its human rights commitments and restore its international standing. Our submission reflects the aspirations of the Hong Kong people who want to build a fairer and more equal society for all. Importantly, it also echoes Chief Executive Carrie Lam’s vision of making Hong Kong a more inclusive city.”

Engagement with civil society is crucial to the UPR process. Unfortunately, quite often, civil society is on the sidelines when it comes to major legal and policy developments, in contrast to the Chief Executive’s pledge to “connect”. “Having meaningful consultations, meeting regularly and promptly responding to correspondence will go a long way in engaging civil society. The UPR is a test for the government to show that it is truly committed to protecting Hong Kong’s core values. We look forward to working with the government to implement these recommendations,” he concluded.
This makes Open Data an important part of this submission, as it should help provide more scrutiny and connectedness, facilitating meaningful consultation and data driven decision making in Hong Kong, and is great to see these issues taken to the UN. We’ve previously hosted meetups covering open data for political advocacy and legislation, and this takes discussion to a global level. The upcoming UPR on China, including Hong Kong and Macau, will take place in Geneva in early November 2018. The UPR Working Group, which consists of the 47 member states of the UNHRC, will conduct the review.
The submission is available online at:

HKMA missing opportunity to introduce a foundation for FinTech

Last week the Hong Kong Monetary Authority launched a consultation for “Open API Framework”. The consultation is part of “seven initiatives announced by the HKMA to prepare Hong Kong to move into a New Era of Smart Banking” with the aim to “facilitate the development and wider adoption of API by the banking sector, thereby stimulating innovations and improving financial services through collaboration between banks and tech firms”.
Despite opening up consultation mid January 2018, the original target date to “finalise the policy on Open API” was “by the end of 2017” so it’s unclear if this goal-post is being delayed or not or how serious the HKMA really is for this consultation. More is unclear – under the label of “smart banking”, it’s unclear just what that is. A label left undefined is a label that is hard to argue against. Even a speech by Mr Norman Chan, Chief Executive of the HKMA shepherding “A New Era of Smart Banking” was unclear on the definition beyond listing the laundry list of seven initiatives lacking connective tissue to give a picture of the whole. The convenience of the “Smart Banking” monicker is that it is open-ended enough of a term to be subject to interpretation, and, like other “Smart anything” labels that governments like to attach to initiatives (like “Smart City”), is it sounds good enough to sound like progress no matter what is done.
With the “Open API framework”, this initiative sounds just geeky enough for an industry analyst to gloss over the details and miss out on its importance, and lost opportunity. The Open API framework has roots in the UK, as part of “Open Banking”, a label that already has a recognised definition and history. The HKMA doesn’t have the baggage of Open Banking by using the ambiguous “Smart Banking” term instead, and define it – or not define it – as it sees fit. Taking effect this year, the UK introduced legislation mandating the opening and connectivity of top UK banks, making it more efficient for consumers (personal and commercial accounts) to compare banking products and services, as well as connect third-party banking services (such as lenders and accounting software). It creates the foundation for a more competitive banking environment, as well as an ecosystem to support fintech, opening up much opportunities for achieving the goals shared by HKMA for smart banking: “[facilitate] very rapid innovations that help provide personalised services and much better customer experience”. The UK initiative is taking off worldwide, with much attention. Without much choice, the top UK banks are cooperating on implementing Open Banking, with agreement on its DNA: the Open API framework, which is the set of standards and protocols for data connectivity between banks, third-parties, with full control by the consumer.
The mandate of Open Banking is to “improve competition, efficiency and stimulate innovation” leveraging how “data could be used to help people to transact, save, borrow, lend and invest their money”. Security is built-in from the ground-up, giving consumers control (through permissioning and revoking permissions) over their data. Built on principles of open data, the framework opens up connectivity for the fruits of fintech to flourish, among the connective tissue in an ecosystem of banks, consumers, and data. Without Open Banking, everything remains as it is: a negotiation, trial-and-error, and barriers to exchange and connectivity.  
This is clear in the review of the HKMA Open API consultation. HKMA is skipping the UK’s Open Banking practice of adopting legislation to force banks to participate and cooperate (and compete). As spelled out in the HKMA consultation, the intended benefits and goals are disconnected from the mechanisms to get us there. The HKMA’s Open API framework is a broken set of vague objectives and  inadequate standards. Banks are left to voluntarily adopt their own standards – which are not standards at all – on their own timelines and roadmaps.
The consultation document is honest in its comparison of the UK’s Open Banking initiative and HKMA’s Smart Banking initiative: HKMA will not risk bank profits (or “competitivity”) with Open API, but help nudge them enough to stay up-to-date with international trends:
“The UK needs to adopt this model because there is a specific mandate to address – to allow personal customers and small businesses to compare products and switch between banks – and therefore a set of focused and standardised Open API could be specified“ (paragraph 61) this coincides with “…the policy objectives of Open API for the Hong Kong banking sector to maintain its competitiveness and to offer innovative/convenient service to improve customer experience are general in nature.” (paragraph 64).
If done effectively, Open Banking lays the foundation for consumers and the industry to benefit with competition and connectivity to a digital economy. Without it, and loose strands of Open API standards and disconnected initiatives such as those under HKMA’s “Smart Banking” direction, at best the benefits remain to be seen.

Hong Kong’s parking data failure

Screen Shot 2018-01-18 at 15.16.02Parking is a big deal in HK. It’s a nauseating vicious cycle on the streets: cars circling in search for parking spaces add to congestion, making it even more difficult to find available parking spaces. Compounded by ever-increasing number of drivers and cars on roads in an already dense city bursting at the seams, congestion and traffic delays have a detrimental effect on city efficiency, navigation, and liveability. Some drivers give up, with little consequence, parking illegally and gambling on getting an affordable parking ticket that just makes it worth it to occupy illegal spaces anywhere they can find and park.
The solution seems straight-forward: if drivers can know where their needed parking space is available, then they can just get to them and be on their way with their day. It’s nothing new, really, many cities around the world already employ this tactic and take steps to easing their congestion. Not for lack of trying, this is not the case in Hong Kong. Still stuck behind the times, the government has adopted a paradigm of pushing their own apps (which are already under fire for being a waste of resources to develop) instead of focusing on the accessibility of the data for users, app developers, and anyone else willing to mine the data for patterns that can become insights to diagnose the problem and arrive at solutions. The effect has been locking valuable data behind an app, and leaving an industry of app developers and collaborators in the dark to duplicate government efforts to make the same information available, resulting in a lacking experience for users alike.
A year ago, Hong Kong’s Transport Department underestimated the ease with collecting and disseminating parking space availability for users via their “eRouting app” stating “only slight adjustments to its existing traffic information system is needed such that real-time information on parking vacancies can be disseminated to the public through websites and mobile applications. The implementation of this arrangement will probably not require a huge amount of resources“. A press release half a year ago says the app “contains information provided by car park operators on real-time parking vacancies at some car parks. The [Transport Department] will continue to encourage more car park operators to provide their parking vacancy information so as to help the motorists searching for parking spaces via ‘Hong Kong eRouting’.”
It would be hard to complain about the eRouting app if it even modestly delivered on its commitment to provide “parking information: on-street parking space and car park information (including real-time parking vacancy information of some car parks)”.
By my count:

  • Out of 1465 total parking spaces or parking lots listed in the app:
    • A total of 135 (9.3%) can tell you if they have space available or not.
      • only 112 of these actually indicate the number of spaces in “real-time” (7.6%), with:
      • the remaining 24 only indicating if they have spaces or not (1.6%)

How useful is an app that can tell you what you need to know roughly 9.3% of the time? Not very.
This doesn’t make the eRouting government app very useful. What would be useful is if the government made this data at least accessible publicly, instead of locked away, behind Terms and Conditions (called “Important Notices”) that states:

“any reproduction, adaptation, distribution, redistribution, dissemination, modification, copying, uploading, transmission, retransmission, commercial exploitation of the Work, publication, or making available of the Work to the public is strictly prohibited”

In no uncertain terms, even if you could crack this data, you are not permitted to do anything with it unless given permission. Oddly, the government does provide “real time parking data” to the public, but sadly, this data is even worse than the app data, demonstrating further the Hong Kong government’s break with the aims of other governments, that open data provided to the public should be as useful for the government itself to use.
Hosted on the government’s “Public Sector Information” portal of data (at, the oddly-named government’s supply of “Real-Time Parking Vacancy Data” is anything but real-time parking vacancy data. It’s a list of parking lots – and a plethora of other data (like location, opening hours, website, contact info, charges) – except the number of spaces available real-time. It’s useless unless you want to plot the parking lots of HK. It is unclear who this data can be useful for. This seems to further reaffirm that the Transport Department has not made progress in understanding what Open Data is.
This data set is altogether counter-intuitive, considering the data portal was built on a “[belief] that the community, particularly technology startups, can develop more innovative and practical applications and services with their creativity and ingenuity. The PSI initiative helps to open up new business opportunities, bring convenience to the public and benefit society as a whole.“. The irony here is we have two government departments working either in opposition, or both failing to deliver on goals and commitments.
The lack of useful real-time parking data on the PSI portal can already claim on parking app victim: TingPark, hailed as a savior to driver’s parking woes and a startup darling and incubated project of Cyberport, has gone bust after spending a few hundred thousand dollars in Cyberport seed money. The app simply couldn’t use the PSI portal’s data in any useful way, or even gather the data from parking lots directly. It’s unfortunate that the app developers couldn’t build on the Transport Department’s progress or resources either.
The recommendation here is straight-forward:

  • Provide the same data set supporting the eRouting app as data on the government’s PSI portal, and have this data support the app (and not the other way around);
  • Deliver on the commitment to gather and disseminate real-time parking vacancies across Hong Kong, as data at a minimum, or provide incentives for the community to gather and disseminate this data.

Further reading

Guest Post: A Data Driven Look At Refugee Crime

Like many societies, Hong Kong is having a heated discussion about immigration. Especially in regards to refugees. A common believe here is that refugees commit more crime than the general population and that most criminals are of South East Asian ethnicity. Further some have suggested the increase in refugees has let to a general increase in crime within Hong Kong. This has let to strong comments by some politicians (e.g. Dominic Lee in Sham Shui Po calling for internment camps). However, there is surprisingly little public data available to base these on.
Therefore, Open Data Hong Kong has attempted to acquire some data on the topic, especially Scott Edmunds who has spend a lot of time collecting the data by contacting individual police districts and police regions in Hong Kong through accessinfo requests. So here I will take a look at the data and see if I can find some answers.

The data

First I should mention something about refugees in Hong Kong in general. I was unable to find some accurate numbers on the total numbers of asylum seekers in Hong Kong. According to the immigration department there were around 9 618 people claiming asylum in HK in 2014, 10,922 in 2015, and 9,981 in 2016.

HK never joined the UN Convention on the Status of Refugees, and asylum seekers can only apply under the UN Convention Against Torture. Or at least cite as a reason for protection. Furthermore, the recognition rate is very low. About 0.16% of applicants are accepted (the global average is 27%). The application process is quite slow as well. This results that many applicants stay in the city for years and many asylum seeker whose application have been rejected cannot be deported due to a lack of extradition agreement with the corresponding home countries. During their stay applicants, as well as those who are rejected, are not allowed to work, but the government provides some minimal rental, food and medical subsidy (HK allocated HK$450 Million in the budget of 2013/2014). Some have suggested that these subsidies are too low to maintain a living in HK and provide incentives to be involved with criminal activities. The majority of claimants are from South and South East Asia.

To asses crimes committed by refugees in HK I took a look at the data provided by Open Data Hong Kong, as well as publicly available census data and crime statistics. Unfortunately, not all police districts in HK were able to provide the criminal statistics of refugees. In fact only West Kowloon region was able to provide a complete picture across their district. Furthermore, these numbers are arrest statistics and not convictions (ODHK has collected data showing roughly 50% of arrests result in convictions). So any conclusions should be viewed with care.

Is there an increase in arrests?

This question is relatively easy to answer and I have plotted the overall number of arrests for each region by year below.

As you can see there seems to be no overall dramatic increase in arrests for all of the regions. However, there is a slight increase in crime Kowloon East and West, but in general the trend points downwards. This would suggest crime in HK is not increasing.

Arrests of refugees

Since I only have limited data available about refugees in HK I was only able to look at Kowloon West. Hence I compared the number of arrests of refugees with the total number of arrests within this region.

Let me explain this plot in bit more detail. I used data available for 2014 and 2015. Since Hong Kong does not use the phrase refugee as Hong Kong does not recognise the UN Refugee Convention, so the exact legal classifications are a bit vague. Nevertheless, some police stations have called refugees “Form 8” (F8) holders, so I will use this phrase here as well. Thus the plot above shows the number of arrests made in Kowloon West by F8 holders and all arrests between 2014 and 2015.
So comparable those arrests rate look quite small. Indeed in 2014 and 2015 the proportion of arrests of F8 holders was 4% and 5% respectively. So these numbers seem rather stable and would suggest no major change between 2014 and 2015, despite a slight increase in the number of refugees.

Do refugees commit more crime than others?

This question turned out to be much more difficult to answer than I thought. One problem is that I do not know how many refugees live in Kowloon West, further police districts are not the same as council districts. This makes it difficult to get an population estimate since the census data from 2011 only looked at council districts. Thus I am unable to answer this question with the current data. Only the availability of the exact arrest numbers of refugees for the whole of Hong Kong or the exact numbers of refugees living in Kowloon would help to answer this question.


There is no evidence of an increase in crime in Hong Kong (at least from the data available), also there seems to be a slight increase from 2014 to 2015 (looks more like random noise to me however). Arrests of F8 holders was relatively stable between 2014 and 2015. Intuitively I think the proportions of arrests of F8 holders are higher than one would expect given a small population of around 10,000, but one needs to keep in mind that arrests are not convictions. In general the data is not really sufficient to make a conclusive statement. Except that HK is incredibly safe compared to other major cities (0.4 murders per 100,000 people in 2016; one of the lowest in the world).
Thanks for Open Source HK for putting together the ODD event this year. See the interesting round table discussion with three LegCo members on our periscope stream. For more on this particular project, interact with this data via some Shiny visualisations and the raw data being collected at ODHK’s CKAN installation.