How AI helped improve crowd counting in Hong Kong protests

  • >>KK Rebecca Lai, Jin Wu and Lingdong Huang, The New York Times
    Published: 2019-07-04 10:26:44 BdST


Crowd estimates for Hong Kong’s large pro-democracy protests have always been a point of contention for years. The organisers and police often release vastly divergent estimates. This year’s annual pro-democracy protest on Monday was no different. Organisers announced 550,000 people attended; the police said 190,000 people were there at the peak.

But for the first time in the march’s history, a group of researchers combined artificial intelligence and manual counting techniques to estimate the size of the crowd, concluding that 265,000 people marched.

The high density of the crowd and the moving nature of these protests make estimating the turnout very challenging. For more than a decade, groups have stationed teams along the route and manually counted the rate of people passing through to derive the total number of participants.

Since 2003, Paul Yip, a social sciences professor at Hong Kong University, has been producing a count of the size of protests held annually on July 1, the anniversary of Hong Kong’s 1997 handover from Britain to China. With the hopes of creating a more robust estimate this year, Yip teamed up with Edwin Chow from Texas State University and Raymond Wong from C&R Wise, a local technology company, to use artificial intelligence to count the crowd at the march.

How AI Can Be Used to Count Crowds

Using open source software, The New York Times developed a computer model to illustrate how artificial intelligence could be used to recognise people and objects moving within a video.

Analysing a short video clip recorded Monday, The Times’ model tried to detect people based on color and shape, and then tracked the figures as they moved across the screen. This method helps avoid double counting because the crowd generally flowed in one direction.

Accurately detecting and tracking a moving object in such a high-density condition is very difficult. Overlapping of people in the crowd and obstructions like umbrellas, protest signs and large backpacks can often confuse the AI system.

On the day of the protest, Yip and the AI team used technology that is much more advanced. They spent weeks training their program to improve its accuracy in analysing crowd imagery.

How AI Was Used to Count the July 1 Protest

On Monday, the AI team attached seven iPads to two major footbridges along the march route. Volunteers doing manual counts were also stationed next to the cameras, to help verify the computer count.

The team’s software ran multiple models with different parameters to track people in the crowd. People could be counted when they crossed a counting line within the frame of the video. Throughout the day, the team monitored the results and adapted to changing variables like the density of people, the speed of flow and different lighting conditions.

“It’s all based on visual cues, such as the brightness value, color, shape, geometry of the pixels, all tied together,” said Wong, explaining how their system distinguished human figures from objects like umbrellas.

The team began exploring the possibility of doing a count with artificial intelligence in June, when protests in Hong Kong were gaining momentum. The newspaper Ming Pao and Cable News, both based in Hong Kong, provided additional support to the team.

“What we deserve is a more accurate, more precise and more verifiable number,” Yip said.

Before the demonstration on July 1, Yip and the AI team tested their software at other marches and crowded places to train and fine-tune the AI model developed by Wong, and to test out the best angles to shoot from. They did a trial run June 16, another day of big protest in Hong Kong, when hundreds of thousands of people marched along the same route from Victoria Park to the government headquarters.

These trials helped them prepare for issues like obstruction and overlapping bodies in the crowd and gave them ideas on how to do a comprehensive count on July 1. For example, the team decided to train their system to detect umbrellas because they are common in Hong Kong protests.

Wong also rewrote the program to run on iPads rather than industrial computers, in order to scale up the operation. The team was then able to set up the devices at more locations. They also had several spares ready to be used in case there were significant numbers of people spilling over onto nearby streets, as happened June 16.

Yip set up a team that surveyed more than 8,000 people on the ground to determine how many people joined the procession halfway, to help calibrate the final result.

Why the Count Matters

Crowd estimates of the protests held each year on July 1 in Hong Kong are often seen as politically contentious, because the turnout is often used as a barometer to measure the strength of the pro-democracy movement.

“From the angle of the social movement, this number represents power,” said Joseph M. Chan, a journalism professor at the Chinese University of Hong Kong.

“From the stance of CHRF, the bigger the number the better,” said Chan, referring to the Civil Human Rights Front, which organised the march. “For the government, they hope for a lower turnout, because then there is less pressure on them.”

Protest organisers often give relatively large numbers, while police do not estimate a total participation figure — but instead release a count of how many people were at the march at a single point in time. This approach does not account for everyone who attends a march, which can continue for many hours. The one Monday lasted more than five hours.

This year’s count was highly anticipated because it came after several large protests in June against a proposed bill that would allow extradition to China.




© 2019 New York Times News Service