Detecting scraper bots through scroll behaviour

Ever since I first read "Burstiness and Memory in Complex Systems" by Kwang-Il Goh, I have been obsessed with the two formulas showcased in the paper. Burstiness (B), and its just-as-important counterpart Memory (M), let us understand the dynamics of event-based systems. 

We can use them to analyse the behaviour of sent emails, texts or even heartbeats when only the time at which those events happened is known. This allows us to clearly establish which patterns are human-like and which aren't based on a dataset of already classified data. We know humans reply to texts in a bursty manner (the time they take to answer is not uniform) while simple bots respond as fast as possible, thus they have different B and M coefficients.

Goh, K.-I., & Barabási, A.-L. (2008), Figure 4

In previous personal research, I used these two values to differentiate between human and bot sessions based on the timings of their requests, which usually worked, nonetheless, this approach identified sessions as human if they were sent from any browser that loaded CSS and JS files, allowing more advanced bots like ClaudeBot to appear human by using a headless browser. It essentially distinguished between crafted requests versus requests sent from a real browser, instead of bot and human.

The scroll wheel

Some time ago I noticed a distinctly human behaviour that I believe cannot be easily imitated by bots, scrolling. When I am scrolling a page looking for information using the scroll wheel, I usually don't scroll down linearly until I find what I am looking for, instead, I do so with a bursty pattern. For example, that pattern might more closely resemble time series a or e rather than c.

Goh, K.-I., & Barabási, A.-L. (2008), Figure 1

I think most scraping bot developers still haven't honed their scrolling pattern, so I decided to test if burstiness and memory applied to the inter-event times of scroll events were variables that could have predictive power in a machine learning model, in order to distinguish between humans and bots.

To do so, I used the dataset provided by the paper "FP-Agent: Fingerprinting AI Browsing Agents" by Ethan Wang, et al., in which the authors explained the process of how they created a machine learning model that could distinguish between different AI browsing agents in a controlled environment (a website made specifically for the purpose). They recorded data of different agents and humans navigating through their website. Their model gave good results, but I don't think it would be effective in a real-world setting when applied to different websites, which involves lots of variation. Nevertheless, they made the dataset available for download on OSF.io.

I calculated the burstiness and memory values for the JavaScript "scroll" events of each agent in each page (one burstiness and memory value per page visited). These are the results:

We can observe that the human distribution is clearly different from the other agents', presenting a higher Burstiness coefficient and almost no Memory; the only agent that sometimes resembles humans is ChatGPT Agent. This data seems promising, but to further prove if the two values have predictive value I trained a LightGBM (decision tree gradient boosting) model to classify each single-page interaction.

The model only had two features available, B and M, and it had to predict which agent performed the interactions. The result was an accuracy of 73.4%, which is not that bad. As expected, the model mainly confused Humans and ChatGPT Agent, most likely due to low feature count or a small dataset (~150 data points per agent is too small), the model would need other features to be able to distinguish them further.

Conclusion

Scroll behaviour seems to be a relevant data point for distinguishing bots from humans, and Burstiness and Memory have proven to have predictive power in a controlled environment. This approach will obviously not be effective on websites that don't require scrolling; on the other hand, blogging or information sites like Wikipedia are the ones most able to take advantage of this method.

The model shown in this blog post is not accurate enough, though, if combined with more features extracted from other actions such as mouse movement or typing, I believe it is possible to create a general model capable of protecting websites from scraper bots.

I will keep posting my research in this website. Next up, I will be taking a look at mouse movement patterns.