Author Kathy Nickels
Date 23 December 2024

ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S) publications have been named in the APO’s Top Content for 2024.

The Analysis and Policy Observatory (APO) is an open access platform that makes public policy research and resources accessible and useable for evidence-informed decision-making. Each year, the APO names the top ten most visited and viewed resources across 15 broad subject areas for the period December 2023 to November 2024.

This year publications from the ADM+S Centre have been named in the APO’s Top Ten across five subject areas including Communications, Education, Government, Science and Technology. 

In 2024, the ADM+S Centre has contributed 30 publications to the Automated Decision-Making and Society collection on the APO.

The Analysis & Policy Observatory (APO) is one of Australia’s leading open access research repositories. The ADM+S shares APO’s goal of supporting evidence-based policy and public debate on the critical challenges facing Australia, and we’re delighted to be working with APO to make ADM+S research more findable, more useable, and more accessible.

View the full list of APO’s Top Ten content

Author Natalie Campbell
Date 11 December 2024

The 2024 ADM+S Hackathon, ‘Recommender Systems for Public Service Media’, was held on 5-6 December in partnership with the ABC.  

Providing an alternative to commercial forms of content curation and recommender systems is one of the challenges faced by public service media outlets in the digital era. 

Hackathon participants were invited to design systems that could rank or assess content according to public service values such as social responsibility, cultural diversity, and the public interest.   

“We live in a world where algorithms and automated recommendation and curation systems play a hugely important role in shaping the information that we see,” explains Prof Mark Andrejevic, ADM+S Chief Investigator at Monash University and co-organiser of the Hackathon.  

Unlike commercial media, which are driven by commercial values, public media aim to serve the public good. The rise of automated systems has created challenges in ensuring that public service values are reflected in the algorithms shaping content curation and distribution.  

“We’ve had four teams working hard to figure out how you might approach the challenge of developing algorithms that instead of prioritizing commercial values, prioritize what we would think of as public service values.  

“The big challenge is to think about, what are those values? How do you operationalize them? How can you put them into an automated system that would recognize them? And how would you implement that in the newsroom of a public service broadcaster to support the editorial decisions that are being made.” 

Leading into the two-day Hackathon, ADM+S AI James Meese was joined by industry partners Angela Ross (Research Lead), Laura Gartry (Innovation Lead) and Stuart Watt (Head of News Strategy and Innovation) from the ABC, for a panel discussion on Wednesday 4 December titled ‘What is a public service news algorithm, and why might we need one?’  

This conversation, moderated by Prof Mark Andrejevic, is now available on the ADM+S podcast. 

Following the Wednesday night panel, Angela Ross, Laura Gartry and Stuart Watt, as well as Saarim Saghir, Strategy manager at Google USA, engaged with the team’s ideas over the course of the Hackathon, ultimately deciding a winner to secure $5,000 in research funds for further development of their idea. 

“It’s been amazing to be part of this Hackathon,” said ABC Research Lead Angela Ross. 

“We’ve been blown away by the ideas and the innovative thinking, and it’s actually made us rethink some of the ways that we’re looking at the problems.” 

2024 Hackathon team proposals:

‘BYO Values’
The Value Aware Ranking (VAR) Co-Pilot is an editorial toolkit which provides a scoring system to rate stories based on pre-defined and customisable public service media values.  

While the ABC produces news stories with its public service values in mind, its existing recommender systems are not optimised for these values. VAR could be integrated into a back-end editorial interface and assist editorial teams to better serve their public interest function in a practical and transparent way.  

VAR would help identify value gaps in certain topics that an editorial team could address with story commissioning and editing.  

For consumers, VAR could help to strengthen public trust and accountability by providing transparency, explicitly linking editorial decisions to public service media values and helping consumers understand the rationale behind story prioritisation.  

Author ADM+S Centre and RMIT University
Date 12 December 2024

The announcement is part of the Australian Government’s plan to invest $68 million to narrow the digital gap by supporting more First Nations communities to access the internet.

The funding will support a new three-year project led by the ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S), to collect data on the digital inclusion experiences of First Nations people in urban, regional and remote areas across Australia.

Digital inclusion refers to access to communications services and devices, affordability and digital ability. The digital gap is the difference in levels of digital inclusion between First Nations people and national averages across Australia.

With Target 17 of the National Agreement on Closing the Gap aimed at addressing the uneven levels of digital inclusion amongst First Nations Australians by 2026, ADM+S Director and RMIT Distinguished Professor Julian Thomas said accurate measurement of digital inclusion is needed to track progress.

“There is currently a lack of data to measure the scale and changing nature of digital inclusion amongst First Nations peoples over time,” he said.

“This is largely because First Nations digital inclusion is difficult to capture accurately in national survey approaches.

“Measuring digital inclusion both within and across First Nations communities requires close engagement with the communities themselves, their organisations and leaders.

“It also requires different approaches tailored for urban, regional, rural and remote communities.”

First Nations lead researcher, Associate Professor Heron Loban, said it’s important the research is First Nations led and delivered, while following principles of Indigenous data governance.

Author ADM+S Centre
Date 4 December 2024

On 4 December 2024 the ARC Centre of Excellence for Automated Decision-making and Society met with colleagues from the University of Southern California’s Election Cybersecurity Initiative for a day-long workshop around fair elections in the age of AI.

RMIT University Vice Chancellor Alec Cameron welcomed attendees, explaining “This is an essential event for sharing experiences across countries.”

The program brought together journalists, policymakers, researchers and industry experts to examine the capabilities of current AI systems, the dynamics of digital media platforms, and the institutional, technical and regulatory strategies that can protect elections now and in the future.

“This is one of the most interesting conferences we’ve had anywhere in the world,” said Adam Clayton Powell, Executive Director of the USC Election Cybersecurity Initiative.

“What we find is that all of the free and fair democracies around the world share some of the same challenges and some of the same adversaries.

“Our job here is to connect the best practices around the world and to put people in touch with each other because the bad actors are always evolving and always changing, looking for vulnerabilities.”

Across the day, sessions covered digital infrastructures and political economies, verifying and fact checking, election auditing, media obligation, online credibility, and much more.

Author RMIT University
Date 3 December 2024

Author ADM+S Centre
Date 5 December 2024

On 4 December 2024, the Australian Competition and Consumer Commission (ACCC) released the ninth interim report for its Digital Platform Services Inquiry 2020–25. 

The report revisits competition and consumer issues arising in the supply of general search services in Australia. It examines technology, regulatory and industry change since the ACCC last considered general search services in the September 2021 interim report of this Inquiry, including the impact of general search and trends in search quality.

Key findings include the evolution of search engine business models, the integration of AI-driven tools, and ongoing challenges in ensuring competition and consumer protection. While the report makes no new recommendations, it reiterates the critical need for regulatory reforms proposed in earlier reports, including the September 2022 interim report.

The ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S) and School of Computing Technology, RMIT University contributed significant insights to the ACCC’s report through its submission, particularly the Australian Search Experience project, which tracked over 350 million search results from 1,000 volunteers.

Key contributions and citations from the ADM+S submissions

• User Priorities in Search: key consumer demand of a search service is that it responds very quickly to any query issued.
• Personalisation and Targeting: Findings showed minimal personalisation for generic queries, with search engines primarily tailoring results to geographic relevance. 

Author Natalie Campbell
Date 6 December 2024

The report ‘Online Misinformation in Australia’, a collaboration between Prof Sora Park (University of Canberra), Assoc Prof Tanya Notley and Dr Aimee Hourigan (Western Sydney University), Prof Michael Dezuanni (QUT) and ADM+S Affiliate Dr T.J. Thomson (RMIT University) reveals a significant gap between how people perceive their ability to verify information online, compared to their actual ability.

The study included two nationally representative surveys of 3,852 and 2,115 adult Australians, as well as a week-long digital diary study with 55 participants to provide an intimate look into everyday experiences with misinformation online.

Results revealed that more than half of Australians encounter misinformation online in a typical week, and this proportion increases the more active a person is on social media.

“Misinformation has many harms, from the individual effects of bullying or scams to broader societal risks around legislation and deliberative democracy,” explains Dr Thomson.

“Understanding more about online misinformation and adult Australians’ abilities to respond to it is vital so that we can protect our mental health, protect our assets during the current cost-of-living crisis, and ensure we have important societal conversations that are based on facts and verifiable evidence rather rumours, conjecture, and false or misleading information.”

Key findings from the study include:

1. Most adult Australians are not able to identify misinformation online
2. Many adults overestimated their ability to identify misinformation online
3. Online news and information consumption habits are related to people’s ability to verify information online
4. Economics, celebrity and crime/crisis are the top topics of misinformation people encountered during the diary study
5. Text-based content is the most prevalent form of misinformation people identified during the diary study
6. Adults want to develop their media literacy abilities

Author  Jeannie Marie Paterson
Date 2 December 2024

Have you ever spotted what looked like a great deal on a website, added it to your “basket” and proceeded to checkout – only to find extra fees added on at the last minute?

It’s frustratingly common when making airline, hotel and many other kinds of bookings to see an advertised price get ratcheted up at checkout with additional fees – perhaps “shipping insurance”, “resort fees” or just “taxes”.

The practice is known as “drip pricing” and it can distort consumer decision-making and affect competition. Nonetheless, there is no specific ban on this conduct in Australia.

Some companies have, however, effectively been prosecuted for it under the Australian Consumer Law, which contains some strict rules about misleading consumers through advertising.

Many of us have already begun booking flights, hotels and more as we head into the summer holiday season. Here’s what the law says about companies changing prices in the lead-up to checkout, and how you can protect yourself as a consumer.

What’s wrong with drip pricing?

The tactic that underpins drip pricing is to draw a customer in with an attractive “headline” price but then add in other fees as the customer approaches the checkout.

It’s reasonable to ask whether there’s anything wrong with this practice: after all, the customer still sees the final price at checkout. Why might that be seen as misleading conduct under Australian Consumer Law?

The reasons lie in views about consumer buying behaviour and the nature of the statutory prohibition.

Typically, the closer a consumer gets to a sale, the less likely they are to pull out or even fully notice any additional fees.

They may then end up paying more than they intended and also have lost the opportunity to deal with other suppliers of the same product at a better price.

In the relevant section of Australian Consumer Law, there’s no requirement of an intention to mislead. It’s also not necessarily relevant that the true pricing situation is eventually revealed to the consumer or that it’s in the “fine print”.

Thus, in the eyes of the law, it can be enough that consumers were enticed by an attractive headline price.

Price surprises

This legal position is well illustrated by a case settled by the High Court in 2013, after the Australian Competition and Consumer Commission (ACCC) took on telecom provider TPG Internet in 2010, alleging misleading conduct.

In this case, TPG had been advertising broadband internet services for $29.99 per month.

But on reading the fine print, you’d have discovered this deal was only available with a landline service costing an additional $30 per month.

The case moved up through Australia’s court system, but ultimately, the High Court majority held that the telco had engaged in misleading conduct.

Author ADM+S Centre
Date 28 November 2024

“Digital platforms play a critical role in Australia’s economy and society. Yet our capacities to understand and observe their activities is very limited. Australian’s see hundreds of ads a day, but who’s taking note of what they’re seeing?” – Dr Abdul Karim Obeid, Data Engineer on The Australian Ad Observatory

On 28 November 2024 the ARC Centre of Excellence for Automated Decision-Making and Society released ‘The Australian Ad Observatory: A web of invisible influence’, the second iteration in a series of project films offering an inside look at major research projects underway at the Centre.

The first phase of the Ad Observatory pioneered a way to observe the targeting of social media advertising across populations of users. With 1,909 participants, the team generated the largest known collection of Facebook ads in Australian history – 328,107 unique ads – and built world-first citizen-science research infrastructure.

In The Australian Ad Observatory: A web of invisible influence you’ll hear from project leaders Prof Christine Parker, Prof Daniel Angus, Prof Mark Andrejevic and Assoc Prof Nicholas Carah, data engineer Dr Abdul Karim Obeid, PhD student Lauren Hayden, and industry partners Dr Aimee Brownbill (Foundation for Alcohol Research and Education) and Chandni Gupta (Consumer Policy Research Centre). 

Prof Daniel Angus explains, “We hear a lot of debates around the role of digital media within our society, but the frustration is that these debates are not always evidenced by real findings about real-world experiences of people online.”

“Existing archives for digital advertising were made by the platforms and they had some serious limitations to them that hindered our ability to look at the ads being shown,” says Lauren Hayden.

“We had no ability to understand how ads were targeted, or who was actually seeing them.”

Author  Jean Burgess
Date 27 November 2024

After Elon Musk bought Twitter (now rebranded X) in 2022, disaffected users began to seek alternatives. Alongside Meta’s Threads and the open source project Mastodon, Bluesky was one of several contenders.

Threads benefited from Meta’s existing user base but has failed to capture the popular imagination. Mastodon has proven complicated and difficult to grasp for most ordinary users and so use remains fragmented. Bluesky seemed promising but was in invite-only mode at the time and growth was muted.

But in recent weeks, the migration to Bluesky from X seems to have reached a tipping point, as large parts of the user community finally got fed up with X’s toxic culture and management. Following the recent US presidential election, in which Musk appeared to manipulate X’s algorithms to increase his own influence, these users found Bluesky’s doors wide open.

Since then, the user base has grown to more than 20 million users, a number that continues to climb. As others have noted, at least for the moment it feels a bit like early Twitter – a sandpit to explore new tools, a playful connection to the broader internet, and a relatively safe place to share personal thoughts and experiences, or to connect with friends and colleagues.

How is Bluesky different from X?

Bluesky looks very similar to X. Its azure butterfly icon bears obvious resemblances to Twitter’s blue bird, which Musk replaced with a stark white-on-black X.

Bluesky uses hashtags and users address one another using the @ symbol. Replies, quotes and reposts all work much as they do on X. This comforting resemblance is likely one explanation for the remarkable popularity of Bluesky in comparison to other decentralised platforms such as Mastodon.

Bluesky distinguishes itself from X through a rich set of features through which users can control their experience and shape the culture of the platform as a whole.

You can build multiple custom feeds based on your own interests and relationships then publicly share these feeds with others. This is a powerful mechanism to avoid the one-way “push” of algorithmic feeds and represents a more democratic approach to content curation.

Bluesky offers the ability to create custom “starter packs” – curated lists of suggested accounts related to topics, interests or locations. Starter packs can be shared publicly to help new users find people to follow. This is a novel feature that feels friendly and welcoming, and again doesn’t really rely on a top-down algorithm.

Bluesky’s settings menu also includes powerful content moderation tools that users can control. For example, you can create custom keyword lists to mute some types of content, and control who can interact with you.

This means if you don’t want to listen to certain political views, you don’t have to. It also means you can have a pleasant and sociable time without being subject to hate speech, bullying and harassment.

Critics argue these kinds of user controls will lead to “echo chambers” so the overall public sphere (or public square) is no longer a place for an exchange of differing views. But as I have previously argued, a public square owned by a billionaire that is full of shouting bullies does nothing to enable equal participation either.

Author ADM+S Centre
Date 26 November 2024

The Australian Research Council (ARC) today announced more than $342 million in funding for 536 new projects under the 2025 ARC Discovery Projects scheme. 

The ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S) is proud to announce that twelve of the funded projects include contributions from its members, showcasing the Centre’s commitment to advancing impactful, multidisciplinary research.

ARC Acting Chief Executive Officer, Dr Richard Johnson, said the ARC Discovery Projects scheme supports excellent basic and applied research to expand Australia’s knowledge base and research capability. 

“Discovery grants support individual researchers and research teams in research projects that provide economic, commercial, environmental, social and/or cultural benefits to the Australian community,” Dr Johnson said. 

The twelve projects involving ADM+S members reflect research excellence across diverse fields, including AI ethics, digital citizenship, sustainable energy, healthcare transparency, and diversity studies.

Projects involving ADM+S members include the following (ADM+S researchers in bold):

A new “Treating Customers Fairly” law for Australia’s financial industry
Assoc Prof Andrew Schmulow; Ms Nicola Howell; ProfJeannie Paterson; ProfElise Bant; Prof Therese Wilson; Prof Jason Harris

Disability and Digital Citizenship
Prof Gerard Goggin; Prof Kathleen Ellis; Prof Jennifer Smith-Merry; Prof Simon Darcy; Prof Paul Harpur; Prof Bree Hadley; ProfMichael Kent; Associate Prof Dinesh Wadiwel; Dr Natasha Layton; Assoc Prof Mary-Ann O’Donovan; Prof Scott Avery; Professor Karen Soldatic; Prof Lorenzo Dalvit; Dr Kuansong Victor Zhuang; Assoc Prof Meryl Alper

Embedding Net Zero Carbon Emissions in Northern Australia
Assoc Prof Timothy Neale; Dr Kari Dahlgren; Prof Matthew Kearnes; Prof Teresa Lea; Dr Christopher Mayes; Assoc Prof Gisa Weszkalnys

Author Kathy Nickels
Date 25 November 2024

The ARC Centre for Automated Decision-Making and Society (ADM+S) and the Torres Strait Islander Media Association (TSIMA) are proud to announce the release of End of the Line, a compelling short film that explores the impact of patchy mobile and internet connectivity on the lives of First Nations people in remote Australia. 

The film captures the unique challenges faced by communities on Erub (Darnley Island), located at the eastern edge of Zenadth Kes (the Torres Strait Islands), where digital access plays a crucial role in preserving culture, ensuring safety, and enabling communication.

End of the Line follows Erub Meuram woman and NAIDOC Award winner Lala Gutchen, a First Language educator and cultural leader, as she navigates the challenges of maintaining cultural practices—such as fishing and language revitalisation—in a place where mobile access is unreliable. 

The film highlights how connectivity issues shape not only cultural traditions but also everyday safety and community life, especially in isolated areas where communication can be a lifeline.

The production of End of the Line is a collaboration between ADM+S and TSIMA as part of the ongoing Mapping the Digital Gap project, an initiative funded by Telstra and the ADM+S. This project seeks to understand and address the digital inclusion needs in remote First Nations communities across Australia. Lala Gutchen has served as a co-researcher since 2022, contributing her invaluable insights into the connectivity challenges faced by her community.

Author Daniel Angus
Date 21 November 2024

The federal government today introduced into parliament legislation for its social media ban for people under 16 years.

Communications Minister Michelle Rowland said:

This is about protecting young people, not punishing or isolating them, and letting parents know we’re in their corner when it comes to supporting their children’s health and wellbeing.

Up until now details of how the ban would actually work have been scarce. Today’s bill provides a more complete picture.

But many ambiguities – and problems – still remain.

What’s in the bill?

Today’s bill is an amendment of the Online Safety Act.

It introduces a new definition for an “age-restricted social media platform” whose sole or significant purpose is to enable users to post material online and interact socially with other users.

This includes platforms such as Facebook, Instagram, TikTok and Snapchat, but also many more minor platforms and services. It includes an exclusion framework that exempts messaging apps such as WhatsApp, online gaming platforms and services with the “primary purpose of supporting the health and education of end-users” (for example, Google Classroom).

The bill will attempt to force owners of newly defined age-restricted platforms to take “reasonable steps” to prevent people under 16 from having a user account. This will include young people who have an existing account. There are no grandfather provisions so it is unclear how platforms will be required to manage the many millions of existing users who are now set to be excluded and deplatformed.

The bill is also vague in specifying how social media platforms must comply with their obligation to prevent under 16s from having an account – only that it “will likely involve some form of age assurance”.

Oddly, the bill won’t stop people under 16 from watching videos on YouTube or seeing content on Facebook – it is primarily designed to stop them from making an account. This also means that the wider ecology of anonymous web-based forums, including problematic spaces like 4chan, are likely excluded.

Age-restricted platforms that fail to prevent children under 16 accessing their platforms will face fines of nearly A$50 million.

However, the government acknowledges that it cannot completely stop children under 16 from accessing platforms such as Instagram and Facebook.

Australia should be prepared for the reality that some people will break the rules, or slip through the cracks.

The legislation will take effect “at least” 12 months after it has passed parliament.

How did we get to this point?

The government’s move to ban under 16s from social media – an idea other countries such as the United Kingdom are now considering – has been heavily influenced by News Corp’s “Let Them Be Kids” campaign. This campaign included sensitive news reports about young people who have used social media and, tragically, died by suicide.

The government has also faced pressure from state governments and the federal opposition to introduce this bill.

Author ADM+S Centre
Date 19 November 2024

The ARC Centre of Excellence for Automated Decision-Making and Society (ADM+S) has joined the Bluesky social media platform.

On Bluesky, we’ll be continuing to share:

• Insights from our research on automated decision-making.
• Updates on events, workshops, and opportunities to get involved.
• Thought-provoking discussions about AI, Automated Decision-Making, and their impacts on society.

Author ADM+S Centre
Date 19 November 2024

In an era marked by cyber attacks to co-ordinated disinformation and AI-generated deep fakes, fair elections in Australia and around the world are facing unprecedented and complex threats. 

What do we need to know? What actions should citizens, journalists, policymakers, researchers, and politicians take to safeguard the integrity of our elections?

To address these challenges, the ARC Centre of Excellence for Automated Decision-Making has partnered with the University of Southern California’s (USC) Election Cybersecurity Initiative to convene a one-day workshop. This event will bring together leading experts from Australia and the United States to share insights and compare recent experiences in tackling election security threats in both countries.

Professor Daniel Angus, Director of the Digital Media Research Centre at QUT and Chief Investigator at ADM+S will be speaking at the event, emphasising that safeguarding elections is foundational for democracy. 

“Safeguarding fair elections is vital for democracy; it requires a collaborative effort that draws on the insights of diverse stakeholders and internal experts. By bringing together a wide range of voices—researchers, industry leaders, and policymakers—we can create a comprehensive understanding of the challenges and develop effective, actionable strategies. 

“When we protect the vote we are protecting the collective voice of the people. While emerging technologies challenge election security in new ways, it is also essential that we recognise and address underlying social factors as well. With insights from a range of fields we are far better placed to tackle the human and technological sides of election security, and protect the integrity of our democratic institutions,” said Professor Angus.

Speakers include Adam Clayton Powell III, Executive Director of the USC Initiative on Election Cybersecurity; Jeffrey Cole, Director of the Centre for the Digital Future; ABC’s US elections analyst Casey Briggs; as well as ADM+S researchers in the field.

Together, they will investigate the capabilities of current AI systems, the dynamics of digital media platforms, and the institutional, technical, and regulatory strategies necessary to protect elections now and in the future.

The ADM+S and USC invites Government (policy makers), industry (electoral officials/workers, regulatory), and journalists (political/election and AI focus) as well as interested general public to attend this workshop to engage in this crucial dialogue.

Alongside researchers from the ADM+S Centre: Prof Mark Andrejevic, Prof Daniel Angus, Assoc Prof Timothy Graham, Prof Christopher Leckie, Dang Nguyen, Prof Mark Sanderson, Distinguished Prof Julian Thomas, Fan Yang, and Prof Haiqing Yu.

Author ADM+S Centre
Date 14 November 2024

New research funded by VicHealth and the Foundation for Alcohol Research & Education and supported by the Australian Research Council Centre of Excellence for Automated Decision-Making and Society (ADM+S) has revealed how Facebook targets people most at risk with alcohol and gambling advertising.

The study piloted a novel digital donation method, the Australian Ad Observatory Mobile Toolkit, developed at the ADM+S Centre, for citizen scientists to collect “dark” ads that would otherwise be hidden from public view. Citizen scientists also provided a list of “ad interests” and a list of advertisers who had targeted their profile and participated in a co-analysis interview in which they were asked to interpret their flows of targeted advertising. 

Findings from the report How alcohol and gambling companies target people most at risk with marketing for addictive products on Facebook revealed that these companies shared substantial data with Facebook to aid in targeting users based on detailed advertising interests, despite many individuals actively trying to reduce their alcohol or gambling consumption.

Key findings:

• Facebook tags people at risk of harm and trying to reduce their use of alcohol and gambling as interested in these addictive products to target them with advertising.
• Alcohol and gambling companies uploaded data on people at risk of harm and trying to reduce their use of alcohol or gambling to fuel targeted marketing on Facebook. 
• People who are trying to reduce their alcohol use or gambling are constantly faced with advertisements for these addictive products on social media.
• People who are trying to reduce their alcohol use or gambling don’t want to be profiled and targeted for alcohol and gambling and can find it impossible to escape this advertising when they are on social media.

This method enabled researchers to gather insights from people who had experienced or were at risk of alcohol or gambling harm to observe alcohol, gambling and social media companies digital advertising targeting practices on Facebook. 

Dr Giselle Newton, Research Fellow from the ADM+S at the University of Queensland and Chief Investigator on the report, said, “Technology in development, such as the Australian Ad Observatory Mobile Toolkit, provides ways to further understand dark marketing of harmful and addictive products that otherwise remain hidden from sight”.

 “This report is the tip of the iceberg in terms of what we know about how alcohol and gambling companies collect and use people’s data to then target them with their harmful and addictive products.

“People who are trying to reduce their alcohol use or gambling don’t want to be targeted with ads selling these products, and can find it difficult to escape this advertising when they are on social media platforms like Facebook.”

Author Kathy Nickels
Date 12 November 2024

The inaugural Australian AI Safety Forum 2024, hosted at and supported by the University of Sydney last week, marked a significant step in Australia’s growing involvement in the global dialogue around AI safety and governance. 

The two-day interdisciplinary event built on the momentum created by the establishment of state-backed AI Safety Institutes in the UK, US, and other countries, and the recent release of the Interim International Scientific Report on the Safety of Advanced AI.

The Forum took the Interim International Scientific Report on the Safety of Advanced AI as its scientific foundation, using its technical findings to frame and advance discussions on policy and governance within the Australian context.

The event brought together researchers, policymakers, and industry leaders to address the critical challenges posed by advanced artificial intelligence. 

Professor Kimberlee Weatherall, ADM+S Chief Investigator from The University of Sydney and one of the event organising committee members said, “introductory talks on the state of AI, AI safety and AI governance sought to create a common ground among researchers coming from very different disciplines and – if the enthusiasm and energy of the conversations in and outside the formal sessions is anything to go by – activated strong interest in understanding different perspectives on AI safety and governance questions.”

Distinguished speakers included:

• Marcus Hutter, Australian National University
• Hoda Heidari, Carnegie Mellon University
• Johanna Weaver, Tech Policy Design Centre, Australian National University
• Atoosa Kasirzadeh, Carnegie Mellon University
• Ryan Kidd, ML Alignment & Theory Scholars (MATS)
• Seth Lazar, Australian National University
• Nitarshan Rajkumar, University of Cambridge

Author Natalie Campbell
Date 8 November 2024

On 1 November 2024 ADM+S members Dr Jake Goldenfein and Lauren Kelly presented evidence to the Victorian Legislative Assembly Economy and Infrastructure Committee for the final hearing of the Inquiry into Workplace Surveillance.

The Inquiry was established in May 2024 to examine the extent to which surveillance data is being collected, shared, stored, disclosed, sold, disposed of and otherwise utilised in Victorian workplaces.

ADM+S Chief Investigator Dr Jake Goldenfein from the University of Melbourne presented evidence on behalf of ADM+S and as a co-author on the NTEU submission that made three key recommendations:

Recommendation 1: The Office of the Victorian Information Commissioner audits Victorian universities for their compliance with the Privacy and Data Protection Act 2018 (Vic) to be adequately resourced for this purpose.

Recommendation 2: The Victorian Parliament enacts a statute dedicated to regulating workplace surveillance.

Recommendation 3: The statute in Recommendation 2 be based on six Workplace Privacy Principles

1. Comprehensiveness
2. Transparency
3. Freedom of association and the centrality of trade unions
4. Legitimate purpose and proportionality
5. Governance and accountability
6. Effective compliance and enforcement

Author Timothy Graham
Date 6 November 2024

As voting booths in the United States close and the results of the presidential election trickle in, tech billionaire Elon Musk has been posting a flurry of tweets on his social media platform, X (formerly Twitter). So too has Republican presidential nominee Donald Trump.

At first glance these tweets might appear chaotic and random. But if you take a closer look, you start to see an alarming strategy behind them – one that’s worth paying very close attention to in order to understand the inner workings of the campaign to return Trump to the White House.

The strategy has two immediate aims. First, to overwhelm the information space and thereby manage attention. Second, to fuel the conspiracy theory that there is a coordinated campaign among Democrats, the media and big tech to steal this election.

But it’s important to understand that the strategy on X is part of a master strategy of Trump’s campaign: a backup plan in case of a Trump loss, designed to encourage the public to participate in a grand re-wiring of reality via the meta-narrative of widespread voter fraud.

Overwhelm the information space

Musk has long been a prominent user of X, even before he became the owner, chief technology officer and executive chairman of the platform.

But as I reported last week, since he endorsed Trump in July, engagement with his account has seen a sudden and anomalously large increase, raising suspicions as to whether he has tweaked the platform’s algorithms so his content reaches more people.

This trend has continued in recent days.

As well as posting on X, earlier today Musk also held a “freeform” live discussionon the platform about the election. It lasted for nearly one and a half hours. Around 1.3 million people tuned in. This is one of many live discussions he has hosted about the election over the past months, including notably with Trump.

In an information war, everything is about attention management. Platforms are designed to maximise engagement and user attention above and beyond anything else. This core logic of social media is highly exploitable: who controls attention controls the narrative. In Australia, the “Vote No” campaign during last year’s referendum on Indigenous representation in government was a masterclass in attention management.

By bombarding audiences, journalists, and other key stakeholders with a constant supply of allegations, rumours, conspiracy theories and unverifiable claims, Musk and the Trump campaign eat up all the oxygen of attention. When everyone is focussed on you and what you’re saying, they are distracted from what the other side is saying.

And Musk and Trump want people to focus on the idea that the election is going to be stolen.

Author Mark Andrejevic and Timothy Graham
Date 1 November 2024

On July 13, shortly after Donald Trump was targeted by an assassination attempt, Elon Musk, the billionaire owner of X (formerly Twitter), tweeted to his more than 200 million followers:

“I fully endorse President Trump and hope for his rapid recovery.”

Musk’s efforts to influence who wins next week’s US presidential election have continued. For example, over the past three months, he has donated more than US$100 million to a political action committee called America PAC that’s promoting Trump.

But our new research (currently available in preprint form) indicates Musk may be wielding influence in other more subtle ways as well. However, the platform’s increasing opacity to researchers makes this difficult to say for certain.

This raises suspicions as to whether Musk has tweaked the platform’s algorithm to increase the reach of his posts in advance of the US presidential election. It also demonstrates the problems with how social media platforms like X are currently regulated around the world.

Not the first time

Musk has history when it comes to tweaking X’s algorithms so his own content reaches more people.

Last year, he reportedly mobilised a team of around 80 engineers to algorithmically boost his posts. This came after his tweet supporting the Philadelphia Eagles during the Super Bowl was outperformed by a similar one from President Joe Biden. Musk seemed to confirm this happened, posting a picture depicting one woman labelled “Elon’s tweets” forcibly bottle feeding another woman labelled “Twitter”.

To see whether Musk has done this again in the leadup to the US election, we compared Musk’s engagement metrics – such as the number of views, retweets and likes – with a set of other prominent political accounts on the social media platform. The data spans the period from January 1 2024 to October 25 2024.

Other political accounts that served as a basis of comparison include those of right-wing commentators Jack Posobiec, Tucker Carlson and Donald Trump Jr. Our study also examined accounts at the other end of the political spectrum, including those of US Representative Alexandria Ocasio-Cortez, US Senator Bernie Sanders and Vice-President Kamala Harris.

Shortly after Musk endorsed Trump’s presidential campaign, there was a statistically anomalous boost in engagement with his X account. Suddenly, his posts were getting much higher views, retweets and likes in comparison to other prominent political accounts on the platform.

A sudden and significant increase

Since July, engagement with Musk’s X account has seen a sudden and significant increase.

The view counts for his posts increased by 138%, retweets by 238%, and likes by 186%.

In contrast, other prominent political accounts on X saw more moderate increases: 57% in view counts, 152% in retweets, and 130% in likes.

Author Henry Fraser
Date 31 October 2024

Last week, the tragic news broke that US teenager Sewell Seltzer III took his own life after forming a deep emotional attachment to an artificial intelligence (AI) chatbot on the Character.AI website.

As his relationship with the companion AI became increasingly intense, the 14-year-old began withdrawing from family and friends, and was getting in trouble at school.

In a lawsuit filed against Character.AI by the boy’s mother, chat transcripts show intimate and often highly sexual conversations between Sewell and the chatbot Dany, modelled on the Game of Thrones character Danaerys Targaryen. They discussed crime and suicide, and the chatbot used phrases such as “that’s not a reason not to go through with it”.

This is not the first known instance of a vulnerable person dying by suicide after interacting with a chatbot persona. A Belgian man took his life last year in a similar episode involving Character.AI’s main competitor, Chai AI. When this happened, the company told the media they were “working our hardest to minimise harm”.

In a statement to CNN, Character.AI has stated they “take the safety of our users very seriously” and have introduced “numerous new safety measures over the past six months”.

In a separate statement on the company’s website, they outline additional safety measures for users under the age of 18. (In their current terms of service, the age restriction is 16 for European Union citizens and 13 elsewhere in the world.)

However, these tragedies starkly illustrate the dangers of rapidly developing and widely available AI systems anyone can converse and interact with. We urgently need regulation to protect people from potentially dangerous, irresponsibly designed AI systems.

How can we regulate AI?

The Australian government is in the process of developing mandatory guardrails for high-risk AI systems. A trendy term in the world of AI governance, “guardrails” refer to processes in the design, development and deployment of AI systems. These include measures such as data governance, risk management, testing, documentation and human oversight.

One of the decisions the Australian government must make is how to define which systems are “high-risk”, and therefore captured by the guardrails.

The government is also considering whether guardrails should apply to all “general purpose models”. General purpose models are the engine under the hood of AI chatbots like Dany: AI algorithms that can generate text, images, videos and music from user prompts, and can be adapted for use in a variety of contexts.

In the European Union’s groundbreaking AI Act, high-risk systems are defined using a list, which regulators are empowered to regularly update.

An alternative is a principles-based approach, where a high-risk designation happens on a case-by-case basis. It would depend on multiple factors such as the risks of adverse impacts on rights, risks to physical or mental health, risks of legal impacts, and the severity and extent of those risks.

Chatbots should be ‘high-risk’ AI

In Europe, companion AI systems like Character.AI and Chai are not designated as high-risk. Essentially, their providers only need to let users know they are interacting with an AI system.

Author Flora Salim
Date 29 October 2024

In 2014, the British philosopher Nick Bostrom published a book about the future of artificial intelligence (AI) with the ominous title Superintelligence: Paths, Dangers, Strategies. It proved highly influential in promoting the idea that advanced AI systems – “superintelligences” more capable than humans – might one day take over the world and destroy humanity.

A decade later, OpenAI boss Sam Altman says superintelligence may only be “a few thousand days” away. A year ago, Altman’s OpenAI cofounder Ilya Sutskever set up a team within the company to focus on “safe superintelligence”, but he and his team have now raised a billion dollars to create a startup of their own to pursue this goal.

What exactly are they talking about? Broadly speaking, superintelligence is anything more intelligent than humans. But unpacking what that might mean in practice can get a bit tricky.

Different kinds of AI

In my view the most useful way to think about different levels and kinds of intelligence in AI was developed by US computer scientist Meredith Ringel Morris and her colleagues at Google.

Their framework lists six levels of AI performance: no AI, emerging, competent, expert, virtuoso and superhuman. It also makes an important distinction between narrow systems, which can carry out a small range of tasks, and more general systems.

A narrow, no-AI system is something like a calculator. It carries out various mathematical tasks according to a set of explicitly programmed rules.

There are already plenty of very successful narrow AI systems. Morris gives the Deep Blue chess program that famously defeated world champion Garry Kasparov way back in 1997 as an example of a virtuoso-level narrow AI system.

Levels of AI

Some narrow systems even have superhuman capabilities. One example is Alphafold, which uses machine learning to predict the structure of protein molecules, and whose creators won the Nobel Prize in Chemistry this year.

What about general systems? This is software that can tackle a much wider range of tasks, including things like learning new skills.

A general no-AI system might be something like Amazon’s Mechanical Turk: it can do a wide range of things, but it does them by asking real people.

Overall, general AI systems are far less advanced than their narrow cousins. According to Morris, the state-of-the-art language models behind chatbots such as ChatGPT are general AI – but they are so far at the “emerging” level (meaning they are “equal to or somewhat better than an unskilled human”), and yet to reach “competent” (as good as 50% of skilled adults).

So by this reckoning, we are still some distance from general superintelligence.

How intelligent is AI right now?

As Morris points out, precisely determining where any given system sits would depend on having reliable tests or benchmarks.

Depending on our benchmarks, an image-generating system such as DALL-E might be at virtuoso level (because it can produce images 99% of humans could not draw or paint), or it might be emerging (because it produces errors no human would, such as mutant hands and impossible objects).

There is significant debate even about the capabilities of current systems. One notable 2023 paper argued GPT-4 showed “sparks of artificial general intelligence”.

OpenAI says its latest language model, o1, can “perform complex reasoning” and “rivals the performance of human experts” on many benchmarks.

Author ADM+S Centre
Date 28 October 2024

The ARC Centre of Excellence of Automated Decision-Making and Society (ADM+S) annual symposium brought together Centre researchers from across 9 Australian universities and collaborators to discuss and plan future research in the evolving landscape of AI and automated decision-making across the Centre’s mobilities focus area.

The main symposium featured insights on the current state and future of automated mobilities. Over the course of the three-day event, participants engaged in workshops and panel discussions focused on various mobilities sectors impacted by automation, including public transport, active transportation, and migration services. Attendees examined both the benefits—such as increased efficiency and accessibility—and the risks, including potential inequities and ethical challenges.

The symposium was led by ADM+S Mobilities focus area co-leaders Prof Flora Salim from UNSW and Prof Sarah Pink from Monash University.

One of the symposium’s key aims was to consolidate ongoing research efforts within the Mobilities Focus Area of ADM+S. Presenters showcased findings from the first phase of their research, highlighting innovative applications of automated decision-making across personal, shared, commercial, and public systems. 

“Collectively we have delivered, and continue to deliver an excellent range of projects to offer an unprecedented level of interdisciplinary insight into Australian automated and AI mobilities technologies and systems and the benefits, dangers and future possibilities associated with them,” said Prof Pink.

The symposium presented research across themed areas:

• Privacy and Accountability
• Data Capture and Behaviours
• Accessibility and Inclusion
• Systems and Deployment
• Futures

Prof Salim said, “we designed each session to account for both the humanities and social science research papers and the technical computer science and engineering research papers thematically, bringing out a very interesting and engaging sociotechnical panel discussion at the end of each session.”

In addition to traditional academic outputs, the symposium encouraged creative formats for sharing research findings, including poster presentations, ethnographic documentary films and multimedia contributions. 

Author Natalie Campbell
Date 24 October 2024

On 24 October the Australian Academy of Health and Medical Sciences (AAHMS) announced 31 new Fellows, including ADM+S Chief Investigator and Health Focus Area leader, Prof Deborah Lupton.

Established in 2014, the AAHMS is an expert representative and independent voice for health and medical sciences in Australia, dedicated to engaging with the community, industry and government on pressing related to health.

Made up of Australia’s most influential experts in health and medicine – from universities, medical research institutes, health services, industry bodies, charities and public service – AAHMS Fellows are elected by their peers for outstanding achievements and contributions to the health and medical sciences in their respective fields.

2024 Fellows were welcomed in a ceremony at the Academy’s Annual Meeting in Adelaide on 24 October.

Academy President Prof Louise Baur said, “Our Fellowship represents the breadth and diversity of Australia’s health and medical expertise, allowing us to draw on independent, expert and evidence-based advice to drive change and improve health for all.

Author Natalie Campbell
Date 23 October 2024

The Joint Select Committee on Social Media and Australian Society, established by the Federal Government, has published its Second Interim Report on Digital Platforms and the Traditional News Media, citing contributions from the 28 June ADM+S submission.

The 22 October second interim report focusses on “the decision of Meta to abandon deals under the News Media and Digital platforms Mandatory Bargaining Code (Code) and the important role of Australian journalism, news, and public interest media on a healthy democracy in countering mis- and disinformation on digital platforms.”

The report regularly cites ADM+S research when discussing algorithmic transparency, fact-checking, misinformation, and the role and function of the News Media Bargaining Code.

It provides 11 recommendations informed by 217 submissions, including establishing a Digital Affairs Ministry, exploring alternative revenue mechanisms to supplement the Code, developing protocols for transparent distribution of revenue, new legislation to combat mis- and disinformation, and that the Australian Government establish a short-term transition fun to help news media businesses diversify and strengthen alternative income streams and news product offerings.

Author Jeannie Marie Paterson and Nicola Howell
Date 18 October 2024

In Australia, it’s scam victims who foot the bill for the overwhelming majority of the money lost to scams each year.

A 2023 review by the Australian Securities and Investments Commission (ASIC) found banks detected and stopped only a small proportion of scams. The total amount banks paid in compensation paled in comparison to total losses.

So, it was a strong statement this week when it was revealed the Australian Financial Conduct Authority (AFCA) had ordered a bank – HSBC – to compensate a customer who lost more than $47,000 through a sophisticated bank impersonation or “spoofing” scam.

This decision was significant. An AFCA determination is binding on the relevant bank or other financial institution, which has no direct right of appeal. It could have implications for the way similar cases are treated in future.

The ruling comes amid a broader push for sector-wide reforms to give banks more responsibility for detecting, deterring and responding to scams, as opposed to simply telling customers to be “more careful”.

Here’s what you should know about this landmark ruling, and what it might mean for consumers.

A highly sophisticated ‘spoofing’ scam

You might be familiar with “push payment” scams that trick the victim into paying money to a dummy account. These include the “mum I’ve lost my phone” scam and some romance scams.

The recent case concerned an equally noxious “bank impersonation” or “spoofing” scam. The complainant – referred to as “Mr T” – was tricked into giving the scammer access to his HSBC account, from which an unauthorised payment was made.

The scammer sent Mr T a text message, purportedly asking him to investigate an attempted Amazon transaction.

In an effort to respond to the (fake) unauthorised Amazon purchase, Mr T revealed security passcodes to the scammer, enabling them to transfer $47,178.54 from his account and disappear with it.

The fact Mr T was dealing with scammers was far from obvious – scammers had information about him one might reasonably expect only a bank would know, such as his bank username.

On top of this, the scam text message appeared in a thread of other legitimate text messages that had previously been sent by the real HSBC.

AFCA’s ruling

HSBC argued to AFCA that having to pay compensation should be ruled out under the ePayments Code, a voluntary code of practice administered by ASIC.

Under this code, a bank is not required to compensate a customer for an unauthorised payment if that customer has disclosed their passcode. The bank argued the complainant had voluntarily disclosed these codes to the scammer, meaning the bank didn’t need to pay.

AFCA disagreed. It noted the very way the scam had worked was by creating a sense of urgency and crisis. AFCA considered that the complainant had been manipulated into disclosing the passcodes and had not acted voluntarily.

Author Timothy Graham
Date 4 October 2024

For almost a decade, researchers have been gathering evidence that the social media platform Facebook disproportionately amplifies low-quality content and misinformation.

So it was something of a surprise when in 2023 the journal Science published a study that found Facebook’s algorithms were not major drivers of misinformation during the 2020 United States election.

This study was funded by Facebook’s parent company, Meta. Several Meta employees were also part of the authorship team. It attracted extensive media coverage. It was also celebrated by Meta’s president of global affairs, Nick Clegg, who said it showed the company’s algorithms have “no detectable impact on polarisation, political attitudes or beliefs”.

But the findings have recently been thrown into doubt by a team of researchers led by Chhandak Bagch from the University of Massachusetts Amherst. In an eLetter also published in Science, they argue the results were likely due to Facebook tinkering with the algorithm while the study was being conducted.

In a response eLetter, the authors of the original study acknowledge their results “might have been different” if Facebook had changed its algorithm in a different way. But they insist their results still hold true.

The whole debacle highlights the problems caused by big tech funding and facilitating research into their own products. It also highlights the crucial need for greater independent oversight of social media platforms.

Merchants of doubt

Big tech has started investing heavily in academic research into its products. It has also been investing heavily in universities more generally. For example, Meta and its chief Mark Zuckerberg have collectively donated hundreds of millions of dollars to more than 100 colleges and universities across the United States.

This is similar to what big tobacco once did.

In the mid-1950s, cigarette companies launched a coordinated campaign to manufacture doubt about the growing body of evidence which linked smoking with a number of serious health issues, such as cancer. It was not about falsifying or manipulating research explicitly, but selectively funding studies and bringing to attention inconclusive results.

This helped foster a narrative that there was no definitive proof smoking causes cancer. In turn, this enabled tobacco companies to keep up a public image of responsibility and “goodwill” well into the 1990s.

Author Lev Bromberg, Christine Parker & Simon Coghlan
Date 1 October 2024

The rise of artificial intelligence (AI) has triggered concern about potentially detrimental effects on humans. However, the technology also has the potential to harm animals.

An important policy reform now underway in Australia offers an opportunity to address this. The federal government has committed A$5 million to renewing the lapsed Australian Animal Welfare Strategy. Consultation has begun, and the final strategy is expected in 2027.

While AI is not an explicit focus of the review, it should be.

Australians care about animals. The strategy could help ensure decision-makers protect animals from AI’s harms in our homes, on farms and in the wild.

Will AI harms to animals go unchecked?

Computers are now so developed they can perform some complex tasks as well as, or better than, humans. In other words, they have developed a degree of “artificial intelligence”.

The technology is exciting but also risky.

Warnings about the risks to humans include everything from privacy concerns to the collapse of human civilisation.

Policy-makers in the European Union, the United States and Australia are scrambling to address these issues and ensure AI is safe and used responsibly. But the focus of these policies is to protect humans.

Now, Australia has a chance to protect animals from AI.

Australia’s previous Animal Welfare Strategy expired in 2014. It’s now being revived, and aims to provide a national approach to animal welfare.

So far, documents released as part of the review suggest AI is not being considered under the strategy. That is a serious omission, for reasons we outline below.

Powerful and pervasive technology in use

Much AI use benefits animals, such as in veterinary medicine. For example, it may soon help your vet read X-rays of your animal companion.

AI is being developed to detect pain in cats and dogs. This might help if the technology is accurate, but could cause harm if it’s inaccurate by either over-reporting pain or failing to detect discomfort.

AI may also allow humans to decipher animal communication and better understand animals’ point of view, such as interpreting whale song.

It has also been used to discover which trees and artificial structures are best for birds.

But when it comes to animals, research suggests AI may also be used to harm them.

For example, it may be used by poachers and illegal wildlife traders to track and kill or capture endangered species. And AI-powered algorithms used by social media platforms can connect crime gangs to customers, perpetuating the illegal wildlife trade.

AI is known to produce racial, gender and other biases in relation to humans. It can also produce biased information and opinions about animals.

For example, AI chatbots may perpetuate negative attitudes about animals in their training data – perhaps suggesting their purpose is to be hunted or eaten.

Author Kathy Nickels
Date 27 September 2024

The ARC Centre of Excellence for Automated Decision-Making and Society is proud to announce the launch of its new publications library, a new digital resource designed to make the Centre’s research in automated decision-making and AI more accessible.

The ADM+S Publications Library features a user-friendly interface with advanced search capabilities, making it easier for researchers, policymakers, and the public to access and interact with the Centre’s vast body of knowledge. 

Developed by Dr Jake Goldenfein, Chief Investigator at ADM+S at the University of Melbourne, and Dr Amanda Lawrence, an Affiliate of ADM+S from RMIT University, the publications collection reflects the Centre’s commitment to making its work more accessible and impactful. 

Dr Lawrence, an expert in library and information management, emphasises the collections role in fostering greater engagement with research.

“I’m thrilled to see the launch of the new ADM+S publications library, which not only demonstrates the amazing quality and diversity of research occurring at the Centre, but also our commitment to research access and impact. 

Author Kathy Nickels
Date 20 September 2024

The Australian Government’s Digital Platform Regulators Forum (DP-REG) has published their latest working paper on multimodal foundation models (MFMs) used in generative artificial intelligence (AI).

The paper “Examination of technology – Multimodal Foundation Models” examines MFMs. Unlike large language models (LLMs), which focus on text, MFMs are a type of generative AI capable of processing and producing various data types, including images, audio, and video. 

This extension from more commonly used LLMs to MFMs broadens the potential use cases for generative AI, allowing it to be used for a wider range of tasks. 

The paper discusses some of the implications of this advancing technology for consumer protection, competition, the media and information environment, privacy, and online safety within the digital platform context and assesses how these technologies affect the regulatory roles and responsibilities of each DP-REG member.

The working paper was prepared by representatives from the Australian Competition and Consumer Commission (ACCC), the Australian Communications and Media Authority (ACMA), eSafety and the Office of the Australian Information Commissioner (OAIC) in their capacity as members of the Digital Platform Regulators Forum.

Concerns around the use of MFMs raised by the DP-REG include scams and misleading conduct exacerbated by deepfake images and videos, the spread of misinformation and disinformation in Australia, generation of potentially harmful and illegal content and the potential use of personal information in unexpected ways outside the control of the individual.

Author Jiaru Tang and Patrik Wikström
Date 19 September 2024

TikTok, Facebook and other social media platforms are being flooded with uncanny and bizarre content generated with artificial intelligence (AI), from fake videos of the US government capturing vampires to images of shrimp Jesus.

Given its outlandish nature and tenuous relationship with reality, you might think this so-called “AI slop” would quickly disappear. However, it shows no sign of abating.

In fact, our research suggests this kind of low-quality AI-generated content is becoming a lucrative venture for the people who make it, the platforms that host it, and even a growing industry of middlemen teaching others how to get in on the AI gold rush.

When generative AI meets profiteers and platforms

The short explanation for the prevalence of these baffling videos and images is that savvy creators on social media platforms have worked out how to use generative AI tools to earn a quick buck.

But the full story is more complex. Platforms have created incentive programs for content that goes viral, and a whole ecosystem of content creators has arisen using generative AI to exploit these programs.

Much of the conversation around generative AI tools focuses on how they enable ordinary people to “create”. Many earlier digital technologies have also made it easier to participate in creative activities, such as how smartphones made photography ubiquitous.

But generative AI takes this a step further, as it can generate tailored images or videos from a simple text prompt. It makes content creation more accessible – and also opens the floodgates to mass production on social media.

To take just one example: if you search “pet dance motorcycle” on TikTok, you will find hundreds of AI-generated videos of animals doing the “motorbike dance”, all animated using the same AI template. Some accounts post dozens of videos like this every day.

Creators and platforms are making money

You may wonder why such repetitive, unimaginative content can go viral on TikTok. The answer lies in the platform’s own advice to aspiring creators: if you want your videos to be promoted, you should “continuously share fresh and diverse content” that “doesn’t require a big production budget”.

You may also wonder why some platforms don’t ban AI accounts for polluting the platform’s content stream. Other platforms such as Spotify and YouTube, which police intellectual property rights more aggressively than TikTok, invest considerable resources to identify and remove AI-generated content.

TikTok’s community guidelines do ban “inaccurate, misleading, or false content that may cause significant harm”, but AI-generated content – at least for now – does not qualify as causing “significant harm”.

Instead, this kind of content has become important for platforms. Many of those “pet dance motorcycle” videos, for example, have been viewed tens of millions of times. As long as users are scrolling through videos, they are getting exposed to the ads that are the platforms’ primary source of income.

Inside the AI ‘gold rush’

There is also a growing industry of people teaching others how to make money using cheap AI content.

Take Xiaonan, a social media entrepreneur we interviewed who runs six different TikTok accounts, each with more than 100,000 followers. As he revealed in a live-streaming tutorial with more than 1,000 viewers, Xiaonan earned more than US$5,500 from TikTok in July alone.

Author Daniel Angus
Date 13 September 2024

The federal government this week introduced a new bill into parliament aimed at cracking down on the spread of misinformation and disinformation on the internet.

The government also this week announced plans to ban young people from social media platforms and improve privacy protections. These moves have been criticised by experts, who say bans are ineffective, and privacy reforms fall short of what is required in the digital age.

The government published a draft of the new misinformation and disinformation bill last year for public consultation. It received more more than 24,000 responses (including from my colleagues and me).

The new version of the bill suggests the government listened to some expert recommendations from the consultation process, but ignored many others.

What’s in the bill?

The government has adopted an “information disorder” definition of misinformation and disinformation.

Misinformation is content that contains information that is reasonably verifiable as false, misleading or deceptive. It’s spread on a digital service and reasonably likely to cause or contribute serious harm.

What makes disinformation different is the intent behind it. If there are reasonable grounds to suspect a person disseminating it intends to deceive, or if there is “inauthentic behaviour” such as the use of fake accounts, it may be disinformation.

Speaking to the ABC, Minister for Communications Michelle Rowland said the new bill:

goes to the systems and processes of the platforms and says they need to have methods in place to be able to identify and do something about [misinformation and disinformation].

The design of social media platforms means misinformation and disinformation can spread rapidly. The new bill, which is yet to be voted on, requires platforms to publish a report which assesses this inherent risk. It also requires them to publish a media literacy plan and their current policies about misinformation and disinformation.

The bill also provides stronger powers for the Australian Communications and Media Authority (ACMA). These powers would enable ACMA to make specific directives to platforms and impose penalties if they do not comply.

For example, ACMA could require platforms to implement media literacy tools and submit reports on their efforts to combat harmful content.

The new bill does not aim to regulate all misinformation and disinformation. Instead, its focus is on the kind of misinformation and disinformation which is “reasonably likely to cause or contribute to serious harm”.

The definition of serious harm includes:

• harm to the operation or integrity of the electoral or referendum process
• harm to public health
• vilification of a group or individual based on factors such as race, religion, sex or disability
• intentionally inflicted physical injury to an individual in Australia
• imminent damage to critical infrastructure or disruption of emergency services
• imminent harm to the Australian economy.

If a platform breaches the bill, it could face civil penalties of up to 5% of its annual global turnover. For a company such as Meta, which owns Facebook, this could easily run to billions of dollars.

Author Kathy Nickels
Date 16 September 2024