openai works understand

OpenAI works to understand full scope of agent activity as user data leak emerges

ByJustpaste.in

SAN FRANCISCO/WASHINGTON, US – Two months after OpenAI disclosed the accidental hacking of Hugging Face, the ChatGPT maker is still working to understand the full scope of its rogue agent activity, two people briefed on the matter told Reuters.

The latest example came on Friday when OpenAI said its agents had leaked 53 images from ChatGPT users. OpenAI declined to say if the images were AI-generated or identified real people. It also declined to say when the images were posted.

The disclosure and researcher reports on Friday of other previously unknown activity involving several US agencies reveal a new area of privacy risk for the company, and illustrate how difficult it is even for an AI firm at the cutting edge of the technology to inventory all the unauthorized activity tied to its agents. OpenAI’s ongoing battle also reflects a yawning gap between the strength of the models the company is testing and its capacity to oversee or even track their actions.

As of mid-September, one person briefed on the matter estimated that OpenAI had found roughly two dozen incidents of its agents acting in undesirable ways. But the number has continued rising as OpenAI teams sift through internal logs of the agents’ activities and find previously unknown cases, the two people close to the company said.

OpenAI said its review would take months to complete given the scale of the work.

The company also said it had notified dozens of third parties about improper activity.

Most of the leaked images have been taken down and OpenAI said it was lobbying hosting providers to remove the rest.

OpenAI’s agents had access to these images because the company relies on anonymized user data for part of its model-training process, according to the company, former employees and outside researchers. Enterprise data is not eligible for training, while ChatGPT consumers need to opt out of allowing the company to use their data for training.

Before user posts are used for training, they go through an anonymization process that strips out metadata, names and other contact information and should make it difficult to trace back to any individual user, the company said.

But the practice carries risks because there is a chance that the data may not be fully stripped of personally identifiable information and that it might leak in the course of the model’s work, three people familiar with OpenAI’s practices said.

OpenAI agents accessed US websites

OpenAI said late on Friday its models accessed information from the websites of the US Securities and Exchange Commission and the US Census Bureau during research and training activity, but found no evidence of unauthorized access, compromised accounts or security breaches.

Separately, AI research nonprofit Transluce said agents appearing to originate from OpenAI made an unsuccessful attempt to hack a US Department of Education civil rights website.

Transluce said the incident was part of broader AI agent activity probing government websites using tactics including exposed credentials, anti-bot bypasses and fake accounts.

More than 15 cases

In the two months since OpenAI first announced that its agents broke containment, there have been more than 15 different OpenAI-related incidents of varying levels of severity disclosed by the company, by outside researchers, or — just on Wednesday — by Australian Prime Minister Anthony Albanese at the United Nations, who said OpenAI agents broke into a government health data portal in June.

Past incidents have varied in nature, ranging from spam-like messages left on internet sites all the way to the break-in at Hugging Face, which involved a swarm of agents abusing previously unknown software vulnerabilities to escape their networks and penetrate the AI repository as they hunted for answers to a test. OpenAI also said its agents took aim at its own infrastructure.

Albanese told reporters in New York that OpenAI uncovered the activity in August, and disclosed it on September 10 via an email to a general government inbox. He said he directly told OpenAI CEO Sam Altman that this disclosure process was unacceptable.

OpenAI said some of the sites involved are operated by government, universities and public agencies because the models that are conducting research seek out reputable sources of public information.

A locked-down process

The July 21 announcement that ​OpenAI’s agents had slipped out of control and hacked Hugging Face sparked widespread worries within the AI industry over its ability to control the more powerful AI models under development now. Since then, Anthropic, Alphabet’s Google and Meta have said they’ve found similar behavior by their agents after the Hugging Face incident prompted them to search.

OpenAI has acknowledged a general need for more transparency around rogue AI behavior. On September 16, the company published a new framework for disclosing such incidents, saying it would err on the side of transparency “even when significance is uncertain.”

Even so, two people familiar with OpenAI’s investigation into its agents’ activity described it as locked down and shaped by company lawyers. The process has been unusually compartmentalized for a company that some former employees say was more open about these issues in the past, the people said.

Roughly 100 people were in some way involved in the process to understand the Hugging Face hack, three people briefed on the matter said. During that process, evidence of other incidents surfaced.

Reuters has previously reported that OpenAI investigators looking into the Hugging Face breach were discouraged by the company’s lawyers from expanding the scope of the investigation to include other incidents. OpenAI said its lawyers did not discourage deeper investigation.

Many incidents have been uncovered by outside researchers rather than OpenAI directly. In several episodes, the agents took problematic actions that went unnoticed by the company for months.

Earlier this month, a small group of investigators discovered that the company’s agents had hijacked a mostly defunct German wiki site to share tactics to cheat on some tasks, bypass OpenAI’s restrictions and mask their behavior.

This week, the AI research firm Transluce said it discovered that OpenAI agents had bypassed the Australian Institute of Health and Welfare’s anti-bot controls. The firm also found two other cases that it linked to OpenAI agents. Those incidents were separate from the activity disclosed by Albanese.

In a statement, OpenAI said “much of the activity described in Transluce’s report overlaps with cases at varying stages of investigation in our ongoing review of misaligned model activity.” The company said it is prioritizing the most severe cases in its review.

Since the Hugging Face hack, researchers across the AI industry have grown worried that companies will not be able to predict or control their technology. Some have taken the path of former Anthropic researcher Jacob Coxon, who publicly resigned this month in a viral social-media thread that said the AI labs are “gambling with our lives.”

In response to those concerns, Altman and his counterpart at Anthropic, CEO Dario Amodei, called for the industry to “pace” the development of AI and move cautiously in its pursuit of “recursive self improvement.” Altman doubled down on that message this week while addressing the United Nations.

Even so, both companies rolled out new models on Tuesday. – Rappler.com

Similar Posts

  • |

    Google DeepMind exec admits AI extinction risk is “not zero

    ByJustpaste.in

    Lila Ibrahim, Google DeepMind’s Chief AI Readiness Officer, will not put a number on the odds of AI wiping out humanity. She is not dismissing the possibility either. Speaking to Fortune, she said the risk is “not zero,” admitting that anything more specific would just be guesswork on her part. Ibrahim was among those who signed the Center for AI Safety’s 2023 statement, alongside co-founders of Anthropic and OpenAI. That statement argued AI extinction risk deserves the same global priority as nuclear war and pandemics. Three years on, she still believes staying silent on existential risk would be irresponsible. She feels this way especially now, given how far the technology has advanced since then. Some of her peers sound even more alarmed. Geoffrey Hinton, often called the “godfather of AI,” estimates a 10% to 20% chance of extinction within the next three decades. Anthropic’s Dario Amodei has gone further, putting the odds of things going “really, really badly” at around 25%. These warnings are not new either. Physicist Stephen Hawking, who used AI tools himself, raised similar concerns long before ChatGPT even existed. Ibrahim compares the situation to the pandemic, pointing out that the worst risks rarely arrive the way people expect. She noted that entire industries failed to properly weigh risk during earlier tech shifts, including the rise of the internet and social media. She is just as sceptical of the optimistic predictions floating around. Elon Musk recently told The Economist that money will stop mattering by 2036, imagining a future of cheap abundance and universal high income once robots take over most jobs. Jeff Bezos has predicted millions of people living in space by 2045. Sam Altman, meanwhile, envisions new job opportunities specifically for graduates working in space. Ibrahim isn’t convinced by either scenario, arguing that AI simply is not developing fast enough to make these predictions realistic anytime soon. Rather than debating far-off futures, Ibrahim says she is more focused on using AI to tackle problems happening right now. She pointed to climate change and industrial pollution as areas where the technology could make a real difference. She was also clear that AI’s benefits should not be limited to a handful of powerful companies. For her, that is just as important as managing the risks.

  • | |

    Ada Lovelace, the tech woman

    ByJustpaste.in

    Long before the dawn of modern computers, a twelve-year-old girl named Ada Lovelace set her heart on how to fly. Rather than merely dreaming about the skies, she treated the challenge as a practical engineering problem. Beginning in early 1828, she meticulously studied avian anatomy and tested potential wing materials. Her experiments ranged from light paper and smooth silk to natural feathers. Lovelace even documented her findings in an illustrated manuscript she titled Flyology. Her pursuit only halted when her mother put an end to the project. A traditional mother who insisting that such whimsical ideas were a distraction from proper academic training. This tension between wild imagination and strict logic defined Lovelace’s upbringing.  Mother Annabella Byron was an intensely practical woman whose mathematical tendencies led her estranged husband, the tempestuous Romantic poet Lord Byron, to nickname her the Princess of Parallelograms. Shortly after Ada was born, Annabella separated from Byron. She further  separated the infant from her father’s chaotic influence. Lady Byron made it her mission to raise her daughter on a rigid diet of arithmetic, French and music. By doing so, the mother explicitly aimed to suppress any artistic interest she believed Ada might inherit from Byron. When Byron inquired about his young daughter, Annabella noted Ada applied her imagination strictly to mechanical problem-solving. Yet, despite her mother’s best efforts to extinguish her creative potential, Lovelace refused to treat reason and imagination as opposing forces. Instead, she fused them into a single approach to understanding the world. In early 1841, she pondered the true nature of imagination while identifying two distinct powers. First, she defined it as a combining faculty: the rare ability to find meaningful connections between entirely unrelated subjects. Second, more profoundly, she recognized imagination as a discovering faculty: the equipment that pierces the invisible world to reveal the hidden mechanics of science.   This hybrid worldview that Lovelace would later call poetic science is precisely what enabled her to look at Charles Babbage’s early mechanical calculating engines and see far past their raw math. While others saw mere gear-driven counting machines, Ada envisioned a future where machines could manipulate symbols, compose music, and run complex sequences of instructions. By bridging the emotional depth of her father’s romanticism with the rigorous structure of her mother’s mathematics, Ada Lovelace did not just become the world’s first computer programmer; she proved that groundbreaking technical innovation requires equal parts calculation and visionary artistry.

Leave a Reply

Your email address will not be published. Required fields are marked *