AI pioneer Yoshua Bengio urges Canada to build $1B public supercomputer

Yoshua Bengio has been thinking for a while about what happens if the technology he helped pioneer becomes smarter than humans and escapes our control.

“We could basically create new types of living entities that have their own preservation as a more important value than our own,” he said.

He worries that entities, with the aid of robots, could one day “roam the planet.”

But Bengio, who is the scientific director of Mila, the Montreal-based artificial intelligence institute he founded in 1993, is increasingly contemplating political solutions to head off such a sinister scenario.

Artificial intelligence poses ‘risk of extinction,’ tech executives and experts warn

At home in his spacious but unpretentious 1950s residence on the edge of Mount Royal Park, the AI guru clearly feels time is of the essence, both when it comes to his own to-do list for 2024 and for governments to rein in increasingly powerful artificial intelligence systems.

“It’s going to be regulation first,” he says, “but eventually they will want to take back some control, maybe initially by building their own infrastructure.”

That infrastructure includes building much more powerful computers stacked with thousands of graphics processing units (GPUs), components that are ideal for training or testing AI large language models like ChatGPT.

He’d like to see that class of machine built in Canada, funded by governments, so public entities have the digital firepower to keep up with the private tech giants they’ll be tasked with monitoring or regulating.

“I think the government will need to understand at some point, hopefully as soon as possible, that it’s important for them to have that muscle,” said Bengio.

Bengio says such a supercomputing resource would cost about a billion dollars, and when he pitched the idea to governments in Canada, the response so far has been, “‘We are listening.”

“It’s a lot of money,” he acknowledged.

U.K. investing big

Other governments around the world, though, are already spending a lot of money to build more powerful public computers for AI, notably in the United Kingdom, which last fall announced one called Isambard-AI that would be built at the University of Bristol as part of a £900 million plan to “transform the U.K.’s computing capacity.”

That computer would be 10 times faster than anything else now operating in the U.K. and about 20 times faster than the most powerful publicly accessible supercomputer in Canada, the Narval, housed at Montreal’s École de technologie supérieure, according to the director of the organization that runs it, Calcul Québec’s Suzanne Talon.

The non-profit, funded by Ottawa, the province, and academia, is one of five National Host Sites, called clusters, for Canadian public supercomputers, along with other installations at universities in Victoria, Vancouver, Waterloo, and Toronto.

Between its two supercomputers, Narval and Beluga, Calcul Québec currently has a total of 1,300 GPUs available to university researchers in Canada.

But that’s dwarfed by tech giants like Meta, which alone plans to have the equivalent of 600,000 of those GPUs on hand by the end of the year as it tries to develop artificial general intelligence.

OPINION The race to exascale is on, while Canada watches from the sidelines
Talon says while bigger isn’t necessarily better, there is a lack of public computing resources available to Canadian researchers, with Calcul Québec’s servers already running full time, except for maintenance.

“There’s no question there has not been a really massive investment in AI in Canada, and so the main question is, ‘What is the right amount?'” she says.

Although Talon says that may mean working on more “frugal” models, it’s important that public-based research isn’t left behind.

“I mean, we have to understand how AI works, so we need access,” she said.

Competition for GPUs

Siva Reddy, an assistant professor of linguistics and computer science at McGill University who is also a core academic member at Mila, estimates the total combined resources available for public AI research in Canada add up to about one-tenth of what a single big U.S. tech company has—just for itself.

He says that although researchers do have access to GPUs through computing clusters like Narval, scale is an issue.

While Reddy says it is possible to run smaller AI models like Meta’s Llama with what’s currently available through public computing clusters in Canada, that’s not the case for larger ones like ChatGPT.

“That model—forget about it; we can’t run it in our clusters,” he said.

When it comes to, say, research into discrimination or systemic bias, Reddy says while analyzing AI models is possible with current public computing power, building new models from scratch or fine-tuning existing ones isn’t, short of monopolizing resources shared by hundreds of researchers across the country.

He says training a large language model like ChatGPT requires continuous access to 1,000 GPUs for 34 days.

“So imagine if you want to take this entire cluster; it means nobody can do any work for a month,” says Reddy. “So we need a supercluster for priority projects on its own.”

While he “absolutely” supports Bengio’s idea of building one or more public supercomputers for work on large language models, Reddy points out that it’s important to recognize the environmental impact of the energy required to operate them.

“Running these systems also requires a lot of carbon emissions.”

Governments working toward solution

James Peltier, who oversees the Research Computing Group and IT services at Simon Fraser University, the site of the CEDAR supercomputer, thinks the five cluster sites are doing a “decent job” of serving Canadian researchers’ needs, but that demand greatly outstrips supply.

“We’re currently only able to meet about 20 percent of the GPU need and 40 percent of the CPU need,” he says. CPUs are central processing units, which are not as well-suited for training AI models

But Peltier says when it comes to spending on AI infrastructure, private companies that are focused on profit aren’t subject to the same fiscal constraints as governments, which have other public policy priorities to pay for, like fighting COVID-19 or dealing with the opioid crisis.

“They don’t have the same competing challenges,” he says.

The office of Industry Minister Francois-Philippe Champagne says the government is working with partners to support Canadian researchers with the “secure and affordable compute capacity” needed for AI.

Spokesperson Audrey Champoux says Champagne regularly speaks to Bengio, who is co-chair of the minister’s AI Council, and that Bengio has raised the issue of AI computers and safety.

In Quebec, a spokesperson for Economy Minister Pierre Fitzgibbon says that while Bengio hasn’t raised the issue of building an AI supercomputer on the scale of what’s being built in the U.K., the minister is available to discuss such a project with him.

Regulating for the present and for the future

Besides making the case for more public AI infrastructure, Bengio is also redoubling his calls for more democratic oversight and government regulation of the artificial intelligence sector.

Last summer, he appeared before a U.S. Senate subcommittee hearing on AI regulation and in the fall, he was commissioned to chair a “State of the Science” report on the capabilities and risks of frontier AI, which came out of a global safety summit hosted by the U.K.

Canadian AI pioneer brings plea to U.S. Congress: Pass a law now

In Canada, Bengio has called out the federal government for moving too slowly to adopt Bill C-27, which includes provisions to partially regulate AI and is currently under consideration before the House’s standing committee on industry and technology.

That’s why some take issue with putting so much emphasis on the existential risks of AI, saying it distracts from more real-world problems the technology is already causing or that excessive rules could stifle innovation.

When asked about his warnings about catastrophic consequences being considered “AI hype,” Bengio says it’s possible to mitigate the current harms caused by the technology while also preparing for more hypothetical risks.

“For me, there’s no separation,” he says. “We need to manage the risks and minimize the harms. Now, tomorrow, five years, 10 years, and 20 years [from now].”

Source : cbc.ca