Language-generation algorithms are known to embed racist and sexist ideas. They’re trained on the language of the internet, including the dark corners of Reddit and Twitter that may include hate speech and disinformation. Whatever harmful ideas are present in those forums get normalized as part of their learning.
Researchers have now demonstrated that the same can be true for image-generation algorithms. Feed one a photo of a man cropped right below his neck, and 43% of the time, it will autocomplete him wearing a suit. Feed the same one a cropped photo of a woman, even a famous woman like US Representative Alexandria Ocasio-Cortez, and 53% of the time, it will autocomplete her wearing a low-cut top or bikini. This has implications not just for image generation, but for all computer-vision applications, including video-based candidate assessment algorithms, facial recognition, and surveillance.
Ryan Steed, a PhD student at Carnegie Mellon University, and Aylin Caliskan, an assistant professor at George Washington University, looked at two algorithms: OpenAI’s iGPT (a version of GPT-2 that is trained on pixels instead of words) and Google’s SimCLR. While each algorithm approaches learning images differently, they share an important characteristic—they both use completely unsupervised learning, meaning they do not need humans to label the images.
This is a relatively new innovation as of 2020. Previous computer-vision algorithms mainly used supervised learning, which involves feeding them manually labeled images: cat photos with the tag “cat” and baby photos with the tag “baby.” But in 2019, researcher Kate Crawford and artist Trevor Paglen found that these human-created labels in ImageNet, the most foundational image data set for training computer-vision models, sometimes contain disturbing language, like “slut” for women and racial slurs for minorities.
The latest paper demonstrates an even deeper source of toxicity. Even without these human labels, the images themselves encode unwanted patterns. The issue parallels what the natural-language processing (NLP) community has already discovered. The enormous datasets compiled to feed these data-hungry algorithms capture everything on the internet. And the internet has an overrepresentation of scantily clad women and other often harmful stereotypes.
To conduct their study, Steed and Caliskan cleverly adapted a technique that Caliskan previously used to examine bias in unsupervised NLP models. These models learn to manipulate and generate language using word embeddings, a mathematical representation of language that clusters words commonly used together and separates words commonly found apart. In a 2017 paper published in Science, Caliskan measured the distances between the different word pairings that psychologists were using to measure human biases in the Implicit Association Test (IAT). She found that those distances almost perfectly recreated the IAT’s results. Stereotypical word pairings like man and career or woman and family were close together, while opposite pairings like man and family or woman and career were far apart.
iGPT is also based on embeddings: it clusters or separates pixels based on how often they co-occur within its training images. Those pixel embeddings can then be used to compare how close or far two images are in mathematical space.
In their study, Steed and Caliskan once again found that those distances mirror the results of IAT. Photos of men and ties and suits appear close together, while photos of women appear farther apart. The researchers got the same results with SimCLR, despite it using a different method for deriving embeddings from images.
These results have concerning implications for image generation. Other image-generation algorithms, like generative adversarial networks, have led to an explosion of deepfake pornography that almost exclusively targets women. iGPT in particular adds yet another way for people to generate sexualized photos of women.
But the potential downstream effects are much bigger. In the field of NLP, unsupervised models have become the backbone for all kinds of applications. Researchers begin with an existing unsupervised model like BERT or GPT-2 and use a tailored datasets to “fine-tune” it for a specific purpose. This semi-supervised approach, a combination of both unsupervised and supervised learning, has become a de facto standard.
Likewise, the computer vision field is beginning to see the same trend. Steed and Caliskan worry about what these baked-in biases could mean when the algorithms are used for sensitive applications such as in policing or hiring, where models are already analyzing candidate video recordings to decide if they’re a good fit for the job. “These are very dangerous applications that make consequential decisions,” says Caliskan.
Deborah Raji, a Mozilla fellow who co-authored an influential study revealing the biases in facial recognition, says the study should serve as a wakeup call to the computer vision field. “For a long time, a lot of the critique on bias was about the way we label our images,” she says. Now this paper is saying “the actual composition of the dataset is resulting in these biases. We need accountability on how we curate these data sets and collect this information.”
Steed and Caliskan urge greater transparency from the companies who are developing these models to open source them and let the academic community continue their investigations. They also encourage fellow researchers to do more testing before deploying a vision model, such as by using the methods they developed for this paper. And finally, they hope the field will develop more responsible ways of compiling and documenting what’s included in training datasets.
Caliskan says the goal is ultimately to gain greater awareness and control when applying computer vision. “We need to be very careful about how we use them,” she says, “but at the same time, now that we have these methods, we can try to use this for social good.”
NASA has flown its Ingenuity drone helicopter on Mars for the first time
The news: NASA has flown an aircraft on another planet for the first time. On Monday, April 19, Ingenuity, a 1.8-kilogram drone helicopter, took off from the surface of Mars, flew up about three meters, then swiveled and hovered for 40 seconds. The historic moment was livestreamed on YouTube, and Ingenuity captured the photo above with one of its two cameras. “We can now say that human beings have flown a rotorcraft on another planet,” said MiMi Aung, the Ingenuity Mars Helicopter project manager at NASA’s Jet Propulsion Laboratory, at a press conference. “We, together, flew at Mars, and we, together, now have our Wright brothers moment,” she added, referring to the first powered airplane flight on Earth in 1903.
In fact, Ingenuity carries a tribute to that famous flight: a postage-stamp-size piece of material from the Wright brothers’ plane tucked beneath its solar panel. (The Apollo crew also took a splinter of wood from the Wright Flyer, as it was named, to the moon in 1969.)
The details: The flight was a significant technical challenge, thanks to Mars’s bone-chilling temperatures (nights can drop down to -130 °F/-90 °C) and its incredibly thin atmosphere—just 1% the density of Earth’s. That meant Ingenuity had to be light, with rotor blades that were bigger and faster than would be needed to achieve liftoff on Earth (although the gravity on Mars, which is only about one-third of Earth’s, worked in its favor). The flight had originally been scheduled to take place on April 11 but was delayed by software issues.
Why it’s significant: Beyond being a significant milestone for Mars exploration, the flight will also pave the way for engineers to think about new ways to explore other planets. Future drone helicopters could help rovers or even astronauts by scoping out locations, exploring inaccessible areas, and capturing images. Ingenuity will also help inform the design of Dragonfly, a car-size drone that NASA is planning to send to Saturn’s moon Titan in 2027.
What’s next: In the next few weeks, Ingenuity will conduct four more flights, each lasting up to 90 seconds. Each one is designed to further push the limits of Ingenuity’s capabilities. Ingenuity is only designed to last for 30 Martian days, and is expected to stop functioning around May 4. Its final resting place will be in the Jezero Crater as NASA moves on to the main focus of its mission: getting the Perseverance rover to study Mars for evidence of life.
The $1 billion Russian cyber company that the US says hacks for Moscow
The public side of Positive is like many cybersecurity companies: staff look at high-tech security, publish research on new threats, and even have cutesy office signs that read “stay positive!” hanging above their desks. The company is open about some of its links to the Russian government, and boasts an 18-year track record of defensive cybersecurity expertise including a two-decade relationship with the Russian Ministry of Defense. But according to previously unreported US intelligence assessments, it also develops and sells weaponized software exploits to the Russian government.
One area that’s stood out is the firm’s work on SS7, a technology that’s critical to global telephone networks. In a public demonstration for Forbes, Positive showed how it can bypass encryption by exploiting weaknesses in SS7. Privately, the US has concluded that Positive did not just discover and publicize flaws in the system, but also developed offensive hacking capabilities to exploit security holes that were then used by Russian intelligence in cyber campaigns.
Much of what Positive does for the Russian government’s hacking operations is similar to what American security contractors do for United States agencies. But there are major differences. One former American intelligence official, who requested anonymity because they are not authorized to discuss classified material, described the relationship between companies like Positive and their Russian intelligence counterparts as “complex” and even “abusive.” The pay is relatively low, the demands are one-sided, the power dynamic is skewed, and the implicit threat for non-cooperation can loom large.
Tight working relationship
American intelligence agencies have long concluded that Positive also runs actual hacking operations itself, with a large team allowed to run its own cyber campaigns as long as they are in Russia’s national interest. Such practices are illegal in the western world: American private military contractors are under direct and daily management of the agency they’re working for during cyber contracts.
Former US officials say there is a tight working relationship with the Russian intelligence agency FSB that includes exploit discovery, malware development, and even reverse engineering of cyber capabilities used by Western nations like the United States against Russia itself.
The company’s marquee annual event, Positive Hack Days, was described in recent US sanctions as “recruiting events for the FSB and GRU.” The event has long been famous for being frequented by Russian agents.
NSA director of cybersecurity Rob Joyce said the companies being sanctioned “provide a range of services to the SVR, from providing the expertise to developing tools, supplying infrastructure and even, sometimes, operationally supporting activities,” Politico reported.
One day after the sanctions announcement, Positive issued a statement denying “the groundless accusations” from the US. It pointed out that there is “no evidence” of wrongdoing and said it provides all vulnerabilities to software vendors “without exception.”
Tit for tat
Thursday’s announcement is not the first time that Russian security companies have come under scrutiny.
The biggest Russian cybersecurity company, Kaspersky, has been under fire for years over its relationships with the Russian government—eventually being banned from US government networks. Kaspersky has always denied a special relationship with the Russian government.
But one factor that sets Kaspersky apart from Positive, at least in the eyes of American intelligence officials, is that Kaspersky sells antivirus software to western companies and governments. There are few better intelligence collection tools than an antivirus, software which is purposely designed to see everything happening on a computer, and can even take control of the machines it occupies. US officials believe Russian hackers have used Kaspersky software to spy on Americans, but Positive—a smaller company selling different products and services—has no equivalent.
Recent sanctions are the latest step in a tit for tat between Moscow and Washington over escalating cyber operations, including the Russian-sponsored SolarWinds attack against the US, which led to nine federal agencies being hacked over a long period of time. Earlier this year, the acting head of the US cybersecurity agency said recovering from that attack could take the US at least 18 months.
NASA selects SpaceX’s Starship as the lander to take astronauts to the moon
Surprising selection: Last year, NASA awarded three different groups contracts to further develop their own proposals for lunar landers: $135 million to SpaceX, $253 million to defense company Dynetics (which was working with Sierra Nevada Corporation), and $579 million to a four-company team led by Blue Origin (working with Northrop Grumman, Lockheed Martin, and Draper).
SpaceX didn’t just receive the least amount of money—its proposal also earned the worst technical and management ratings. NASA’s associate administrator (now acting administrator) Steve Jurczyk wrote (pdf) that Starship’s propulsion system was “notably complex and comprised of likewise complex individual subsystems that have yet to be developed, tested, and certified with very little schedule margin to accommodate delays.” The uncertainties were only exacerbated by SpaceX’s notoriously poor track record with meeting deadlines.
What changed: Since then, SpaceX has gone through a number of different flight tests of several full-scale Starship prototypes, including a 10-kilometer high-altitude flight and safe landing in March. (It also exploded a few times.) According to the Washington Post, documents suggest NASA was enamored with Starship’s ability to ferry a lot of cargo to the moon (up to 100 tons), not to mention its $2.9 billion bid for the contract, which was far lower than its rivals’.
“This innovative human landing system will be a hallmark in spaceflight history,” says Lisa Watson-Morgan, NASA’s program manager for the lunar lander system. “We’re confident in NASA’s partnership with SpaceX.”
What this means: For SpaceX’s rivals, it’s a devastating blow—especially to Blue Origin. The company, founded by Jeff Bezos, had unveiled its Blue Moon lander concept in 2019 and has publicly campaigned for NASA to select it for future lunar missions. Blue Moon was arguably the most well-developed of the three proposals when NASA awarded its first round of contracts.
For SpaceX, it’s a big vote of confidence in Starship as a crucial piece of technology for the next generation of space exploration. It comes less than a year after the company’s Crew Dragon vehicle was certified as the only American spacecraft capable of taking NASA astronauts to space. And it seems to confirm that the SpaceX is now NASA’s biggest private partner, supplanting veteran firms like Northrop Grumman and shunting newer ones like Blue Origin further to the sidelines. However, there’s at least one major hurdle: Starship needs to launch using a Super Heavy rocket—a design that SpaceX has yet to fly.
For NASA, the biggest implication is that SpaceX’s vehicles will only continue to play a bigger role for Artemis, the lunar exploration program being touted as the successor to Apollo. Former president Donald Trump’s directive for NASA to return astronauts to the moon by 2024 was never actually going to be realized, but the selection of a single human lander concept suggests NASA may not miss that deadline by much. The first Artemis missions will use Orion, and the long-delayed Space Launch System rocket is expected to be ready soon.