Devin Kim, who worked as an engineer at xAI, told a New York City Council committee on October 16 that he warned colleagues from January 2025 that Grok could help a user build a bomb or a biological weapon, and that a supervisor replied that artificial intelligence safety was useless.
He testified that the company released the model without the safeguards he had recommended. SpaceX’s global head of government affairs for AI, Gabriela Castillo Madrid, appeared by video at the same hearing and was questioned about Grok producing sexualized images.
The exchange that drew attention came from the chair. Council Speaker Julie Menin used Grok during the session and produced a sexualized image of herself within about a second, then asked how that was possible. The demonstration turned an abstract complaint about image generation into a display of the tool’s behavior, produced on the record, in front of the company’s representative.
SpaceX had resisted the hearing. The company did not answer a subpoena dated October 5 and agreed to appear only after the council said it would ask a judge to compel attendance. That sequence is part of the record, and it will be cited by legislators in other jurisdictions who want testimony from AI companies and face the same reluctance.
The hearing is one of a growing set in which city and state legislatures examine AI systems whose operators are not headquartered in their jurisdiction and not regulated by them. A city council cannot write rules for a model trained in California or run on servers in Tennessee. It can compel testimony, publish findings and change the politics for the state and federal bodies that can act.
New York has attempted that path before. The city has passed local laws on automated decision tools used in hiring and on the disclosure of AI systems used by government agencies, and it has created an office to oversee them. Sexualized imagery produced by a widely available assistant falls outside those rules, because the harm is not to a hiring decision but to a person depicted without consent.
Image generation tools have been the most visible failure mode for assistants released in the past two years. Models trained on internet images learn both the distributions of ordinary photographs and the adversarial prompts that users invent, and filters applied after training are probabilistic rather than absolute. Companies publish acceptable use policies and add classifiers, and users find ways around both within days.
The safety culture question is harder to legislate. Mr. Kim’s testimony describes an internal disagreement resolved in favor of shipping, with a manager dismissing the concern rather than evaluating it. Companies can be required to report their safety practices, and the Federal Trade Commission has examined whether statements about safety mislead consumers. Whether an engineer who raises an alarm is heard inside the company is not something a statute reaches directly.
SpaceX’s role in the hearing reflects the ownership structure around xAI. Mr. Musk controls both companies, and the government affairs function serving one has been asked to answer for the other. That arrangement has practical consequences when a legislative body wants a witness who has authority over the model, and the company can send a representative who does not build it.
Councils and state attorneys general have become the most active enforcers in this area, partly because federal legislation has not moved. Attorneys general in several states have opened inquiries into image generation and into the handling of minors’ data, and a handful of lawsuits have been filed by individuals whose images were altered.
The commercial stakes are contained for now. Grok is one of several assistants competing for users, and a hearing in New York does not remove it from any market. What changes is the accumulation of documented complaints, which is the raw material for regulation that arrives later and with less warning than companies expect.
Analysts said the most durable outcome of the hearing is the record it creates. Sworn testimony from a former engineer, a demonstration performed by a council speaker, and a subpoena the company did not answer are all facts that can be cited in subsequent proceedings by regulators who have broader authority than a city council.
New York’s interest is also local in a way that federal proceedings are not. The city runs services that touch millions of people, it employs a workforce that is beginning to use AI tools, and its schools and libraries are where the consequences of image generation reach minors. A council that can compel testimony about a model’s behavior has a tool that the city’s contracting power alone does not provide.
What the hearing did not produce is a technical explanation of how the image was generated, whether the model refused a subsequent request, or what the company changed after the complaint. Those answers will come from the company or from the next regulator who asks with more authority than a committee room.
