Whether you are attending the International Journalism Festival or the International Internet Preservation Consortium’s conference, we would love to connect with you. Here is where you can find us:

International Journalism Festival

We will be participating in crucial conversations about how reporters can protect the integrity of the data they collect in conflict zones.

Public Panel: Witness to war: how journalists can safeguard digital evidence for justice

From Ukraine to Sudan, journalists are often among the first to document human rights abuses and war crimes. This panel will explore the technical, ethical, and logistical realities of capturing and safeguarding sensitive digital evidence in conflict zones so that it can eventually be used for justice.

We are also co-hosting a closed-door session discussing how journalistic documentation of atrocity crimes can be preserved, verified, and managed to support future accountability efforts. In partnership with: Airwars, IrpiMedia, Paper Trail Media, and Videre.

IIPC General Assembly & Web Archiving Conference

Location: Royal Library of Belgium, Brussels | Dates: 20–23 April 2026

We are honored to be leading the Closing Keynote Panel at this year’s IIPC event, and to bring to the event a fresh perspective on web archiving: the rapidly evolving, high-stakes use case of using web archives as a critical tool for legal accountability and human rights documentation.

Closing Keynote: Archiving for accountability: new frontiers in open source intelligence and digital evidence

Starling Lab’s own Basile Simon will be chairing the session, and we have invited Emily Tripp (exec director of Airwars), Marvin Milatz (researcher / OSINT expert at Der Spiegel), and Friedhelm Weinberg (director of programmes at Mnemonic).

If you plan on attending either of these events, please reply to this email or send us a message so we can find time to connect. We are always looking forward to meeting with fellow technologists, journalists, and legal experts to share knowledge and build better standards for digital evidence.

Safe travels, and we hope to see you there!

The Reality Collapse: Certainty as the Premium Product

In an AI-polluted media landscape increasingly defined by what we call the “reality collapse,” our work this winter focused on moving beyond mere prototypes to establish “certainty as the premium product.” This effort was perhaps best articulated by Adam Rose in his recent op-ed for Poynter, where he argued that the existential threat of AI is not merely the content it generates, but the resulting systemic collapse of trust in all digital information.

This philosophy of “Authenticity by Design” is our north star, guiding our contributions to international human rights frameworks and the future of investigative journalism. This vision was further articulated in Basile Simon’s 2026 prediction for Nieman Lab, which noted: “If confusion is the commodity, certainty is the premium product.”

The Lab on the International Stage

The Lab continues to bridge the gap between deep tech and global policy through high-level convenings. In collaboration with Stanford HAI, we recently co-sponsored an event featuring Tom Friedman and Craig Mundie, which served as a vital forum for discussing the geopolitical implications of the current AI revolution.

Ann Grimes recently delivered a talk at the Stanford Symbolic Systems Forum (SSP), outlining the foundational principles of Starling Lab and our “Authenticity by Design” framework to a cross-disciplinary audience of technologists and researchers.

Basile Simon has spearheaded several key initiatives:

• His submission, on behalf of the Lab, to the UN Special Rapporteur on Counter Terrorism was formally cited in their position paper.

• A piece for OpinioJuris titled “When Tech Disrupts Faster Than Rules Adapt,” providing emergency guidance for handling AI-affected evidence, in collaboration with our colleagues from the Fénix Foundation, and following our summer 2025 gathering in The Hague.

• Contributed to the Filecoin Foundation for the Decentralized Web (FFDW) Digest, using accessible analogies to explain how a decentralized web of trust can safeguard against digital authoritarianism.

2026 Global Roadmap

On the international stage, the Lab is preparing for a series of high-profile engagements that place our tools in the hands of those on the front lines of justice. At the upcoming International Journalism Festival in Perugia, we will take part in a panel on preserving digital evidence alongside partners from Airwars and Global Rights Compliance, moderated by Frederik Obermaier of Paper Trail Media.

We will then take to RightsCon in Lusaka, Zambia where, in partnership with the Guardian Project, we will lead a workshop session on authenticity tools and capturing authenticated media, including a paper prototyping of provenance UIs workshop.

We were also delighted by the decision of the International Internet Preservation Consortium (IIPC) to elevate our upcoming session on the various practices of web archiving to the closing keynote of the annual web archiving conference, which will take place in Brussels, Belgium.

Preserving the Digital Record

As we look toward the rest of 2026, our focus remains on ensuring that the digital record—from oral histories to social media captures—remains verifiable, immutable, and resilient. Our most recent effort in this direction is the preservation of several terabytes of 3D scans of heritage sites in Nagorno-Karabakh at risk of degradation, through the lab’s data preservation efforts and collaboration with the USC Digital Repository and Libraries.

Things we’ve seen

• Camera Bits’ Photo Mechanic support for C2PA integrated into the verification process for Olympic photography.

• Time Magazine plummeted into the “uncanny valley” with unsettling historical recreations using generative AI.

• The launch of the AP Verify dashboard, providing a bouquet of verification tools for reporters.

• The problematic use of AI to “unmask” and “enhance” crime footage by home sleuths.

• Meedan’s brand evolution and the introduction of Suwali, an AI-powered tool designed for public interest.

• The Internet Archive’s WordPress plugin aimed at automatically solving the problem of link rot.

• Bellingcat founder Eliot Higgins on why “playing whack-a-mole” with disinformation is a losing strategy.

• ProofMode’s new partnership with the Jack Dorsey-backed DiVine social network to support “human-made” content.

• High-stakes instances of government-led image manipulation and archival blurring.

• Technical horizons in privacy with Midnight, a new partner chain for selective data disclosure.

With gratitude,

Basile Simon and the Starling Lab team

PS If you’d rather not receive these occasional updates, just reply to this email or click the link below to unsubscribe. No hard feelings!

This collaboration represents more than just data storage: it is a vital effort to protect cultural memory in the face of conflict and erasure.

Preserving “Digital Twins” of Endangered Sites

The data we have preserved consists of high-fidelity 3D scans – including photogrammetry and laser scan data – of Armenian cultural heritage sites located in the Artsakh (Nagorno-Karabakh) region. The collection includes original raw photographs and 3D models of monasteries, churches, and monuments, many of which are currently inaccessible or at risk of destruction due ethnic cleansing policies carried out by Azerbaijani authorities in the region (see EU Parliament resolution and Caucasus Heritage Watch at Cornell University for more).

By creating and preserving these “digital twins,” TUMO, Armenian Cultural Heritage Institute and Starling Lab are ensuring that even if the physical sites are damaged or erased, accurate and immersive records will survive for future generations of historians, researchers, and the public. This initiative directly counters the threat of cultural erasure by creating an immutable, decentralized record of this heritage.

Powered by World-Class Preservation Infrastructure

To ensure these records are resilient against censorship, data loss, and single points of failure, we have stored them on the USC Filecoin Node.

More than a simple server, this node is in fact a massive 22-petabyte storage facility integrated into both the USC Digital Repository and the Filecoin network. It leverages cryptographic proofs to guarantee data durability while offering the immense scale required for high-fidelity 3D historical archives.

It is operated by us at Starling Lab and the USC Digital Repository, a service of the USC Libraries. While the Filecoin node represents the cutting edge of decentralized storage, it is just one part of the Libraries’ deep expertise in preservation and archiving. By housing this node within a leading research university, we combine the innovation of Web3 protocols with the rigorous preservation standards developed over decades by archivists and librarians.

We have committed to preserving this archive for 20 years, ensuring that the history of Armenian heritage in Artsakh (Nagorno-Karabakh) remains accessible in the long term – at no cost to TUMO, the custodians of that archive.

The Frontier of Spatial Intelligence

This preservation effort also dovetails with our broader research into 3D and spatial intelligence. As we move beyond 2D images, technologies like Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting allow us to transform standard photographs into fully immersive, navigable 3D environments.

By applying our “Authenticity by Design” framework to these 3D datasets, we are not just storing files; we are establishing a root of trust for spatial experiences. This allows us to verify that a 3D reconstruction of a heritage site is faithful to the original scans and has not been manipulated. As we continue to develop these tools, the TUMO collection will serve as a foundational dataset for testing how we can authenticate and experience history in the metaverse.

Preserving such a collection is a powerful demonstration of how decentralized technology can serve humanity. By locking these records on the blockchain, we are ensuring that the history of Armenian heritage in Artsakh (Nagorno-Karabakh) can never be deleted, denied, or forgotten.

• Journalism programs director Ann Grimes attended the Partnership on AI Summit in San Francisco and led a breakout session with Stanford HAI at the Getty Images sponsored conference: “Growing and Protecting the Creative Economy with AI”.

• “Sanctions, Scams, and Deepfakes”, an investigative story by Airwars and IStories, with support from IJ4EU, published in early November. Starling Lab worked with these organizations to capture and preserve web content and Telegram posts, bringing in about 7,500+ web pages crawled; we also published our own case study highlighting the tools and techniques used in assisting this investigation.

• Human Rights Program Director Basile Simon joined the newly formed University of Colorado Boulder Visual Evidence Lab, focused on advancing equitable and responsible technology in pursuit of justice, as well as The Reckoning Project’s event at Yale “Catching a War Criminal in the 21st Century”, offering guidance on improving legal processes regarding the use of video and AI tools as evidence.

• As our archiving work grows, the Starling Lab team onboarded collections of US declassified battle damage assessments related to claims of 1,500+ civilian casualties (announcement and details to follow), as well as a dataset of photogrammetric and 3D scans of Armenian heritage sites located in Nagorno-Karabakh and at risk of physical degradation. We’re also working with the Armenian Heritage Project to assess their preservation/authentication needs.

  • Check out one of our recent Dispatch posts offering practical steps to securing records with the USC Digital Repository and their newly-operational Filecoin node.

As we look ahead to 2026, we’re exploring journalism collaborations with Rolling Stone and the BBC, academic partnerships with Goldsmith University’s Centre for Research Architecture, and designing the syllabus for the next Authenticity by Design course at Stanford. We’re also exploring collaborations with several news organizations to preserve and authenticate their archives, allowing them to better utilize their vast collections to serve their communities and constituents.

If you have questions, ideas, or would like to explore how Starling Lab’s work in journalism, law, and archiving can advance your efforts, please get in touch via info@starlinglab.org. Wishing you a happy and peaceful holiday season!

With gratitude,

Basile Simon and the Starling Lab team

PS If you’d rather not receive these occasional updates, just reply to this email or click the link below to unsubscribe. No hard feelings!

Today, we are proud to announce the publication of “Sanctions, Scams, and Deepfakes,” a major investigation by our partners at Airwars into the illicit flow of luxury cars from Europe to Russia. Starling Lab worked in support of the investigative team to preserve the digital evidence supporting their findings, while the reporters traced the physical movement of vehicles across borders.

At least, that’s what they originally set out to do.

At the center of this investigation into the apparent smuggling operation of cars into Russia laid a red herring. The investigative team initially believed they were tracking real sanctions-evasion schemes bringing luxury cars from Germany to Russia, but instead discovered the operation was largely a scam targeting Russians themselves. The deepfaked video of a legitimate Russian car dealer explaining the smuggling process, the paid actors posing as satisfied customers, and the geolocated cars in Germany were all elaborate deceptions designed to make the fraud appear credible – with potential victims losing thousands of euros to scammers likely operating from Ukraine rather than to actual smugglers delivering sanctioned vehicles.

The investigation was supported by IJ4EU, a grant scheme for cross-border investigative journalism in Europe. The Starling Lab did not participate in the grant application, nor did it receive any funds. Our support of the investigation was purely pro bono and limited to that project.

Technical Stack Deployed

During the investigative phase, the research and investigative team needed to browse hundreds of online sources, from Belarusian border cam footage to social media and social messages.

The core technical challenge was clear: How do we allow investigators to move fast while maintaining a forensic chain of custody? A simple screenshot is insufficient for legal or historical proof. We needed a system where an investigator could claim, “at this time and date, I browsed this unique URL which contained precisely this content,“ and be able to back it up with cryptographic proof.

Fortunately, we have been working with state-of-the-art web archiving tools for years – and have even produced a whitepaper on best practices for web archiving. The approach described in this project aims to ensure the collected material meets the high bar for authenticity and probative value required in legal proceedings. It directly addresses the belief that these tools and techniques can be cumbersome or slow down the investigative process.

This perception being a key barrier to adoption, we set out to build a bridge between consumer-friendly tools and forensic-grade archiving. We recommended the team use Raindrop.io, a lightweight browser extension, to bookmark relevant links and add annotations. This allowed the investigators to simply “click and save” without leaving their browser.

Behind the scenes, however, a preservation pipeline was built: on schedule, Github Actions would trigger Typescript scripts tasked with fetching new bookmarks that might have been added to Raindrop; and where appropriate, to schedule their individual crawling in the Starling Lab Browsertrix Cloud account.

In total, we preserved more than 9,000 unique URLs, totalling 98 GiB in compressed form.

A major early lead was the case of the border crossing between Lithuania and Belarus, which the team understood as a key milestone in the supply of cars into Russia. The border crossing is monitored by webcams that refresh every 10 minutes or so. By running our crawls on schedule, we were able to provide the team once a day with a collage of captures – in the hope of substantiating that one of the known delivery lorries was doing the journey, as the Telegram channel seemed to claim e.g. below:

Despite monitoring the webcams for a month and a half, reviewing about 3,800 photographs (two shots every 10 minutes most hours of the daytime), we were not able to find one of the known trucks.

We, however, witnessed the spectacle of political opposition Mikalai Statkevich sitting in this no-man’s land after having been freed from Belarus, reportedly refusing to walk to Lithuania and go into exile.

Learnings

Proactive engagement

The earlier and the more closely we are involved, the better our chances to do good work. This oft-repeated canon of collaboration is worth its salt for a reason.

In the context of this investigation, we think we were able to support the collection of web content to a fair extent. Frequent touchpoints with a team helped identify developing parts of their workflow - for example the need to monitor the webcam photographs, which were being overwritten relatively frequently.

Time and opportunity to engage

As the story pivoted from tracking the supply chain to the realisation that key elements had been fabricated and were deepfakes or shallowfakes, the pressure to make a looming deadline led to a tightening of the loop on the investigative team’s side. The result of that was material received by investigators in the last days, prior to publication, was not shared with us for authentication and preservation.

Furthermore, we had prepared for a planned field trip to dealerships in Lithuania, and ran a short training for reporters on using the Proofmode app to take photographs, cryptographically seal them, and share them on – this trip however never materialized as the story pivoted away.

Seeing as the investigation relates to both forgeries and inauthentic material, we regard these omissions as missed opportunities.

The need for redactions

The pre-publication legal review sought by Airwars led to a cautious treatment of the evidence collected. The story was complex, with a good deal of uncertainty about what was real, and which of the persons and organisations involved were in full possession of the facts.

While technically well within reach, we opted to not publish full embeds of the Telegram posts and messages, as well as to redact some metadata from them. We are also not making the archive publicly accessible. These measures aim to protect car sellers, dealerships, clients, and anyone else whose photographs might have been used against their will and in support of the scam.

This case study further underscores the need for verifiable redaction technology to be part of the feature set of web archiving tools for journalists. In service of this requirement, we have deployed techniques from the field of Zero-Knowledge Proofs in our Rolling Stone investigation into war crimes in Bosnia. This technology allows publishers to redact sensitive information (such as names in documents or metadata in digital files) while generating a mathematical proof that certifies only specific pixels or data fields were obscured.

Scaleable

This pilot project allowed us to test and refine a preservation mechanism that is both robust and non-intrusive to the investigative workflow. We are eager to further deploy the processes developed during this collaboration, and this model more broadly, for future investigations and other partnerships with journalistic organizations.

This walkthrough offers concrete steps to strengthen the integrity and provenance of your materials. If you want to ensure your valuable digital assets remain authentic and verifiable for years to come, but aren't necessarily a cryptography expert, these straightforward practices will prepare your collection for proper long-term preservation.

At its simplest, this means hashing and signing your data before placing a copy of it at the Repository. Think of it as sealing a letter with a wax stamp – ensuring that, as it travels, the letter’s provenance is unbrokenly attached to it.

The Essentials: What We Need

1. The files themselves.
   These are your original digital artifacts, whether photographs, recordings, documents, or any other data types. Your collection is only as trustworthy as its unaltered source, so it’s critical to start with the most original (raw, unmodified) files.

2. Signed hashes of the files.
   Hashes act like digital fingerprints – they are unique cryptographic identifiers computed for each file. By signing these hashes, you attach a verifiable mark of authenticity and provenance to them: “this public key was in possession of this specific file.”

These authenticity markers serve three purposes:

• Prove data integrity: Comparing hashes before and after transport confirms the file hasn't been altered—if the hashes match perfectly, you're looking at an exact copy of the original. This verification process ensures that every piece of your collection retains its evidentiary or cultural value.

• Establish chain of custody: Digital signatures cryptographically link each hash to a specific key and its owner, creating an unbroken record of who possessed the file. This verifiable trail of ownership strengthens the legal and archival standing of your materials.

• Enable long-term verification: Timestamped hashes can be anchored to immutable record systems like blockchains for enhanced legal robustness. This approach provides independent, third-party verification that can withstand scrutiny years or decades into the future.

By combining the raw files with their signed hashes, you can create a robust method for maintaining data integrity and establishing provenance. This is the key to ensuring that every piece of your collection is preserved exactly as it was originally produced, safeguarding its evidentiary or cultural value for years to come.

Grouping & Organizing Your Data

Before diving into cryptographic operations, it’s vital to group and organize your data in a thoughtful, deliberate way. Performance considerations aside, the idea is to preserve the semantic relationships between files.

Grouping data also minimizes the number of cryptographic operations needed. Instead of computing and verifying hundreds or thousands of individual hashes, you can authenticate entire clusters of related files with a single hash and signature. This is particularly useful when dealing with large datasets that can be gigabytes of information.

In addition to this reduction in computational overhead, it also simplifies future verification: With clearly defined data clusters, one hash can authenticate an entire group, speeding up the verification process and making it easier to pinpoint any discrepancies

To determine these groupings, consider the inherent value and context of your files:

• Individual Units: For important singular files, such as field photographs or witness testimony recordings, it’s often best to treat them as standalone units. Each file’s individual authenticity can then be verified directly, ensuring high evidentiary value.

• Semantic Clusters: Other data only gains meaning when viewed alongside its peers. For example, code dependencies, build files, or a directory of thumbnails only tell their full story when examined as a group. Grouping these files preserves their semantic proximity, ensuring that their collective integrity is maintained.

Choosing Your Approach: Individual Hashes vs. Merkle Trees

For smaller collections or when you need to verify individual files independently, traditional hashing tools like sha256sum work well – simply generate one hash per file and sign each one.

However, for larger datasets or collections with natural groupings, Merkle trees (using tools like merkdir) offer significant advantages: they create a single "root hash" that can verify an entire directory structure, dramatically reducing the number of signatures you need to manage while still allowing you to pinpoint exactly which files have changed.

Choose individual hashes when simplicity matters and file counts are manageable; choose Merkle trees when you're dealing with hundreds of files, complex folder structures, or want to authenticate entire project directories as unified collections.

In Practice: How It Might Work

Before transferring your collection, you will:

✅ Group & organize files meaningfully (e.g., individual vs. grouped datasets).
✅ Hash files using tools like sha256sum or merkdir (Merkle trees).
✅ Sign hashes using lightweight cryptographic tools (minisign, PGP, or x509 certs).
✅ Verify the integrity after transport—confirming that your hash from t0 (before transfer) matches the hash at t1 (post-transfer).

This simple but powerful process keeps history intact. It ensures photographs, recordings, or other digital assets retain their evidentiary value, even years from now.

To get started 🚀, check out this GitHub Gist from our software engineer Cole Capilongo, outlines the use of the above hashing and signing tools. We’d also be happy work hand-in-hand with you to support your project. Just reach out to <info@starlinglab.org>

As we work together to safeguard truth in the digital age, your data’s integrity matters. Let’s make sure it’s preserved—securely and verifiably.

Limitations: What This Guide Doesn’t Cover

While this guide lays out the essentials for ensuring data integrity and establishing provenance through hashing and signing, it doesn’t cover every aspect of digital preservation. In particular, it stops short of detailing advanced verification methods and external validation systems. For instance, the complexities of Public Key Infrastructure (PKI)—with its certificate management and intricate trust hierarchies—are beyond the scope of this primer. Similarly, emerging standards like C2PA attestations that provide enhanced content provenance by linking digital assets to verified identities are not discussed in depth here. Moreover, while third-party record holders, such as blockchain services or other immutable ledger systems, can offer an additional layer of public verification, this guide doesn’t elaborate on how to integrate these external record-keeping systems into your preservation workflow. Consider this a focused introduction to the core practices of data integrity, with many advanced topics available for further exploration.

"Never ask a question in court you don't know the answer to" is advice lawyers heed. On Saturday 19 July, 2025, I took part in a training session for British barristers built as a mock hearing. The objective: practicing cross-examining an expert witness about open-source intelligence (OSINT) as evidence.

OSINT is increasingly available to prosecutors thanks to the popularity of cell phone cameras and social media, but finding the photographer may prove difficult – if not impossible. Generative AI creates additional challenges to admissibility.

The fictional scenario we designed involved all the staples of open-source methodologies (geolocation, chronolocation, etc.) and was the first time C2PA metadata was being interrogated in a court-like setting for its reliability and probative weight.

The Case Itself

The fictional case derives from an alleged war crime in Yemen (in fact, an active investigation of Bellingcat). In our training scenario, the prosecution argues that a jet fighter pilot conducted a "double-tap" airstrike on June 12, 2023 at approximately 10:30 AM – first dropping one bomb, then returning to strike again and killing at least six civilians who were conducting rescue operations. Mr. Smith, the defendant, claims he conducted only a single, legitimate military strike at 9:00 AM targeting a military communications center when no civilians were present.

Ultimately, the case is about the second strike (which makes it a war crime). If its criminal nature can’t be established beyond a reasonable doubt, Smith won’t face consequences for an alleged strike against a possibly legitimate target.

The Evidence Presented

The mainstays of open-source investigations are relied upon to interrogate when the acts might have taken place:

- Firstly, chronolocation aims to reliably and accurately estimate the time of the incident.

- Secondly, contextual verification is put forward by the prosecution to corroborate a piece of open-source evidence with other open-source evidence. (The scenario deliberately doesn't include direct testimony, to focus the trainees on open-source investigations only.)

- Finally, and most relevantly for us at Starling Lab, some material includes C2PA metadata, a technical standard for provenance which is presently untested in judicial settings. We presented the trainees with a video including two Manifests: one with the media's authentication metadata produced straight from the C2PA-compatible Sony camera that captured it; the second added to the video after the fact by an investigative journalism collective, claiming to have verified the video, in a similar fashion to what BBC Verify have experimented with.

In addition to contributing to the scenario design and the case papers, I took part in the training as an expert witness interrogated and cross-examined by a group of six barristers (three on each side). I testified to the whole of the prosecution expert report as though the case was real and it were my own report. It was an intense four hours!

Impressions and takeaways

I found it fascinating to get an insight into what barristers – with no pre-existing familiarity with digital investigations methods – make of this bundle of evidence. It appeared quite corroborating, all-pointing-in-one-direction, but perhaps somewhat circular. In a trial, nothing is taken for granted.

As an expert witness the reliability of what you say is under assault – all answers contribute to your own credibility on the stand, and in your report.

The experience highlighted the paramount importance of intellectual honesty for an expert witness. Rather than serving the interests of the party who commissioned their report, the expert's primary responsibility lies with the court. This means providing unbiased, accurate, and complete information, even if certain nuances or findings might not directly support the commissioning party's case. The mock trial served as a practical exercise in upholding this principle, underscoring that the expert's credibility and the integrity of the judicial process depend on their unwavering commitment to the truth, irrespective of who is paying their fees.

The purpose of relying on expert witness testimony is to inform the court, but it comes with the responsibility of having to effectively communicate findings and opinions. Too long and you'll be cut out; too in the weeds and jargony and you'll lose the room. Analogies and relatable explanations are the name of the game there.

From a substance perspective, I found myself unable to get to some of the nuances I wanted to touch on. Because the exercise was quite focused on how much time was allocated to the metadata questions, we got somehow stuck on how C2PA data can be removed from an image, and how difficult it might be to acquire an X.509 certificate.

I had laid a few hooks for the defense to pick up on, and left a question or two unanswered. For example, the prosecution expert did not include a detailed verification report of the second organisation's cert (they really should have).

Prior to this mock trial, I took a week-long "Courtroom Testimony for Expert Witnesses" course from LEVA and Jonathan Hak KC. This gave me a hunch that I was trying to be too clever: as a training exercise it's not optimal to leave stones unturned, since no-one in the court is more of an expert than we are. In other words: The training aims to bring the lawyers to having some familiarity with the substance, and to practice the techniques – it is not for the defense to pick up on minute, detailed, very precise concerns from the report themselves.

Indeed, we did not stray far from the expert reports themselves due to lack of time. I'll be proposing a different, much more back-and-forth structure in a future training, so that we can derive must-dos and mainstays of _any_ C2PA verification.

I'm very grateful to the following people and organisations for giving me this opportunity to contribute to this exercise, and for inviting me to take part on the day. Walking the small alleyways of Temple, Blackfriars, and Fleet St is so humbling for this former law undergrad who spent a bit of time in various newsrooms, and in Hackney. So a huge thank you to:

- Professor Yvonne McDermott from Swansea University and the TRUE Project;

- Judge Korner KC, presently at the International Criminal Court (ICC)

- The Honourable Society of the Inner Temple

- The Institute for International Legal Advocacy Training

Why We Built the CAR Content Locator

As decentralized storage solutions become more prevalent, the need for robust, user-friendly tools to manage and verify content availability is critical. Our work with the USC Digital Repository highlighted a significant challenge: tracking individual files within CAR files (Content Addressable aRchives) and verifying their continued preservation without relying on public indexing services.

Organizations storing large datasets on Filecoin and IPFS face a practical problem: once dozens of files are bundled into CAR archives, how do you later find and verify specific ones?

Existing public indexing services can’t help with private or institutional data, and unfortunately our work with the USC Digital Repository – which looks after 1.2PiB+ of private archives – highlighted this challenge.

This led us to develop a tool that:

• Maps file CIDs (Content Identifiers) to their locations within CAR files, especially when CAR metadata and file data exist in separate CAR files.

• Extracts specific files using either a file identifier or CID across multiple CAR files.

• Connects Piece CIDs and Deal IDs on the Filecoin network without requiring a full indexer.

Feature Highlights

The CAR Content Locator codebase is now published on GitHub under the MIT License, and can easily be deployed using Docker Compose. It features:

• A web-based file browser that displays file integrity information (i.e. CIDs) and their availability across Filecoin and IPFS networks;

• For Filecoin storage information, a Singularity-based workflow is required as it relies on its databases to keep track of Piece and Deal information;

• For IPFS content availability, the tool relies on delegated routing endpoints to be available;

• Although the tool is preconfigured with several public indexers, such as Pinata and Storacha, it also supports private IPFS networks, and similarly, private Filecoin nodes with the required Singularity databases.

Explore the tool in action at our demo instance!

Learnings & Next Steps

During development of the CAR Content Locator, we also conducted research on how different organizations – particularly those managing large datasets like the USC Digital Repository – address similar use cases for tracking their files across CAR-compatible networks.

We found little consistency in how users interact with Filecoin and IPFS, and workflows and toolings are usually bespoke and tailored to specific organizational needs. We hope that publishing this open source tool will also participate to the conversation on building common tooling across similar organizations.

We’d greatly appreciate your input:

1. Share with us how your organization tracks files stored on Filecoin and/or IPFS;

2. File tickets on starlinglab/car-content-locator on what features would make this tool helpful to you.

The CAR Content Locator's development has been significantly shaped by the specific needs of the USC Libraries, and it continues to keep track of its 1.2 PiB+ of private archives on their Filecoin node. Our team also works closely in parallel with the Internet Archive’ Filecoin operations.

This work is made possible by the Open Impact Foundation and the Filecoin Foundation for the Decentralized Web, as well as those who advised / contributed feedback: Mosh, Robin, Bumblefudge, Lidel, Arkadiy, Casey, Sankara, Ian, and the whole USC Digital Repository and Libraries team.

Resulting 3D reconstructions have serious implications for fields like law, journalism, and history. General consumer audiences may associate “virtual reality” with video games and similar engaging but otherwise low-stakes experiences. In practice, those same VR headsets are already being worn in court by judges in the United States.

On screens around the world, audiences are visiting the “digital twins” of undersea wrecks like the Titanic and Shackleton’s Explorer, or the glittering Doge Palace in Italy and St. Peter’s Basilica at the Vatican.

New opportunities in 3D storytelling and evidence presentation also bring a significant danger: the potential to mistake compelling and believable content for genuine reality. We face the crucial challenge of ensuring people can distinguish between authentic representations and algorithmically generated content.

Three Approaches to 3D Reconstruction: A Reliability Spectrum

Today, we can identify three distinct categories of 3D reconstruction tools, each with different implications for trust and authenticity:

1. Photogrammetric Processes: These established techniques require hundreds or thousands of images but produce highly reliable – and to an extent, deterministic – results with minimal algorithmic interpolation. Courts, forensic experts, and industry (insurance, etc.) currently rely on these methods to a significant extent due to their verifiable nature and transparent methodology.

2. Neural Radiance Fields (NeRFs) and 3D Gaussian Splatting (3DGS): These emerging techniques can create compelling 3D environments from significantly fewer images. While they don't completely fabricate content, they do algorithmically "fill in the gaps" between captured data points, creating a middle ground between documented reality and generation.

3. Generative AI Video Tools: These technologies can create entirely synthetic environments with minimal or no authentic source material. These approaches can extend a single image into video (popularized by commercial services like Sora, etc.) into the opposite end of the reliability spectrum, producing content that may be visually compelling but risks bearing little relation to documented reality. Selected key frames from these videos can then be fed into NeRF or 3DGS models.

The Double-Edged Sword of Advanced 3D Reconstruction

NeRFs and 3DGS currently occupy a middle position on this spectrum, differing from earlier photogrammetric techniques (based on producing 3D polygons) by representing scenes as point clouds with view-dependent characteristics. This allows them to accurately depict complex visual effects, such as reflections and transparency, much like photographs with depth.

Think of it like taking many photos of a statue from different angles. Traditional methods would stitch these photos together to create a 3D model made of flat surfaces (like a paper model). NeRFs and 3DGS, however, treat the statue as a collection of tiny, colorful dust particles that change how they look depending on your viewpoint, capturing subtle details like how light reflects off the surface.

However, these technologies are evolving at breathtaking speed, with new papers, algorithms, and tools published weekly. The rapid advancement of NeRF and 3DGS techniques presents a concerning trajectory: as they require fewer source images and incorporate more generative capabilities, the boundary between authentic representation and algorithmic fabrication becomes increasingly blurred. There's a significant risk that these tools—or their successors—will soon cross into a "danger zone" where viewers cannot reliably distinguish between documented reality and synthetic creation.

Researching Authenticity in Immersive Environments

Understanding the boundaries between authentic, verifiable reality and algorithmically-generated data is crucial – especially in the context of evidence or historical reconstruction. The relative ease of creating convincing reconstructions thanks to NeRF and 3DGS underscores the need for focused research into how viewers perceive authenticity, how trust is established in immersive media, and the implications of blurred boundaries between the real and synthetic.

In today's models, it's relatively clear at which point the model breaks down. Such failures in realism are easy to spot. Accurate portions of a scene based on authentic 2D photographs are relatively clear, while areas that might be hallucinated are either missing or fuzzy (sometimes blurring into oblivion). Tomorrow's technologies may not present such clear indicators. Unlike generative AI, which can create obvious misdirections, NeRF and 3DGS occupy a more nuanced position that demands careful study.

Above video: Four rendering techniques for the same scene of a burned residential area from the 2025 Los Angeles wildfires. Note in NeRF and Gaussian Splat the tell-tale signs of missing data: blurring, artefacts, wrong colors. Credit: Mike Caronna, for ASU

Our research at the Starling Lab is addressing these very questions, exploring how methods like "authenticity anchors" can serve as tools to verify authenticity. These anchors are cryptographic integrity markers linking elements within the 3D space (such as 2D photographs) back to securely authenticated source data (using industry technical standards like C2PA and a variety of cryptographic techniques). Although they are a practical solution, their greatest value lies in supporting deeper investigations into user interactions with authenticity and the effectiveness of transparency mechanisms in XR environments.

Building Trust in XR: Practical Research Directions

We are seeing 3D reconstruction approaches rapidly evolve at the same time we see them being introduced into high-stakes environments like courtrooms. As a result, we propose the following axes of research demand immediate attention to ensure authenticity infrastructure is integrated:

• Perception of Authenticity: Investigating how users perceive and differentiate between authentic and generated content in immersive experiences, and at which point trust breaks down along the spectrum.

• Verification Methods: Studying which verification techniques are most effective for each category of 3D reconstruction, from cryptographic proofs for photogrammetry to clearer "authenticity boundaries" for NeRF/3DGS. In addition, the blossoming field of spatial intelligence may help navigate this nuance by providing a way to assess whether the algorithmic "filling in the gaps" is reasonable and aligned with real-world expectations. In other words: if the model's spatial understanding is sound, it increases confidence in the reconstructed environment.

• Transparent Interpolation Indicators: Researching methods to visually indicate when and where algorithmic interpolation has occurred in NeRF and 3DGS reconstructions, allowing users to understand the boundary between captured and generated content.

• Spatial Intelligence in reconstruction: By enabling models to "understand" the 3D space, the blossoming field of spatial intelligence may enhance the overall reliability of these immersive experiences. If a model can identify and reason about objects within the scene, it suggests a deeper level of processing and a lower likelihood of "hallucinating" or misrepresenting elements of the environment. This directly impacts trust, as users are more likely to believe and rely on a system that demonstrates an understanding of what it's depicting, perhaps even similar to the way a human witness would evaluate a scene.

Building upon this research, our goal is to translate the insights gained into practical, real-world solutions. These solutions will likely encompass a combination of technical authentication, user experience (UX) design considerations, educational programs, and public awareness initiatives.

Looking Ahead

As immersive technologies advance, the critical question of authenticity will become even more prominent. Focused research in this area is essential not just for maintaining trust and transparency but also for safeguarding the integrity of digital storytelling and documentation. By clearly understanding and navigating the fine line between the real and the synthesized, we can harness the extraordinary capabilities of emerging technologies like NeRF and 3DGS responsibly and effectively.

We are honored to see our work featured in Mary Lawlor's UN Special Rapporteur report to the Human Rights Council: "Out of Sight: Human Rights Defenders in Rural, Remote and Isolated Contexts."

The report notes how Starling Lab supported efforts in Brazil’s Pantanal wetlands with tools that work even in low-connectivity environments, which is needed where the “digital divide hits many human rights defenders very hard."

In a speech this month presenting her findings, Lawlor observed how some of those most at-risk include “journalists covering human rights issues at the local level.” Starling’s methodology was used by journalists to document environmental destruction – and confront climate change denialism. Our submission to Lawlor’s office focused on data authentication, decentralized storage, and cryptographic verification. Together, these ensure documentation remains tamper-evident and accessible, even when governments seek to dismiss the truth.

Her report includes several valuable recommendations to governments and other international actors, two of which resonate with our work:

• "Expand access to the Internet and secure communication tools, including by increasing funding for such digital security resources as encrypted communication applications and secure reporting channels.”

• "Support efforts to enable human rights defenders to store and safeguard their information securely, without fear of unlawful surveillance or data breaches, including putting in place robust legal safeguards to prevent the misuse of digital tools to suppress dissent or target defenders and ensure that their digital rights are protected.”

We appreciate being included among so many other human rights defenders, and remain grateful to Pablo Albarenga for his photojournalism in Brazil, as well as to Inside Climate News for publishing this important coverage.

And we hope this underscores the vital role of trusted digital evidence in defending human rights and environmental justice.

🔗 Read the Special Rapporteur’s remarks;
📄 Read the full UN report;
📕 Don’t miss our own case study;
📰 And the Inside Climate News article

Starling Lab was previously referenced in a report to the UNHRC in 2023 by the Rapporteur on the right to education, who acknowledged similar methods used by the lab as emerging “good practices” for documenting war crimes against civilian objects like schools.

Update (April 16, 2025): Our submission is also echoed in a new resolution from the UN Human Rights Council, which addresses the protection of human rights defenders in the digital age (full text: A/HRC/58/L.27/Rev.1).

The resolution emphasizes "universal connectivity" and "meaningful connectivity" as essential for defenders' work, calls for "technical solutions for strong encryption and anonymity," and advocates for secure information storage "without fear of unlawful surveillance." It specifically recognizes the "growing number of digital attacks" on defenders and acknowledges the "protection gap" caused by lack of accountability.

These address key points from our submission on data integrity and authentication technologies for remote areas.

We're particularly encouraged by the HRC’s recognition that "new and emerging digital technologies can hold great potential for strengthening democratic institutions and the resilience of civil society, empowering civic engagement and enabling the work of human rights defenders, public participation and the open and free exchange of ideas, and for the exercise of all human rights." This aligns perfectly with our mission at Starling Lab to harness technology to establish trust in digital records.

Our earlier submission outlined innovative approaches to produce documentary evidence and combat digital denialism. This includes cryptographic methods that authenticate evidence from the point of capture, enabling defenders in remote areas to establish credibility despite connectivity challenges. The submission also referenced ongoing work on telecommunication technologies, including 5G, to further enhance these capabilities.

This past July, Starling Lab organized and participated in the Public Records and Human Rights  track at IPFS Camp 2024 in Brussels, Belgium.

The conference brought together a community of builders, operators, researchers, and more to talk about applications on the the peer-to-peer, content-addressed data access network and protocol IPFS. For our focused track, Starling Lab invited Numbers Protocol, Hala Systems, Ceramic Network, and the Guardian Project to come and share their important work (see videos in the presentation playlist) related to distributed data availability, open-source software, and what applications they were building to address data provenance, data accessibility, open source, and more.

Authenticated Attributes Demo

A highlight of this track was a demo unveiling of Starling Integrity Backend v2, our data processing pipeline, along with the command line interface (CLI)  tool built by developer Cole Anthony Capilongo. It’s our latest delivery of a prototype workflow that involves multiple tools and software libraries to solve problems around data integrity and preservation. 

A major feature of this new CLI tool is Authenticated Attributes, which is a cryptographically signed and timestamped database with a flexible schema that has the ability to show relationships between different data points (including assets like digital media files). In our work over the past several years, Starling Lab has encountered many challenges related to processing and preserving different types of data and metadata across separate systems. This new feature set enables the Lab to have the flexibility to capture, preserve, and update authenticity data about digital media while maintaining the integrity of the information.

The lab works with several processes and software: from capture with different applications, to the various strategies we use to hash and assign content identifiers, to preserving and distributing data on decentralized systems, to presenting verifiable data. We needed an easy to use tool to streamline the process of putting together modular authentication workflows.

Since the workflows we prototype deal with a variety of different inputs, we needed the ability to process them in different ways. We also perform different operations on this data, such as encrypting certain data fields, adding C2PA manifests, adding signatures, and placing data in decentralized systems such as IPFS, Hypercore, Filecoin, and – of course – registering hashed ‘fingerprints’ of data on various distributed ledgers. The CLI tool makes it possible to pick and choose which kinds of attributes we want to modify and add to our data store.

See the presentation to learn more about the different tools we use, and how they are brought together with the Starling Integrity Backend v2.

Authenticity Tools in Action

Securing Taiwan’s Election Integrity with Blockchain Technology

In this talk, Natalie Wang from Numbers Protocol shared their work on preserving images of historical and social significance, including the Mom I See War and the Taiwan Elections projects, as well as updates about their suite of authenticity tools designed for journalists. Challenges are heightened in an election year, fraught with threats of mis- and dis- information from bad actors and AI generated content. Numbers has been working with Taiwanese journalists and news organizations to develop a camera app, SDK, and dashboard that enables them to A) assign identities and identifiers to registered photographs, and B) secure the provenance of these artifacts on-chain. In a year when more than half of the world’s population has the opportunity to vote in a high-stakes election, Numbers and Starling Lab are working together to combat mis- and disinformation that could sway the course of history.

IPFS in Action: Conflict Zone Use-case 

Begoña Sesé de Lucio joined us from Hala Systems, which develops data-driven technological solutions to protect conflict-affected populations and strengthen accountability for international crimes. In the presentation, Begoñia shared the work they have been doing documenting the destruction of schools in the Ukraine-Russian conflict, as well as shared how they are using decentralized technology in developing ways to aid international investigators to collect data – with integrity and a provable chain of custody – that can be handed off to prosecutors. 

Hala Systems has created a product focused on human rights observation that includes a user interface for file upload and registration that allows users to easily upload and interact with the registration information. Users can ensure, with this interface, that steps have been taken to curate and store information for the long term, generate assets with integrity metadata, and register asset bundles to create an immutable chains of custody so that the valuable witness evidence being preserved can be useful, now and in the future.

Humanitarian App Distribution & Demo

Internet outages and blockages are becoming a more common tactic of authoritarian regimes. Paul Mayero from the Guardian Project explained that the internet “is now a basic human right.”  that needs to be protected. He is an expert in developing applications to help those who don’t have free and open access to data and digital information. Oftentimes, the places where humanitarian apps that give communities the tools they need to access information and services aren’t available because of internet blockages, or simply because of the fact that apps aren’t available on Apple or Google Play stores. 

FDriod is an alternative source to download apps – and can publish apps on the peer-to-peer IPFS network to provide access to those who might otherwise not have it. This makes it possible to bypass internet shutdowns - including downloading apps from behind such a blockage, that make it possible to gain access to data and communication tools once internet has been walled or cut off.

Peer-to-peer systems offer a valuable, resilient, and censorship-resistant solution to make digital information available in places where suppression of free speech can block access to basic necessities: services, resources, applications, and data that many of us take for granted.

The set of talks from this track present compelling evidence for the need for continued support for IPFS technology, and show the resilient types of solutions that can be engineered with decentralized technologies like these.

Event Recap

Our expert panel, featuring Prof. Yvonne McDermott Rees from Swansea University and Jonathan W. Hak KC PhD, a seasoned Crown Prosecutor from Canada, led us through a comprehensive exploration of the challenges and opportunities presented by generative AI in the realm of digital evidence.

Introduction and Context: We began with a discussion on the rise of generative AI and its broad accessibility to all. With AI tools now available for as little as $20/month, the creation of photorealistic but inauthentic evidence is no longer limited to advanced disinformation campaigns by nation-states. This democratization of technology has significant implications for the authenticity of OSI, as evidence already subjected to intense scrutiny due to the unavailability of the original author to testify is now further cast into doubt by potential reflexive calls of fakery.

Core Topics:

• Admissibility vs. Weight: Considering our mixed audience including non-legal professionals, our panelists began by debunking the notion that admissibility of digital evidence is the primary issue, emphasizing instead the importance of weight and probative value of the evidence, most notably in international jurisdictions. Both emphasized that instead of aiming to “make it” to admissible evidence, investigators ought to “aim as high as possible” to not be caught out. They referred to the recent Al Hassan ICC judgment (26 June 2024) to highlight how little guidance practitioners might receive from the court with regards to admissibility and weighting. Best come prepared!
  

• Prejudice and Authenticity: The discussion then moved to the potential prejudicial effects of inauthentic material on judicial processes. Our panelists emphasized the concept of "seeing is believing," highlighting how visual evidence can heavily influence jurors and judges. The power of images to create lasting impressions makes it difficult for fact-finders to "unsee" evidence, even if its authenticity is later questioned. This challenge is compounded by the strong emotional responses visual materials can evoke. To mitigate these prejudicial effects, rigorous technical interrogation and verification of digital evidence are essential before presenting it in court, helping protect proceedings from being compromised by misleading visuals–which are by now too likely.
  

• Technical Interrogation: Further to the point above, we explored the necessity of comprehensive technical interrogation before viewing evidence, discussing its legal and ethical implications. Our panelists referenced the Berkeley Protocol on Open Source Investigations, which notes the importance of technical analysis in assessing the authenticity of digital evidence. We believe this analysis should include verifying the provenance and integrity markers (the objects of our research at Starling), supporting the verification of digital materials.
  However, while rigorous technical interrogation might help prevent prejudice by filtering out falsified evidence, it also risks discounting authentic evidence that may lack sophisticated markers due to technical limitations or resource constraints. Balancing the thoroughness of technical verification with the inclusivity of all relevant evidence is crucial in maintaining the integrity of judicial processes.

Audience Q&A

The event concluded with an engaging Q&A session where we tackled questions about:

• The need for legal education and training to adapt to these new challenges for all involved in proceedings, be they judges and fact-finders but also experts and investigators. 

• The role of the Confrontation Clause in U.S. settings, as guaranteeing a defendant's right to confront their accuser, which could (should?) include the possibility to challenge the authenticity and reliability of digital evidence presented against them.

We sincerely hope that you found the discussion as insightful and thought-provoking as we did. Your participation and questions enriched the conversation and highlighted the importance of community engagement in navigating these complex issues.

Thank you once again for joining us. We look forward to seeing you at our future events as we continue to explore the intersection of technology and justice. I have several such events planned and hope to convene us again in August and September.

Until then,
Basile

The article, titled “Old Doubts, New Doubts: Evaluating Digital Open Source Imagery in the Courtroom” gave me the opportunity to consider pressing issues surrounding the verification of digital open source imagery (OSI). While raising the bar for evidence standards is often seen as beneficial, it also raises concerns about excluding genuine evidence that may not yet meet these higher thresholds. It’s an existential question for our research at the Lab!

That’s because the work of the Law program is committed to enhancing the baseline for digital evidence verification and increasing our readiness to interrogate all forms of digital evidence. Our approach includes:

• Supporting the development of stronger verification techniques, as detailed in Yvonne’s group’s recently launched Guide for Judges and Fact-finders.

• Promoting the awareness that audio-visual content does not "speak for itself."

• Encouraging the adoption and literacy of industry standards and innovations for provenance and content integrity.

I have long admired Yvonne’s work at the TRUE project and have been an avid reader of Opinio Juris, so contributing to this publication is truly a privilege. Huge thanks to Sarah Zarmsky, Jessica Dorsey, and the OpinioJuris team for their support in bringing this article to life.

Read the full article and join the conversation on the future of digital evidence in the courtroom!

Best.
-basile

Most of Starling Lab’s work involves trusted timestamping. Normally timestamping is very easy: you simply add  a timestamp field to your metadata and call a function such as time.Now(). Trusted timestamping, by contrast, is concerned with proving that the provided timestamp is actually correct, and wasn’t forged after the fact.

This is simple to do in one direction, in order to prove some data existed after a certain point in time. That can be easily achieved by including unguessable current information, such as news headlines and stock prices. A classic pop culture example is kidnappers proving a hostage is still alive by having them pose with that day’s newspaper.

But trusted timestamping is concerned with the opposite direction: proving data existed before a certain time. This is not so easily done, but it is possible, and the use cases are quite important.

Purpose

There are three main reasons for doing trusted timestamping.

1. Verifying old cryptographic signatures

2. Data integrity

3. Proof of technology ✨

In addition to these there is the obvious one: authenticity. Trusted timestamping can prove that a document that claims to have been made on a certain date was actually created on that date.

Verifying Signatures

The mainstream industry use case is verifying that a signature was added on a given date. For example, Apple requires timestamping as part of its code-signing process. The reasoning is simple: sometimes private keys get leaked or revoked. This scenario can break the guarantees signatures provide, namely that you can be sure of the author of the signed data.

Once a key is leaked, can we validate any previous signatures? They could have been made by a third party using the leaked key. But if the signature was timestamped, we can know whether the signature was created before the key was made invalid. This is a real problem that trusted timestamping solves.

Data Integrity

A basic method of ensuring data integrity is storing hashes. But what if those hashes are tampered with? Using signatures instead allows us to verify that no one but a person/org we trust has modified the data. But what if we don’t want to trust that person, or their ability to guard their keys? Using signatures adds an extra moving part and security risk that in some cases may not even be needed. If we can agree that the data was valid and unmodified at a certain point in time, trusted timestamping will provide data integrity forever. An example use case is backups, or storing org-internal archival data like legal agreements.

Proof of Technology

This is a new, modern use case. As AI-generated content gets better and better, being able to prove that a piece of media was generated before certain AI software even existed will become extremely important. Trusted timestamping has the ability to end all these new uncomfortable arguments that such-and-such picture was created whole-cloth by some inscrutable algorithm, and instead bring us back into old school discussions about whether something looks “PhotoShopped”.

Of course, it can be that AI content generation has advanced too far for trusted timestamping to be useful. But the technology is not perfect, nor has it peaked. As the saying goes, the best time to plant a tree timestamp data was 20 years ago, the second best time is now.

Software

There currently exist two viable trusted timestamping methods I am aware of. The first is OpenTimestamps, and the second is the Time-Stamp Protocol, or RFC 3161. The former timestamps using Bitcoin (but efficiently, not one timestamp per block) and the latter uses the signature of a trusted service (Time Stamp Authority).

Here is a quick comparison:

You can learn more about OpenTimestamps on the website. Beyond the actual RFC linked above, you can learn more about RFC 3161 by playing around with OpenSSL, use the CLI and type the command man openssl-ts

I’m not going to dive any deeper into these tools than the above table, but my general recommendation is to use RFC 3161 with a good third-party unless not needing to trust anyone is actually a requirement for your project. And you could always do both!

Usage

OpenTimestamps has a CLI tool and libraries in several programming languages, see here. It also has a GUI on the website.

RFC 3161 is supported by OpenSSL as a CLI tool with the openssl ts command. To make RFC 3161 timestamping on the CLI easier (as it’s not just one OpenSSL command), I’ve created a bash script each for timestamping and verifying. You can get them here.

The only GUI option for RFC 3161 I’m aware of is on the freeTSA.org website, although personally I have no reason to trust freeTSA. Better than nothing though, for anyone who doesn’t use the command line.

For trusted third-parties there are various options, including reputable companies such as DigiCert.

Conclusion

It’s time for trusted timestamping! The cost of this is basically zero (storing a few kilobytes of proof at most), but the advantage can be very large. It’s one of those things that you’ll wish you had done at the start.

Many groups can act on this by adding timestamps for their documents and media. In increasing order of importance:

• General public: timestamp your important documents and media to prove their authenticity. You can do this today on your phone with ProofMode, and on the desktop using CLI or GUI tools as discussed above.

• Archivists: timestamp everything you archive, for integrity and authenticity. For redundancy consider using both methods mentioned above and many RFC 3161 timestamp authorities.

Developers: if you work on software where data or signature integrity is at all important, add trusted timestamping to your application. And if you use Git signing, you can easily start timestamping all those signatures, seehere.

Leveraging blockchain technology, the initiative, led by Numbers Protocol in collaboration with the Starling Lab, Taiwanese news outlets, and journalists assigns unique digital identities to images and videos to ensure their authenticity and immutability. This approach enhances public confidence in media reporting by documenting the presidential election's key moments with traceable and secure data, thereby countering misinformation in the politically and geopolitically sensitive context of Taiwan.

This article provides technical details for any developer interested in creating a similar project. For more information about the Taiwan 2024 election project, please visit the project's Taiwan 2024 `website . 

Why Blockchain?

The technical flow of the Taiwan 2024 election project builds off of  other projects Numbers Protocol collaborated on with Starling Lab, including 78 Days, The DJ And The War Crimes, and Cryptographic Submission of Evidence of War Crimes in Ukraine (Project Dokaz). One key difference between these innovative projects and traditional archive projects is the adoption of blockchain. Before going further, the first question we need to address is why blockchain is necessary.

There are already many metadata standards that help establish the provenance of digital media such as IPTC and C2PA, and they all have value. However, blockchain registration provides additional advantages, making sure the content history records are immutable and transparent and can be connected to the broader Web3 ecosystem. Once a transaction is written to the blockchain, it cannot be altered or deleted. This immutability guarantees the permanent accessibility of records, ensuring data remains untampered.

Products Used By This Project

Numbers Protocol builds a provenance infrastructure compliant with IPTC and C2PA. The products include an SDK to help developers quickly adopt the provenance standards. We have the  Capture Cam, which you can easily download from the App Store or Google Play. It is a mobile tool that can help users take photos with integrity assured. The Capture Dashboard also allows content owners to manage content and its metadata with ease.

Technical Flow Overview

There are three main steps involved in the technical flow

• Register content to the blockchain

• Verify on-chain information via the Verify Engine or the information widget

• Archive the project content in the decentralized storage using the Filecoin network to ensure long-term preservation of this collection.

Step 1: Register to Blockchain

Media outlets and journalists capture the images and videos from the election campaigns. The original file and their provenance metadata are uploaded via Dashboard (for individual journalists) or SDK (for mass uploads of the content files). The tools extract the decentralised content identifier (NID) from the original file. Thanks to the IPFS technology, the same content identifier is also the unique address of the content on the decentralised web. Once the NID is generated, the smart contract executes the “commit” function to take the NID and the provenance metadata and create a transaction on the blockchain. 

Commit and ERC-7053

The “commit” function of the smart contract follows ERC-7053 (Ethereum Improvement Proposals) standard. Suppose you are familiar with the development tool Git, similar to the updating of source code. In that case, ERC-7053 allows you to do the same thing to update the metadata of content by linking multiple metadata to the same content ID. Just like Git commits. These commits can happen on the same blockchain or different blockchains. As long as they all follow 7053, you can use the content ID to query and get the full content history. In the Taiwan 2024 election project, all records are registered to the EVM-compatible Numbers Mainnet.

Civic Participation

Another inspiring aspect of this project is the embrace of civic engagement. We all know the important role that authentic voices and perspectives from people play in elections. However, distinguishing real human contributions from AI-generated deepfakes presents a significant challenge and reduces trust in the internet. In the Taiwan 2024 election project, we introduced the Capture Cam technology, facilitating the contribution of verified media by the general public.

Capture Cam, accessible on both Android and iOS platforms, simplifies the process of capturing and blockchain-registering photos and videos through a user-friendly interface. The secure signature for the content metadata is generated right after the photo is taken and sealed onto the blockchain. This allows users to easily record media with a tap and make basic edits before blockchain registration, thus maintaining the content's authenticity and integrity. 

The content uploaded by the general public is listed in the "Civic Participation" tab on the project website. One user used Capture Cam to record the election counting process at the polling station, which gave the polling station a permanent record of the counting process on the blockchain. More details of the Capture Cam can be found here. 

Provenance and metadata are only useful if they can be verified. In the following paragraphs, I will focus on the verification pipeline. 

Verification Pipeline

When images are shared across social media, they are often cropped, compressed, or modified. By visiting the Numbers Verify Engine and uploading the content, the AI compares it against existing images, highlighting the matched content. Clicking the matched content (or any similar ones that AI found), the Verify Engine will navigate you to the asset profile (explained below) of the content. This verification process is akin to a digital fingerprint, ensuring your content's uniqueness and legitimacy.

Asset Profile

Once uploaded, content can be verified through Numbers Protocol's AI-powered engine. Each piece of content on Capture has its profile, displaying comprehensive information similar to a LinkedIn profile or a company's front page. This profile includes creator details, creation time and location, and a direct link to the blockchain transaction, ensuring transparency and trust in the digital realm. 

In some cases, the media outlets and the journalists may want to display the content using the link on their social media or website in order to keep the content traffic. Asset Profile also supports the display of social media content, and here is an example from the Taiwan 2024 election project.

AssetTree files

As mentioned in the previous paragraphs, the "commits” follow the ERC-7053 standard to ensure the traceability of the content history. In every blockchain transaction associated with the content, you can find the AssetTree file in the Metadata column. AssetTree files describe an asset's properties, including its creator, creation time, license, and other relevant details. 

When changes are made to an asset, a new Commit is created on the blockchain. This process also generates a new AssetTree file reflecting the updated state of the asset. The provenance area of the asset profile displays the merged AssetTree files from all commits. You may find more information about AssetTree in our developer documentation. 

The usedBy field in the AssetTree is especially important to this project. It records how the content is used and may help the viewer of this asset profile explore more news related to the image or video.

Information Widgets (Capture Eye)

Another way to allow users to verify the provenance information is to install the information widgets on the website. Capture users can either choose C2PA Content Credentials pin or Capture Eye. Capture Eye is a simple HTML widget that can be installed on any website. It takes the content identifier (NID) as the input information. Once the user clicks the widget, the blockchain information is shown. This allows the media outlets to highlight the blockchain provenance on their website and to enhance the content authenticity. 

Capture Eye not only displays on-chain information but also supports advanced features like direct license NFT purchases through a "COLLECT" button. This offers a streamlined approach for audiences to engage with and support authentic content directly from publishers' websites and increases the publishers' monetisation channels.

Final Archive

All the videos uploaded by media outlets, journalists and civic participants are packaged as zips with the content metadata and archived on the Filecoin network by Starling Lab. The decentralised storage like Filecoin to archive the Taiwan Election project content ensures long-term preservation  and integrity. The decentralised and crypto-incentivized  features of the Filecoin network offer a secure solution for storing data across multiple nodes globally, reducing risks of censorship and data loss. This strategy ensures that essential election content is preserved for extensive periods, promoting trust and reliability in digital archives.

In addition to the Filecoin archive, journalists also used a customized instance of Uwazi asset management system with a prototype of an Authenticated Attributes backend, developed by Starling Lab in the Taiwan 2024 election project to prototype a collaborative journalism investigation.  

Uwazi is an open-source tool developed by HURIDOCS that is often used for collecting evidence or building an archive. Authenticated Attributes uses an authenticated database, alongside modern cryptography to enable individuals and groups to securely store and share media and metadata. The outcome is a tool that is useful for journalists and investigators to build a better understanding of events through collaboration. More details about the final archive and Uwazi will be shared in another blog.

New Digital Era with Content Provenance

Leveraging blockchain and decentralized storage technologies, the Taiwan Election project marks a significant step towards a future where the provenance and integrity of digital content are highly valued, and election coverage is protected against denialism and mis- and disinformation.This approach is crucial not only for combating misinformation but also for enabling news outlets and journalists to emphasize their brand values. 

Similar technologies have been adopted in the Indonesian and Indian elections, and more election collaborations are coming, showcasing a global trend. This initiative emphasizes the importance of protecting electoral integrity and digital information. These approaches promise greater transparency, trust in election results, and a precedent for employing tech to uphold democratic principles and verify digital archives globally.

Right now we’re using AI to make funny pictures, but the future is a little more painful. Your estranged spouse might create a fake recording of you to take your kids away, and anyone can make the President say whatever they want. 

The internet is full of tips on how to spot AI-generated photos, but this approach isn’t foolproof. Today in AI-generated photos the Pope’s glasses may merge with his eyelid, but they won’t for long. Artificial intelligence will continue to get better, and unfortunately, our intelligence will, well, stay the same. Generating AI images and detecting them is an ongoing arms race with no end in sight. 

Surviving the future of technology requires a different approach. Surprisingly, surviving the future involves using the methods of those who record the past: archivists. Archivists like the USC Shoah Foundation are well known for their efforts in recording the stories that need to be told, so that the past is not forgotten. It’s this same archival process that can save us, if we use the right technology.

Here’s the different approach: instead of taking a photo and trying to find flaws with it, what if we were able to ask the Internet where the image came from and trust the answers? If we were able to ask who exactly is claiming the photo is true? And when was the photo taken? By whom? And was it edited?

What if we were able to choose which news organizations and researchers we want to trust, and ask them specifically about the photo? This different approach shifts the burden of proof to better fit a world where AI is cheap and investigations are expensive. When there’s a constant firehose of AI generated content, we can’t assume photos are real until they’re proven fake. Instead, we can create platforms, spaces, and resources that people can turn to, to find content that is verifiable, with attestations from those we trust.

Knowing which images we should trust is no small task. But there are elements of blockchain technology (cryptographic hashes, timestamping services, and digital signatures) that can help. Note I didn’t say blockchains themselves. Take that argument out into the hall, please. It’s the cryptographic tools – the things that blockchains are built with – those are the things I want to use. 

Cryptographic hashes, timestamping services, and digital signatures are nothing new. Timestamping services have been around since 1991 and are a part of the original Bitcoin whitepaper. Digital signatures were introduced back in the 1970s. It seems cryptographers have a peculiar way of talking: what they say is only heard by the rest of the world decades later, if at all. So if you haven’t heard about any of these things before, that’s entirely okay. Hardly anyone else has either. 

The easiest way to explain all this is with a truck seal. It looks like this:

In the trucking industry, a truck seal is a plastic loop used to seal the cargo of a semi-truck. It has a serial number on it that is recorded by the shipper. At receiving, the receiver cuts the truck seal off and compares the serial number with the number that was transmitted separately by the shipper. If they match, we know the truck wasn’t tampered with. 

Because a truck seal could have been cut off and replaced in transit by a different seal, it’s important to compare the serial numbers.  It’s also important that the serial number is transmitted separately. We don’t want the recorded serial number to be easy to change if the truck seal is cut and a new seal is applied in transit. That would defeat the purpose of the check.

We can use this same concept to secure images and other forms of data instead of trucks. And instead of plastic seals, we can use a cryptographic hash. 

A cryptographic hash of an image is like a truck seal, except the hash isn’t a serial number. It’s made of numbers, but it’s a unique id that is a short encoded summary, or digest, of everything that is in the image. If anything at all in the image changes, a pixel is added or subtracted, then a different hash is produced. Compared to an assigned serial number, this property has the added benefit that if you have the image data, you also can derive its cryptographic hash.

Like a truck seal, recording the cryptographic hash of a file allows us to ensure that the contents of the file have not changed since we first recorded the hash. And like a truck seal, we can’t guarantee anything about tamper-evidence until we record the hash. 

So what does this get us? If anyone asks us if an image has been tampered with, and we’ve previously hashed the image and recorded the hash, we can look through our computer, find the hash that we’ve recorded, and compare that old hash to the new hash of the current image. Hashing enables us to say “this image hasn’t changed since I saw it last,” and “this photo is the same as the one from the defendant’s Facebook page,” which is pretty helpful compared to having to visually look at the two images. 

Using just a cryptographic hash has some flaws. For one thing, you could lie about the hash that you recorded earlier. “Yup, this [tampered-with] image hashes to the same thing I recorded! Look here!”, just like the trucker could cut the truck seal and then sneakily replace the serial number, if they were the only one with the paperwork. So we need to record the hash of the image someplace else.

Just hashing has a second flaw: when we’re trying to investigate faked media, we usually care about time. For instance, consider this deepfake of Jim Carrey starring in The Shining. If we had two timestamped images and no other information, one of Jack Nicholoson starring in The Shining in 1980, and one starring Jim Carrey in 2019, which should we believe? We should probably believe the earlier one, because tampered images don't go back in time. The original is by definition the oldest. This same logic was at work when Reuters proved that this image of Dutch farmers protesting was actually from 2019, not 2022, and therefore was not an accurate depiction of the protests in 2022. 

So, we need to hash images, and store the hashes somewhere else — somewhere else that records the time. Luckily, this is exactly what a timestamping service does. You can (very crudely) think of a timestamping service as a service that records the serial numbers of truck seals in the classified ads of the New York Times. Of course, no one does that exactly, but let’s examine this. 

If you record the serial number of a truck seal in the classified ads, then you know the truck was sealed before the newspaper was printed. How does this apply to images? If you record the cryptographic hash of an image (the guaranteed-to-be-unique id that creates a pixel-perfect fingerprint of an image, that is so “random” it effectively can’t be known before the image exists) in a newspaper, then you know that the image existed before the newspaper was printed, and more than that, everyone else knows it too. It is very unlikely that the NYT is going to go around and collect everyone’s newspapers and replace them with a new version with a different hash value. And if they print newspapers with conflicting values, that can be proved afterwards with just two differing copies. 

This is all great, but it’d be impossible to print hashes of all the images in the New York Times classified section – it’s just too many and it’d be a terrible read. Luckily, we already know how to fix this. Remember how a cryptographic hash is a digest or summary of something? We can go a step further, and hash the hashes to create a fingerprint of a collection of fingerprints. Then, the Timestamping Service just has one hash to record in the newspaper – the one that summarizes everything they’ve received in that time period. 

A number of timestamping services exist, and one of them did actually record their hashes in the New York Times. But others like OpenTimestamps use blockchains instead as the place to record the hash “somewhere else.” 

With cryptographic hashes of images stored with timestamping services, we can trust that an image existed at a certain point in time, without needing to trust the person telling us so. We can prove it to ourselves by taking a copy of the original file, hashing it, and checking that the output is the same as what was originally registered on the timestamping service. In a world where we are increasingly losing trust in institutions and may not know who to trust, especially as AI-generated propaganda becomes more prevalent, it is essential that we can prove it ourselves. 

This is fantastic, but we’ve still got some problems. For example, if we are both investigating an image, how do we know it’s the same image we’re referencing, and not an altered version? And if you tell me that Reuters says the image is fake, how do I know that you’re not lying about what Reuters is saying? And what about privacy concerns?

Cryptography can help us with these problems too. Remember again how the cryptographic hash of an image was a unique id for it? We can use that id to refer to the image, rather than describing it by its filename and location. Using the cryptographic hash as the id is called content addressing, as in content addressed storage. Now we can know for sure whether we’re talking about the same image. And we can easily talk about one image being an edited version of another. 

Using the cryptographic hash as the id helps us with privacy concerns as well. While content will always need to be removed for a variety of reasons, we can timestamp or talk about an image without ever revealing the image itself, because the cryptographic hash doesn’t reveal the underlying image.

And in the case of you telling me that Reuters says the image is fake, I can know that you’re telling me the truth even when you’re relaying second-hand information, if Reuters digitally signs the information. Using digital signatures makes it much easier to double check this information, as they allow us to “go to the source” even when we get the information second-hand. A digital signature is also different from an electronic signature, although they sound like the same thing. An electronic signature is the kind of image you make when you sign on the tablet at the grocery store. A digital signature is cryptography that lets you create something analogous to a branded seal, a seal that can only be created by you because you know the secret. And anyone who looks at the branded seal can confirm that you stamped it. 

Digital signatures allow us to “stamp” information, in a way that cannot be forged. And unlike a paper signature, they can’t be cut and applied to a different document. This means that we can share information from sources, and if the sources have digitally signed that information, and we know that they said it, even if we don’t quite trust the person who relayed the information. 

With these three tools (cryptographic hashes, timestamping services, and digital signatures), we as the general public can act as archivists for our future. By recording images as they exist now, and sharing digitally signed and timestamped information about the images, we can create a world in which we can ask the internet about an image. A world in which we can automatically get information about what is real from researchers we choose to trust, with information that we can independently verify. 

As part of my work with Starling Lab, we built a new tool called Authenticated Attributes to enable researchers to do exactly that: share authenticated information from their investigations so that people have spaces to turn to, to find content that is verifiable, with attestations from those they trust. You can learn more about Authenticated Attributes in this video, and find the demo code in this Github repository. 

The solution to the problem of AI-generated images isn’t to try to detect the “tells” of AI. That’s a battle that’s lost every time AI improves. The solution is to use technology to record what’s real, and share it with each other for our future.

Help us by taking this survey, and (optionally, at the end of the form) volunteer for a 30 minute user interview to add your perspective on how we can most effectively communicate information about the authenticity of digital media.

Starling Lab is working to bridge the gap between those who are experts in cryptography and data provenance with everyday readers of online content. Coordinating with Georgia Tech graduate students Kenneth Chen and Chris Hovey on this project, we are working on the analysis and design of user interface (UI) displays in news articles that communicate additional information about the source, provenance, and changes made to photographs and other assets in online publications.

Some of the question we are exploring are:

• What do readers want to see that will help them understand the contents and source of a photograph?

• What information is valuable front and center, and what information should only be revealed to those who want a deeper dive?

• How can we assist and enhance the process of inspecting and verifying information about photographs?

• Can we draw readers into an informed and critical inspection of the provenance of digital media?

• What is the best way to display information about the authenticity of an image without overwhelming users?

• How can we best communicate technical information, and why it is relevant to the user?

By experimenting, gathering feedback, and using an iterative process to incorporate it, we hope to be able to deepen both trust in media, and an understanding of the capture, store, verify framework for authenticity. The cryptographic tools and authentication workflows are nuanced and complex, and the right presentation is key to encourage critical readership and help users to understand how distributed tools add value to the journalistic process.

Take the survey to view and give your feedback on earlier authenticity displays. The last part of the form has an optional section where you can elect to participate in a user interview.

This collaboration surfaced records related to the beating of Tyre Nichols in Memphis, Tennessee, and were a part of the research completed by Big Local News. These records were sourced from the City of Memphis website, and videos were sourced from the City of Memphis Vimeo account. They were archived and preserved by the Starling Lab for Data Integrity using authentication technologies for video and web archiving.

These tools underscore the importance of accuracy, preserve immutable records of evidence at risk of disappearing or being unavailable through other means, and serve to combat mis- and disinformation. 

By using emerging cryptographic methodologies and decentralized systems, these public records can now be protected from disappearance and made available for the public to review, should the source or authenticity of the digital media be called into question. This problem is becoming more acute with the rise of Generative Artificial Intelligence. For example, should the video released by the City of Memphis be manipulated in the future, we now have that original video authenticated and stored.

Decentralized tools can create an audit trail (or “chain of custody”) that identifies the history (known as provenance) of a piece of digital content. This serves to reduce information uncertainty and bolster trust in digital records. In the process of creating audit trails, Starling Lab surfaces metadata, which is data about digital media.

When Starling archived these public records, an immutable digital ‘fingerprint’ called a hash was created using a mathematical formula to establish a snapshot of the records captured. If any single byte of the data is changed, be it a pixel of an image or the timestamp of when it was collected, one can use the hash to verify if a copy is different or manipulated. A hash mismatch is an indicator of altered data. It functions like a tamper-evident seal.

A hash is nearly impossible to fake, and the hashed data can also be signed with a cryptographic signature. This cryptographic tool makes it so we can attest to exactly what was created and when. Signing with a cryptographic key (pair) makes it possible to sign these records with a known identity, and give your ‘stamp of approval’ to a set of metadata and media. Hashes are also used to create the CIDs (Content IDentifiers) that we use to name the files with original videos, websites, and metadata related to the media.

A hash protects an original version of the public record that can be used down the line if the source of this research, the videos and archives of webpages, is called into question. It also helps protect important records like these from disappearing. 

These records, or the hashes of the content, were also registered on various blockchains to establish exactly what content existed, and when it existed.

Archives

These records are stored in a distributed, peer-to-peer data sharing system called IPFS (InterPlanetary File System) from which users can download and view copies of the archives.

These same files are preserved on a distributed archival storage network called Filecoin in a long-term agreement with Filecoin providers that store this data. Users can inspect these deals and the identity of the nodes who are archiving this data.

Last, metadata records for each of the files were registered on the blockchain, so users can validate what content existed, and when we established the record of their existence.

For example, this registration on the Numbers blockchain of one of the videos posted of Memphis PD body cams contains a pointer to the metadata file on IPFS labeled as “assetTreeCid”. In the metadata file you will find both a CID for the original content, as well as a Content ID for the entire archive.

Users can validate these records against the version of the videos they have to see if that version has been altered, faked, or modified.

• IPFS File Copies

  • Original Body Cam Videos – Video 1 | Video 2 | Video 3 | Video 4

  • Website Archives – Vimeo Webpages  |  Memphis PD Website

• Filecoin Archive Information:

  • Records of Storage – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages  |  Memphis PD Website

• Blockchain Registrations:

  • Numbers – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages  |  Memphis PD Website

  • Avalanche – Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages  |  Memphis PD Website

  • ISCN on Likecoin– Video 1 | Video 2 | Video 3 | Video 4 | Vimeo Webpages  |  Memphis PD Website

How to Download and Inspect Records

• Click on the IPFS Registration link to download this from the distributed storage network

• Locate where you downloaded the file. It should be in your Downloads folder and have a name that looks like babyf6g3sdb…

• For videos, change the name by adding a .mp4 the end of the filename for body cam videos, and for the web archives add a .wacz to the end of the filename

• You can open the .mp4 file on your computer

• You can open the web archive files by visiting the website https://replayweb.page/ and dragging and dropping the .wacz files into the web interface.

  

• Validate the videos and web pages you have a copy of are the same ones registered on the blockchains by looking at the metadata record called "assetTreeCid". You can view this record by using an IPFS gateway with one of the assetTreeCids from a blockchain registration record, such as https://ipfs-pin.numbersprotocol.io/ipfs/<assetTreeCid>.

  Example: https://ipfs-pin.numbersprotocol.io/ipfs/bafkreiauefa46ksrrtvws7g6wszvck7jaf5oorwb3wf7dv7dykyljo3kq4

The Mom, I See War (MISW) digital museum, part of the newly launched Mom, I See Foundation, launches its global platform dedicated to documenting artwork created by children who find themselves in the midst of a full-scale invasion of their country. With a vast collection of over 15,000 children’s drawings, the digital museum provides a powerful testament to the impact of war on young lives. Notably, the platform will also serve as a means to raise funds for educational programs and scholarships benefiting children affected by war.

Mom, I See War (MISW) is the first known digital archive to document the way children experience war by preserving high-resolution jpeg scans of their drawings, metadata about the drawings, and encrypted versions of these that contain private, identifying information (the last names) of the children. MISW has a collection of 10,026 single-image assets from Russia’s invasion of Ukraine which Starling Lab has registered on the NEAR blockchain and archived on distributed web protocols.

• First, records of these assets were hashed and registered on the NEAR blockchain using the Numbers Protocol. 

• Second, versions of these drawings and separate metadata files (with the full names of the children redacted) were stored on IPFS.

• Copies of the metadata were injected with information according to the C2PA standards, which enables users to inspect and understand more about what’s been done to these records, where it’s been, and who’s responsible.

• An encrypted archive of the drawings (assets) and full metadata were also sealed and preserved as archives on Filecoin.

The four versions of records of these assets each serve a different purpose. The registration of a hash (or an immutable ‘fingerprint’ of the drawing and metadata) on the NEAR blockchain networks establishes a record of exactly what was stored, and a date that it was stored on, without revealing any other information. 

The publication of the full drawings and metadata on IPFS (without individual identifying information like the last name of the child who made the drawing) makes these assets available on a publicly owned, peer-to-peer system which is resilient against infrastructure deprecation and various forms of censorship, and the assets can easily be hosted by anyone. 

The C2PA version of the registered assets was created according to standards set by a coalition for standards development: The Coalition for Content Provenance and Authenticity (C2PA). These standards guide the development of tools, supported within Adobe products such as Photoshop, that create public and tamper-evident records that can be attached to an image. These images can then be used with different tools for inspecting and making edits that preserve the understanding of what’s been done to modify assets, where it’s been, and who’s responsible.

Finally, the archiving on Filecoin (drawings are bundled with full metadata and encrypted) helps preserve these drawings across multiple storage providers in a way that can outlast current media storage and publishing methods and technology.

Numbers Protocol Registration

Drawing & Metadata Registration

To register these assets, Starling Lab collaborated with the Numbers Protocol and also used the Starling Integrity pipeline to process the data and create publicly available registration records, publishing images and metadata records on IPFS. The records created include images of the drawings, along with a limited (redacted) set of metadata, which did not include certain identifying information, such as the children’s last names. See the example on Numbers Search.

The Numbers cross-chain search engine has an interface to display registration information alongside the drawings & metadata.$

The initial drawing and metadata record were both stored on IPFS with their IPFD CIDs stored on the NEAR blockchain and a registration of this record was added to Numbers Mainnet. Using the Numbers Protocol search, we can use the drawing’s IPFS CID to find the transaction containing all the information.

C2PA Registration

Once the regular versions of the drawing and metadata were registered, C2PA-compliant versions of these assets were also created.The presence of an icon next to indicates that an asset is one with C2PA metadata, and links back to the initially registered asset. The C2PA standard is a way of adding metadata that points versions of an asset back to the original asset, making an audit trail. 

This makes it possible for anyone who may want to create verifiable, processed or modified versions (such as versions that are cropped or color enhanced for publication) of the drawings to create them, and connect them back to the original image with a record of any changes made. 

The metadata record for the C2PA version of the asset points back to the initial asset registered on Numbers

The C2PA metadata added to the image is a standardized method used by tools such as Adobe Photoshop which allows each incremental edit to add a signed thumbnail and edit log to an image. 

Starling Integrity Pipeline

All of the assets for this project were also bundled together, then registered and preserved using the Starling Integrity pipeline. This is a data processing pipeline that Starling Lab uses to bundle images with metadata, add registration certificates, encrypt, and add to various distributed storage and archival systems.

For this particular project, we used the Starling Integrity pipeline to zip complete copies of the image assets with private metadata including the last names of the children who made the drawing, then included authsign and OpenTimestamps certificates, and encrypted the file bundle. We then added these signed and encrypted files on Filecoin for archival storage, a cryptocurrency-collateralized archival storage system that will store the files for a duration committed by the set of storage providers, in this case 18 months, and run intermittent checks on the content to verify that the data is continuously preserved with the specified redundancy.

The archive itself is registered on NEAR using Numbers Protocol, as well as on the LikeCoin blockchain with using the ISCN sepcification.

How To: Searching and Viewing Assets

The records for each drawing were archived by Starling Lab and the Numbers Protocol, with the CIDs & NEAR blockchain transaction IDs for the original drawing, the original metadata, as well as versions of these that have C2PA credentials, which allows one to understand the history and identity data attached to images that may need to be modified or edited.

For each drawing, you can view each of these four records using Numbers Search to:

1. See the blockchain registration of these assets using a block explorer for NEAR or Numbers Mainnet.

2. Inspect the images and sets of metadata on IPFS using the Numbers IPFS gateway.

3. Use Filecoin explorer to see the archive of the assets.

4. Download images and view C2PA-credentialed assets with the Verify tool.

Search for a Drawing and Metadata with a CID

In order to search and view the NEAR blockchain registrations with the Numbers Search tool, a user needs to have the asset CID provided from the list of archives. In order to search in a web browser, type in https://nftsearch.site/asset-profile?nid= followed by a CID, such as bafybeibazzhn2unnxtymqva5jceimoohjtesh7l6lknudrdkltmnepikfq

The resulting URL of the above C2PA-credentialed asset would look like: https://nftsearch.site/asset-profile?nid=bafybeibazzhn2unnxtymqva5jceimoohjtesh7l6lknudrdkltmnepikfq

Blockchain Registrations

The assets were registered on two blockchains. The basic information with the CID of the content stored on IPFS, plus a hash of the data (a check you can use to see if content you have, such as a copy of the drawing is the same as this registered version) is added to the Numbers Jade blockchain network. Next both the basic registration and more complete version of the asset’s metadata, which includes C2PA data, was registered on the NEAR Aurora blockchain.

The transactions where these are registered are linked from the Numbers Search interface and can also be found by appending the transaction ID to the following URLS

• Jade: https://mainnet.num.network/tx/<transaction ID>

• Aurora: https://explorer.mainnet.aurora.dev/tx/<transaction ID>

These registrations provide an immutable, timestamped record of the sha256 hash of the content, and can be used to verify whether or not any copy you may have of a drawing or it’s metadata are in fact identical to the version that was archived on the blockchain by Starling Lab and Numbers.

Viewing Data on IPFS

IPFS is a peer-to-peer protocol for sharing and hosting data and media. This is a resilient alternative to the client-server http-based web2 internet that can be accessed from web browsers using a gateway that bridges the http and IPFS networks.

Numbers Protocol hosts a gateway which you can use to view the IPFS copies of the drawings and copies of metadata. To access these copies, you can click on a link in Numbers Search, or type in the gateway address along with the CID in any web browser: 

https://ipfs-pin.numbersprotocol.io/ipfs/<CID>

Example: https://ipfs-pin.numbersprotocol.io/ipfs/bafybeigda2fjjeta7725hbicpfzldygszl4gbllcaybwuetlemraauazwm

Having this published on IPFS makes it easy to retrieve and inspect the images and metadata that has been registered on the different blockchain networks.

Viewing Filecoin Registration

Filecoin is a cryptocurrency (FIL) backed archival storage system. This system ensures archival storage on their network by the miners of their cryptocurrency. The protocol runs intermittent checks against sets of data that are archived, and if data isn’t stored, the miners (also known as storage providers) lose FIL collateral that they stake when they made the storage deal. The Starling Integrity pipeline used this to prepare and store these encrypted versions of the archive. 

With the Filecoin CID Checker, you can view information about an archive, such as the identity of the node storing the data, the status of that archive, the deal made for the preservation, and the immutable content identifier or the payload that was submitted for storage. See the information about the MISW collection that was archived on Filecoin  (Filecoin piece CID baga6ea4seaqdqgywa3n55hughvnfkroou6nm6qisqalzew7vy2vdqe3jfcr6wci), which includes all files, metadata, and a spreadsheet listing all assets.

Viewing C2PA assets with the Replay Tool

The Content Authenticity Initiative (CAI) is a group working together to add a layer of verifiable trust to all types of digital content. Members of this group include those from media and tech companies, NGOs, academics, and more. 

In February 2021, Adobe, Arm, BBC, Intel, Microsoft, and Truepic launched a formal coalition for standards development: The Coalition for Content Provenance and Authenticity (C2PA). These standards guide the development of tools, supported within Adobe editing tools such as Photoshop, that create public and tamper-evident records that can be attached to an image to understand more about what’s been done to it, where it’s been, and who’s responsible. These same standards were used to create metadata and records for the MISW collection.

With the Verify tool, you can download an image via IPFS from the MISW collection (adding a .jpg file extension), drop the image in the online tool, and inspect the image and data added according to the C2PA standards. If you were to edit the image with Photoshop, the hash of this record would change - however, you now have a tool that can tie this image back to the original, as well as show and compare any edits and changes made in Photoshop.

This image has C2PA Content Credentials. If you make edits in Photoshop, a record of the edits and activity can be viewed with the Verify tool.

Conclusion

The project demonstrates the archival process for a valuable set of assets that create a record of the war in Ukraine to help future generations that want to understand this pivotal event in history, and helps create a reliable narrative of the past that may be used by future generations to help understand, make claims, and hold individuals and governments accountable and build a better future for humanity.

This set of tools not only creates an immutable archive of these assets, it also demonstrates the tools that can be used for the creation of an archive that is searchable and verifiable. It  creates a set of assets that can be used in journalistic, legal, and historical depictions that allow those who may want to make minor edits to share these images with others by cropping, improving color, tone, and contrast, or make other minor edits, in a way that makes it possible for the public to understand the accuracy and veracity of these copies. Learn more about the collaboration with the Mom I See War Research and Numbers Protocol.

As business rivalries go, the story of Suen-nada and Ennum-Ashur was pretty routine. They both claimed ownership of valuable intellectual property, wrestled over control of accounts, and accused each other of theft.

Their case went to court. Witnesses were present. Testimony was delivered and became crucial evidence.

A fire burned down their Assyrian colony in 1836 B.C., but a record of their testimony survived and is now available for review at The Metropolitan Museum of Art in New York.

We know about the trial because, thousands of years ago, the world’s most advanced technologists figured out how to use a crude implement to etch markings into stone and clay. This data storage breakthrough would be used to record crop yields, trades, weddings, births, deaths, wars, legends, and other data that was critical to evolving human civilization.

While they are museum pieces today, at the time there was a logic to using hard materials for persistence. Even back then, this method wasn’t the fastest way to record information (papyrus was invented at least a millennium before) but it would last and, most importantly, be difficult to alter. Modern technologists still find this approach important — and the efficiency tradeoffs familiar.

(…)

Evidence has always needed to stand the test of time and the test of scrutiny. Long-established concepts can still help, including decentralization and cryptography (which was likely around even in the days of Suen-nada and Ennum-Ashur). A related modern concept can also help: blockchains.

In the spring of 2022, Russian artillery shells ripped through the walls of several schools in Kharkiv, Ukraine. The whole region was under heavy fire from advancing Russian armed forces, but an intentional attack on civilian targets is a war crime. The United Nations specifically defines education as a human right. Turning sanctuaries of learning into a battlefield creates a vicious cycle of illiteracy and poverty.

(…)

Starling Lab set out to authenticate and preserve these vulnerable assets. (…)

Our investigators made web archives of social media posts from Kharkiv, verified using state-of-the-art OSINT techniques. Decentralized storage deals preserve the collection on thousands of servers around the world, and the redundant recording of hashes and cryptographic signatures permit trustless inspection of the items and their audit log.

(…)

Open source intelligence (like social media posts) may still be questioned, so Starling arranged for photographers to visit two of the schools. They used the context-rich capture app ProofMode, from the Guardian Project, to include corroborating metadata (including time, GPS coordinates, surrounding cell network, phone locale, etc.). These bundles were cryptographically sealed with the images and their integrity proofs registered to several blockchains for safekeeping.

Starling Lab has since made a pair of submissions to the Office of the Prosecutor at the International Criminal Court, including an analysis of how these methodologies can establish credibility of the evidence.

(…)

The same approach – using our Starling Framework of Capture/Store/Verify – has helped us to preserve testimonies of Holocaust survivors, store thousands of examples of Russian misinformation, document living conditions for the homeless in California, record promises by politicians about government surveillance, and save examples of climate change impacts in the Amazon.

Cryptography, decentralization, and blockchains are the tools we used to preserve these important records in humanity’s collective memory. These projects have created immutable records to stand up against challenges from the wide-scale adoption of generative AI, sophisticated disinformation campaigns, and changing digital custody practices.

Today’s courts and other civic institutions must confront similar challenges that undermine trust in their own critical records. By embracing similar innovations, there’s a chance for digital evidence to become as resilient as ever – the modern equivalent of being etched in stone.