Research & Analysis · Forensics and investigation · 21 min read

Which Cloud Logs Must You Preserve Before an Incident?

Before a cloud account compromise, preserve the identity events, administrative changes and selected data-access records needed to reconstruct consequential actions in your critical services. Include changes to logging itself, and retain enough resource and permission context to interpret the events later. Test that these records reach a protected destination, survive the required investigation window, and can be retrieved by an authorized responder. A provider offering audit logs does not establish that your organization has usable evidence.

The difficult question is not which product collects the most events. It is whether the available records can support a particular conclusion: a credential was used, a privilege changed, a sensitive object was requested, or a logging control stopped working. Those conclusions require different observations. Treating them as interchangeable produces false reassurance precisely when a decision carries the greatest consequences.

I use an evidence-path model to examine that problem: generation, selection, routing, preservation, retrieval and interpretation. This is an analytical framework, not an empirical maturity score. The examples below are hypothetical; no customer investigation or live cloud test is represented. The aim is a defensible answer to a specific operational question, with its uncertainty still visible.

Begin with the conclusion you may need to defend

Choose a critical service and describe the incident decision in plain language. For a document repository, the question might be whether a compromised workload identity accessed a particular collection of sensitive files. For an administration platform, it might be whether a new privileged role allowed persistence after the original account was disabled. A generic requirement to retain security logs leaves both questions underdefined.

Build backward from the conclusion. Establishing a role change requires a record of the change, its target and its effective scope. Establishing subsequent use requires activity under the resulting authority. Connecting that activity to a person requires additional evidence about credential possession and session context. A single successful authentication event cannot carry all three arguments.

NIST's cloud forensic reference architecture supports examining specific functional capabilities against forensic challenges and adapting the resulting analysis to the organization's architecture (Herman et al., 2024). That matters because evidence availability is a property of a service arrangement, not merely a cloud brand. A managed application, an infrastructure account and a federated identity service expose different parts of the same action.

I would document the following mapping before selecting retention settings. The final column is as important as the first two: it prevents an investigation from making a stronger claim than its collection design permits.

Investigation questionRecords and context to identifyClaim the records alone cannot establish
Which credential initiated the activity?Authentication, token/session context and stable principal identifiersWhich human controlled that credential
What authority changed?Role, policy, application-consent or administrative events; relevant configuration versionsThat the new authority was subsequently exercised
Which sensitive resources were accessed?Applicable data-access events, resource identifiers and operation outcomesEvery downstream use or disclosure of returned data
Did evidence collection change?Logging configuration events, destination permissions and independent delivery observationsCompleteness of periods without trustworthy collection

This mapping also identifies alternatives. If a service cannot expose a needed data operation, an application audit record or another independently relevant source may help. The substitute needs its own argument about coverage. Calling it compensating evidence does not make it equivalent.

Separate identity, control-plane and data-plane evidence

Identity logs describe authentication and related identity activity. Administrative or control-plane logs describe operations on infrastructure, permissions and configuration. Data-plane records concern operations on the data or service resources themselves. The boundaries vary by provider and service, so record the actual event categories and methods rather than imposing a universal taxonomy on every product.

AWS offers a concrete warning against conflating these layers. CloudTrail event history contains the last 90 days of management events for an account in an AWS Region. It is not an organization-wide view and does not contain data events, Insights events or network activity events (Amazon Web Services [AWS], n.d.-d). Seeing an administrator's bucket-policy change in that interface therefore does not establish visibility into object reads.

CloudTrail trails and event data stores do not log data events by default. AWS documents separate configuration and additional charges for those events, including operations such as S3 object reads and writes (AWS, n.d.-b). The readiness question is which resources and operations the actual selectors cover. Enabling a data-event category somewhere in an account is not proof that the sensitive bucket under investigation was included throughout the relevant period.

Google Cloud has its own boundaries. Its documentation distinguishes Admin Activity, Data Access, System Event and Policy Denied audit logs. Admin Activity logs are always written; Data Access logs are disabled by default except for BigQuery, and access to publicly available resources has documented logging exceptions (Google Cloud, 2026a). These specifics should drive a service-level coverage check, not a claim that every provider behaves like AWS or Google.

For each critical action, preserve the stable resource identity, operation, recorded outcome and available correlation identifiers. Interpret an outcome using that operation's semantics. An accepted request for asynchronous work may require a completion record before you claim the work finished. That is a question for the relevant service specification and test, not a universal interpretation of a field named success.

Treat routing as a separate failure boundary

An event can be generated correctly and still be absent from the investigation store. Source selection, destination configuration, permissions and downstream processing introduce additional opportunities for loss or exclusion. A dashboard showing an enabled audit feature addresses only part of this path.

Azure Monitor diagnostic settings define which supported data to collect from a resource and where to send it; Microsoft describes a separate diagnostic setting for each resource whose data is being collected (Microsoft, 2026a). The practical inventory should therefore identify resources and their settings, not simply name a central workspace. A newly created service can fall outside an otherwise functional collection design unless its enrollment is checked.

Google's Log Router provides another useful distinction. Sinks apply inclusion and exclusion filters, and temporary router storage buffers against disruptions but does not protect against configuration errors (Google Cloud, 2026b). A healthy destination cannot recover an event that an upstream selection rule deliberately omitted. Nor should a temporary buffer be mistaken for the organization's evidence-retention commitment.

I would compare three observations for a controlled action: what the source exposes, what reaches the preservation destination, and what the analyst's search returns. If the first two contain the event but the third does not, the immediate problem may be normalization or query scope. If only the source contains it, inspect selection and transport. If no source record exists, investigate event eligibility and configuration before blaming the search platform.

Use the same transaction or operation identifiers where available, while documenting their scope. Matching only a display name and approximate timestamp risks joining different actions. Conversely, identical records delivered more than once should not be counted as separate business actions. The purpose of this comparison is to locate the missing observation, not to manufacture an apparently complete timeline by merging loosely related events.

Detect silent event loss while the collection path is running

A successful readiness exercise needs continuing support from the live collection path. I would combine source-specific synthetic actions, boundary counters and reconciliation of identifiable records. Each observes a different failure. A collector heartbeat proves that something is responding; it does not prove that the required data-access category is selected. A record injected directly into the downstream collector tests a shorter path than a controlled action that should generate an audit event at the source. Name that coverage difference before calling either a completeness check.

Protocol success is another bounded observation. OTLP describes acknowledgements between one client/server pair, not an end-to-end guarantee across intermediary collectors. For OTLP/HTTP, a 200 OK response can contain partial success and a count of rejected log records; a client must not retry a request whose partial_success field is populated. The specification also recognizes duplicate delivery when an acknowledgement is lost and a client resends (OpenTelemetry Authors, n.d., “OTLP/gRPC Response,” “OTLP/HTTP Response” and “Known Limitations”). This makes HTTP status alone inadequate evidence that every submitted record survived. Distinguish the rejected-record count from a warning accompanying zero rejected records.

At a collector, inspect metrics for accepted records, export success, enqueue failures and receive refusal. OpenTelemetry's internal-telemetry guidance distinguishes these observations and warns that export-send failures do not inherently mean data loss because retries may succeed. Its documented queue-capacity metric is measured in batches, while log-record counters count records (OpenTelemetry Authors, 2026a, “Queue length,” “Receive failures” and “Data flow”). Do not subtract those values as though they described the same population or unit. Check the deployed version, metric labels, counter resets and processor configuration before comparing intervals.

For a bounded cohort with stable source identifiers, use a proposed accounting rule: expected selected records must be classified as received uniquely, still pending, deliberately excluded under the declared policy, or unexplained. Make the categories disjoint at a stated observation time. Preserve duplicate multiplicity separately. If the cohort starts at the collector, it cannot account for events the source never generated; if it starts from known source records, it can test the subsequent collection path but still cannot prove that all relevant business actions were logged.

Consider an explicitly hypothetical cohort of 1,000 selected source records. The destination has 970 rows, but identifier reconciliation shows 960 distinct cohort members and ten duplicate deliveries. Another 25 members remain pending, five were removed by the documented policy, and ten are unexplained. The accounting is 1,000 = 960 + 25 + 5 + 10. Reporting only a 30-row difference would mix pending work, exclusions, duplication and possible loss. Reporting the 960 unique arrivals as complete would be worse. These figures illustrate the method; they are not a measured pipeline result or a universal loss threshold.

An unexplained record is not immediately a permanently lost record. It may arrive late, be quarantined, reside in another partition or be omitted by the retrieval query. Define how long the observation remains provisional, using the actual delivery behaviour and remaining recovery opportunity. Then investigate the unresolved population rather than treating the expiration of a dashboard window as proof of loss. Where source identifiers are not reliably unique, retain the weaker evidence and the ambiguity instead of silently collapsing repeated events.

Queue capacity buys a bounded opportunity to recover; it does not create unlimited durability. OpenTelemetry documents queue overflow, exhausted retry periods and loss of in-memory data on collector failure. Persistent queue storage can survive a restart but still has disk and retry limits (OpenTelemetry Authors, 2026b, “Sending queue,” “Persistent storage” and “Circumstances of data loss”). Record the configured behaviour rather than copying a documentation default into an availability promise. A queue draining successfully after an outage also does not prove that earlier overflowed records returned.

The response clock should be tied to the last feasible recovery action. Identify the affected source and interval, the oldest unresolved cohort, whether source replay is supported, and how long the source will retain the relevant records. A backlog alert that arrives after that window closes may accurately describe failure while offering no recovery. Prefer an alert that names the remaining action and owner: restore permission, correct a selection rule, recover a failed export, or preserve the source interval before its retention expires. Do not assume all cloud audit feeds offer replay.

Synthetic events should carry an unambiguous test identifier and a predefined expected route, transformation and arrival condition. Rotate through the critical categories and routes the business actually relies on; one successful administrative canary cannot establish coverage of object reads or a different account. Observe both arrival and interpretation. A canary whose identifier survives while its operation field is stripped may show transport health without proving that the resulting evidence supports the intended investigation. Keep synthetic payloads free of real customer data and avoid creating sensitive values solely for monitoring.

Finally, test the monitoring failure itself in a controlled environment. Exercise a rejected record, an interrupted downstream destination and a collector restart under the intended persistence configuration. Establish whether the failure observation reaches an operator through a path that remains available when the main logging route is impaired. This is a proposed acceptance exercise, not a test performed for this paper. Its result should bound what was delivered, delayed, deliberately excluded or left unknown, and explain whether replay or reconstruction restored the missing evidence. “The collector recovered” is a component result; the investigation still needs an account of the affected records.

Design retention around the earliest evidence you need

The useful retention period begins with the earliest event needed to answer the investigation question. That may precede the alert: a credential might have been created well before its suspicious use. Designing only around the time between alert and analyst response can omit the preparation phase of the incident.

Microsoft's current retention table illustrates why product and license details matter. Standard Entra audit and sign-in reports retain seven days under the Free tier and 30 days under P1 and P2. Other reports have different rules. Microsoft also states that upgrading a license does not recover activity that has already expired unless it was previously archived (Microsoft, 2026b). Those values are not a statement about all Microsoft security logs or every separate archive.

For a rolling source whose retention is measured from event time, a simple planning inequality is R ≥ D + P + M. Here, R is source retention; D is the delay from the earliest needed event until recognition of the need to preserve it; P is the time from that recognition until preservation completes; and M is a deliberate margin. This is my planning model, not a provider guarantee or a prescribed regulatory period.

Suppose the earliest needed event occurs on day 0, recognition occurs on day 12, and export finishes on day 14. At completion, a seven-day rolling source cannot supply the earliest seven days of that interval. A 30-day source has 16 days of arithmetic headroom for the day-0 event. Neither result proves that the relevant event was generated, selected or exported. Boundary inclusivity and the provider's actual expiration behavior also need checking.

Treat source lookback, collection delay and archive retention as separate clocks. Continuous archival can preserve an event beyond its source lifetime, provided collection succeeded in time. An outage longer than the recoverable source window can create an unrecoverable gap even when the archive itself retains years of data. Do not add ingestion delay twice if it is already included in P; define the timing points before calculating margins.

Preserve context that changes after the incident

A log's meaning often depends on facts outside the record. A principal identifier may survive while the account's display name changes. A resource may be deleted and recreated with a similar name. A permission assignment at investigation time may differ from the assignment under which the recorded operation occurred. Reconstructing only the current configuration can silently rewrite the historical context.

I would preserve relevant identity mappings, resource inventories, authorization configuration and logging configuration at useful change points. This does not mean copying every secret or every object. It means retaining the minimum contextual evidence needed to explain consequential identifiers and authority. A role identifier without the relevant permission definition may prove that an assignment existed while leaving its practical effect uncertain.

Keep event time, ingestion time and collection time distinguishable. Event time describes when the source says something happened; ingestion time describes arrival in a particular processing system; collection time describes the investigator's acquisition. Delayed delivery can make arrival order differ from action order. Preserve the original values and their documented semantics, and record any timezone conversion applied during analysis.

Where timing is uncertain, use intervals and relationships instead of false precision. A common request identifier may establish that two records concern one operation even when their displayed timestamps disagree. An ordered provider sequence may establish precedence without proving an exact elapsed duration. If the evidence cannot distinguish two possible orders, both should remain possible in the finding.

NIST's forensic-process guidance separates collection, examination, analysis and reporting, and cautions through its examples that recorded account activity does not itself identify the person acting (Kent et al., 2006). That older publication is useful for evidential reasoning, not current cloud configuration. The same discipline applies here: distinguish the original observation, the transformation used to interpret it and the conclusion drawn from it.

Distinguish completeness, integrity and attribution

Completeness asks whether the required observations are present. Integrity asks whether the retained material has changed relative to an established reference. Attribution asks what entity or person the activity can be associated with. These are separate claims. A perfectly preserved export can be incomplete, and a complete set of credential events can still leave human attribution uncertain.

CloudTrail's integrity mechanism makes this distinction tangible. When enabled, it supplies signed digest files containing hashes of delivered log files. Enabling that feature delivers the digests; it does not itself perform validation. AWS describes validation using its CLI at the original delivery location and custom tooling for moved logs (AWS, n.d.-c). A readiness exercise should therefore include actual validation of the preserved artifacts and retain the result.

The mechanism supports checking delivered files. It does not, by itself, prove that your event selectors covered every relevant operation or that a credential belonged to the person using it. My evidential rule is to state the validation claim narrowly: which files, which interval, which validation method and what result. Do not replace those particulars with a general declaration that the investigation is cryptographically proven.

NIST's preservation guidance recommends recording a file's source and creation or transfer history, hashing evidence close to collection, and protecting the resulting hashes separately from the evidence they describe (Guttman et al., 2022, Section 3.2). Its primary audience is evidence-handling professionals. Applying those principles to enterprise incident work does not establish admissibility or replace the organization's legal and investigative procedures.

For an export, I would retain the raw bytes, collection time, requesting identity, source and scope, query or API parameters, pagination status, reported errors and tool version. Record any later decompression, normalization, enrichment or redaction as a derivative with its own identity. The analyst can search a convenient representation while another reviewer can trace a disputed conclusion back to the preserved source material.

Make preservation survive the compromise you are investigating

Putting logs in another bucket is not automatically separation of authority. Determine whether the compromised administrator can change source collection, alter the destination, delete retained versions, bypass a retention mode, or deny access to the keys needed for retrieval. These are different failure paths and may require different controls.

AWS S3 Object Lock illustrates why the mode matters. It applies retention protections to object versions. Governance mode can be bypassed by appropriately authorized users; compliance mode prevents protected-version deletion or overwrite during retention, including by the account root user, subject to the documented account-deletion exception (AWS, n.d.-a). The word immutable without its mode, scope and administrative boundaries is too imprecise for an evidence claim.

A retention control on stored bytes cannot retrospectively generate an omitted event. Nor does its stated protection alone establish that decryption keys and retrieval permissions will remain available. Those are separate dependencies to evaluate. A design can successfully prevent deletion yet fail the investigation because no authorized responder can read the preserved material when the primary tenant is unavailable.

I would separate routine analysis access, preservation administration and emergency retrieval as far as the organization's size and risk justify. The access test should include the loss of the usual operator's account. Verify the alternate authorized path without granting broad standing access merely for convenience. Record how emergency access is approved and audited, and how its use ends.

Preservation also needs an exit plan. A provider change, account closure or obsolete export format can make retained evidence difficult to use. NIST's preservation discussion explicitly includes future key availability and transition from a cloud storage provider (Guttman et al., 2022, Section 3.2). Test an authorized retrieval and interpretation path before treating the retention configuration as an achieved capability.

Prove the path with a bounded reconstruction exercise

A useful readiness test starts with a known action and ends with an independently reviewable account of that action. In a controlled environment, choose a harmless administrative change and access to a designated test object. Specify expected identifiers, event categories, sources and destinations before execution. Keep a separate ground-truth record of what the test operator did.

The reconstruction task is then concrete: another authorized participant retrieves the evidence using the documented incident path and explains what happened, to which resource, under which recorded principal and within what timing uncertainty. They should state what the records do not establish. Success requires an interpretable result, not merely a screenshot containing the test account's name.

Test boundaryEvidence to retainAcceptance question
Generation and selectionKnown action, configuration version, expected category and observed source recordWas the selected operation actually observable?
Delivery and preservationDestination record, receipt times, retained original and integrity resultDid the expected record survive the complete path?
RetrievalAuthorized role, query scope, pagination and errorsCould a responder obtain the required interval and population?
InterpretationReconstruction with source identifiers and explicit uncertaintiesDoes the conclusion follow without inventing missing context?
Failure recoveryControlled fault, detection, restoration and gap assessmentWas the loss recognized, bounded and handled as designed?

Repeat selected paths under safe, authorized faults: remove a test collector's permission, apply a deliberately excluding test filter, or exercise alternate retrieval while the normal operator is unavailable. Keep these faults confined to test resources; production changes need their own operational authorization. The result should identify the affected interval and whether missing events can be recovered from the source.

A clean test is evidence about that path, configuration and time. It is not a measured completeness percentage for the entire estate. Reassess when the service, identity model, logging categories or collection architecture changes. A new critical operation can invalidate the coverage argument without breaking any existing collector health check.

Use absence carefully and measure the right failures

An empty search result has several possible explanations: the action did not occur, the event was never generated, the event was excluded, retention expired, retrieval was incomplete, or the query failed to match the stored representation. An investigation should eliminate relevant alternatives before translating absence into reassurance.

For a bounded test, the denominator can be known: the designated actions and expected eligible records. For ordinary production, the total population of relevant actions may be unknown. Reporting 100 percent log completeness because every received file passed a hash check confuses integrity with coverage and uses the wrong denominator. Report the narrower observed result instead.

I would track outstanding critical-action coverage gaps, age of the last successful reconstruction, unresolved delivery interruptions and remaining source lookback during a collection failure. Each measure should name the service and event scope. Aggregate storage volume can help capacity planning, but it cannot tell an executive whether object access is reconstructable for the compromised application.

Detection testing is a related but separate task. A detection rule may perform well on the records it receives while missing an uncollected category entirely. Conversely, a preserved record may be valuable to an investigation without supporting a useful real-time alert. Keep forensic availability and detection effectiveness as distinct acceptance arguments, with shared telemetry dependencies made explicit.

Balance privacy, cost and containment without hiding the tradeoff

Collecting every available event indefinitely is not a neutral default. Logs can reveal identifiers, resource names and sensitive operational patterns; some formats also contain request details that warrant tighter treatment. More retained material increases the scope of information that must be governed, searched and protected. The collection decision should follow the investigation question and sensitivity of the source.

I would prioritize consequential administrative operations and data access affecting critical resources, then justify broader coverage where the threat and investigative task require it. For each source, record the purpose, access population, retention rationale and disposal mechanism. Where redaction is appropriate, distinguish the restricted original from the analyst-facing derivative and document what information the transformation removes. Do not assume a redacted copy can answer questions requiring removed identifiers.

There is also a cost to waiting for perfect evidence during an active compromise. If ongoing access creates immediate harm, containment may need to proceed while preservation occurs in parallel. Capture readily available volatile context when safe and authorized, but do not turn a collection checklist into an unconditional delay on disabling a dangerous account or stopping a damaging operation.

NIST's current incident-response profile calls for recording investigation actions and preserving the integrity and provenance of incident data and metadata. It also places evidence retention within the organization's procedures and policies (Nelson et al., 2025, RS.AN-06 and RS.AN-07). I would apply that by recording containment actions, their timing, the authority used and any evidence knowingly lost. An explained constraint is more defensible than a retrospective claim that nothing was sacrificed.

Responsibility still needs an owner. The cloud incident accountability discussion addresses who must act across organizational and provider boundaries. The incident-response tabletop framework addresses whether those handoffs work under pressure. This paper's narrower acceptance task is proving that the required evidence is available and usable when those decisions arise.

Limits of the method and the decision it supports

The strongest objection is that a small organization cannot sustain a separate forensic architecture for every service. That is reasonable. Start with the few incident questions whose answers materially change business decisions, and test representative critical paths. Managed collection can reduce operational burden. It cannot remove the need to establish which questions the service can answer, which remain unresolved and who accepts the resulting exposure.

A second objection is that provider audit records should already be trusted. Provider-generated records can be valuable without being sufficient for every conclusion. The relevant issue is the scope of the claim, including configuration, retention, access and interpretation. Additional evidence should resolve a real uncertainty rather than duplicate storage ceremonially.

This framework has not been validated through a controlled study, and it makes no quantified claim about reduced investigation time or incident loss. Its source base combines official forensic guidance and current provider documentation, not fifteen independent empirical studies. Provider behavior and licensing can change. The particular services, editions, operations and contractual access available to an organization must be checked before implementation. Legal preservation duties and admissibility require the appropriate professional review.

What the method does support is a bounded readiness decision. For a named critical action, you can identify the required observations, demonstrate their preservation and retrieval, explain what they establish, and expose the remaining uncertainty. If one of those steps fails, the result is a specific engineering or governance task rather than an unsupported declaration of readiness.

If your team cannot reconstruct a consequential cloud action from records it already controls, a cloud forensic readiness assessment can focus on that gap: map the required evidence for selected critical services, inspect collection and retention boundaries, and design an authorized reconstruction exercise. The useful deliverable is an evidenced account of what you can investigate today, what you cannot, and which changes would make the difference.

References