Mandate new final assertion format
Bug: 441727732
Test: none
Change-Id: I054d317dd8bedb5e8855e9eb0320200c6415b7a8
diff --git a/example/AGENTS.md b/example/AGENTS.md
index 172e49e..c6a7636 100644
--- a/example/AGENTS.md
+++ b/example/AGENTS.md
@@ -20,6 +20,8 @@
The final assertion message (for a vulnerable device) **should include the captured artifact** as proof when possible.
+The assertion message **must** begin with the exact prefix `AUTOREPRO_VULNERABILITY_PROVEN:`. The message following the prefix **must** contain a concise description of the vulnerability and include any captured artifacts (e.g., leaked data, error codes, unexpected state) that prove the exploit was successful.
+
**Good Example:**
```java
// Step 3: Trigger the vulnerability, then read the file again.
@@ -27,11 +29,10 @@
String fileContent = readFile("/data/data/com.victim.app/files/secret.txt");
// The test fails because the device is vulnerable, and the failure message
// contains the proof (the content of the secret file).
- Assert.fail("Vulnerability Verified: Successfully read protected file. Content: '" + fileContent + "'");
+ Assert.fail("AUTOREPRO_VULNERABILITY_PROVEN: Successfully read protected file. Content: '" + fileContent + "'");
} catch (ThisVulnSpecificException e) {
Log.i(TAG, "protected against reading protected file", e);
}
-
```
By following this pattern, the resulting test provides a clear, verifiable, and high-impact demonstration of the vulnerability.
diff --git a/templates/app-instrumented-only/AGENTS.md b/templates/app-instrumented-only/AGENTS.md
index 172e49e..c6a7636 100644
--- a/templates/app-instrumented-only/AGENTS.md
+++ b/templates/app-instrumented-only/AGENTS.md
@@ -20,6 +20,8 @@
The final assertion message (for a vulnerable device) **should include the captured artifact** as proof when possible.
+The assertion message **must** begin with the exact prefix `AUTOREPRO_VULNERABILITY_PROVEN:`. The message following the prefix **must** contain a concise description of the vulnerability and include any captured artifacts (e.g., leaked data, error codes, unexpected state) that prove the exploit was successful.
+
**Good Example:**
```java
// Step 3: Trigger the vulnerability, then read the file again.
@@ -27,11 +29,10 @@
String fileContent = readFile("/data/data/com.victim.app/files/secret.txt");
// The test fails because the device is vulnerable, and the failure message
// contains the proof (the content of the secret file).
- Assert.fail("Vulnerability Verified: Successfully read protected file. Content: '" + fileContent + "'");
+ Assert.fail("AUTOREPRO_VULNERABILITY_PROVEN: Successfully read protected file. Content: '" + fileContent + "'");
} catch (ThisVulnSpecificException e) {
Log.i(TAG, "protected against reading protected file", e);
}
-
```
By following this pattern, the resulting test provides a clear, verifiable, and high-impact demonstration of the vulnerability.
diff --git a/templates/app-with-activity/AGENTS.md b/templates/app-with-activity/AGENTS.md
index 172e49e..c6a7636 100644
--- a/templates/app-with-activity/AGENTS.md
+++ b/templates/app-with-activity/AGENTS.md
@@ -20,6 +20,8 @@
The final assertion message (for a vulnerable device) **should include the captured artifact** as proof when possible.
+The assertion message **must** begin with the exact prefix `AUTOREPRO_VULNERABILITY_PROVEN:`. The message following the prefix **must** contain a concise description of the vulnerability and include any captured artifacts (e.g., leaked data, error codes, unexpected state) that prove the exploit was successful.
+
**Good Example:**
```java
// Step 3: Trigger the vulnerability, then read the file again.
@@ -27,11 +29,10 @@
String fileContent = readFile("/data/data/com.victim.app/files/secret.txt");
// The test fails because the device is vulnerable, and the failure message
// contains the proof (the content of the secret file).
- Assert.fail("Vulnerability Verified: Successfully read protected file. Content: '" + fileContent + "'");
+ Assert.fail("AUTOREPRO_VULNERABILITY_PROVEN: Successfully read protected file. Content: '" + fileContent + "'");
} catch (ThisVulnSpecificException e) {
Log.i(TAG, "protected against reading protected file", e);
}
-
```
By following this pattern, the resulting test provides a clear, verifiable, and high-impact demonstration of the vulnerability.
diff --git a/templates/app-with-activity/submission/appTest/src/main/java/com/android/security/DeviceTest.java b/templates/app-with-activity/submission/appTest/src/main/java/com/android/security/DeviceTest.java
index 37dd0be..33eed7e 100644
--- a/templates/app-with-activity/submission/appTest/src/main/java/com/android/security/DeviceTest.java
+++ b/templates/app-with-activity/submission/appTest/src/main/java/com/android/security/DeviceTest.java
@@ -101,7 +101,7 @@
// we fail the test and report the artifact as proof. This test SHOULD fail on a
// vulnerable device.
if ("SECRET_DATA_ACCESSED".equals(finalArtifact)) {
- fail("Vulnerability proven: Leaked sensitive data: '" + finalArtifact + "'");
+ fail("AUTOREPRO_VULNERABILITY_PROVEN: Leaked sensitive data: '" + finalArtifact + "'");
}
// If the exploit failed, the artifact will be "ACCESS_DENIED". In this case, the
diff --git a/templates/same-app-id/AGENTS.md b/templates/same-app-id/AGENTS.md
index 172e49e..c6a7636 100644
--- a/templates/same-app-id/AGENTS.md
+++ b/templates/same-app-id/AGENTS.md
@@ -20,6 +20,8 @@
The final assertion message (for a vulnerable device) **should include the captured artifact** as proof when possible.
+The assertion message **must** begin with the exact prefix `AUTOREPRO_VULNERABILITY_PROVEN:`. The message following the prefix **must** contain a concise description of the vulnerability and include any captured artifacts (e.g., leaked data, error codes, unexpected state) that prove the exploit was successful.
+
**Good Example:**
```java
// Step 3: Trigger the vulnerability, then read the file again.
@@ -27,11 +29,10 @@
String fileContent = readFile("/data/data/com.victim.app/files/secret.txt");
// The test fails because the device is vulnerable, and the failure message
// contains the proof (the content of the secret file).
- Assert.fail("Vulnerability Verified: Successfully read protected file. Content: '" + fileContent + "'");
+ Assert.fail("AUTOREPRO_VULNERABILITY_PROVEN: Successfully read protected file. Content: '" + fileContent + "'");
} catch (ThisVulnSpecificException e) {
Log.i(TAG, "protected against reading protected file", e);
}
-
```
By following this pattern, the resulting test provides a clear, verifiable, and high-impact demonstration of the vulnerability.
diff --git a/templates/single-ndk/AGENTS.md b/templates/single-ndk/AGENTS.md
index 172e49e..c6a7636 100644
--- a/templates/single-ndk/AGENTS.md
+++ b/templates/single-ndk/AGENTS.md
@@ -20,6 +20,8 @@
The final assertion message (for a vulnerable device) **should include the captured artifact** as proof when possible.
+The assertion message **must** begin with the exact prefix `AUTOREPRO_VULNERABILITY_PROVEN:`. The message following the prefix **must** contain a concise description of the vulnerability and include any captured artifacts (e.g., leaked data, error codes, unexpected state) that prove the exploit was successful.
+
**Good Example:**
```java
// Step 3: Trigger the vulnerability, then read the file again.
@@ -27,11 +29,10 @@
String fileContent = readFile("/data/data/com.victim.app/files/secret.txt");
// The test fails because the device is vulnerable, and the failure message
// contains the proof (the content of the secret file).
- Assert.fail("Vulnerability Verified: Successfully read protected file. Content: '" + fileContent + "'");
+ Assert.fail("AUTOREPRO_VULNERABILITY_PROVEN: Successfully read protected file. Content: '" + fileContent + "'");
} catch (ThisVulnSpecificException e) {
Log.i(TAG, "protected against reading protected file", e);
}
-
```
By following this pattern, the resulting test provides a clear, verifiable, and high-impact demonstration of the vulnerability.
