Date: Tue, 19 Mar 2024 03:30:29 -0400 (EDT) Message-ID: <219622148.13.1710833429719@wilbert.sei.cmu.edu> Subject: Exported From Confluence MIME-Version: 1.0 Content-Type: multipart/related; boundary="----=_Part_12_1896531185.1710833429717" ------=_Part_12_1896531185.1710833429717 Content-Type: text/html; charset=UTF-8 Content-Transfer-Encoding: quoted-printable Content-Location: file:///C:/exported.html
The CERT manifest files (=E2=80=9Ccert_manifest.xml=E2=80=9D and= =E2=80=9Ccert_juliet.csv=E2=80=9D) are authored by Lori Flynn, David Svobo= da, and Andrew Kotov.
Download them (version 1) here.
These files can be used by static analysis tool developers to test their= coverage of (some of the) CERT Secure Coding Rules for C, using many of 61= ,387 test cases in the Juliet test sui= te v1.2. The format of =E2=80=9Ccert_manifest.xml=E2=80=9D is a sli= ghtly-modified version of the SARD man= ifest format (https://samate.nis= t.gov/SRD/resources/sard_schema.xsd), designed to enable users of t= he SARD manifest to easily also use this new CERT manifest.
To determine which test suite data could be used for CERT Secure Coding = Rules, we used precise mappings (see here and here for precise mapping information) between CERT= Secure Coding Rules and CWEs, combined with analysis of the Juliet Test Su= ite metadata, and occasionally examining the test suite files (e.g., to che= ck if the type of a variable was a short or a long).
The file =E2=80=9Ccert_manifest.xml=E2=80=9D is modeled after Juliet ent= ries in the SARD manifest. It differs in two major ways:
Generally, we tried not to modify the order of existing attribute fields= , so users of the SARD manifest could most easily also use our manifest (in= case their parsers rely on attribute order). Also, we used test suite attr= ibutes with values as in the original Juliet SARD manifest, so the particul= ar version of Juliet files can be identified.
Additional details about how our manifest differs from the SARD manifest= , with reasons:
alternate-taxonomy
. (alternate-taxo=
nomy=3D=E2=80=9CCERT-C-Standard"
) Purpose is to indicate which alter=
nate code flaw taxonomy (eg. CERT rules, CWEs, MISRA rules, etc.) that info=
rmation will be provided for, as opposed to the code flaw taxonomy that the=
test suite was originally designed to test.SubmissionDate-alternate-taxonomy
. (SubmissionDate-alternate-taxonomy=3D2018-09-28
) Purpose is to indic=
ate the date of submission of this manifest to SARD, for potential publicat=
ion on the NIST SARD test suite website. The=
similarly-named attribute SubmissionDate
is specific to the t=
estcase itself, and that was used for all manifest entries.alternate-taxonomy-author
. (alternate-taxonomy-author=3D"Lori Flynn and David Svoboda and And=
rew Kotov"
) Purpose is to identify authors of the new manifest entri=
es. The similarly-named author
attribute is specific to the te=
stcase itself, and that was used for all manifest entries.False
verdicts, we did particular things for=
the following fields and attributes (in bold):=20
fixed
field (same as in the or=
iginal SARD manifest) that identifies where the identified CERT secure codi=
ng rule is not violated=20
verdict
attribute, we use t=
he value False (verdict=3D=E2=80=9DFalse=E2=80=9D
).numberOfFiles. (numberOfFiles=3D"1"
) The =
purpose of this field for file entries with True
verdicts is t=
o indicate how many files are in a testcase. As an initial estimate, in checksum. (checksum =3D=E2=80=9D<SHA1_HASH>=
;=E2=80=9D)
The purpose of this attribute is to uniquely identify th=
e file. The other SARD file entries for checksum were derived using SHA1, s=
o we derived a checksum value by running sha1sum.size. (size =3D=E2=80=9D<SIZE>=E2=80=9D) The purpose of this attribute is to identify the number of bytes in t=
he file. To get this number, we ran the following command in a bash shell: =
wc -c
id
, (id=3D"10000000"
)=
The purpose of this field is to uniquely identify the testcase ID. Initial=
ly, we start with the first ID at 10000000 (a number larger than any id in =
the current SARD manifest), then increase each by 1. These are placeholders=
, as SARD assigns their own testcase ids.testcase
field: =
id=3D=E2=80=9D86=E2=80=9D, submissionDate=3D"2013=
-05-20",
status=3D"Candidate"
True
(=E2=80=9Cmixed=
code>=E2=80=9D) verdicts: alternate-taxonomy, SubmissionDate-alternate-taxonomy,
and alte=
rnate-taxonomy-author
.
The file =E2=80=9Ccert_juliet.csv=E2=80=9D contains True and False infor= mation for Juliet test suite and CERT Secure Coding Rules, but in a sparser= format than the .xml file.
It has entries of 2 types:
<CERT_RULE>, True, <JULIET_FILEPATH>, <SINGLE_LINE=
>, <CWE>
<CERT_RULE>, False, <JULIET_FILEPATH>, <LINE_RANGE=
>, <CWE>
The filepaths for type-2 (False
) are different from t=
he filepaths for type-1 (True
), for the same files. The type-1=
filepath is taken from the Juliet test suite's manifest XML file obtained =
from the SARD site (https://samate.nist.gov/SRD/testsui=
te.php) by downloading the SARD-type manifest, The type-2 filepath=
is taken from the filepath starting at the "testcases"
d=
irectory from the source code in the Juliet test suite obtained the Juliet-=
standalone-test-suite way. To explain: the key to getting =
the different versions (Juliet-standalone-test-suite or SARD-type) of sourc=
ecode and manifest is how you download (from the same webpage!) from here: =
https://samate.=
nist.gov/SRD/testsuite.php