RayveLabs/RayveNet
A network-discovery walkthrough

See what's really on your network.

Practice on a simulated network where ground truth is known. Then scan your own and see why a real network won't let you measure what you missed.

In gratitude to Dr. Ravinder Mallarapu · Analytical Tools for Cyber · SEAS-8414 · DC9 · The George Washington University
GWU · SEAS-8414 · Lecture

Vulnerability Assessment, Correlation & Prioritization

A lecture overview by Dr. Ravinder Mallarapu

Analytical Tools for Cyber · SEAS-8414 · DC9 · The George Washington University

Discovery tells you what is on the network. This lecture is about the harder question that follows: of everything you found, what actually puts the mission at risk, and in what order should you fix it? We move from raw asset fingerprints to a defensible, ranked remediation plan.

By the end of this lecture you should be able to
  • Turn an asset fingerprint into a set of candidate vulnerabilities
  • Read a CVSS vector and explain what actually drives the score
  • Separate severity from real-world exploitation likelihood
  • Confirm a finding with direct evidence instead of trusting a guess
  • Rank findings so the most dangerous work rises to the top

1 From discovery to assessment

A discovered host is a hypothesis, not a finding. Knowing that a device runs a particular product and version is the starting point — assessment is the work of asking which known weaknesses that software carries, and whether they are actually reachable and relevant here. Skipping this step is how teams drown in noise.

2 Identification — CPE and the CVE match

We name software with a structured identifier (a CPE) so it can be matched against the public vulnerability record (the CVE list). The match is powerful but blunt: version ranges overlap, back-ported patches hide real state, and one string can map to dozens of CVEs. Treat the result as a candidate list to be narrowed — never a verdict.

Watch for: false positives from version-only matching — a vendor may patch the flaw without changing the banner.

3 Severity is not likelihood

A CVSS base score describes how bad an exploit would be — remote? no privileges? no user interaction? It says nothing about whether anyone is actually using it. EPSS estimates the probability a vulnerability will be exploited in the near term, and the CISA KEV catalog lists what is being exploited right now. The three answer different questions, and you need all three.

4 Correlation — confirming with evidence

The strongest finding is corroborated by more than one independent signal: the version match, a service banner you pulled yourself, observed protocol behavior, traffic patterns. When several evidence paths agree, confidence rises and false positives fall. Direct evidence beats inference every time.

5 Prioritization — the remediation queue

Finally we triage like an operator with limited hours. Anything actively exploited (KEV) jumps the line; then weight by exploitation likelihood (EPSS) and by raw severity (CVSS) scaled to how critical the affected asset is to the mission. The output is a single ordered list that tells the team exactly what to touch first.

Continue to the lab →
Original lecture overview prepared for rayvelabs.com. It summarizes the concepts of the SEAS-8414 module in original wording and does not reproduce Dr. Mallarapu’s slides, notes, or graded materials.
GWU · Dr M Lab

The Phase 3 lesson, rebuilt to run.

A from-scratch recreation of the vulnerability-assessment lesson — the full reasoning path from a device fingerprint to a defensible remediation queue, with every concept made runnable: a written walkthrough, a Python toolkit you can paste, and a live console. The graded coursework stays on Blackboard; everything on this page is original.

Instructor Dr. Ravinder Mallarapu · Analytical Tools for Cyber · SEAS-8414 · DC9 · The George Washington University
The lesson · rebuilt to teach

From a fingerprint to a defensible queue

Phase 3 is not “run a scanner and read the score.” It is a reasoning discipline: turn what you know about a device into a list of candidate weaknesses, test each one until the evidence is honest, then rank what survives by what an attacker can actually reach and break. Seven steps — each one runnable in the toolkit and console below.

00
Four words that keep you honest

Most bad vulnerability reports come from collapsing these four states into one. Hold them apart and the rest of the method follows on its own.

01Candidate

A weakness that might apply — a CPE match, a banner, a template hit. A lead, never a verdict.

02Confirmed

A candidate backed by direct or strongly corroborating evidence on the actual host.

03False positive

A candidate that does not survive triage — wrong version, backported patch, feature not enabled.

04Prioritized

A confirmed finding chosen for action now, under real exposure and consequence.

Every finding also carries five things you should be able to name on demand: observation (what a tool saw), state (source, method, confidence, validation status), decision (candidate / confirmed / false positive / accepted), guardrail (no promotion without applicable evidence), and an action handoff.

·
Five assumptions you work under
  • No single method is authoritative. A database match, an active probe, and a successful login each see something the others cannot.
  • Zero CVEs is not zero risk. It can mean a weak CPE, thin database coverage, or a firmware nobody indexed — not a clean host.
  • A high score is not an automatic first action. A 9.8 nobody is exploiting can wait behind a 7.5 that is already in the wild.
  • An untested host is not a safe host. “Found nothing,” “never ran,” and “skipped by policy” are three different sentences.
  • A working default login is your strongest single signal. It is direct proof, not a hypothesis — treat it as top of queue.
01
Turn identity into applicability — the CPE

A CPE 2.3 string is how you ask the world, “is this exact product known to be vulnerable?” Its precision decides everything downstream: too broad and you drown in unrelated advisories, too narrow and you miss adjacent builds.

What raises CPE qualityWhy it matters
Source strengthAn ONVIF or TLS identity beats an HTTP header, which beats a generic OUI guess.
Version precisionA specific build sharpens the CVE set; a wildcard version blurs it into noise.
Layer claritySoftware, firmware, and hardware CPEs imply different fixes — keep them distinct.
Kept alternativesWhen identity conflicts, preserve both candidates rather than averaging them away.

One practical guardrail: do not feed OS-level CPEs (a broad linux or android match) into actionable CVE lookups — they pull thousands of advisories that have nothing to do with the device in front of you.

02
Ask the database — NVD, read carefully

Query NVD with the exact cpeName first; on a miss, fall back to a broader virtualMatchString prefix. Cache by CPE and rate-limit politely — identical CPEs across hosts should resolve to a single lookup. But the real skill is reading the record:

FieldHow to read it
Affected productsThe field that matters most. It names the CPE criteria the CVE actually applies to — check your version against it.
DescriptionA public claim, often product-level not version-level. Necessary, never sufficient.
Severity + vectorA standardized estimate. Keep the vector; you will decompose it next.
CWEA weakness-category hint, useful for grouping and generic mitigations.
ReferencesPatch > Vendor Advisory > Mitigation tags give you the fix hierarchy.

A recent modified date does not mean newly exploitable, and an old published date does not mean low priority. Dates organize; they do not rank.

03
Read CVSS as structure, not a number

A 9.8 means nothing until you can explain it. The base vector is a claim about the attack path and the impact — learn to read it metric by metric so two findings with the same score can still earn different priorities.

MetricReads as
AV — Attack VectorNetwork / Adjacent / Local / Physical — how close must the attacker be?
AC — Attack ComplexityLow / High — are special conditions required?
PR — Privileges RequiredNone / Low / High — authenticated or not?
UI — User InteractionNone / Required — must a victim do something?
S, C, I, AScope, then Confidentiality / Integrity / Availability impact — what is actually lost.

Bands (0·1–3·9 Low, 4–6·9 Medium, 7–8·9 High, 9+ Critical) are general indicators, not rules. AV:N/AC:L/PR:N/UI:N means remote, trivial, unauthenticated, no victim action — that is the 9.8.

04
Gather evidence from four different angles

No method is authoritative, so you run several and compare. They trade breadth against directness, and passivity against intrusiveness:

MethodWhat it observesPosture
NVD / CVECPE-to-CVE association from a public database.Passive, broad
OpenVASNetwork probe + vulnerability-test result.Active
NucleiTemplate-driven check of target behavior.Active, precise
Default credsA successful authentication on the host.Direct proof

A CPE match is a hypothesis; a banner you pulled yourself, or a login that succeeded, is evidence. The closer a method gets to the host, the more it can confirm — and the more carefully you must run it. Active and credential checks stay off by default for a reason.

05
Correlate — without double-counting

Three things can happen when methods meet on one host. Name which one you are looking at before you write the finding.

Convergence

Different methods support the same claim (CPE match and an active template fires). The strongest case — but count it once.

Unique coverage

One method finds what no other can — a default-credential login with no CVE attached. Still fully actionable.

Contradiction

Methods disagree (CPE says vulnerable, the active probe fails). Do not average — record the conflict and resolve identity first.

Rank the strength of what you have on a confidence ladder — a scaffold for judgment, not a replacement for it:

Evidence you holdConfidence
Successful authenticationHigh
Active behavior check + matching identityHigh
Single active tool, clear target behaviorMedium–high
Exact CPE + version, no active confirmationMedium
Wildcard CPE, no versionLow–medium
Broad family match + conflicting identityLow
06
Triage the false positives

A candidate that does not apply is noise that costs trust. Most false positives come from one of five places — each with a cheap check that resolves it:

SourceWhat happenedThe check
CPE too broadThe product family is wider than this device.Is the device in the CVE’s affected-CPE list?
Backported patchThe banner is stale; the fix is already in.Vendor firmware / patch-branch history.
Stale bannerThe service string is simply outdated.A second, independent version source.
Config-dependentVulnerable only when a feature is enabled.Is that setting actually present?
Misread probeAn active response was interpreted wrongly.A safe second check.

Suppress at the (cve_id, host) pair level, never globally — and write the reason. “False positive” is weak; “CVE applies to firmware <V5.5.800, device reports V5.7.1, vendor confirms patched branch” is reusable.

07
Prioritize into action bands

Rank by exploitation reality, not raw score. Five inputs decide the band: severity, evidence strength, exposure (can an attacker reach it?), device role (what it controls), and consequence (what breaks if it goes).

Act nowHigh consequence + strong evidence. Patch, isolate, rotate credentials, or disable the exposure today. A camera taking default RTSP creds on a user subnet lives here.
Validate nextSevere but uncertain. Gather one safe piece of confirming evidence before you commit. Broad-family CVEs on an isolated VLAN belong here.
ScheduleReal finding, lower urgency. Add it to the maintenance plan with an owner and a date.
MonitorWeak or externally dependent. Track it until more evidence arrives — e.g. a cloud-only device with sparse local signal.
AcceptKnown risk under policy. Record the owner, the reason, and a review date. Acceptance is a decision, not a shrug.
What you hand forward

The next phase (attack-graph analytics) is only as good as the packet you pass it. An edge built on a confirmed default login is worth far more than one built on a wildcard CPE — so the handoff carries provenance, not just a verdict:

  1. Identity + confidence for the host.
  2. CPE candidates with their precision.
  3. CVE candidates with source and affected-product evidence.
  4. CVSS base score + vector.
  5. Active findings (OpenVAS, Nuclei) with method provenance.
  6. Default-credential findings as direct authentication proof.
  7. Validation status — confirmed / false positive / accepted / unresolved.
  8. Priority band + rationale.
  9. Unresolved contradictions, stated plainly.
Lesson → Lab

Where each idea becomes hands-on

The lab is the lesson, made concrete. The course runs it as six exercises — each one takes a concept from above, gives you a real call to make, and produces evidence you reason over (the skill is interpretation, not API tourism). Below: the exercise, the lesson it practices, the canonical lab endpoint, and a button that drops you into the matching view of the live console.

Ex 1
Pre-commitment — pick hosts, state expectations
Practices the four states & the evidence model — decide what would confirm or break each guess before you look.
GET /v1/scanning/smart-scan/{id}/results
Ex 2
Broad vs. narrow CPE
Practices 01 CPE applicability + 02 NVD lookup — watch the CVE set widen and blur, or sharpen and miss.
POST /v1/cves/lookup
Ex 3
Decompose the CVSS vector
Practices 03 CVSS as structure — open any asset and read the vector as an attack path, not a number.
GET /v1/cves/{cve_id}
Ex 4
Correlate four evidence methods on one host
Practices 04 evidence streams + 05 correlation — spot convergence, unique coverage, and contradiction.
GET /v1/cves/search?keyword=
Ex 5
Explain a zero-result host
Practices 06 false-positive triage — is “no CVEs” a clean host, a weak CPE, or a coverage gap?
GET /v1/cves/search?keyword=
Ex 6
Write the analyst memorandum
Practices 07 prioritization + the handoff — separate output, rationale, evidence strength, and action into a ranked queue.
results → ranked queue
How to read this: work top to bottom. Each row’s button jumps the console to the view where that exercise’s output is visible — so you can move from the idea, to the command, to the evidence without leaving the page.
Phase 3 · Week 03 · Hands-on
Run the workflow on one host
SEAS-8414· Worked walkthrough· public APIs & tooling, no backend

The lesson above is the reasoning; this is the practice. Take a single host through the whole flow against live, public data sources — match software to a CPE, decompose the CVSS vector, correlate exploit signals, confirm with direct evidence, then assemble the queue. Running example throughout: CVE-2021-41773 (Apache httpd 2.4.49 path traversal). Swap in your own host as you go.

What you'll practice
  • Comparing broad vs. narrow CPE matching strategies
  • Decomposing a CVSS vector by hand instead of trusting the score
  • Weighing evidence methods on a single host (CVE/NVD, EPSS, CISA KEV, direct probing, nuclei)
  • Diagnosing weak or empty CVE results
  • Treating default credentials as high-confidence evidence
  • False-positive triage, then correlation and convergence
  • Building a prioritized remediation queue
The walkthrough

Six short steps, each with a command you can run now. The graded runtime and write-up are Dr. Mallarapu’s coursework on Blackboard — these are an original, public-tooling worked version.

Ex 1Match the software to a CPE — broad vs. narrow

From observed evidence (a banner reading Apache/2.4.49) you can build a broad CPE that matches the whole product line, or a narrow one pinned to the exact version. Query NVD both ways and watch how the CVE count changes — broad over-reports, narrow can miss adjacent versions.

// narrow — exact version curl 'https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:a:apache:http_server:2.4.49:*:*:*:*:*:*:*' // broad — whole product curl 'https://services.nvd.nist.gov/rest/json/cves/2.0?cpeName=cpe:2.3:a:apache:http_server:*:*:*:*:*:*:*:*'
Ex 2Read the CVSS vector as structure, not a score

The 9.8 is a summary; the vector is the argument. Decompose CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H metric by metric:

AV:N attack vector = Network -> reachable remotely AC:L attack complexity = Low -> no special conditions PR:N privileges required = None -> unauthenticated UI:N user interaction = None -> no victim action needed S:U scope = Unchanged C:H I:H A:H full confidentiality / integrity / availability loss

Read together: trivially exploitable, remote, unauthenticated, total impact — that's the 9.8.

Ex 3Correlate exploit signals — EPSS + CISA KEV

Base score says how bad if exploited. EPSS estimates the probability it will be, and the CISA KEV catalog tells you it already is, in the wild. Pull both:

// EPSS probability for this CVE curl 'https://api.first.org/data/v1/epss?cve=CVE-2021-41773' // is it on CISA's Known Exploited Vulnerabilities list? curl -s https://www.cisa.gov/sites/default/files/feeds/known_exploited_vulnerabilities.json \ | grep -i CVE-2021-41773

A KEV hit is the strongest single signal you have — it outweighs a high base score with no real-world activity.

Ex 4Confirm with direct evidence on the host

API data is inference; a response from the host is evidence. Two high-confidence checks — a default-credential probe and a version confirmation:

// default credentials accepted? 200 = high-confidence finding curl -s -o /dev/null -w '%{http_code}\n' -u admin:admin http://TARGET/ // confirm the server banner you matched on curl -sI http://TARGET/ | grep -i '^Server:'

Default creds are treated as high-confidence evidence: little ambiguity, directly actionable.

Ex 5Scan one host with nuclei

Template scanners map known CVEs to detection logic. Run the high/critical CVE templates against your one host:

nuclei -u http://TARGET -tags cve -severity critical,high

Treat hits as leads to verify, not verdicts — carry each one back through the CVSS / EPSS / KEV read above.

Ex 6Build a prioritized remediation queue

Rank by exploitation reality, not raw score. A 7.5 that's actively exploited outranks a 9.8 nobody is touching:

1. On CISA KEV -> patch now 2. High EPSS (>= ~0.5) -> patch this cycle 3. High CVSS, low EPSS -> schedule 4. Default creds present -> rotate immediately (independent of CVE) 5. Everything else -> backlog
Walkthrough screencast
Drop your own screencast here — replace this slot with a <video> or embed once you've recorded your run.
Academic integrity: the graded homework and the official lab runtime are Dr. Mallarapu's coursework, distributed through Blackboard. None of it is reproduced here — this lesson, walkthrough, and toolkit are original, written from the public method (NVD, FIRST.org EPSS, the CISA KEV catalog) the course exercises.
Python toolkit · copy & run

The assessment pipeline, in five short scripts

The console above shows the result. These scripts show the method — the same vulnerability-assessment, correlation and prioritization flow the lab asks you to build, written from scratch against public data sources so you can run each step and watch it work.

1 Fingerprint → candidate CVEs

Turn an asset fingerprint into a CPE 2.3 string, then ask NVD which CVEs are recorded against that exact product and version.

import requests def cpe_for(vendor, product, version): return f"cpe:2.3:a:{vendor}:{product}:{version}:*:*:*:*:*:*:*" def cves_for_cpe(cpe): r = requests.get( "https://services.nvd.nist.gov/rest/json/cves/2.0", params={"cpeName": cpe, "resultsPerPage": 50}, timeout=30, ) r.raise_for_status() vulns = r.json().get("vulnerabilities", []) return [v["cve"]["id"] for v in vulns] cpe = cpe_for("apache", "http_server", "2.4.49") print(cves_for_cpe(cpe)) # -> ['CVE-2021-41773', ...]
2 Decompose the CVSS vector

A score of 9.8 means nothing until you can explain it. Pull the base metrics out of the v3.1 vector so you can say why it is critical — remote, no privileges, no user action.

def parse_cvss(vector): body = vector.removeprefix("CVSS:3.1/") parts = dict(p.split(":") for p in body.split("/")) legend = { "AV": {"N": "Network", "A": "Adjacent", "L": "Local", "P": "Physical"}, "AC": {"L": "Low", "H": "High"}, "PR": {"N": "None", "L": "Low", "H": "High"}, "UI": {"N": "None", "R": "Required"}, "C": {"H": "High", "L": "Low", "N": "None"}, "I": {"H": "High", "L": "Low", "N": "None"}, "A": {"H": "High", "L": "Low", "N": "None"}, } return {k: legend.get(k, {}).get(v, v) for k, v in parts.items()} print(parse_cvss("CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:H/A:H")) # {'AV': 'Network', 'AC': 'Low', 'PR': 'None', 'UI': 'None', ...}
3 Enrich with EPSS + CISA KEV

Severity is not likelihood. EPSS gives the 30-day exploitation probability; the CISA KEV catalog tells you what is already being exploited in the wild. Correlate both.

import csv, io, requests def epss_score(cve): r = requests.get("https://api.first.org/data/v1/epss", params={"cve": cve}, timeout=30) rows = r.json().get("data", []) return float(rows[0]["epss"]) if rows else 0.0 def kev_set(): url = ("https://www.cisa.gov/sites/default/files/csv/" "known_exploited_vulnerabilities.csv") text = requests.get(url, timeout=30).text return {row["cveID"] for row in csv.DictReader(io.StringIO(text))} kev = kev_set() for cve in ["CVE-2021-41773", "CVE-2019-10929"]: print(cve, f"epss={epss_score(cve):.3f}", "KEV" if cve in kev else "-")
4 Confirm with direct evidence

A CPE match is a hypothesis. Grab the live service banner and verify the version that is actually running before you raise a finding — this is what kills false positives.

import socket def banner(host, port, probe=b"HEAD / HTTP/1.0\r\n\r\n"): with socket.create_connection((host, port), timeout=5) as s: s.sendall(probe) return s.recv(2048).decode("latin-1", "replace") def confirms(host, port, needle): return needle.lower() in banner(host, port).lower() print(confirms("10.20.7.42", 80, "Apache/2.4.49")) # True -> CVE applies
5 Build the prioritized remediation queue

Finally, triage like an operator: anything in KEV jumps the line, then weight by EPSS likelihood and by CVSS scaled to how critical the affected asset is.

CRIT_WEIGHT = {"low": 1.0, "medium": 1.5, "high": 2.0, "critical": 3.0} def priority(f): score = f["cvss"] * CRIT_WEIGHT[f["asset_criticality"]] score += f["epss"] * 10 # exploitation likelihood if f["in_kev"]: score += 100 # actively exploited -> top of queue return score queue = sorted(findings, key=priority, reverse=True) for f in queue[:10]: print(f"{f['cve']:<16} {f['hostname']:<22} score={priority(f):6.1f}")
Sources: NVD CVE API, FIRST.org EPSS, and the CISA Known Exploited Vulnerabilities catalog — all public. This is original teaching code written for this site; it does not reproduce Dr. Mallarapu's graded assignment, only the public method it exercises.
Lab runtime · interactive

A working build of the Chapter 3 Discovery & Evidence workflow — six linked views from asset inventory through to a live, ranked remediation queue. Data is deterministic and generated locally in your browser; there is no backend and nothing leaves this page.

Chapter 1 · RayveNet Lectures of Dr. Ravinder Mallarapu · SEAS-8414 · GW · interactive lab by Rayve Malhotra, student of the course · approved by Dr. Mallarapu

Network Discovery & Asset Inventory

The first substantive act of network defense. Before vulnerability matching or attack-path reasoning, defenders answer four foundational questions — simple in enterprise IT, hard in IoT/OT.

What devices exist?
What evidence supports each claim?
Which blind spots remain?
How much uncertainty?
SECTION 01

The Fish-Tank Problem

Why the most boring device is often the entry point.

At 2:13 a.m., a casino's security team did not recognize the fish-tank thermometer as a network asset. Cameras, payment terminals, guest records — those were classified critical. The thermometer was overlooked. The breach did not begin with a missed patch; it began with a lapse in attention. An ordinary device became the critical access point.

This pattern recurs. Mirai exploited a vast supply of embedded systems with default credentials. Verkada showed that cameras governed as devices, not as a fleet, could become a globally reachable surveillance substrate. Oldsmar and Target showed the same structural weakness outside consumer IoT — building systems and vendor links become strategically central the moment segmentation and asset awareness fail.

Match each incident to the weakly-governed device that enabled it. Click an option for each. The reveal explains what discovery would and would not have addressed.

2017 · Casino · North America
The Casino Breach
Which device became the foothold into the high-roller database?
CCTV camera Fish-tank thermometer Badge reader VPN gateway
Discovery would have made the thermometer visible as an inventoried device with a known vendor and protocol surface. It would not have prevented the breach by itself — segmentation, credential governance, and monitoring still had to follow. The lesson: discovery is the precondition, not the control.
2016 · Botnet · Global
Mirai
What device class was hijacked at massive scale?
IoT cameras & DVRs (Telnet + default creds) Home routers POS terminals VOIP phones
Mirai's strength came from scale, not technical novelty. Inventory and exposure mapping would have made the population visible before abuse at scale. Vendor firmware hardening remained the controlling fix — discovery sets the conditions for it.
2021 · Camera platform · Global
Verkada
What turned ordinary cameras into a global surveillance substrate?
Centralized fleet with shared privileged credentials Firmware malware Physical access Zero-day exploit
Verkada cameras were known locally but not governed as a fleet with blast-radius visibility. Discovery would have surfaced scope and exposure; credential governance remained the controlling failure.
2021 · Water plant · Oldsmar FL
Oldsmar SCADA
What was the exposed device class?
SCADA / HMI with remote management & weak access Smart meter PDF reader Email server
Discovery could have surfaced the exposed interface and its cross-zone placement. Segmentation and multifactor authentication were the controlling fixes — discovery alone would not have prevented the intrusion attempt.
2013 · Retail · United States
Target
What device class hosted the initial vendor foothold?
HVAC vendor access (a heating & cooling system) Store cameras POS terminals (initial vector) Customer Wi-Fi
The vendor link was not "important" by executive ranking. It was important because it sat on a path. Discovery and cross-zone mapping would have surfaced the dependency; segmentation and least privilege were the controlling fixes.

Counterfactual · what discovery would, and would not, have prevented

Discovery is a precondition for many controls — it is rarely the control itself. Honest read of each incident:

Incident
Discovery would have surfaced
Discovery alone would NOT have prevented
Mirai
The exposed Telnet population at scale → vendor firmware hardening could follow.
The default-credential design choice itself. Discovery is detection, not policy.
Verkada
The fleet's centralized blast radius — every camera was reachable from one credential.
The credential-governance failure. A privileged token misused remains misused even on a perfectly inventoried fleet.
Oldsmar
The exposed HMI / remote-management interface and its cross-zone placement.
The weak access control and shared credentials. Segmentation + MFA were the controlling fixes.
Target
The HVAC vendor link and the cross-zone path it sat on.
The poor segmentation that let one foothold reach payment systems. Discovery surfaces dependencies; least-privilege design enforces boundaries.
Casino
The thermometer's presence on the flat network with weak defaults.
The flat-network design and absence of monitoring. An inventoried thermometer on a flat segment is still a foothold.

The Chapter 1 stance: discovery sets the conditions for every later control. It is not a substitute for any of them.

SECTION 02

The Four Foundational Questions

Look easy in enterprise IT. Become hard in IoT and OT.

In enterprise IT, asset lists, agents, and directories make these questions easy. Embedded environments break those assumptions. Devices arrive through facilities, contractors, clinical engineering, building management, vendor drop-ins, and stale projects that never reached formal intake.

Click each question to see why it gets hard.

Question 01
What devices exist?
Sounds obvious — until you can't agent-install.
A cabling closet pump, a clinical infusion device, a contractor-supplied display, a Wi-Fi badge reader. None speak agent. None appear in your CMDB. Existence is inferred from network behavior, not looked up.
Question 02
What evidence supports each claim?
Provenance is the difference between a defensible inventory and a wishful one.
A stale ARP entry, a router observation, a TCP connect success, and a service banner each provide different qualities of evidence. They are not interchangeable. The host record must preserve which method found the host, when, and from where.
Question 03
Which blind spots remain?
A clean scan is not the same as a complete one.
Devices may exist behind routers, behind filters, on different multicast scopes, or simply not have responded during the observation window. Discovery measures what was visible — not what is there.
Question 04
How much uncertainty?
A host found by one stale cache is not the same as one corroborated five ways.
Every later chapter — vulnerability matching, attack-path reasoning, remediation — inherits the inventory's confidence. Discovery must record uncertainty, not launder it into apparent certainty.
SECTION 03

The Analytics Ladder

Discovery is the descriptive base. Every higher rung depends on it.

A vulnerability inventory can only reason over devices first counted. An attack graph can only model paths through devices first counted. A remediation plan can only govern assets first made visible. When Chapter 1 undercounts the environment, every later chapter inherits the error.

Click each rung to see what it asks and which chapter answers it.

DESCRIPTIVE
What exists?
The device population.
Chapter 1 · You are here
DIAGNOSTIC
What is it?
Identity, vendor, model, role.
Chapter 2 · Cascade Lab
DETECTIVE
What's wrong with it?
CVEs, defaults, credential risk.
Chapter 3 · TBD
PREDICTIVE
What paths exist?
Attack-graph reasoning.
Chapter 4 · TBD
PRESCRIPTIVE
What to do?
Offensive testing, remediation, simulation.
Chapter 5+ · TBD
SECTION 04

Why Traditional Scanners Fail in IoT & OT

The closed-world assumption is the problem.

Conventional enterprise scanners assume a closed world: assets are already known, agent-capable, directory-linked, and recorded in a current management system. Pick an environment and watch which assumptions hold and which break.

The environment should guide the sensible choice of tools — not the other way around. In IoT and OT settings, a mix of passive observation, infrastructure queries, and targeted active probes is necessary.

The five structural reasons enterprise scanners hit a wall

Click each to see what it means in practice and what discipline it demands.

REASON 01
No agent support
Many embedded systems cannot host endpoint agents at all.
A camera, an infusion pump, a smart thermostat, a building controller — these run firmware, not Windows or Linux you can install software on. The entire agent-based scanning model assumes a controllable, agent-capable endpoint. When that assumption breaks, you fall back to network-side evidence: ARP, mDNS, SSDP, TCP probes. The scanner has to meet the device on the wire, not on the OS.
REASON 02
Protocol diversity
Devices live on mDNS, SSDP/UPnP, RTSP, ONVIF, MQTT, CoAP — not the unicast TCP patterns scanners prefer.
A scanner that only probes 22/80/443/3389 walks past a fleet of cameras, badge readers, sensors, and pumps without seeing them. Each protocol family has its own discovery surface. mDNS uses multicast UDP/5353. SSDP uses UDP/1900 M-SEARCH. RTSP cameras live on TCP/554. MQTT brokers on TCP/1883 or 8883. CoAP on UDP/5683. Discovery in IoT means speaking the dialects of the device population.
REASON 03
Identity is inference, not lookup
No authoritative inventory means identity is reconstructed from OUI prefixes, hostnames, banners, port signatures.
In a managed enterprise, you look up a host in the CMDB and get its model, owner, OS, criticality. In unmanaged fleets, every identity claim has to be inferred from evidence the device emits. An OUI of D4:F5:47 says "this MAC was assigned to Google Inc." — not which Google product. A realm of "Hik" hints at Hikvision firmware — not which model. Identity becomes an evidence problem that Chapter 2 (Cascade Lab) solves with cascading corroboration.
REASON 04
Topology constrains visibility
ARP doesn't cross routers. SNMP needs credentials. Multicast stays local. Every fallback alters evidence quality.
Each method has a scope. ARP sees only the local broadcast domain. mDNS sees only the local multicast scope. Reverse DNS depends on the resolver. SNMP requires community-string or v3 credentials. TCP connect probes only the ports you picked. When you fall back from one method to another, you don't just lose coverage — you change the kind of evidence you have. A row that says "discovered by TCP probe on 192.168.1.0/24 from the gateway" is a different claim from "discovered by local ARP on a laptop joined to Wi-Fi."
REASON 05
Downstream depends on prior discovery
CPE matching, CVE lookup, attack-path reasoning — they all reason over the population Chapter 1 counted.
If a camera never makes it into the inventory, the vulnerability scanner cannot fail to match it — it never knows to look. The cost of an undercounted Chapter 1 cascades through every later chapter. A clean-looking vulnerability report on the wrong population is worse than a noisy report on the right one. This is why discovery is the descriptive foundation: every higher rung inherits its blind spots.
SECTION 05

Discovery as a Measurement Problem

Existence ≠ identity. A scanner does not return truth. It returns partial views.

Defenders gather partial evidence, combine multiple perspectives, estimate what is visible, and consider what may remain hidden. The true device population is D. The defender never directly observes D. Instead, the defender sees method-specific subsets M1, M2, ..., Mk. The discovered population is the union: D̂ = M1 ∪ M2 ∪ ... ∪ Mk. What remains is the discovery gap: D \ D̂.

Meet 192.168.86.42. Right now this is a single ARP line:
? (192.168.86.42) at d4:f5:47:xx:xx:xx on en0 ifscope [ethernet]
Something exists on the local subnet. We have no idea what. As we work through this chapter, evidence accumulates — by the §11 handoff record we'll have a defensible, well-bounded claim about what this host actually is.

D vs D̂ · interactive

Adjust how many devices are out there and how many your methods catch. The gap is what an attacker hunts.

D · real D̂ · found 8 missed 18 caught 2 phantom
C = 0.69 · S = 0.90 · gap = 0.31

Evidence classes · what counts as a host claim?

Not every network hint is sufficient to create an inventory entry. Each class of evidence has a different strength and a different way of going wrong.

CLASS 01 · LOCAL CACHED STATE
ARP cache entry
A recent local mapping between an IP and a MAC address. The OS saw this device on the local broadcast domain.
May be stale or local-only. Cache retention varies. Can't see across routers.
CLASS 02 · INFRASTRUCTURE STATE
Router ARP, SNMP ipNetToMediaPhysAddress
A gateway or managed switch observed a device or binding. Crosses subnet boundaries.
Requires management access. May not cover all segments. Credentials must be governed.
CLASS 03 · LOCAL-LINK SELF-DESCRIPTION
mDNS / SSDP advertisement
A nearby device voluntarily advertised a name, service class, or device description.
Absence is weak — multicast can be filtered, misrouted by VPN tunnels, or suppressed by IGMP snooping.
CLASS 04 · ACTIVE LIVENESS
ICMP echo, TCP connect success
The host responded to a direct probe. Existence is corroborated.
Failure does not prove absence. Many devices deliberately drop ICMP while staying reachable on service ports.
CLASS 05 · SERVICE EVIDENCE
Open port, banner, certificate, device description
The host has a reachable service surface. Identity and vulnerability claims become possible.
Still no identity proof on its own. The Chapter 1 contract preserves the evidence; Chapter 2 interprets it.

The recall formula · why complementary methods matter

If method m detects any given device with probability p_m, and the methods are conditionally independent for a given device, the expected recall R satisfies:

R = 1 − m (1 − p_m) = 0.94

Toggle methods on, slide each one's per-device detection probability, and watch the expected recall update. Notice that each additional method multiplicatively reduces the miss probability — that's why combining cheap, partial methods beats any single tool.

Local ARP
Router / SNMP
mDNS
SSDP
fping ICMP
TCP pilot
Per-method miss probabilities multiplied: 0.06
Expected recall: 0.94
Methods active: 4
Reality check: methods are rarely truly independent. ARP and router ARP are differently scoped observations of the same network state, not independent views. The formula is an upper bound planning approximation — useful for reasoning about complementarity, not as a guarantee.
SECTION 06

Vantage Point — Same Network, Different Worlds

Methods do not just differ in speed. They differ in what kinds of worlds they can see.

A laptop on Wi-Fi, a scanner on a switch port, a host with gateway access, a central VPN collector, and a distributed sensor inside a remote segment will return very different inventories from the same network. The shape of the inventory carries the shape of the vantage point that produced it.

SEGMENT A · 192.168.1.0/24 SEGMENT B · 10.0.10.0/24 SEGMENT C · 10.0.20.0/24 gateway laptop printer cam phone HVAC badge NVR pump PLC HMI sensor switch YOU YOU YOU YOU · over VPN collector
SECTION 07

The Passive-First Discipline

Start with the safest evidence. Escalate only when cheaper methods leave uncertainty.

In sensitive environments — hospitals, factories, water treatment — unexpected network traffic can cause real problems. Even in benign environments, starting with passive evidence improves interpretive quality: a host appearing in cached state versus only after active probing tells you something different about its presence pattern.

Drag the methods into the correct order, from least disturbing (top) to most disturbing (bottom). Hit Check when you're done.

Pool · drag from here
Your ordering · drop here

Method posture · how to read a positive or negative result

The same answer means different things depending on which method gave it. Print this and stick it to the wall before your next discovery run.

Method
Question it asks
Traffic posture
When silence is uninformative
arp -an
Has this host appeared in recent local mappings?
Passive · no new packets. Reads OS-maintained state.
Cache may be empty or stale. Cannot cross routers. Recent rebuilds blank it.
router ARP / SNMP
Did the infrastructure observe the host?
Low-disturbance · single management-plane query.
Access or scope may be missing. SNMP needs credentials. Some segments may be out-of-view.
mDNS · UDP/5353
Did the device announce itself on the local link?
Low-disturbance · passive listen + bounded multicast.
Multicast filtered or misrouted. IGMP snooping on managed switches. Interface binding wrong.
SSDP · UDP/1900
Did the device respond to an M-SEARCH?
Low-disturbance · single multicast or bounded unicast.
SSDP disabled on device. Filtered at the segment edge. VPN tunnel hijack of the default route.
ICMP / fping
Does a direct probe elicit a response?
Active · host-directed traffic.
Filtering. Port choice. Many IoT devices deliberately drop ICMP. No-retry, short-timeout sweeps fail more often.
TCP connect
What reachable service surfaces exist?
More active · bounded port list, Chapter 2 boundary.
You only tested the ports you picked. Closed != absent. RST != alive on other services.
SECTION 08

Silence ≠ Absence

"No response" is one of the easiest signals to misread.

Negative evidence is useful only when the method could have seen the device, assuming it existed and behaved normally. That condition breaks often. ARP cannot cross routers. mDNS cannot cross multicast boundaries. ICMP can be blocked. TCP probes test only the ports you picked.

Your ICMP sweep across 192.168.1.0/24 returned no responses from .42. The other 14 hosts answered. What's the most defensible interpretation?

What does silence from each method actually mean?

Map the alternatives explicitly so you don't collapse them into "the host isn't there."

arp -an
Device has not generated traffic recently (cache aged out)
Device is on a different subnet (ARP doesn't cross routers)
Cache was just cleared (reboot, interface restart)
Wrong interface — VPN or virtual adapter intercepted the route
Device uses MAC randomization and the prior entry is stale
mDNS / SSDP
Device doesn't self-advertise (many OT devices don't)
Multicast filtered by managed switch (IGMP snooping)
Wrong interface bound — Zeroconf is going to a tunnel, not the LAN
VLAN boundary blocks multicast scope
Browse window too short for the device's advertisement interval
ICMP / fping
Firewall on the host or network blocks ICMP
Device deliberately drops echo (very common in IoT)
Short timeout, no retries — packet loss caused a miss
Route asymmetry — reply was lost
Device is alive but only on application protocols
TCP connect
You probed the wrong port set for this device family
Stateless scanner lost the response under packet loss
Firewall returns RST or drops silently
Service is up but bound to a different IP
Asymmetric routing or NAT artifact
SELF-TEST

Five questions to check yourself

Click an answer to see if it's right and why. Score updates as you go.

QUESTION 1 / 5
Your ICMP sweep returned no responses from one host on a /24. The other 14 hosts answered. Which interpretation is defensible?
Correct: B. Many IoT devices deliberately drop ICMP while remaining reachable on HTTP, RTSP, or vendor management ports. ICMP non-response narrows neither existence nor identity. A and D claim certainty the method cannot support; C is unsupported speculation.
QUESTION 2 / 5
Which combination most strengthens a host claim through structural independence?
Correct: B. ARP (link layer), mDNS (multicast self-description), and TCP connect (active service evidence) come from different protocol families — independent observations of the device. A is two views of the same link state. C and D are same-method repetition: they reinforce nothing.
QUESTION 3 / 5
A MAC address 02:1A:2B:3C:4D:5E appears in your ARP cache. What can you confidently infer?
Correct: C. The second-lowest bit of 02 is set, which marks the address as locally administered. There is no IEEE OUI assignment. Identification must fall back to behavioral evidence — JARM, TCP fingerprint, service pattern. B and D are possible causes of locally-administered MACs but not the inference from the MAC alone.
QUESTION 4 / 5
What is the discovery gap?
Correct: B. D is the true device population; D̂ is what your methods saw. The gap is what's real but unseen. It's the part of the network an attacker is most likely to exploit precisely because you don't know it exists.
QUESTION 5 / 5
Why does Chapter 1 refuse to make identity claims, even when the evidence looks compelling?
Correct: C. Same vendor prefix and port set can belong to many products. Chapter 1's job is to admit hosts and preserve evidence — preferring honest uncertainty to false precision. Chapter 2 (Cascade Lab) takes the identity question; Chapter 3 follows.
Drawn from Dr. Ravinder Mallarapu's Chapter 1 lectures · Analytical Tools for Cyber · SEAS-8414 · GW · interactive lab by Rayve Malhotra · approved by Dr. Mallarapu.
SECTION 09

Five Methods, Hands-On

Each tool runs in your browser. Paste real data if you have it.

No method is sufficient alone. Each one carries the shape of its vantage point. Switch between tools below to see how the running case at 192.168.86.42 gains evidence — or stays silent — under each.

ARP Cache Harvester
read existing local state · no new packets
Reading the ARP cache is the best initial step on a local IPv4 subnet — it uses information the operating system already maintains. Paste the output of arp -an from your Mac, arp -a from Windows, or ip neigh from Linux. The tool will parse it, normalize MAC formatting, run an OUI lookup, and filter broadcast/multicast artifacts.
Paste your ARP output
Parsed hosts
Output will appear here

Why normalize? Operating systems format MACs differently: c:ae:7d:1:2:3 vs 0C:AE:7D:01:02:03. The same device shows up as two records if you don't zero-pad and uppercase before merging. Vendor lookups also fail on short-form prefixes.

Try this on YOUR network
# prime the cache (ping every host on your subnet) — about 4 sec for ip in $(seq 1 254); do ping -c 1 -W 50 -q 192.168.1.$ip & done; wait 2>/dev/null # read the populated cache arp -an # copy lines and paste them into the tool above
# prime the cache for ip in $(seq 1 254); do ping -c 1 -W 1 -q 192.168.1.$ip & done; wait 2>/dev/null # read it (modern Linux) ip neigh # or arp -an
# prime the cache in PowerShell 1..254 | ForEach-Object { Test-Connection 192.168.1.$_ -Count 1 -TimeoutSeconds 1 -Quiet } > $null # read it arp -a
Replace 192.168.1 with your own subnet (look at ifconfig / ip addr / ipconfig). The ping pass forces sleeping devices to respond, then ARP shows you the cache. Nothing leaves your machine.

MAC address edge cases that break naive inventory work

An OUI prefix says "this MAC was assigned to vendor X" — not which product, not which user, not whether the device is even there now. Five edge cases that bite real fleets:

02:xx:xx:xx:xx:xx
Locally-administered MAC
The second-lowest bit of the first octet is set. There is no IEEE OUI — the MAC was assigned by software (a hypervisor, a container runtime, a docker bridge, a randomized phone). Vendor lookup returns nothing because there is nothing to look up.
DA:xx:xx:xx:xx:xx
MAC randomization (iOS, Android)
Modern phones rotate their Wi-Fi MAC per-SSID and on reconnects. Two ARP entries with different randomized MACs may be the same device reconnecting. Treat per-MAC counts on mixed user segments as noisy.
00:50:56:xx:xx:xx
VMware virtualized interface
OUI resolves to VMware, but the device behind it is a guest VM — possibly a Windows server, a Linux container, or a vendor-supplied appliance image. The OUI tells you about the hypervisor, not the workload.
FF:FF:FF:FF:FF:FF
Broadcast — never a host
This is the link-layer broadcast address. If it shows up paired with .255 or as an "extra host" in your inventory, your pipeline isn't filtering. Strict admission excludes it; lenient admission flags it as needing review.
empty / incomplete
Intermittent device · stale cache
Battery-powered sensors and low-power radios may only respond briefly. ARP shows them, then ages out. Treat their evidence as timing-dependent, not as proof of constant presence. Repeat discovery is the only honest answer.
Router / SNMP Viewport
when local ARP can't see beyond the broadcast domain, the gateway becomes the witness
A router's ARP table or SNMP ipNetToMediaPhysAddress can expose devices on adjacent segments with far less traffic than broad sweeps. Toggle between local laptop view and gateway view of the same network. Some devices appear only from the gateway.
Local cache · 192.168.86.0/24
Gateway ARP · cross-segment

Three devices visible only from the gateway. On a flat residential network, local ARP would be enough. On a segmented hospital VLAN — building management, lab instruments, wireless bridges — the gateway is often the only witness.

Try this on YOUR network
# walk ipNetToMediaPhysAddress on your gateway (needs SNMP enabled + community string) snmpwalk -v 2c -c public 192.168.1.1 1.3.6.1.2.1.4.22.1.2 # example output: # IP-MIB::ipNetToMediaPhysAddress.1.192.168.1.42 = STRING: d4:f5:47:11:22:33 # IP-MIB::ipNetToMediaPhysAddress.1.10.0.10.5 = STRING: 0:80:a3:11:22:33 # the second row is on a DIFFERENT subnet — visible only from the gateway
# OpenWrt / dd-wrt / typical home router ssh root@192.168.1.1 'cat /tmp/dhcp.leases' # dnsmasq-based ssh root@192.168.1.1 'cat /var/lib/misc/dnsmasq.leases' # each line: timestamp MAC IP hostname client-id
# if your DHCP server is dnsmasq on a Linux box cat /var/lib/misc/dnsmasq.leases # if it's ISC dhcpd cat /var/lib/dhcp/dhcpd.leases | grep -E 'lease|hardware ethernet|client-hostname' # Windows DHCP via PowerShell Get-DhcpServerv4Lease -ScopeId 192.168.1.0
SNMP is the highest-value method in managed environments and the least available in ad-hoc ones. If you don't have an SNMP community string, lease-file scraping over SSH or a DHCP query is the practical fallback.
mDNS Service Browser
UDP/5353 multicast · devices voluntarily announce themselves
mDNS surfaces hostnames, service types, and local-link identity for devices that participate in zero-configuration ecosystems. A response advertising _googlecast._tcp is not a full device identity — it's a capability hint. Click each observed service to see what it implies and what it does not.

The Chapter 1 discipline: admit the host, preserve the service evidence, refuse to overstate. Chapter 2 (Cascade Lab) decides what identity claim, if any, follows from these strings.

Try this on YOUR network
# browse a single service type for 4 seconds dns-sd -B _googlecast._tcp local. # browse everything (very chatty — Ctrl-C after a few seconds) dns-sd -B _services._dns-sd._udp local. # resolve a specific instance found above dns-sd -L "Living Room" _googlecast._tcp local.
avahi-browse -a -t -r # breakdown: # -a all service types # -t terminate after the first complete browse cycle # -r resolve each instance (hostname, IP, port, TXT)
# pip install zeroconf python3 - <<'PY' from zeroconf import Zeroconf, ServiceBrowser import time class Listener: def add_service(self, zc, type_, name): info = zc.get_service_info(type_, name) print(f"{type_:32s} {name} {info.parsed_addresses() if info else ''}") zc = Zeroconf() for svc in ["_googlecast._tcp.local.", "_airplay._tcp.local.", "_ipp._tcp.local.", "_printer._tcp.local.", "_smb._tcp.local."]: ServiceBrowser(zc, svc, Listener()) time.sleep(5) zc.close() PY
If you have a VPN active, mDNS browsing may bind to the tunnel interface instead of your LAN — you'll see nothing. Check with route -n get default (mac) or ip route show (linux) and rebind explicitly if needed.
SSDP / UPnP Parser
UDP/1900 M-SEARCH · device descriptions in XML
SSDP can expose device class and service descriptions. The flow: send M-SEARCH, get a LOCATION header back pointing to a device description URL, fetch the XML, parse it safely (using a hardened XML parser — production code uses defusedxml). Click Run M-SEARCH to walk the flow.
STEP 01
Send M-SEARCH
Bounded unicast to 192.168.86.42:1900
STEP 02
Extract LOCATION
Parse HTTP-over-UDP response for device description URL
STEP 03
Fetch XML
GET the location, treat response as untrusted input
STEP 04
Parse safely
Use a hardened XML parser; record device class
(click Run M-SEARCH to walk the flow)

Why defusedxml? UPnP device descriptions are fetched from devices that may be poorly maintained or hostile. A naive XML parser exposes the scanner to XXE and entity-expansion attacks. The Chapter 1 discipline: treat all third-party XML as untrusted.

Try this on YOUR network
# multicast M-SEARCH (run as root or with raw socket cap) gtimeout 3 bash -c ' echo -e "M-SEARCH * HTTP/1.1\r\nHOST: 239.255.255.250:1900\r\nMAN: \"ssdp:discover\"\r\nMX: 2\r\nST: ssdp:all\r\n\r\n" \ | nc -u -w 3 239.255.255.250 1900 '
# pip install defusedxml requests python3 - <<'PY' import socket, re, requests from defusedxml import ElementTree as ET REQ = ('M-SEARCH * HTTP/1.1\r\n' 'HOST: 239.255.255.250:1900\r\n' 'MAN: "ssdp:discover"\r\nMX: 2\r\nST: ssdp:all\r\n\r\n') s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) s.settimeout(3) s.sendto(REQ.encode(), ('239.255.255.250', 1900)) seen = set() try: while True: data, addr = s.recvfrom(2048) m = re.search(r'LOCATION:\s*(\S+)', data.decode(errors='ignore'), re.I) if m and m.group(1) not in seen: seen.add(m.group(1)) try: xml = requests.get(m.group(1), timeout=2).text root = ET.fromstring(xml) # <-- defused ns = {'u':'urn:schemas-upnp-org:device-1-0'} dev = root.find('.//u:device', ns) print(addr[0], dev.findtext('u:friendlyName', '', ns), dev.findtext('u:manufacturer', '', ns)) except Exception as e: print(addr[0], 'parse-error', e) except socket.timeout: pass PY
# quick probe of one device only — no multicast needed echo -e 'M-SEARCH * HTTP/1.1\r\nHost:192.168.1.42:1900\r\nMan:"ssdp:discover"\r\nMX:1\r\nST:upnp:rootdevice\r\n\r\n' \ | nc -u -w 2 192.168.1.42 1900
The Python snippet uses defusedxml — the same protection production discovery pipelines apply. Don't use the stdlib xml.etree.ElementTree on responses from third-party devices.
TCP Connect Pilot
durable fallback when passive methods leave uncertainty
A successful TCP connect ties the host to a reachable service endpoint — stronger than generic liveness. Pick from the Breakwater port list. The simulation runs against 192.168.86.42 and reports which respond, refuse, or filter. Breadth tradeoff: narrow pilot = cheap, may miss sparse subnets; broad pilot = more visibility, more traffic.
Pick ports to probe · click to toggle

An open port is still not a device identity. It is stronger evidence than ICMP echo, but a camera-like port set (80, 443, 554) could be a camera, an NVR, a media bridge, a test container, or a vendor appliance. The Chapter 1 contract: admit the host with port evidence; hand identity claim off to Chapter 2.

Try this on YOUR network
# cheap broad liveness — about 4 seconds for a /24 fping -a -g 192.168.1.0/24 -q -r 0 -t 50 # breakdown: # -a only show alive hosts # -g generate IPs across the range # -q quiet (no per-host status) # -r 0 no retries (single shot) # -t 50 50ms timeout per probe
# stateless high-speed probe — narrow port set, capped rate sudo masscan 192.168.1.0/24 \ -p22,80,443,554,8080 \ --rate 500 --wait 1 # search-space reducer: # subnets that respond → run deeper discovery # subnets silent → DON'T conclude empty; mark as "needs review"
# TCP connect probe across the Breakwater port set nmap -sT --open -p 22,80,443,445,554,631,1400,1883,3000,5000,8001,8008,8080,8443,8883,9100,10001,37777 \ 192.168.1.0/24 # breakdown: # -sT TCP connect (no raw socket needed) # --open only show open ports # -p the Breakwater IoT-leaning port list
In OT environments, never run masscan or aggressive nmap during business hours without stakeholder approval. fping is the safest broad sweep; nmap with -sT and a narrow port list is the safest active follow-up.
SECTION 10

The Streaming Pipeline

Time-to-first-truth · producer-consumer · hosts appear while discovery is still running.

Discovery should not force the analyst to wait for a monolithic batch result if useful evidence is already present. As soon as hosts appear, the pipeline normalizes MAC formats, filters artifacts, merges observations, and persists provisional host records. The Breakwater pipeline runs three async queues — discovery → enrichment → identification — each operating concurrently.

Discovery queue 0
Enrichment queue 0
Identification queue 0
Time to first host:
Stable count at:
Total runtime:

Why early partial truth matters. A streaming pipeline delivers provisional rows with explicit incompleteness. That is preferable to long silence followed by output that looks final because it arrived at the end. Operators cancel, resume, and inspect scans while they're still in motion. Toggle to batch mode above to feel the difference.

Publication gate
Before you publish a recall, precision, or harmonic-mean number — produce the underlying artifacts.

A clean-looking percentage on a slide with no scoring script behind it is not evidence — it is theatre. The PDF (Table 1.9) requires the following before any discovery metric goes into a report:

Recall
Scored host-claim output against a named ground truth. State the denominator and whether transient devices were in scope.
Precision
Raw observations + the final admitted inventory. State how stale ARP, broadcast, multicast, and phantom entries were filtered.
Method contribution
Per-host source provenance from the scan output. State whether each method uniquely admitted a host or only strengthened an existing claim.
Runtime
Timestamped phase events or logs. Separate time-to-first-host, time-to-stable-count, and end-to-end completion.
Baseline
Matched-scope runs for each baseline tool. Hold target scope, vantage point, and traffic policy constant — otherwise the comparison is unfair.
Confidence interval
The count table and the interval script or formula. State the confidence level and sample size.

A missing artifact should remove the number, not invite a guess.

SECTION 11

The Host Record as Analytic Contract

Evidence + provenance + confidence. The output of Chapter 1 is not "a device label" — it is a defensible claim.

The Chapter 1 contract is narrower than the answer Chapter 2 will give. A camera-like port set can belong to a camera, an NVR, a media bridge, a test container, or a vendor appliance. A Google OUI can belong to a Chromecast, a Nest device, a Wi-Fi product, or a virtualized interface. The inventory becomes stronger when it refuses to overstate.

Premature conclusion
ip:      192.168.86.42
device:  Chromecast
model:   2nd gen
status:  confirmed
# identity locked in too early
# vendor hint + port set ≠ proof
Defensible Chapter 1 handoff
ip:           192.168.86.42
mac:          d4:f5:47:xx:xx:xx
vendor_hint:  Google Inc.
hostname:     unknown
open_ports:   [80, 443]
sources:      [arp_cache, router_arp, tcp_connect]
icmp:         no response observed
multicast:    no useful response observed
confidence:   existence well-supported
identity:     unresolved · hand off to Ch.2

The calibration ladder. Not every host claim is equally trustworthy. Drag each observation to the confidence level it should carry. Stale ARP alone is not the same kind of evidence as ARP + router corroboration + TCP connect.

ARP + router ARP + TCP connect on 80/443
ARP + mDNS advertisement of _googlecast._tcp
ARP + router ARP only
Stale ARP entry, no other evidence
Multi-source agreement across repeated weekly observations

The formal handoff to Chapter 2

Treat the handoff as an API contract. Chapter 2 (Cascade Lab) consumes this shape — and is entitled to refuse it if required fields are missing or forbidden assertions appear.

interface ChapterOneHandoff · v1
Required fields · contract violation if missing
ipstringIPv4 dotted-quad (IPv6 in v2)
discovery_sourcesstring[]non-empty · each method that admitted this host
first_seenISO 8601timezone-aware
last_seenISO 8601timezone-aware
vantage_pointenumlaptop · switch · gateway · central · collector
confidenceenumlow · moderate · strong · very-strong · durable
Optional fields · preserve when present
macstringnormalized (zero-padded, uppercase, colon-separated)
vendor_hintstringOUI lookup · hint only · not identity
hostnamestringreverse DNS or self-advertised, source labeled
open_portsint[]TCP/UDP tuples preferred
servicesstring[]e.g. _googlecast._tcp · _ipp._tcp · capability hints only
upnp_descriptionobjectparsed via hardened XML parser only
notesstring[]conflicts retained · e.g. "router ARP disagreed with local"
Forbidden assertions · Chapter 2 refuses the row
device_modelidentity is Chapter 2's job · vendor hint ≠ model
firmware_versionrequires banner / service interrogation · not present in Ch.1
vulnerabilitiesChapter 3 boundary · CVE matching needs identity first
business_criticalitygovernance attribute · not derivable from network evidence
device_ownerorganizational attribute · requires CMDB / asset registry

Note * = required. The forbidden fields are not just missing from a Chapter 1 record — they're not Chapter 1's claim to make. A record that asserts them anyway is a discipline failure, not a feature.

SECTION 12 · CAPSTONE

The 26-Device Lab

172.30.0.0/24 · known ground truth · score your discovery against reality.

The lab network has 26 simulated devices: cameras (HTTP + RTSP), media players (mDNS + nonstandard web), NAS (SMB + web), printers (IPP + JetDirect), IoT gateways (narrow service surfaces), and a handful of cloud-only devices with no open ports — discoverable only through lower-level evidence. Pick your methods. The pipeline streams results. You score completeness, soundness, and the discovery gap at the end.

Step 1 · Pick your methods
Local ARP
Read existing local state. No new packets.
Router / SNMP
Cross-subnet visibility from the gateway.
mDNS browse
UDP/5353 local-link self-description.
SSDP / UPnP
UDP/1900 device descriptions.
fping ICMP sweep
Cheap broad liveness. Many devices drop.
TCP connect pilot
22, 80, 443, 554, 8080 — service evidence.
Scorecard · evaluated against ground truth
Completeness · C
found / 26
Soundness · S
real / admitted
Discovery gap · G = 1 − C
% of fleet unseen
Phantom hosts
stale / broadcast / artifact
Method contribution · which method admitted which hosts uniquely
SECTION 13 · DISCIPLINE

The Discovery Discipline Checklist

Before you publish a discovery report — or hand off to Chapter 2 — every line below should be true.

A discovery run that quietly skips any of these produces a report that looks like an inventory and behaves like a guess. Tick each box only when you can answer the question it implies from your own evidence — not from habit, not from hope.

The denominator is named.
You can state the target scope explicitly: which subnet, which VLAN, which time window. "The network" is not a denominator.
The vantage point is recorded.
Whether the scanner ran on a laptop on Wi-Fi, a switch port, a gateway, a VPN, or a distributed collector — and on which interface — is part of every host record's provenance.
Methods are listed with their traffic posture.
Passive vs. low-disturbance vs. active. Operational stakeholders need to know what the scan generated, not just what it found.
Each host record carries its method provenance.
Not just a flat list — every row knows which methods admitted it and when. Conflicts between methods are preserved, not silently resolved.
MAC addresses are normalized.
Zero-padded, uppercased, separator-consistent. c:ae:7d:1:2:3 and 0C:AE:7D:01:02:03 must merge to one row, not two.
Broadcast and multicast artifacts are filtered.
.255, .0, FF:FF:FF:FF:FF:FF, and other phantom-generators are explicitly excluded — not silently included as "extra hosts."
Negative evidence is stated by method, not summarized.
No claim like "the subnet is clean." A defensible statement: "the methods used from this position did not produce evidence during this time."
Identity claims are deferred to Chapter 2.
A vendor OUI is a hint. A port set is a hint. An mDNS service string is a hint. Chapter 1 admits hosts and preserves evidence — it does not label devices.
Time fields are recorded.
First-seen and last-seen, in the timezone you intend to operate in. "When did we observe this?" is part of every claim's strength.
Uncertainty is explicit.
Each host record carries a confidence level — low, moderate, strong, very strong, durable — and the reason for it. Chapter 2 inherits this; downstream analytics weight by it.
Stakeholders were notified before active probing.
In OT, healthcare, hospitality, and any environment where unexpected traffic carries risk — discovery is a coordinated activity, not a unilateral one.
The handoff to Chapter 2 carries questions, not answers.
What device model? Which firmware version? Which vulnerabilities? Is it business-critical? Is it present tomorrow? These remain questions in the Chapter 1 handoff — Chapter 2 begins by answering them, not by re-asking what Chapter 1 should have answered.
Ticked 0 / 12 · sign off only when every box can be defended from your evidence.
Lab 1 vocabulary · concepts you'll see used throughout the tool

These are the terms the Lab 1 lectures use, mapped to what the tool actually shows. Every label, callout, and metric in Learn and Scan Mine comes from this vocabulary — knowing the words is half the discipline.

Foundational concepts
Vantage point
where you observe from
The position from which you observe the network. ARP from your laptop sees a different reality than ARP from the gateway, or nmap from a remote server. Discovery results are always relative to the vantage point. In the diagram, the center node is your vantage point — the rest of the network is what's reachable from there.
in tool · center of the diagram
Existence vs. identity
two different claims
Existence: the device is there. It answered at L2. This is proven.
Identity: what kind of device it is. This is inferred — from MAC vendor (OUI), hostname, or behavioral fingerprinting. Never authenticated, just narrowed down. The Chapter 1 motto: existence ≠ identity.
in tool · "existence" + "type (inferred)" rows in popups
Host record
unified per-device record
The single record per device that builds up as evidence streams in. ARP fills in MAC; mDNS fills in hostname; TCP fills in open ports. Each method merges into the record rather than creating a new one. Records are provisional until enough corroborating signals make you confident.
in tool · the Host Records table
Ground truth
the actual answer
The complete, known-correct list of devices on a network. Available only in simulations (Learn mode), test environments, or networks where you have total visibility. You almost never have ground truth on a real network. That's the whole reason you can't measure completeness in production.
in tool · Learn mode tells you "12 real devices"
Measurement (completeness, soundness, gap)
Completeness
found ÷ real
What fraction of the real devices did your discovery find? Requires ground truth to compute. In Learn mode you can compute it (we provide the denominator); on a real network you cannot — you don't know what you missed.
in tool · Learn scorecard, first metric
Soundness
real ÷ discovered
Of what you discovered, how much is actually real? Phantoms (stale ARP entries, ghosts) drop soundness below 100%. A discovered inventory full of phantoms = poisoned inventory — you're not just incomplete, you're confidently wrong.
in tool · Learn scorecard, second metric
Discovery gap (D − D̂)
what exists but you missed
The set of devices that exist (D) but were not in your discovered set (D̂). The gap is invisible by definition on a real network: if you knew what you missed, you would've found it. The Chapter 1 lesson: the discovery gap is what an attacker hunts.
in tool · Learn scorecard third metric + outer dashed ring
TTFT — time to first truth
how fast you get usable inventory
How quickly your discovery converges to a useful inventory. ARP alone is fast but incomplete; layering mDNS, SSDP, TCP brings TTFT down. Longer TTFT = more time an attacker has the network before you do.
concept · runs underneath the method-by-method merge in Learn
Discovery methods
ARP
Address Resolution Protocol · L2 · active
Resolves IP → MAC on the local broadcast domain. Your machine's ARP cache lists devices it has recently communicated with. Fast and quiet (your machine ARPs anyway), but only sees devices you've actually talked to. Pinging the subnet first forces ARP entries for sleeping devices.
in tool · green edges · "Run ARP" button
mDNS
Multicast DNS · L7 · passive
Devices advertise themselves with .local hostnames over UDP 5353. Apple Bonjour, Linux Avahi, printers, IoT — all common mDNS announcers. Best source of hostnames, which are usually the strongest identity signal.
in tool · violet edges · hostnames in popups
SSDP
Simple Service Discovery Protocol · L7 · passive
UPnP-style passive announcements on UDP 1900. Smart TVs, Roku, Sonos, IoT all use it. Passive means you only listen — you don't generate probe traffic.
in tool · amber edges
TCP probe
L4 · active
Actively send connection attempts to common ports (22, 80, 443, 8080, etc.). Reveals devices that ARP/mDNS missed, but is noisy — generates traffic that defenders and IDS systems can see. Use with care on operational systems (see Oldsmar).
in tool · teal edges
Active vs. passive
do you send packets, or only listen?
Active: you send probes and watch for responses (ARP request, TCP SYN, ICMP echo). Generates traffic. Defenders can see you.
Passive: you only listen for announcements (mDNS, SSDP, DHCP traffic). Silent on the wire — defenders may never notice. Passive misses devices that don't broadcast; active misses devices that don't answer. Real discovery uses both, passive-first.
concept · informs method ordering in Learn mode
Identity signals
OUI
Organizationally Unique Identifier
The first 3 bytes (24 bits) of a MAC address. IEEE assigns these to manufacturers. 00:17:88 → Signify (Philips Hue). Tells you the vendor of the network interface chip, not necessarily the device's actual brand — a Roku has its own OUI, but a Raspberry Pi running Roku software would show the Pi OUI.
in tool · "Vendor (inferred)" column · ~250 entries in lookup table
Randomized MAC (LAA)
locally-administered address
Modern phones (and increasingly all client devices) generate a fresh random MAC per network to defeat tracking. Detectable: bit 1 of the first MAC octet is set. 2nd hex digit ∈ {2, 3, 6, 7, A, B, E, F}. Example: 6a:11:9f:… — the 6a means binary 01101010, bit 1 = 1 → locally administered → randomized. When randomized, OUI lookup yields nothing useful — identity is hidden.
in tool · "(randomized MAC)" in vendor column · dashed nodes
Hostname (mDNS-resolved)
strongest identity signal
The .local name a device announces via mDNS. Often very revealing — iPhone-of-Bob.local, roku-livingroom.local, hue-bridge.local. Trumps OUI for identity inference because users name their devices according to function. Can be spoofed — a device can claim any hostname — so still inference, not authenticated.
in tool · hostname column + node labels when available
Failure modes & pitfalls
Phantom / stale ARP entry
inventory pollution
An entry in your ARP cache pointing to a device that's no longer present — or never was (misconfiguration, ARP spoofing, decommissioned device). Phantoms inflate your discovered count without adding real devices. They drop soundness below 100%. The cleanup discipline is hard: how do you know which entries are stale without ground truth?
in tool · the rose-colored "PHANTOM" node in Learn mode
Inventory poisoning
the cost of bad data
When your inventory contains incorrect or stale records, every downstream decision is wrong. You exclude an attacker's foothold from your "known" list. You allocate IPs to phantoms. You alarm on changes that aren't real. A poisoned inventory is sometimes worse than no inventory because it inspires false confidence.
callout · soundness drop in Learn mode
The Oldsmar lesson
active probing can hurt
In 2021, an attacker accessed the Oldsmar (FL) water treatment plant's HMI. Defenders had no current inventory of what was on the network — partly because previous active scans had crashed operational systems, so scanning had been avoided. The lesson: active discovery on operational tech requires care. Passive-first lets you build inventory without disturbing the environment.
concept · why passive comes before active in the method order
Inferred — not proven
always a hypothesis
Everything in the "type" and "vendor" columns is inference, not authenticated truth. A device claiming iPhone-of-Bob via mDNS could be lying. A MAC with Apple OUI could be spoofed. Always treat identity as a hypothesis, not a fact. Multiple independent signals raise confidence; one signal is just one signal.
in tool · "guess via" reasoning shown in every popup
Operational discipline
Baseline
your "known good" snapshot
A captured discovery result you trust enough to compare future scans against. The point isn't that the baseline is "right" — it's that it's stable. The diff against subsequent scans (new devices, missing devices, changed identity) is where the operational value lives.
future feature · diff against saved scans
Change detection
what's different now
Re-run discovery; compare to baseline; investigate the diff. New device = an unexpected join (could be benign, could be hostile). Missing device = something went offline (or was removed; figure out which). Changed identity = OUI changed for a stable IP — possible MAC spoof or device replacement.
future feature · scheduled re-scans
Structured report output
discovery → operations handoff
The end product of a discovery cycle: not a pretty diagram, but a structured artifact (CSV, JSON) that another system can consume. The Host Records table here is the early-stage version. In production: feeds into asset DBs, vulnerability scanners, network access control.
in tool · Host Records table
Chapter 2 · Cascade Lab Lectures of Dr. Ravinder Mallarapu · SEAS-8414 · GW · interactive lab by Rayve Malhotra, student of the course · approved by Dr. Mallarapu

Service Enrichment & Device Fingerprinting

Identity begins where discovery left off. Chapter 1 told you something is there. Chapter 2 asks what is it — and how much confidence each piece of evidence deserves before that identity becomes Chapter 3's input for vulnerability work.

What is this device?
How sure can we be?
Which evidence is independent?
Where does Chapter 2 stop?
SECTION 01

Identity is not Existence.

Chapter 2 receives a structured handoff. Its job is to turn presence into a defensible identity claim.

The running case from Chapter 1 walks in carrying this record. It is well-supported on existence and intentionally weak on identity:

ip:            192.168.86.42
mac:           d4:f5:47:xx:xx:xx
vendor_hint:   Google Inc.   // OUI · hint only
open_ports:    [80, 443]
discovery:     [arp_cache, router_arp, tcp_connect]
confidence:    existence well-supported
identity:      unresolved · hand off to Ch.2

Chapter 2 is diagnostic analytics. It sits between Chapter 1's measured presence and Chapter 3's vulnerability reasoning. Many tools blur that boundary — spitting out IPs, vendors, service strings, and CVEs in a single confident-looking line. The cost of that blur is real: an inaccurate CPE creates phantom CVEs; a missing one lets real vulnerabilities slip past; a wildcard CPE drowns the report in noise.

Chapter
Question it answers
Output shape
Ch.1 Discovery
What devices appear to exist?
Host record with evidence provenance
Ch.2 Enrichment · you are here
What are those devices? How sure?
Identity artifact with confidence and CPE candidates
Ch.3 Assessment
What weaknesses apply?
Candidate or confirmed vulnerability findings

The artifact you produce here gets reasoned over by everything downstream. Overstate, and you launder weak evidence into apparent certainty. Understate, and you hand the next analyst a vague row they cannot act on. Calibration is the whole job.

SECTION 02

Thirteen Evidence Sources

Each contributes something. Each can mislead in its own way.

Enrichment is detective work under uncertainty. Open ports, service banners, certificates, multicast names, protocol responses, even the absence of replies — every source has a profile. Click each card to see what it tells you, what it cannot, and how it goes wrong.

MAC OUI
Manufacturer prefix lookup
IEEE-registered first three octets identify the vendor that registered the MAC range.
Cheap, near-universal, no probe needed.Broad. Randomized addresses, locally-administered MACs, and recycled modules all defeat it.Treating "Google OUI" as "Chromecast." OUI is the company, not the product.
Hostname
Reverse DNS or self-advertised name
A name attached to the device — by DHCP, mDNS, NetBIOS, or its owner.
Human-readable hints when present.User-set, stale, absent on IoT, or deliberately misleading.Trusting "MyCamera" as device identity when it is just a friendly name.
Open ports
Service surface
Which TCP/UDP services accept connections. Routes which adapters to even try.
Tells you which expensive probes are worth running.Ports do not prove product identity. Many devices share port sets."Has 554, must be a camera" — could also be an NVR, a media bridge, or a vendor appliance.
nmap -sV
Service / product version detection
Probes open ports against nmap's signature database, returns product + version hypothesis.
Excellent for managed services (OpenSSH, nginx, MySQL).Generic for IoT devices behind custom or templated web stacks.Trusting "Neato Botvac Connected" — a notorious false positive for embedded HTTPS servers (Google Nest).
HTTP
Title, server header, auth realm
Quick HTTP probes pull title tags, server headers, and authentication realms.
Fast, often a clear vendor hint when devices brand their interfaces.Easy to spoof. Often names the framework, not the device."Server: lighttpd" or "Server: nginx" assumed as device identity.
TLS certificate
Subject, issuer, SAN, self-signed pattern
Pulled from the handshake. Vendors often stamp factory certificates with characteristic names.
Self-signed factory certs are a fingerprint when the issuer is vendor-specific.May identify the vendor's TLS stack, not the device class.Treating a wildcard TLS issuer as device identity.
JARM
TLS handshake behavior hash
Sends 10 ClientHello variants, hashes the responses. Groups devices by TLS stack behavior.
Groups hosts by stack — useful for finding peers of a well-identified device.JARM collisions across products sharing a library or platform."Same JARM = same product" — could be the same SDK on different products.
RTSP
Camera / media server banner
Lightweight OPTIONS request often reveals Server header or auth realm naming vendor / firmware.
Auth realms frequently leak vendor names ("Hikvision," "IP Camera").Often incomplete; many cameras require auth before useful info.Assuming RTSP only means "camera" — could be an NVR or media server.
ONVIF
Structured camera self-report (SOAP)
GetDeviceInformation returns manufacturer, model, firmware, serial in structured XML.
Highest-confidence camera identity when device answers.Still self-reported — device can lie, lag, or fill fields oddly.Treating ONVIF firmware as ground truth — newer firmware than reported is common.
SSDP / mDNS
Local-link self-advertisement
SSDP returns UPnP device description XML; mDNS broadcasts service types like _googlecast._tcp.
Strong ecosystem signal — _googlecast._tcp narrows to Cast-capable.Link-local, unauthenticated, can be noisy or spoofed.Trusting service strings as model identity — _ipp._tcp says "prints," not "model X."
DHCP / router
Lease name, vendor class
Router-side queries return hostnames and DHCP vendor-class identifiers.
Authoritative for local-network knowledge — the router saw the device.Router-specific access required. Vendor-class often blank or generic.Falling back to "unknown" when SNMP fails instead of trying lease file scrape.
Fingerbank
Optional external classification
Cloud API that classifies devices by MAC + DHCP fingerprint. Off by default.
Useful external corroboration when configured.Requires trust in an external service; results may lag deployment reality.Letting an external score outvote local ONVIF or RTSP self-report.
TCP fingerprint
Stack-behavior heuristic
Raw probe checks TTL, window size, MSS, TCP options — guesses OS family.
Behavior is harder to fake than banners.Permission-sensitive (raw sockets), low confidence ceiling, off by default.Treating OS-family hints as device-class identity.
SELF-TEST

Five questions to check yourself

Click an answer to see if it's right and why.

QUESTION 1 / 5
Two evidence sources implemented in Breakwater share the same 0.85 confidence ceiling. Which two?
Correct: B. Both cap at 0.85 — ONVIF because structured camera self-report is high quality (but still self-reported), and the Fingerprint DB MAC-prefix match because it's a stored, curated fingerprint. SSDP, mDNS, Fingerbank ≥70 cap at 0.75. Hostname rules, HTTP title rules, JARM known hash cap at 0.70.
QUESTION 2 / 5
An ONVIF response and an HTTP title disagree about the device model. What's the right action?
Correct: C. Averaging or silently picking one is the most common discipline failure. The conflict is itself information — it survives in the host record's conflicts array. Chapter 3 reads this and weights downstream queries accordingly.
QUESTION 3 / 5
A JARM hash on Host B matches the hash of a well-identified Hikvision camera (Host A). What does Chapter 2 admit?
Correct: B. JARM groups by behavior, not identity. Two products sharing the same TLS library and config produce the same hash — JARM is propagation evidence, not primary identification. The 0.60 ceiling is implemented for exactly this reason.
QUESTION 4 / 5
A device shows both _airplay._tcp AND _ssh._tcp via mDNS. The Apple SSH guard fires. What does it prevent?
Correct: B. A MacBook advertises both. Without the guard, AirPlay alone would classify it as a media player. The guard encodes the rule that a specific combination dominates a single ambiguous signal — preserving rule precedence over evidence count.
QUESTION 5 / 5
A wildcard CPE comes out of Chapter 2 enrichment for an IoT device. What does Chapter 3 do with it?
Correct: C. Wildcards are admissible with explicit uncertainty. They produce candidate CVEs, not confirmed ones — and Chapter 3's job is to keep that distinction visible. The version_confidence_class tag on the CPE travels into Chapter 3's lookups and prevents wildcard-CPE-matched CVEs from being treated as model-specific evidence.
Drawn from Dr. Ravinder Mallarapu's Chapter 2 lectures · Analytical Tools for Cyber · SEAS-8414 · GW · interactive lab by Rayve Malhotra · approved by Dr. Mallarapu.
SECTION 03

Structural Independence

Three sources agreeing across protocols > ten sources agreeing on the same string.

A handful of HTTP strings from the same web stack is less persuasive than a few agreements from different protocols. When HTTP title, RTSP banner, and ONVIF response all point to the same camera family, that means independent parts of the device — implemented and configured separately — are telling the same story. It is easy to fake one string. It is very hard to forge a collection.

Toggle evidence sources on. Watch how confidence rises faster when the sources are structurally different vs. when they overlap.

Open ports 80, 554, 8000 HTTP title: "Network Camera" HTTP server: "App-webs/8.13" HTTP realm: "Hikvision" RTSP server: "Hikvision-Webs/8.13" ONVIF GetDeviceInformation TLS issuer: "HIK" JARM hash matches Hikvision peers SSDP description: manufacturer Hikvision
Confidence in the claim "this is a Hikvision camera": 0.00
Select evidence above. Watch what happens when you pick three pieces from the same protocol family (HTTP) vs. one piece each from three different families.
SECTION 04

The 12-Adapter Fan-Out

Concurrent. Independent. Failure-isolated.

Breakwater's enrichment pipeline doesn't rely on a single probe. It fans out twelve specialized adapters in parallel via asyncio.gather(). A slow ONVIF response must not block HTTP titles from being collected. A malformed UPnP document must not erase TLS certificate evidence. A JARM timeout must not prevent a strong RTSP banner from reaching the fingerprint service. Missing keys mean "no useful evidence from that adapter" — never "the host disappeared."

Click any adapter to toggle it OFF and watch graceful degradation in action: evidence drops, but the host stays in play.

Running enrichment · 192.168.86.42
ARP
d4:f5:47:xx:xx:xx
mDNS unicast
no reply on 5353
mDNS browse
_googlecast._tcp
SSDP
device description fetched
TLS
cert subject extracted
HTTP
title + server header
nmap -sV
2 services on 80/443
Fingerbank
disabled (no API key)
DHCP / router
lease confirms
RTSP
no 554 open · skipped
ONVIF
no candidate ports
JARM
hash 27d…
Active adapters: 11 / 12
Evidence sources contributing: 7
Host status: in play
SECTION 05

The Adapter Contract

Every adapter implements the same interface. That's why fan-out works.

Modularity needs a contract. Breakwater's enrichment adapters are predictable and replaceable precisely because they all promise the same shape. Any adapter that violates this contract — by raising uncaught exceptions, blocking the event loop, or returning malformed data — breaks failure isolation for every other adapter.

interface EnrichmentAdapter · v1
Input
hostsHostRecord[]discovered hosts with open ports, vantage, topology
Output
returnMap<ip,Evidence>missing key = no useful evidence (NOT "host gone")
Error handling
internal exceptionsisolatedone adapter's failure must not crash the batch
timeoutsboundedreturn what was collected before the timeout, don't hang
missing dependencydegradelog + skip, don't crash (e.g. no scapy → no TCP fingerprint)
Configurability
enabledboolon/off via BREAKWATER_<ADAPTER>_ENABLED env var
credentialsoptionale.g. ONVIF user/pass · empty by default
scopelocal/remotelink-layer adapters skipped when subnet is remote

The contract is what lets the router parallelize without fear. The router doesn't know what evidence ONVIF will return or whether RTSP will get past authentication — it knows it can call the adapter, wait at most N seconds, and merge whatever comes back.

SECTION 06

Port-Driven Adapter Selection

Open ports decide which expensive probes are even worth attempting.

Choosing probes based on open ports keeps the system nimble. No TLS port? JARM is skipped. No 554? RTSP doesn't fire. Remote subnet? Skip multicast — it never crosses routers. These choices keep performance high and measurements clean by avoiding pointless data. Type a port set (or use a preset) and see which adapters fire.

SECTION 07

Worked Trace 1 · Hikvision Camera

Local protocol surfaces · rich, but ONVIF might or might not answer.

The lab camera at 172.30.0.10 has a ground-truth identity (the answer key), an eligible-probe set (what could run), and an actually-observed evidence set (what came back). Real reports go wrong by collapsing these three into a single confident-looking row. Side-by-side comparison below.

172.30.0.10 · Hikvision DS-2CD2143G2-I

c0:56:e3:00:00:01 · ports 80/554/8000 · ssdp
Ground truth · answer key
vendor:    Hikvision
model:     DS-2CD2143G2-I
firmware:  V5.5.52 build 200915
known_cves:
  CVE-2021-36260
  CVE-2017-7921
# the answer key.
# the scanner has to EARN
# this through evidence.
Overconfident · sloppy
vendor:    Hikvision
model:     DS-2CD2143G2-I
firmware:  V5.5.52
vulns:     CVE-2021-36260
              CVE-2017-7921
confidence: 0.98
# ONVIF didn't actually
# answer. Model + firmware
# are fabricated. CVEs are
# Chapter 3's job.
Defensible · evidence-based
vendor:    Hikvision
device_type: camera
model:     DS-2CD family
confidence: 0.75
match_method: port_combo
                + rtsp_server
evidence:
  oui: c0:56:e3
  open_ports: [80,554,8000]
  rtsp_port: 554
  ssdp: present
not_observed:
  onvif: no GetDeviceInfo
version:   family-only
ch3_admit: family CPE
            with review

If ONVIF had answered with manufacturer, model, and firmware, the confidence could rise to 0.85 and the version-confidence class jumps to exact_self_reported. The artifact would record match_method: onvif alongside the existing port/RTSP evidence. The two artifacts have different downstream meanings — the first routes the host into camera-family CPE review; the second supports model-level CPE candidates while still flagging "self-reported." Same device, different defensible claims.

SECTION 08

Worked Trace 2 · Google Chromecast

Ecosystem identity without a traditional management protocol.

The Chromecast at 172.30.0.20 shows that a device can be well-identified without ONVIF, without RTSP, without a vendor management interface. The Cast ecosystem itself — _googlecast._tcp, port 8008/8009, the Eureka metadata endpoint — carries the family claim. The same scanner running from a routed-only vantage produces a much weaker (but still honest) artifact.

172.30.0.20 · Google Chromecast

54:60:09:00:00:01 · ports 8008/8009/8443 · _googlecast._tcp
Ground truth · answer key
vendor:    Google
model:     Chromecast
firmware:  1.68.382489
protocols:
  http/8008
  tcp/8009
  tls/8443
  mdns _googlecast._tcp
# Eureka endpoint exposes
# device metadata when
# HTTP probe succeeds.
Local · rich
vendor:    Google
device_type: media_player
family:    Chromecast /
              Google Cast
confidence: 0.75
match_method: mdns_service_type
supporting:
  oui: 54:60:09
  mdns: _googlecast._tcp
  http_8008: branded
  tls_8443: cert + jarm
limits:
  exact hw model depends
  on Eureka metadata
ch3_admit: family CPE
Routed scanner · degraded
vendor:    Google
device_type: unknown_or_media
confidence: 0.25
match_method: oui_vendor
evidence:
  oui: 54:60:09
  open_ports: [8008,8009,8443]
missing:
  mdns: not visible
        (routed scanner)
  http_title: no branded
  jarm_match: none
ch3_admit: no model CPE
# LOWER score is correct.
# Same device, less evidence.

The system must not hide the difference. The same physical device produces a confident family claim from a local vantage and an honest weak claim from a routed vantage. Conflating those by always reporting 0.75 — "we know it's a Chromecast" — would be analytically dishonest. Calibration is the message.

SECTION 09

The Fingerprint Cascade · Four Rungs

Generic hints first. Specific cross-protocol agreement shapes the final claim.

The fingerprint service is a ladder. Each rung is more specific than the last. A specific mix of signals always takes precedence over a misleading single clue. Click each rung to see what it does and where it caps.

RUNG 01
Vendor hinting
MAC OUI · router lease · Fingerbank classification
Useful for triage and grouping; rarely sufficient for a model-specific claim. Most affected by MAC randomization, locally-administered MACs, recycled modules, and devices that embed network capabilities from other products. A Google OUI is a hint, not an answer.
≤ 0.25
RUNG 02
Service-class inference
Port combinations · generic service products · stack behavior
Many IoT devices reveal their general type by service pattern, even when their names remain hidden. The rule table includes camera-like port sets (80 + 554 + 8000) and Google Nest/Home control ports. Ports reveal services offered, not business function.
≤ 0.65
RUNG 03
Protocol-stated identity
HTTP title · auth realm · RTSP banner · mDNS type · SSDP description · TLS issuer · ONVIF info
Strongest when several surfaces agree (HTTP title and RTSP realm and ONVIF model). Weakest when a single, easily-edited string carries the whole claim. ONVIF self-report on a camera is a different quality of evidence from an HTTP title with the same words.
≤ 0.85
RUNG 04
Normalized output
Stable artifact: vendor · device_type · model · confidence · match_method · evidence · conflicts · CPE candidates
This is where a scanner row becomes an analytic statement. If the output loses the match_method, Chapter 3 can't tell where the CPE came from. If it drops the conflict note, Chapter 3 might mistake a disputed firmware version for a sure thing.
artifact

The Apple SSH guard · why rule precedence matters

A worked example. macOS devices advertise BOTH _airplay._tcp (AirPlay) AND _ssh._tcp. Without rule precedence, the AirPlay evidence would mislabel a MacBook as a media player.

FingerprintService.identify_hosts() · Apple SSH guard
// for any host advertising mDNS _airplay._tcp OR _raop._tcp: if host.has_mdns("_ssh._tcp"): // SSH dominates — likely a Mac, not a speaker skip_media_player_classification() classify_as("workstation_or_developer") else if host.has_mdns("_airplay._tcp"): classify_as("media_player") // The specific mix wins. A single clue ("AirPlay") // is dominated by the presence of another, stronger signal // (SSH) that changes the device class entirely.
SECTION 10

Confidence Ceilings · Table 2.4

A handshake on how the system behaves — not a calibrated probability.

These are the implemented ceilings from FingerprintService.identify_hosts(). They are engineered ceilings, not statistical promises. A 0.75 score is not a calibrated 75% probability of being right across a hand-checked fleet. True calibration would need labeled devices, repeatable scans, scoring scripts, and error analysis. Treat these as handshakes: "if this evidence type fires, the system will cap its claim here."

Evidence source
Ceiling
Why
Fingerprint DB MAC prefix
0.85
Stored fingerprint match
ONVIF manufacturer / model
0.85
Structured self-report from camera
SSDP manufacturer / model
0.75
UPnP device description
mDNS service type
0.75
Service-type rule match
Fingerbank score ≥ 70
0.75
External classification, high confidence
Hostname rule
0.70
Hostname, mDNS name, or SSDP friendly name
HTTP title rule
0.70
HTTP title in flattened enrichment
JARM known hash
0.70
Local known-hash table match
Exact port signature
0.65
Stored port signature exact match
RTSP server header
0.65
Camera / media-server header
TLS certificate upgrade
0.65
Device-specific TLS issuer / CN over generic type
Service product string
0.60
Nmap product or service text
RTSP authentication realm
0.60
RTSP realm pattern
Fingerbank score 50–69
0.60
External classification, medium confidence
JARM grouping
≤ 0.60
Propagation from an identified peer · capped
Subset port signature
0.55
Stored signature subset match
DHCP vendor class
0.50
Router-side DHCP evidence
Fingerbank score 30–49
0.50
External classification, low confidence
Fingerbank score < 30
0.45
External classification, very low
HTTP server header
0.40
Generic server-header clue (nginx, lighttpd)
TCP fingerprint
0.35
TTL, window, MSS heuristics
OUI vendor / device-type hint
0.25
Vendor prefix + broad device-type map

Confidence climbs when evidence is strong, specific, and from independent sources. OUI + mDNS + HTTP beats OUI alone. ONVIF + RTSP + a matching port signature is a stronger case than three HTTP banners. Repeating weak signals doesn't help.

SECTION 11

CPE Construction

Identity becomes vulnerability naming. The version-confidence class travels with it.

CPE (Common Platform Enumeration) is the naming scheme vulnerability databases speak. Chapter 2 builds CPE candidates from service products, HTTP headers, ONVIF details, SSDP fields, RTSP strings, TLS issuers, and fallback identity data. Output must always remember its roots — a CPE built from an ONVIF model + firmware is higher quality than a wildcard CPE based on vendor + device type alone. Both are admissible; Chapter 3 needs to know which is which.

Build a CPE
Built CPE cpe:2.3:o:hikvision:ds-2cd2143g2-i_firmware:*:*:*:*:*:*:*:*
Admissibility: family-level query · review required
Downstream weight: moderate

The golden rule: keep uncertainty in plain sight. A wildcard CPE is fine for general awareness; not as proof of vulnerability. If CPEs conflict (HTTP says firmware A, SSDP says firmware B), the system does not just grab the newest or the simplest — it records the conflict and lets Chapter 3 decide.

SECTION 12

Adversarial Manipulation

What an attacker can fake. What is harder to forge.

Banners, HTTP titles, TLS certificates, and multicast advertisements are cheap to spoof. A compromised device can lie through ONVIF. A bridge can advertise services on behalf of another host. A locally-administered MAC defeats OUI assumptions. The practical defense is disagreement-aware evidence. A claim supported by OUI + HTTP + TLS + RTSP + ONVIF + repeated observations is harder to fake than one supported by a single banner. Click each card to flip from threat to defense.

↻ flip
Banner spoofing
An attacker edits the HTTP title or Server header on a compromised host to impersonate a high-value device. Single-protocol evidence becomes evidence of a different device entirely.
Require structural independence. HTTP title alone caps confidence at 0.70. Cross-validate against RTSP, ONVIF, SSDP, or JARM. A spoofed banner that doesn't change behavior across protocols stands out.
↻ flip
Hostname spoofing
DHCP-assigned hostnames or mDNS friendly names are user-controlled. An attacker sets hostname: NAS-prod-01 on a workstation to evade host-class assumptions.
Hostname caps confidence at 0.70 and is one input among many. A hostname conflicting with port set, TLS issuer, or service evidence becomes a conflict note — not a silent decision.
↻ flip
JARM collision
Two unrelated products built on the same TLS library (e.g. shared SDK or platform image) produce the same JARM hash. Grouping by JARM propagates an incorrect identity to peers.
JARM grouping is capped at 0.60 and is propagation, never primary identification. Each grouped host must still be corroborated by another source before Chapter 3 treats its CPE as model-level evidence.
↻ flip
Locally-administered MAC
An attacker sets the second-lowest bit of the first octet of their MAC (e.g. 02:xx:xx:xx:xx:xx). OUI lookup returns nothing — vendor-prefix-based grouping fails entirely.
No-OUI MACs are explicitly flagged. Identification falls back to behavioral signals (JARM, TCP fingerprint, service pattern, DHCP vendor class) — none of which trust a missing OUI as ground truth.
↻ flip
Compromised ONVIF lies
A compromised camera reports outdated firmware or a different model via GetDeviceInformation. The 0.85 confidence ceiling becomes 0.85 confidence in a lie.
ONVIF self-report is preserved as match_method: onvif with the explicit caveat that the device is reporting itself. Repeated observations + cross-protocol corroboration (RTSP banner version vs. ONVIF firmware) detect drift.
↻ flip
Service-advertising bridge
A device on the local link advertises mDNS services it doesn't actually run, on behalf of (or impersonating) another host. SSDP descriptions are similarly forgeable.
Multicast evidence is corroborated by direct probing. If a host advertises _googlecast._tcp but port 8008 is closed when probed, the advertisement is suspect — the conflict is preserved in the record.
SECTION 13

The Chapter 2 → Chapter 3 Handoff

A typed contract. Identity claim, evidence, CPE candidates, conflicts. No vulnerability assertions.

Chapter 3 (vulnerability assessment) consumes the artifact below. Required fields must be present. Optional fields are preserved when available. Forbidden assertions cause Chapter 3 to reject the row — because identity claims that include CVEs or business-criticality are not Chapter 2's claims to make.

interface ChapterTwoHandoff · v1
Required · contract violation if missing
ipstringinherited from Ch.1
vendorstringfrom OUI / SSDP / ONVIF / fingerbank — best available
device_typeenumcamera · printer · nas · media_player · iot_gateway · workstation · unknown
confidencefloat0.00 — 1.00 · respects ceiling table
match_methodenumonvif · ssdp · mdns · http_title · rtsp_server · oui · port_combo · …
evidenceobjectevery contributing source, with raw value
Optional · preserve when present
model_or_familystring"DS-2CD family" or "DS-2CD2143G2-I" — be explicit
firmwarestringself-reported · carry the caveat forward
cpe_candidatesCPE[]each tagged with version_confidence_class + source
conflictsstring[]e.g. "ONVIF firmware V5.5.52 vs RTSP banner V5.6.0"
version_confidenceenumexact_self_reported · normalized · family_only · wildcard · conflicting
downstream_admissibilityenummodel_level · family_level · vendor_only · do_not_query
Forbidden · Ch.3 refuses the row
vulnerabilitiesCh.3's claim · CVE matching needs identity first
cvessame · Ch.2 names devices, Ch.3 names weaknesses
exploit_pathCh.4 attack-graph reasoning
business_criticalitygovernance attribute · not derivable from enrichment
remediationCh.5+ prescriptive · not in scope
SECTION 14 · CAPSTONE

Enrich the 26-Device Fleet

Same lab subnet as Chapter 1. Pick your adapters. Score identity coverage + CPE admissibility.

The 26 simulated devices at 172.30.0.0/24 (cameras, media players, NAS, printers, IoT gateways, cloud-only hosts) need identity claims. Pick which adapters to enable. The pipeline runs enrichment and scores: how many got an identity above 0.50 confidence, how many CPE candidates were generated at each admissibility class, how many conflicts were preserved, how many hosts pass the §13 handoff contract.

Step 1 · Pick adapters
ARP · MAC corroborate
OUI vendor hint · 0.25 ceiling
mDNS browse + unicast
Service-type rule · 0.75 ceiling
SSDP / UPnP
Device description · 0.75
HTTP probe
Title / server / realm · 0.70
TLS certificate
Issuer / SAN · 0.65 upgrade
JARM
Known hash 0.70 · grouped ≤0.60
RTSP OPTIONS
Server header · 0.65
ONVIF GetDeviceInfo
Structured self-report · 0.85
nmap -sV
Service product · 0.60
Fingerbank
External · 0.45–0.75
DHCP / router
Lease + vendor class · 0.50
TCP fingerprint
Behavioral · 0.35 · raw sockets
Scorecard · enrichment evaluated against ground truth
Identified (≥ 0.50)
/ 26 hosts
Mean confidence
across all admitted
Conflicts noted
preserved, not hidden
Ch.3-admissible
pass the §13 contract
CPE candidate breakdown · by version-confidence class
DISCIPLINE

The Enrichment Discipline Checklist

Twelve sign-off items. Each row should be defensible from evidence — not asserted from habit.
Every identity claim carries its match_method.
"oui" is not the same kind of claim as "onvif." Chapter 3 weights them differently — the method must travel.
Confidence respects the ceiling table.
A claim built from HTTP title alone caps at 0.70. A pipeline that emits 0.95 from a single banner has broken its own contract.
Independent sources required for confidence above 0.65.
Multiple HTTP banners are not independent. ONVIF + RTSP + port combo are. Structural independence is the gate.
Conflicts are preserved, not silently resolved.
If ONVIF says firmware A and RTSP banner says firmware B, record both. Chapter 3 picks.
ONVIF responses carry the self-report caveat.
A compromised camera lies. ONVIF confidence ceiling is 0.85, not 1.0, for a reason — the trust caveat lives in the record.
Missing adapter results are recorded, not inferred.
"ONVIF did not respond" is data. "Therefore this is not a camera" is wrong. Absence ≠ negation.
JARM groupings are propagation, not primary identification.
A peer with the same JARM as an identified Hikvision camera is a hypothesis at ≤0.60. It still needs its own corroboration before model-level CPE use.
Locally-administered MACs trigger the no-OUI flag.
02:xx in the first octet means there is no IEEE OUI to look up. Fall back to behavioral evidence; do not invent a vendor.
CPE candidates declare their version-confidence class.
exact_self_reported / normalized / family_only / wildcard / conflicting. A wildcard CPE without that tag becomes a phantom CVE in Chapter 3.
Apple SSH guard (or equivalent) is honored.
A specific mix of signals (SSH + AirPlay) wins over a single ambiguous clue (AirPlay alone). Rule precedence is a feature, not a bug.
No vulnerability assertions in the artifact.
"Hikvision DS-2CD" is Chapter 2's claim. "CVE-2021-36260 applies" is Chapter 3's claim. Crossing that line silently is the most common discipline failure.
Downstream_admissibility is stated, not implied.
model_level vs family_level vs vendor_only vs do_not_query. Chapter 3 needs this to decide how broadly to search CVE catalogs.
Ticked 0 / 12 · the artifact is defensible when every box can be supported.
Chapter 3 · Threat Hunt Lectures of Dr. Ravinder Mallarapu · SEAS-8414 · GW · interactive lab by Rayve Malhotra, student of the course · approved by Dr. Mallarapu

Vulnerability Assessment & The Detective Layer

A vulnerability identified by an automated tool is an initial indicator — never a definitive conclusion. Its significance is established only when an analyst contextualizes it with device-specific evidence. Chapter 3 is the bridge from identity (what is this device?) to candidate weaknesses (what might apply?) — and the discipline that decides which candidates earn the right to move forward.

Which CVEs actually apply?
How sure can we be?
What's actually being exploited?
Where does Ch.3 stop?
SECTION 01

The Four-Term Boundary

Vocabulary controls everything. A scanner result is not a vulnerability — until it earns the right.

Most vulnerability tools fail by collapsing four distinct states into one number. Dr. M's chapter holds them apart. Click each card to see the definition, the example, and what evidence is required to move a finding between states.

STATE 01↻ flip
Candidate vulnerability
A weakness that may apply, based on identity, CPE, banner, template, or scanner evidence.
Example: NVD returns CVE-2021-36260 for a wildcard Hikvision camera CPE built from an HTTP server header. Affected versions match the rough family — but no firmware-level evidence has been gathered yet.

To promote → confirmed: an active check (Nuclei template hits the vulnerable endpoint) OR a direct corroborating signal (vendor advisory confirms the firmware branch).
STATE 02↻ flip
Confirmed weakness
A weakness supported by direct or strong corroborating evidence — not just identity.
Example: A Nuclei template verifies the vulnerable endpoint and the CPE matches affected criteria and the firmware is on a vulnerable branch. Two independent paths agree.

Also example: Default credentials succeed. That single direct authentication event is gold-standard evidence for authentication-failure, even if no CVE ID applies.
STATE 03↻ flip
False positive
A candidate that does not apply to this device after triage. Recorded, not deleted.
Five mechanisms produce false positives: CPE imprecision · backported patch · stale banner · configuration dependency · active-scan misinterpretation.

The validation note matters: not just "false positive" but "CVE applies to firmware before V5.5.800; ONVIF reports V5.7.1 and vendor advisory confirms patch branch." Future analysts can challenge or retire the decision.
STATE 04↻ flip
Prioritized item
A finding selected for action under operational constraints — not just sorted by CVSS.
Five inputs: severity · evidence strength · exposure · device role · consequence. A medium CVE on a router routing zones beats a critical CVE on an isolated lab camera.

Five action bands: act now · validate next · schedule · monitor · accept temporarily. Each requires a defensible rationale — not "the score said so."
SECTION 02

CVSS Is a Vector, Not a Number

Read the eight metrics. The kind of danger matters more than the score.

A CVSS v3.1 base score summarizes eight metrics. Two findings with the same 8.8 can be operationally different — one breaks confidentiality, the other breaks availability. On a media device with private metadata, confidentiality dominates. On a clinical camera supporting safety monitoring, availability does. Adjust the metrics below and watch the vector and band recompute.

CVSS v3.1 vector CVSS:3.1/AV:N/AC:L/PR:N/UI:N/S:U/C:H/I:L/A:N
8.2
High
0.1–3.9 · low
4.0–6.9 · medium
7.0–8.9 · high
9.0–10.0 · critical
Reading the vector: network-reachable, low complexity, no privileges, no user interaction, unchanged scope, high confidentiality impact. This is the classic "remote unauthenticated information disclosure" shape. On a router carrying credentials in management traffic, this would be act-now. On an isolated lab device with no sensitive data, it would be schedule.
SECTION 03

Five Methods, Different Error Profiles

Each tool observes different things. Each fails differently. Combine accordingly.

Breakwater's vulnerability phase runs five distinct evidence paths. They are not redundant — combining them is how confidence grows, but only when they agree about structurally different things. Combining them without understanding their failure modes is how false certainty spreads.

Method
What it observes
Strength
Failure mode
Safety
NVD lookup
CPE-to-CVE association from a public database
Broad coverage of known vulnerabilities
Depends on identity + CPE precision; database lag
Passive · no device contact
OpenVAS
Active network probes (NVT plugins)
Mature active assessment of service behavior
Slow · feed-dependent · can disrupt fragile devices
Active · maintenance windows + policy approval
Nuclei
Template-driven endpoint checks (JSONL output)
Targeted confirmation of specific weaknesses
Template coverage gaps; partial results on timeout
Active · less invasive than OpenVAS
Default creds
Vendor-aware authentication attempts
Gold-standard direct evidence when succeeds
Lockouts · alerts · false-positive on public pages
Opt-in only · written authorization required
Validation
Analyst decisions on prior findings (pair-level)
Suppresses known false positives without losing history
Stale if devices change · requires ongoing curation
Passive · note-taking

A device left untested by a method is not proven safe. The final report must distinguish at least four states per method per host: completed · disabled · unavailable · failed-or-partial.

SELF-TEST

Five questions to check yourself

Click an answer to see if it's right and why.

QUESTION 1 / 5
NVD returns a CVE for a host whose CPE was built from a vendor OUI wildcard. What state should this finding hold?
Correct: B. NVD's answer is an association, not a confirmation. Candidate → confirmed requires direct or strong corroborating evidence (active probe, vendor advisory agreement, or default-credential success). The four-term boundary is the chapter's vocabulary discipline.
QUESTION 2 / 5
For "act now" prioritization, which list should drive your decisions?
Correct: C. CVSS is a structural score; KEV is empirical evidence of active exploitation. A 9.8 CVSS with no public exploit deserves different urgency than a 7.5 CVE on the KEV list. Severity alone is the wrong prioritization axis.
QUESTION 3 / 5
A default-credential test against a camera at admin:admin returns HTTP 401 Unauthorized. What does this mean?
Correct: B. 401 means "authentication required and the supplied credentials were rejected." This is the entire point of HTTP authentication. A sloppy default-credential adapter that counts 401 as a hit produces false positives. The Breakwater adapter explicitly requires a 2xx response with content distinct from the unauthenticated baseline.
QUESTION 4 / 5
NVD says a host is vulnerable to CVE-X. A Nuclei template targeting that exact vuln returns no findings. What's the correct action?
Correct: C. Active evidence does not always take precedence. Templates have coverage gaps; configurations matter; backported patches happen. The contradiction is itself useful information — it tells the analyst what to investigate next, and Chapter 4 needs to see it.
QUESTION 5 / 5
A default-credential test on a Cisco gateway succeeds with cisco:cisco. NVD returns no CVEs for this host. Why is the credential finding still high-priority?
Correct: A. Default-credential success is gold-standard direct evidence — it's the most concrete vulnerability class because it required no inference. The action is immediate: rotate credentials, restrict management reach, segment the device. A CVE ID is not required for the finding to drive action; credentials and CVEs are different evidence classes.
Drawn from Dr. Ravinder Mallarapu's Chapter 3 lectures · Analytical Tools for Cyber · SEAS-8414 · GW · interactive lab by Rayve Malhotra · approved by Dr. Mallarapu.
PRACTICAL EXERCISE

Run the CPE→CVE Pipeline · Live

Six steps. Real public APIs. Real CVE data fetched in your browser.

Pick a banner, click through six steps. Each step exposes its real API call so you can copy the URL and verify in your own terminal. Cached for 24 hours per banner in your localStorage. NVD API docs · FIRST EPSS API · CISA KEV

Step 1 · Collect evidence no api

Paste a banner string. Where did this come from in real life? nmap -sV output, an HTTP Server header, a TLS certificate CN, an SNMP sysDescr, an SBOM line — any text that names a vendor + product + version.

Step 2 · Normalize no api

Regex + vendor mapping extracts canonical fields. Ambiguous fields stay ? rather than guessed.

Step 3 · Resolve CPE candidates no api

Three candidates from most-specific to least. We'll query NVD with the exact one first; fall back to fuzzy if exact returns nothing. CPE 2.3 syntax · NIST CPE specification.

Step 4 · Query NVD LIVE API

Real fetch to services.nvd.nist.gov. First tries cpeName= (exact); if 0 results, falls back to virtualMatchString= (prefix). Cached 24h locally.

Step 5 · Interpret applicability no api

For each CVE: which versions are affected, does our version fall in range, did a distro likely backport a patch? CVEs that may not apply get flagged.

Step 6 · Score & prioritize LIVE API

CVSS comes from the NVD record. EPSS (Exploit Prediction Scoring System) fetched live from FIRST.org. KEV lookup from the cached CISA catalog. Composite priority = severity × evidence × exposure × consequence.

SECTION 04

Correlation Without Double-Counting

Three patterns: convergence · unique coverage · contradiction.

Pick evidence types to corroborate a Hikvision camera CVE claim. Watch confidence rise when methods are structurally different — and notice what happens with same-method repetition or active/passive disagreement.

NVD exact CPE match NVD wildcard CPE match OpenVAS active probe finding Nuclei template verifies endpoint Default credentials succeed Validation record (analyst confirmed) Second NVD wildcard hit
PICK · evidence
Select evidence above. Notice how a Nuclei template + NVD exact CPE (two structurally different methods agreeing) raises confidence faster than three NVD hits.
LIVE DATA

What's Actively Being Exploited · Right Now

CISA KEV — Known Exploited Vulnerabilities — fetched live, cached 24h. ~1,200 entries vs NVD's ~280,000.

Severity bands tell you what could be bad. CISA's KEV catalog tells you what is actually being exploited in the wild. For prioritization, KEV beats CVSS. Below: the 10 most recently-added entries from cisa.gov, refreshed in your browser at most once every 24 hours.

CISA KEV · most recent additions loading from cisa.gov…
▸ fetching feed · falls back to cached snapshot if offline
GAME 01

🎯 The Triage Room

8 minutes simulated time. ~20 alerts. Pick an action band for each.

Alerts stream in from the simulated SOC. For each, you get device identity (inherited from the Ch.2 capstone), the CVE candidate, CVSS vector + score, exposure class, KEV status, and evidence-source provenance. You assign one of Dr. M's five action bands. Some alerts are phantoms — dismissing them is correct.

SOC · shift active
8:00
Decided: 0
Score:

You are on shift.

Alerts will arrive at irregular intervals. Read the vector, not just the score. Check KEV status. Reference your Ch.2 identity confidence. Wildcard CPE matches in the source field are suspect.

GAME 02

🕵️ Phantom Hunt

Spot the false positives. Identify which of the five mechanisms produced each one.

12 host records, each with a CVE candidate match. Some are real (the CVE truly applies). Some are phantoms caused by one of the five false-positive mechanisms. Drag the right mechanism label onto each phantom record. Mark real ones as "real · keep." The reveal explains the validation note that should be written.

Drag a mechanism onto each record · or use "real" if it actually applies
real · keep CPE imprecision backported patch stale banner config dependency active-scan misread
Score: 0 / 12
GAME 03

🎰 Default Credential Roulette

Read the response. Did the credential actually succeed?

A camera/NAS/printer is probed with vendor-default credentials. The response comes back. Decide: was that a real authentication success, a 401 failure, or a public-page false positive? The default-credential checker's whole job is reading responses correctly — 401/403 are failures, 2xx with matching unauth body is a public page (not a credential success), only 2xx with a different authenticated response counts.

— · —
Click SPIN to attempt a credential
Round 0 / 10
Correct: 0
Best:
WORKED TRACE 01

172.30.0.10 · Hikvision Camera

Identity is rich. CVEs are known. Default credentials are present.

The lab ground truth lists CVE-2021-36260 and CVE-2017-7921 for this camera, plus default credentials admin:12345 and admin:Hikvision. Ground truth is a grading oracle — not an analytical shortcut. The scanner has to earn each finding through real evidence.

Hikvision DS-2CD2143G2-I · firmware V5.5.52

CVE-2021-36260 · CVE-2017-7921 · admin:12345 · admin:Hikvision
Ground truth · oracle
cve_list:
  CVE-2021-36260
  CVE-2017-7921
defaults:
  admin:12345
  admin:Hikvision
firmware: V5.5.52
protocols:
  http/80 · rtsp/554
  tcp/8000 · ssdp
# the answer key.
# scanner must EARN
# every claim.
Overconfident · sloppy
cves:
  CVE-2021-36260
  CVE-2017-7921
status: VULNERABLE
action: act now
confidence: 0.95
# NVD returned these but
# only a wildcard CPE matched.
# No active probe ran.
# Default creds not tested
# (no authorization).
# All listed as confirmed —
# Ch.2 admissibility ignored.
Defensible · evidence-based
cve_candidates:
  CVE-2021-36260
    cpe: hikvision wildcard
    status: candidate
    evidence: nvd_lookup
    confidence: 0.55
  CVE-2017-7921
    cpe: family-level
    status: candidate
defaults_test: not run
  (no authorization)
priority: validate next
next_action:
  refine firmware identity
  via authenticated probe
  before claiming confirmed

If default-credential test runs and succeeds with explicit authorization, the artifact upgrades. The credential success becomes match_method: default_creds_http with confidence 0.95+ for authentication-failure. The priority moves to act now, but the action is rotate credentials and restrict management reach — not "patch the CVE." Different action for different evidence type.

WORKED TRACE 02

172.30.0.20 · Google Chromecast

Sparse evidence. Zero findings ≠ safe.

Ground truth lists CVE-2019-2099, firmware 1.68.382489, no default credentials. The Chromecast trace demonstrates a different problem: zero findings is ambiguous. NVD may return nothing because the CPE was too weak, the product name didn't match, the firmware wasn't represented, or the device truly has no matching records. The artifact has to say which.

Google Chromecast · firmware 1.68.382489

CVE-2019-2099 · no default credentials · http/8008 · tls/8443 · mdns
Ground truth · oracle
cve_list: CVE-2019-2099
defaults: none
firmware: 1.68.382489
device_type: media_player
protocols:
  http/8008 · tcp/8009
  tls/8443 · mdns
# Cast ecosystem device.
# No camera-class probes.
Overconfident · sloppy
cves: none
status: CLEAN
priority: monitor
# "Clean" is a claim.
# Did Nuclei run? No template
# for Cast Eureka endpoint.
# Did OpenVAS run? Disabled.
# Did default-creds run?
# No credentials defined
# for this class.
# Zero findings is a
# coverage statement —
# NOT a safety statement.
Defensible · evidence-based
cve_candidates: []
methods_run:
  nvd: completed
    matches: 0
  openvas: disabled
  nuclei: completed
    matches: 0
    (no cast template)
  default_creds: n/a
    (no creds for class)
finding_state:
  no_known_public_cves
  · coverage_gap_likely
priority: validate next
action:
  monitor for firmware
  changes · request
  Cast-specific templates

Two devices in the same lab. Same assessment phase. Wildly different priorities. The Hikvision case demands credential rotation as the first action. The Chromecast case demands coverage improvement — Cast-specific templates, deeper protocol probing — before any "clean" claim is defensible.

CAPSTONE

🛠 Patch Tuesday · The 26-Device Fleet

40 patch-hours · 4 maintenance windows · 7 days · a chosen attacker model.

Inherits identity + CPE candidates from the Chapter 2 capstone. Pick which methods to authorize (active ones carry safety warnings). Pick an attacker model. Then the simulator runs "the week" — exploits land based on KEV status, your patch order, and the attacker model. Score against Dr. M's four method-completion states.

Attacker model
Budget
40 patch-hours
Windows
4 over 7 days
Authorize methods
NVD lookup
Passive · always safe
OpenVAS
Active · GMP probes
⚠ can disrupt fragile devices
Nuclei templates
Active · targeted endpoint checks
⚠ less invasive than OpenVAS
Default credentials
Opt-in · vendor-aware
⚠ written authorization required
CISA KEV filter
Prioritize what's actively exploited
Validation suppression
Filter known false positives
Scorecard · after 7 simulated days
Incidents prevented
of N possible
KEV CVEs unpatched
attacker priority list
Hours on phantoms
wasted budget
Method coverage
of 5 methods
HANDOFF

Chapter 3 → Chapter 4 Contract

Weighted vulnerability claims · not a raw CVE list.

Chapter 4 (attack-graph reasoning) consumes the artifact below. An edge based on a high-confidence default-credential finding is structurally different from one based on a broad wildcard CPE. The contract preserves that difference.

interface ChapterThreeHandoff · v1
Required
ipstringinherited from Ch.1
identity_confidence_inheritedfloatfrom Ch.2 · CVE applicability depends on it
cve_candidatesCveRef[]each with cpe_source, version_confidence_class, evidence_method
method_completion_statesenum[]completed / disabled / unavailable / failed-or-partial · per method
priority_bandenumact_now / validate_next / schedule / monitor / accept_temporarily
priority_rationalestringexplains the 5-input reasoning
Optional
confirmed_findingsCveRef[]candidates promoted via active corroboration
credential_findingsCredHit[]direct authentication-failure evidence
false_positivesValidation[]pair-level (cve_id, device_ip) with mechanism note
kev_listedCveId[]subset that appears in CISA KEV
unresolved_contradictionsstring[]conflicts that survived triage · preserve, don't average
Forbidden · Ch.4 rejects the row
attack_path_assertionsthat's Ch.4's claim · Ch.3 names weaknesses, Ch.4 connects them
exploit_attempt_resultsrequires explicit authorization beyond Ch.3 scope
business_impact_quantificationgovernance attribute · not derivable from CVE evidence
guaranteed_safety"no CVEs found" ≠ "safe" · Ch.3 never claims this
SOURCES & FURTHER READING

The Actual Documents This Chapter Cites

Every standard, catalog, scanner, and advisory referenced — with direct links.

Vulnerability databases & catalogs

Standards & specifications

Active scanners (referenced in §3)

Operational guidance

The incidents referenced in Ch.1

Ch.2 protocols & tools

This lab itself

  • rayvemalhotra / rayvelabsThe full source — everything you see in your browser is built from this HTML, CSS, and JavaScript.
  • rayvenet.html (this page)Open in GitHub to read the un-minified source. View Source in your browser shows the deployed minified version.
  • LICENSE© Rayve Malhotra · All Rights Reserved. Read freely; copying or submitting as your own work (academic, employment, commercial) is expressly prohibited.
  • SECURITY.mdResponsible-disclosure policy for the lab itself.
  • OPERATIONS.mdDNS records, GitHub Pages setup, recovery runbook — the back-of-house.
  • Build pipelineEvery deploy runs a syntax check before shipping — see the GitHub Actions log.
  • Found a bug or have a suggestion? email Rayve or open an issue.
DISCIPLINE

The Vulnerability Assessment Checklist

Twelve sign-off items. Each must be defensible from your evidence.
No method declared authoritative.
NVD candidates are not "vulnerabilities found." OpenVAS findings are not "the truth." Each is one stream.
Every finding is tagged candidate / confirmed / false positive / prioritized.
The four-term boundary is the chapter's vocabulary. Collapsing them silently is the most common discipline failure.
Wildcard CPE matches were never passed forward as confirmed.
Ch.2's version_confidence_class exists for this reason. A wildcard CPE produces candidates only.
CISA KEV was cross-referenced before any "act now" call.
~1,200 KEV entries vs. ~280,000 NVD CVEs. Severity alone is the wrong prioritization axis. Exploitability + business path are.
No credentials were tested without explicit written authorization.
Default-credential testing can trigger lockouts, generate logs, violate engagement rules. Opt-in always.
Attacker model is stated explicitly.
"Critical" depends on attacker capabilities and motivation. Without a stated model, prioritization is unfalsifiable.
Validation notes name the mechanism, not just "false positive."
Dr. M's example: "CVE applies to firmware before V5.5.800; ONVIF reports V5.7.1 and vendor advisory confirms patch branch." Lets future analysts retire the decision.
Method-completion state is recorded per host.
completed / disabled / unavailable / failed-or-partial. Silent absence is not "no findings."
"No findings" was never claimed as "safe."
Cloud-only devices, sparse evidence, missing templates, unauthenticated probes all produce absence. None proves safety.
Contradictions between methods survived to the handoff.
NVD says vulnerable, Nuclei template fails. Both observations matter. Don't average; preserve.
No attack-path claims or exploit-attempt results in the artifact.
Ch.4 chains weaknesses into paths. Ch.3 names the weaknesses. The boundary is sharp.
CVSS vector was read, not just the score.
Two 8.8s are not equivalent. AV/AC/PR/UI/S/C/I/A determines which device class needs the attention most.
Ticked 0 / 12 · the assessment is defensible when every row can be supported.

Turn raw web logs into privacy-preserving fingerprints.

A real analyst tool. Paste an Apache or Nginx access log, and every request becomes a fingerprint you can group, compare, and publish — without ever exposing the original IPs, timestamps, query values, or user-agent strings.

100% in your browser No uploads No accounts SHA-256 via Web Crypto API Apache · Nginx · Bro/Zeek http.log
01 · Group
Find similar behavior
Requests with the same fingerprint behaved the same way — different IPs, different timestamps, identical pattern.
02 · Compare
Match new against old
Drop in a recent request, check whether it matches a known-bad pattern from before. Token-overlap measured live.
03 · Publish
Share findings safely
The normalized pattern strips identifying detail. Copy a fingerprint to a colleague, a ticket, a report — no PII goes with it.
1 · What is a fingerprint? · the five-step pipeline

A fingerprint is not the raw event. It is built in five explicit steps that strip sensitive details and preserve behavioral patterns. Read the step index first, then follow the same example log line through each stage below.

1
Capture the raw event read
Take one log line from the web server exactly as written, with no changes. This is the input.
2
Extract useful features select
Pull out the structural fields that describe behavior (method, path, status, UA family). Drop the sensitive ones (IP, timestamp, query value, full UA, referrer).
3
Normalize into categories generalize
Replace specific values with broad categories. A 401 becomes 4xx. A 10:01 timestamp becomes business_hours. The path /login becomes login. Combine the categories into a single dot-joined pattern string.
4
Hash the pattern SHA-256
Run the normalized pattern string through SHA-256 to get a fixed-length, deterministic hash. Same input → same hash. Hashes don't reverse.
5
Compare two fingerprints jaccard
Measure token-overlap between two patterns (intersection ÷ union). Same hash = same behavior. Partial overlap = related behavior. The slider in section 5 controls the threshold.

Now watch the same example flow through every stage:

Stage 1 of 5
Capture the raw event
Action:Read one log line from the web server, no processing.
Input:A line in Apache/Nginx combined format.
Output:The same line, captured as-is.
The strikethrough below previews which parts will get stripped in step 2 — IP, exact timestamp, query value, full user-agent, referrer. Nothing is removed yet; we're just marking what's sensitive.
198.51.100.23 - - [25/May/2026:10:01:18 -0400] "POST /login HTTP/1.1" 401 932 "https://example.test/login" "Mozilla/5.0"
Stage 2 of 5
Extract useful features
Action:Pull the structural fields that describe behavior. Drop the sensitive ones.
Input:The raw log line from Stage 1.
Output:A small set of field=value pairs — no IP, no timestamp, no query value.
The privacy filter runs here. Sensitive fields are not just hidden — they are never written to the downstream pattern. Only the structural shape of the request survives.
method = POST
path = /login
status_code = 401
query_present = false
user_agent_family = browser
Stage 3 of 5
Normalize into categories
Action:Replace specific values with broad categories, then join them into a dot-separated pattern.
Input:The selected features from Stage 2.
Output:A single pattern string of generalized tokens.
Status 4014xx. Timestamp 10:01business_hours. Path /login → category login. The behavior label (failed_login_pattern) is derived from the combination of method + path + status + UA family.
request.post.login.4xx.browser.query_absent.business_hours.failed_login_pattern
Stage 4 of 5
Hash the pattern
Action:Compute the SHA-256 of the normalized pattern string.
Input:The pattern from Stage 3.
Output:A 64-character hexadecimal hash. Fixed length. Deterministic. Irreversible.
Same pattern → same hash. Any single-token change → a completely different hash (the avalanche property — see Section 1b). The hash is the comparison key; the pattern is the audit trail.
8c91a1d2 4f7e3b6a 5c8d2e9f 1b4a7c0d e2f5a8c3 1d6b9f2e 4a7c1d8e 3b6a5c9f
Stage 5 of 5
Compare two fingerprints
Action:Compute Jaccard token overlap (intersection ÷ union) between two pattern strings.
Input:Two normalized patterns + a similarity threshold (40–95%).
Output:A percentage overlap and a verdict: Similar / Partial / Different.
Same hash means same normalized behavior — even if the underlying requests came from different IPs, at different times, with different query values. That's the whole point of fingerprinting.
Row A: request.post.login.4xx.browser.query_absent.business_hours.failed_login_pattern
Row B: request.post.login.4xx.browser.query_absent.business_hours.failed_login_pattern

→ Same hash. Same behavior. Different requests at different times, from a different IP, still match.

The core takeaway. A good fingerprint preserves behavior while discarding identity. The hash protects the normalized pattern from direct exposure, but the feature design (step 2) matters more than the hash (step 4). Two events with the same fingerprint behave the same; they may or may not come from the same actor. That distinction is the difference between an analyst tool and an accusation.

1b · The avalanche property · why a tiny change rewrites the hash

SHA-256 is designed so that any single-bit change in the input produces a completely different output. This is critical for fingerprints: it means the hash is sensitive to even the smallest meaningful difference in the normalized pattern.

Pattern A
request.post.login.4xx.browser.query_absent.business_hours.failed_login_pattern
SHA-256
8c91a1d2…3b6a5c9f
A failed login.
Pattern B (one token different)
request.get.login.4xx.browser.query_absent.business_hours.failed_login_pattern
SHA-256
f3c2b8a7…1d4e6f9c
Method flipped from post to get. The hash is entirely different.

Why this matters. If two fingerprints match, the underlying behavior was truly the same. If they don't match, the behavior differed in at least one meaningful way. Hashes can't tell you which token changed — the visible normalized pattern above the hash does that. The hash is the comparison key; the pattern is the audit trail.

Use this with your own data

This page is a real tool. If you operate a web server, you can paste your own access logs into Section 2 below and the lab will convert each request into a fingerprint you can: group (which requests behaved similarly?), compare (does this new request match an old one?), publish (share patterns without exposing the original log).

  1. Tail or copy a slice of your access log (e.g. tail -n 200 /var/log/nginx/access.log).
  2. Paste it into the textarea in Section 2. Click Parse logs.
  3. You'll get a fingerprint per request. Use Select for compare on any two rows to measure behavioral overlap.
  4. To share findings without exposing the original logs: copy just the normalized pattern strings or hashes — IPs, timestamps, and query values are already stripped.

Privacy. Everything runs in your browser. No fetch, no upload, no telemetry. If your logs include sensitive production data, this is the safe place to paste them. Reload the page and every trace disappears.

2 · Paste or upload logs

Apache/Nginx-style combined log format. Sample IPs are from documentation-safe ranges (127.0.0.1, 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24). One request per line.

Bundled sample datasets · click any to load

All bundled samples use documentation-safe IPs only (127.0.0.1, 192.0.2.0/24, 198.51.100.0/24, 203.0.113.0/24). Loaded directly into your browser — no fetches.

Where to find real log files from the internet

When you want to test on data beyond the bundled samples, these are real, public, well-known sources for web access logs and security data. All free. Pick one, download a file, paste it into the textarea above.

Format requirement

This lab only reads Apache/Nginx combined log format — one HTTP request per line with the shape IP - - [timestamp] "METHOD path PROTO" status size "referer" "ua". It does not read Bro/Zeek connection logs, Squid native logs, IIS W3C logs, JSON event streams, or PCAP/IDS captures. Pick files from the sources below carefully — the format matters.

SecRepo · public security data samples

https://www.secrepo.com/ · maintained by Mike Sconzo.

Look for files described as Apache or web-server access logs. Avoid the Bro/Zeek conn/SMTP/HTTP-bro files (their format is tab-separated columns starting with a Unix timestamp like 1331902024.07) — that's Bro, not Apache.

Loghub · academic log archive on GitHub

github.com/logpai/loghub · maintained by the LogPAI research group.

Use the specific file Apache/Apache_2k.log — that one is real Apache access log format and parses cleanly here. The other Loghub datasets (BGL, OpenSSH, HDFS, OpenStack, Spark, etc.) are different formats and will not parse.

The Honeynet Project · forensic challenges

honeynet.org/challenges/ · long-running public CTF-style challenges. Many include web access logs as forensic evidence. Useful when you want logs with a real attack story attached.

Your own server

If you run an Apache or Nginx web server, your real access logs are usually at /var/log/apache2/access.log or /var/log/nginx/access.log. Tail a sample of recent lines (tail -n 100 /var/log/nginx/access.log), copy them, paste them above. Reminder: if these are production logs, only paste them here on your own machine. This page runs in your browser only, but treat real user data with respect.

How to verify the format works. Apache/Nginx "combined log format" is the standard. A valid line looks like: IP - user [timestamp] "METHOD path PROTO" status size "referer" "user-agent". If a line is in a different format (Squid, IIS, CDN logs), this lab's parser won't match it. The parser marks unparseable lines so you can see which ones to drop.

8 · What students should notice

When you parse the sample data and try the toggles, look for these patterns:

  • Two failed-login rows produce the same fingerprint even though their timestamps and response sizes differ. The fingerprint dropped the noise and kept the behavior.
  • The three probing rows from 203.0.113.44 share a fingerprint. They look like reconnaissance, not normal browsing.
  • The two product-page rows share a fingerprint even though the query string changed. Query presence matters; query values don't.
  • Toggling off path_category collapses very different behaviors into the same pattern — a weak fingerprint.
  • Toggling on every possible feature can make even similar requests look different — an overly-specific fingerprint.
9 · Try this · student challenges

Load the sample data first. Then work through these in order.

Analyst note
Fingerprinting is powerful because it converts messy evidence into structured comparison logic. But it is only useful when the selected features represent meaningful behavior. A weak fingerprint may hide important differences. An overly specific fingerprint may fail to match related activity. The work isn't in the hash — it's in deciding which features to include and how to generalize them.
Key takeaways.
  • Fingerprints reduce raw events into safer comparison patterns.
  • Good fingerprints remove sensitive values (IPs, timestamps, full URLs, full user agents).
  • Hashing protects the normalized pattern from direct exposure, but feature design matters more than the hash.
  • Similar fingerprints suggest similar behavior, not guaranteed same attacker.
  • Fingerprinting is useful for grouping, comparison, detection logic, and privacy-aware analysis.
  • Context still matters. A fingerprint is evidence, not a final conclusion.
12 · Limitations
  • This lab uses a simple parser. Real-world logs include malformed lines, custom formats, multi-line entries, and unicode edge cases — handle them carefully in production.
  • The token-overlap similarity metric is intentionally simple for teaching. Real systems use locality-sensitive hashing, MinHash, or learned embeddings.
  • The five behavior patterns here are illustrative. A real detection pipeline has dozens, with continuous review and tuning.
  • Fingerprint stability across deployments depends on the normalization rules. If two pipelines categorize /wp-admin differently, their fingerprints won't match.
  • This lab does not perform attribution. It compares behavior, not identity. Drawing conclusions about specific actors requires additional independent evidence.
13 · Mini glossary
Raw event
A single observation as it appears in the source — for web logs, one full request line including IP, timestamp, path, status, agent.
Feature
A specific extracted attribute of an event — method, path category, status class, etc. Features are inputs to the fingerprint.
Normalization
Replacing specific values with generalized categories. 10:01:14 becomes business_hours; /products?id=4839 becomes product_page · query_present.
Sensitive field
Any value that could deanonymize the requester or leak operational detail — IPs, exact timestamps, usernames, session tokens, query string contents.
Hash
A deterministic, fixed-length transformation of the normalized pattern (SHA-256 here). Same input always produces same hash. Inputs cannot be reversed from the hash.
Similarity threshold
The percentage of token overlap required before the comparison engine calls two fingerprints "similar."
False positive
The fingerprint says two events match but they don't actually describe the same kind of behavior.
False negative
The fingerprint says two events differ but they actually do describe the same behavior. The fingerprint was too specific.
RayveNet · a RayveLabs project · © Rayve Malhotra