This article provides a comprehensive reference for field names and data sources used in Web Scanner queries. Field names are searchable data categories in SPQL (Search and Query Language), enabling precise searches across Web Scanner’s data repositories. Understanding these fields and their associated data sources is essential for constructing effective queries and interpreting scan results.
Overview of Data Sources
Web Scanner organizes scanned data into six data sources each representing a specific type of web data. The table below summarizes each data source and its primary use case:
Data Source | Description |
---|---|
Webscan | Web data from public IPv4 and IPv6 ranges, including HTML, favicons, and SSL data. |
torscan | Data from .onion sites on the Tor network, similar to |
services | Non-HTTP services (e.g., SSH, DNS), including TLS/SSL certificate data. |
opendirectory | Data from open directories, including file and directory metadata. |
webscanhistory | Log of successful scanning attempts, tracking domains and IPs scanned. |
webscanfailure | Log of failed scanning attempts, including reasons for failure. |
Important!
The "Datestring" data type is taken as YYYY-MM-DD
Use Field Names in Queries
Field names are used in SPQL queries to target specific data within a data source. For example:
Query:
domain = crypto* AND datasource = torscan
Searches the
torscan
data source for .onion domains starting with “crypto”.
Query:
ssl.expired = true AND datasource = services
Returns all expired SSL certificates in the
services
data source.
To search across multiple data sources, use square brackets with a comma-separated list:
Query:
domain = payments* AND datasource = [webscan, torscan]
Searches for domains starting with “payments” in both
webscan
andtorscan
.
For guidance on constructing queries, see Manage and Run Queries in Web Scanner.
'webscan' field names
Here's a list of field names you can search across using the webscan
data source.
Field name | Description | Type |
---|---|---|
| Has /ads.txt | Boolean |
| sha256 of /ads.txt | String |
| Has /app-ads.txt | Boolean |
| sha256 of /app-ads.txt | String |
| Has /sellers.json | Boolean |
| sha256 of /sellers.json | String |
| String | |
| String | |
| String | |
| SHA256 hash of the | String |
| SHA256 hash of the | String |
| String | |
| GA4 tag | String |
| UA tag | String |
| SHA256 hash of the | String |
| Chinese ICP LIcense | String |
| List of referenced js files in the HTML content, in the form of "url sha256" | String |
| List of referenced js files in the HTML content, in the form of "url ssdeep" | String |
| List of languages found in the HTML content, comma separated, from most used to least used | String |
| List of onion addresses that are in the HTML response | String |
| Script Hash Value, based on js scripts used by a website. | String |
| A unique hash of the overall scan result | String |
| The final domain that the origin domain that was scanned redirects to | String |
| Visual similarity image hash of the website's favicon file | String |
| Visual similarity image hash of the website's favicon2 file | String |
| Favicon MD5 Hash | String |
| Favicon Murmur3 Hash | String |
| Favicon Path | String |
| Favicon2 MD5 Hash | String |
| Favicon2 Murmur3 Hash | String |
| Favicon2 Path | String |
| List of the URLs of the favicons | String |
| Does URL scanned point to a file | Boolean |
| Hash of file pointed to | String |
| Autonomous System Name (ASN) | Integer |
| AS Organization | String |
| City of IP geolocation | String |
| Continent of IP geolocation | String |
| Country code of IP geolocation | String |
| Country code of IP geolocation | String |
| Country name of IP geolocation | String |
| Designated marketing area | String |
| Latitude value of IP geolocationll | Float |
| Latitude value of IP geolocation | Float |
| Longtitude value of IP geolocation | Float |
| Longtitude value of IP geolocation | Float |
| Postal code of IP geolocation | String |
| Region code of IP geolocation | String |
| Region name of IP geolocation | String |
| Timezone of IP geolocation | String |
| Instructions that control caching in browsers and shared caches (e.g. Proxies, CDNs) | String |
| Whether the network connection stays open after the current transaction finishes | String |
| Size of the message body, in bytes, sent to the recipient. | Number |
| Original media type of the resource (prior to any content encoding applied for sending). | String |
| The ETag (or entity tag) HTTP response header | String |
| String | |
| Software used to serve the HTTP response. If a redirect is present, this field shows data from the server that performed the last redirect | String |
| Value returned from server stating whatv it's powered by | String |
| A hash value based on the header keys | String |
| Hostname of domain that original domain that was scanned redirects to | String |
| A Murmur3 hash of the HTML body | Number |
| Number of bytes in the HTML body | Integer |
| A SHA256 hash of the HTML body | String |
| Percentage difference in the HTML body for this scan versus the previous scan - based on the SSDeep hash | Number |
| SSDeep hash of the HTML body | String |
| HTML Title | String |
| IP hosting URL that origin URL that was scanned redirects to | String |
| JARM Hash fingerprinting the TLS configurations of the host | String |
| Is this an open directory | Boolean |
| Domain that was scanned | String |
| Hostname of domain that was scanned | String |
| IP hosting URL that was originally scanned | String |
| URL path that was originally scanned | String |
| URL port that was originally scanned | String |
| Scheme of URL that was originally scanned | String |
| URL that was originally scanned | URL |
| Path of URL that originally scanned URL redirects to | String |
| Port of URL that originally scanned URL redirects to | Number |
| Does the URL scanned result in a redirect | Boolean |
| Number of URLs involved in a redirect | Integer |
| List of URLs that sit between the origin and destination URLs | String |
| Does the URL scanned result in a redirect to https | Boolean |
| Scan Request Response Code | Number |
| The date that data was scanned | Datestring |
| Scheme of URL that originally scanned URL redirects to | String |
| The authority key identifier (AKI) is an X.509 v3 certificate extension. It contains a key identifier which is derived from the public key in the issuer certificate. | String |
| A fingerprint on how the issuer creates the certificate based on the certificate's issuer/subject/extension RDN keys/wildcard and SANS count | String |
| Has SSL certificate expired | Boolean |
| SSL Certificate Issuer Common Name | String |
| SSL Certificate Issuer Country | String |
| SSL Certificate Issuer Organization | List of strings |
| SSL Certificate Validity End Date | Datetime |
| SSL Certificate Validity Start Date | Datetime |
| SSL Certificate Sans List | List of domains |
| SSL Certificate Sans List Count | Number |
| SSL Certificate Serial Number | String |
| S1SL Certificate SHA1 Hash | String |
| SSL Certificate SHA256 Hash | String |
| SSL Certificate Signature Algorithm | String |
| SSL Certificate Subject Common Name | String |
| SSL Certificate Subject Country | String |
| SSL Certificate Subject Names | List of domains |
| SSL Certificate Subject Organization | String |
| Is this a wildcard SAN certificate, i.e. Sans List references wildcards | Boolean |
| The subdomain value, if it exists, of the final domain that scanned original domain redirects to | String |
| The top level domain of the final domain that scanned original domain redirects to | String |
| The final URL that the origin URL that was scanned redirects to | String |
'torscan' field names
Here's a list of field names you can search across using the torscan
data source.
Field name | Description | Type |
---|---|---|
| String | |
| String | |
| String | |
| SHA256 hash of the | String |
| SHA256 hash of the | String |
| String | |
| GA4 tag | String |
| UA tag | String |
| SHA256 hash of the | String |
| List of referenced js files in the HTML content, in the form of "url sha256" | String |
| List of referenced js files in the HTML content, in the form of "url ssdeep" | String |
| List of languages found in the HTML content, comma separated, from most used to least used | String |
| List of onion addresses that are in the HTML response | String |
| Script Hash Value, based on js scripts used by a website. | String |
| A unique hash of the overall scan result | String |
| The final domain that scan redirects to | String |
| Visual similarity image hash of the website's favicon file | String |
| Visual similarity image hash of the website's favicon2 file | String |
| Favicon MD5 Hash | String |
| Favicon Murmur3 Hash | String |
| Favicon Path | String |
| Favicon2 MD5 Hash | String |
| Favicon2 Murmur3 Hash | String |
| Favicon2 Path | String |
| List of the URLs of the favicons | String |
| Does URL scanned point to a file | Boolean |
| Hash of file pointed to | String |
| Instructions that control caching in browsers and shared caches (e.g. Proxies, CDNs) | String |
| Whether the network connection stays open after the current transaction finishes | String |
| Size of the message body, in bytes, sent to the recipient. | Number |
| Original media type of the resource (prior to any content encoding applied for sending). | String |
| The ETag (or entity tag) HTTP response header | String |
| String | |
| Software used to serve the HTTP response. If a redirect is present, this field shows data from the server that performed the last redirect | String |
| Value returned from server stating whatv it's powered by | String |
| A hash value based on the header keys | String |
| Hostname of domain that original domain that was scanned redirects to | String |
| A Murmur3 hash of the HTML body | Number |
| A SHA256 hash of the HTML body | String |
| SSDeep hash of the HTML body | String |
| HTML Title | String |
| Is this an open directory | Boolean |
| Hostname of domain that was scanned | String |
| IP hosting URL that was originally scanned | String |
| URL path that was originally scanned | String |
| URL port that was originally scanned | String |
| Scheme of URL that was originally scanned | String |
| URL that was originally scanned | URL |
| Path of URL that originally scanned URL redirects to | String |
| Port of URL that originally scanned URL redirects to | Number |
| Does the URL scanned result in a redirect | Boolean |
| Number of URLs involved in a redirect | Integer |
| List of URLs that sit between the origin and destination URLs | String |
| Does the URL scanned result in a redirect to https | Boolean |
| Scan Request Response Code | Number |
| The date that data was scanned | Datestring |
| Scheme of URL that originally scanned URL redirects to | String |
| The authority key identifier (AKI) is an X.509 v3 certificate extension. It contains a key identifier which is derived from the public key in the issuer certificate. | String |
| A fingerprint on how the issuer creates the certificate based on the certificate's issuer/subject/extension RDN keys/wildcard and SANS count | String |
| Has SSL certificate expired | Boolean |
| SSL Certificate Issuer Common Name | String |
| SSL Certificate Issuer Country | String |
| SSL Certificate Issuer Organization | List of strings |
| SSL Certificate Validity End Date | Datetime |
| SSL Certificate Validity Start Date | Datetime |
| SSL Certificate Sans List | List of domains |
| SSL Certificate Sans List Count | Number |
| SSL Certificate Serial Number | String |
| SSL Certificate SHA1 Hash | String |
| SSL Certificate SHA256 Hash | String |
| SSL Certificate Signature Algorithm | String |
| SSL Certificate Subject Common Name | String |
| SSL Certificate Subject Country | String |
| SSL Certificate Subject Names | List of domains |
| SSL Certificate Subject Organization | String |
| Is this a wildcard SAN certificate, i.e. Sans List references wildcards | Boolean |
| The subdomain value, if it exists, of the final domain that scanned original domain redirects to | String |
| The top level domain of the final domain that scanned original domain redirects to | String |
| The final URL that the origin URL that was scanned redirects to | String |
'services' field names
Here's a list of field names you can search across using the services
data source.
Field name | Description | Type |
---|---|---|
| Service banner on a specific port | String |
| A unique hash of the overall scan result | String |
| Fingerprint of the ECDSA public key | String |
| Fingerprint of the ED25519 public key | String |
| Fingerprint of the RSA public key | String |
| Autonomous System Name (ASN) | Integer |
| AS Organization | String |
| IP hosting URL that origin URL that was scanned redirects to | String |
| Port of URL that originally scanned URL redirects to | Number |
| The date that data was scanned | Datestring |
| The authority key identifier (AKI) is an X.509 v3 certificate extension. It contains a key identifier which is derived from the public key in the issuer certificate. | String |
| A fingerprint on how the issuer creates the certificate based on the certificate's issuer/subject/extension RDN keys/wildcard and SANS count | String |
| Has SSL certificate expired | Boolean |
| SSL Certificate Issuer Common Name | String |
| SSL Certificate Issuer Country | String |
| SSL Certificate Issuer Organization | List of strings |
| SSL Certificate Validity End Date | Datetime |
| SSL Certificate Validity Start Date | Datetime |
| SSL Certificate Sans List | List of domains |
| SSL Certificate Sans List Count | Number |
| SSL Certificate Serial Number | String |
| SSL Certificate SHA1 Hash | String |
| SSL Certificate SHA256 Hash | String |
| SSL Certificate Signature Algorithm | String |
| SSL Certificate Subject Common Name | String |
| SSL Certificate Subject Country | String |
| SSL Certificate Subject Names | List of domains |
| SSL Certificate Subject Organization | String |
| Is this a wildcard SAN certificate, i.e. Sans List references wildcards | Boolean |
'opendirectory' field names
Here's a list of field names you can search across using the opendirectory
data source.
Field name | Description | Type |
---|---|---|
| Is a directory | Boolean |
| Autonomous System Name (ASN) | Integer |
| AS Organization | String |
| Hostname of domain that original domain that was scanned redirects to | String |
| IP hosting URL that origin URL that was scanned redirects to | String |
| Last modified date of a file in an open directory | Datestring |
| Filename or directory name of a file in an open directory | String |
| Port of URL that originally scanned URL redirects to | Number |
| The date that data was scanned | Datestring |
| Scheme of URL that originally scanned URL redirects to | String |
| The filesize in bytes | Integer |
'webscanhistory' field names
Here's a list of field names you can search across using the webscanhistory
data source.
Field name | Description | Type |
---|---|---|
| A unique hash of the overall scan result | String |
| The final domain that the origin domain that was scanned redirects to | String |
| Hostname of domain that original domain that was scanned redirects to | String |
| IP hosting URL that origin URL that was scanned redirects to | String |
| URL that was originally scanned | URL |
| The date that data was scanned | Datestring |
| Scheme of URL that originally scanned URL redirects to | String |
'webscanfailure' field names
Here's a list of field names you can search across using the webscanfailure
data source.
Field name | Description | Type |
---|---|---|
| The final domain that the origin domain that was scanned redirects to | String |
| IP hosting URL that origin URL that was scanned redirects to | String |
| Port of URL that originally scanned URL redirects to | Number |
| The reason a scanning failure occurred | String |
| The date that data was scanned | Datestring |
| Scheme of URL that originally scanned URL redirects to | String |
| The final URL that the origin URL that was scanned redirects to | String |
Field name index
Field name | Description | Type | Data source |
---|---|---|---|
| Has /ads.txt | Boolean |
|
| sha256 of /ads.txt | String |
|
| Has /app-ads.txt | Boolean |
|
| sha256 of /app-ads.txt | String |
|
| Has /sellers.json | Boolean |
|
| sha256 of /sellers.json | String |
|
| Service banner on a specific port | String |
|
| String |
| |
| String |
| |
| String |
| |
| SHA256 hash of the | String |
|
| SHA256 hash of the | String |
|
| String |
| |
| GA4 tag | String |
|
| UA tag | String |
|
| SHA256 hash of the | String |
|
| Chinese ICP LIcense | String |
|
| List of referenced js files in the HTML content, in the form of "url sha256" | String |
|
| List of referenced js files in the HTML content, in the form of "url ssdeep" | String |
|
| List of languages found in the HTML content, comma separated, from most used to least used | String |
|
| List of onion addresses that are in the HTML response | String |
|
| Script Hash Value, based on js scripts used by a website. | String |
|
| A unique hash of the overall scan result | String |
|
| The index of the data | String | N/A |
| Is a directory | Boolean |
|
| The final domain that the origin domain that was scanned redirects to | String |
|
| Visual similarity image hash of the website's favicon file | String |
|
| Visual similarity image hash of the website's favicon2 file | String |
|
| Favicon MD5 Hash | String |
|
| Favicon Murmur3 Hash | String |
|
| Favicon Path | String |
|
| Favicon2 MD5 Hash | String |
|
| Favicon2 Murmur3 Hash | String |
|
| Favicon2 Path | String |
|
| List of the URLs of the favicons | String |
|
| Does URL scanned point to a file | Boolean |
|
| Hash of file pointed to | String |
|
| Fingerprint of the ECDSA public key | String |
|
| Fingerprint of the ED25519 public key | String |
|
| Fingerprint of the RSA public key | String |
|
| Autonomous System Name (ASN) | Integer |
|
| AS Organization | String |
|
| City of IP geolocation | String |
|
| Continent of IP geolocation | String |
|
| Country code of IP geolocation | String |
|
| Country code of IP geolocation | String |
|
| Country name of IP geolocation | String |
|
| Designated marketing area | String |
|
| Latitude value of IP geolocationll | Float |
|
| Latitude value of IP geolocation | Float |
|
| Longtitude value of IP geolocation | Float |
|
| Longtitude value of IP geolocation | Float |
|
| Postal code of IP geolocation | String |
|
| Region code of IP geolocation | String |
|
| Region name of IP geolocation | String |
|
| Timezone of IP geolocation | String |
|
| Instructions that control caching in browsers and shared caches (e.g. Proxies, CDNs) | String |
|
| Whether the network connection stays open after the current transaction finishes | String |
|
| Size of the message body, in bytes, sent to the recipient. | Number |
|
| Original media type of the resource (prior to any content encoding applied for sending). | String |
|
| The ETag (or entity tag) HTTP response header | String |
|
| String |
| |
| Software used to serve the HTTP response. If a redirect is present, this field shows data from the server that performed the last redirect | String |
|
| Value returned from server stating whatv it's powered by | String |
|
| A hash value based on the header keys | String |
|
| Hostname of domain that original domain that was scanned redirects to | String |
|
| Number of bytes in the HTML body | Integer |
|
| A Murmur3 hash of the HTML body | Number |
|
| A SHA256 hash of the HTML body | String |
|
| Percentage difference in the HTML body for this scan versus the previous scan - based on the SSDeep hash | Number |
|
| SSDeep hash of the HTML body | String |
|
| HTML Title | String |
|
| IP hosting URL that origin URL that was scanned redirects to | String |
|
| JARM Hash fingerprinting the TLS configurations of the host | String |
|
| Last modified date of a file in an open directory | Datestring |
|
| Filename or directory name of a file in an open directory | String |
|
| Is this an open directory | Boolean |
|
| Domain that was scanned | String |
|
| Hostname of domain that was scanned | String |
|
| IP hosting URL that was originally scanned | String |
|
| URL path that was originally scanned | String |
|
| URL port that was originally scanned | String |
|
| Scheme of URL that was originally scanned | String |
|
| URL that was originally scanned | URL |
|
| Path of URL that originally scanned URL redirects to | String |
|
| Port of URL that originally scanned URL redirects to | Number |
|
| The reason a scanning failure occurred | String |
|
| Does the URL scanned result in a redirect | Boolean |
|
| Number of URLs involved in a redirect | Integer |
|
| List of URLs that sit between the origin and destination URLs | String |
|
| Does the URL scanned result in a redirect to https | Boolean |
|
| Scan Request Response Code | Number |
|
| The date that data was scanned | Datestring |
|
| Scheme of URL that originally scanned URL redirects to | String |
|
| The filesize in bytes | Integer |
|
| The authority key identifier (AKI) is an X.509 v3 certificate extension. It contains a key identifier which is derived from the public key in the issuer certificate. | String |
|
| A fingerprint on how the issuer creates the certificate based on the certificate's issuer/subject/extension RDN keys/wildcard and SANS count | String |
|
| Has SSL certificate expired | Boolean |
|
| SSL Certificate Issuer Common Name | String |
|
| SSL Certificate Issuer Country | String |
|
| SSL Certificate Issuer Organization | List of strings |
|
| SSL Certificate Validity End Date | Datetime |
|
| SSL Certificate Validity Start Date | Datetime |
|
| SSL Certificate Sans List | List of domains |
|
| SSL Certificate Sans List Count | Number |
|
| SSL Certificate Serial Number | String |
|
| SSL Certificate SHA1 Hash | String |
|
| SSL Certificate SHA256 Hash | String |
|
| SSL Certificate Signature Algorithm | String |
|
| SSL Certificate Subject Common Name | String |
|
| SSL Certificate Subject Country | String |
|
| SSL Certificate Subject Names | List of domains |
|
| SSL Certificate Subject Organization | String |
|
| Is this a wildcard SAN certificate, i.e. Sans List references wildcards | Boolean |
|
| The subdomain value, if it exists, of the final domain that scanned original domain redirects to | String |
|
| A date string | String | |
| The top level domain of the final domain that scanned original domain redirects to | String |
|
| The final URL that the origin URL that was scanned redirects to | String |
|