Parallelize URL indicator checks (#844)

This commit is contained in:
besendorf
2026-07-27 15:03:02 +02:00
committed by GitHub
parent 123c9081ed
commit a6c3a805d8
9 changed files with 239 additions and 40 deletions
+4 -3
View File
@@ -37,8 +37,10 @@ export MVT_STIX2="/home/user/IOC1.stix2:/home/user/IOC2.stix2"
## Network Access
When checking URL indicators, MVT follows recognized shortened URLs with an
HTTP `HEAD` request. The following environment variables control these
requests:
HTTP `HEAD` request. URL checks are deduplicated and run concurrently, with at
most 20 requests in progress at a time. Redirects within an individual URL
chain are still followed sequentially. The following environment variables
control these requests:
- `MVT_NETWORK_ACCESS_ALLOWED` enables or disables network requests. It defaults
to `true`. Set it to `false` to prevent MVT from attempting to resolve
@@ -73,4 +75,3 @@ You can automaticallly download the latest public indicator files with the comma
Please [open an issue](https://github.com/mvt-project/mvt/issues/) to suggest new sources of STIX-formatted IOCs.