A library for fast parse & import of Windows Master File Table($MFT) into Elasticsearch.
mft2es uses the Rust library pymft-rs, making it much faster than traditional tools.
mft2es can be executed from the command line or incorporated into a Python script.
$ mft2es /path/to/your/$MFTfrom mft2es import mft2es
if __name__ == '__main__':
filepath = '/path/to/your/$MFT'
mft2es(filepath)mft2es supports simultaneous import of multiple files.
$ mft2es file1/$MFT file2/$MFT file3/$MFTIt also allows recursive import from the specified directory.
$ tree .
mftfiles/
├── $MFT
└── subdirectory/
├── $MFT
└── subsubdirectory/
└── $MFT
$ mft2es /mftfiles/ # The path is recursively expanded to all MFT and $MFT files.--version, -v
--help, -h
--quiet, -q
Suppress standard output
(default: False)
--multiprocess, -m:
Enable multiprocessing for faster execution
(default: False)
--size:
Chunk size for processing (default: 500)
--host:
Elasticsearch host address (default: localhost)
--port:
Elasticsearch port number (default: 9200)
--index:
Destination index name (default: mft2es)
--scheme:
Protocol scheme to use (http or https) (default: http)
--pipeline:
Elasticsearch Ingest Pipeline to use (default: )
--timeline:
Enable timeline analysis mode for MACB format
(default: False)
--tags:
Comma-separated tags to add to each record for identification
(e.g., hostname, domain name) (default: )
--login:
The login to use if Elastic Security is enabled (default: )
--pwd:
The password associated with the provided login (default: )
When using from the command line:
$ mft2es /path/to/your/$MFT --host=localhost --port=9200 --index=foobar --size=500When using from a Python script:
if __name__ == '__main__':
mft2es('/path/to/your/$MFT', host='localhost', port=9200, index='foobar', size=500)With credentials for Elastic Security:
$ mft2es /path/to/your/$MFT --host=localhost --port=9200 --index=foobar --login=elastic --pwd=******With timeline analysis mode:
$ mft2es /path/to/your/$MFT --timeline --index=mft-timelineWith tags for host identification:
$ mft2es /path/to/your/$MFT --tags "WORKSTATION-1,DOMAIN-ABC" --index=host-analysisNote: The current version does not verify the certificate.
An additional feature: 🍣 🍣 🍣
Convert Windows Master File Table to a JSON file.
$ mft2json /path/to/your/$MFT -o /path/to/output/target.jsonWith tags for host identification:
$ mft2json /path/to/your/$MFT --tags "WORKSTATION-1,DOMAIN-ABC" -o /path/to/output/target.jsonConvert Windows Master File Table to a Python List[dict] object.
from mft2es import mft2json
if __name__ == '__main__':
filepath = '/path/to/your/$MFT'
result: List[dict] = mft2json(filepath)mft2es supports timeline analysis mode that creates MACB (Modified, Accessed, Created, Birth) timeline records for forensic investigation.
$ mft2es /path/to/your/$MFT --timeline --index=mft-timelineThis mode creates separate records for each timestamp type (M, A, C, B) from both StandardInformation and FileName attributes, making it easier to analyze file system activity over time.
[
{
"header": {
"signature": [
70,
73,
76,
69
],
"usa_offset": 48,
"usa_size": 3,
"metadata_transaction_journal": 172848302,
"sequence": 1,
"hard_link_count": 1,
"first_attribute_record_offset": 56,
"flags": "ALLOCATED",
"used_entry_size": 416,
"total_entry_size": 1024,
"base_reference": {
"entry": 0,
"sequence": 0
},
"first_attribute_id": 6,
"record_number": 0
},
"attributes": {
"StandardInformation": {
"header": {
"type_code": "StandardInformation",
"record_length": 96,
"form_code": 0,
"residential_header": {
"index_flag": 0
},
"name_size": 0,
"name_offset": null,
"data_flags": "(empty)",
"instance": 0,
"name": ""
},
"data": {
"created": "2019-03-11T16:42:33.593750Z",
"modified": "2019-03-11T16:42:33.593750Z",
"mft_modified": "2019-03-11T16:42:33.593750Z",
"accessed": "2019-03-11T16:42:33.593750Z",
"file_flags": "FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_SYSTEM",
"max_version": 0,
"version": 0,
"class_id": 0,
"owner_id": 0,
"security_id": 256,
"quota": 0,
"usn": 0
}
},
"FileName": {
"header": {
"type_code": "FileName",
"record_length": 104,
"form_code": 0,
"residential_header": {
"index_flag": 1
},
"name_size": 0,
"name_offset": null,
"data_flags": "(empty)",
"instance": 3,
"name": ""
},
"data": {
"parent": {
"entry": 5,
"sequence": 5
},
"created": "2019-03-11T16:42:33.593750Z",
"modified": "2019-03-11T16:42:33.593750Z",
"mft_modified": "2019-03-11T16:42:33.593750Z",
"accessed": "2019-03-11T16:42:33.593750Z",
"logical_size": 16384,
"physical_size": 16384,
"flags": "FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_SYSTEM",
"reparse_value": 0,
"name_length": 4,
"namespace": "Win32AndDos",
"name": "$MFT",
"path": "$MFT"
}
},
"DATA": {
"header": {
"type_code": "DATA",
"record_length": 72,
"form_code": 1,
"residential_header": {
"vnc_first": 0,
"vnc_last": "0x198f",
"unit_compression_size": 0,
"allocated_length": 62390272,
"file_size": 62390272,
"valid_data_length": 62390272,
"total_allocated": null
},
"name_size": 0,
"name_offset": null,
"data_flags": "(empty)",
"instance": 1,
"name": ""
},
"data": null
},
"BITMAP": {
"header": {
"type_code": "BITMAP",
"record_length": 80,
"form_code": 1,
"residential_header": {
"vnc_first": 0,
"vnc_last": 0,
"unit_compression_size": 0,
"allocated_length": 12288,
"file_size": 8200,
"valid_data_length": 8200,
"total_allocated": null
},
"name_size": 0,
"name_offset": null,
"data_flags": "(empty)",
"instance": 5,
"name": ""
},
"data": null
}
},
"tags": ["mft", "WORKSTATION-1", "DOMAIN-ABC"]
},
...
][
{
"@timestamp": "2007-06-30T12:50:52.252395Z",
"event": {
"action": "mft-standardinformation-m",
"category": [
"file"
],
"type": [
"change"
],
"kind": "event",
"provider": "mft",
"module": "windows",
"dataset": "windows.mft"
},
"windows": {
"mft": {
"record": {
"number": 0,
"name": "$MFT",
"path": "$MFT"
},
"header": {
"signature": [
70,
73,
76,
69
],
"usa_offset": 48,
"usa_size": 3,
"metadata_transaction_journal": 77648146,
"sequence": 1,
"hard_link_count": 1,
"first_attribute_record_offset": 56,
"flags": "ALLOCATED",
"used_entry_size": 424,
"total_entry_size": 1024,
"base_reference": {
"entry": 0,
"sequence": 0
},
"first_attribute_id": 6
},
"attribute": {
"type": "StandardInformation",
"macb_type": "M",
"header": {
"record_length": 96,
"form_code": 0,
"residential_header": {
"index_flag": 0
},
"name_size": 0,
"name_offset": null,
"data_flags": "(empty)",
"instance": 0,
"name": ""
},
"data": {
"file_flags": "FILE_ATTRIBUTE_HIDDEN | FILE_ATTRIBUTE_SYSTEM",
"max_version": 0,
"version": 0,
"class_id": 0,
"owner_id": 0,
"security_id": 256,
"quota": 0,
"usn": 0
}
}
}
},
"log": {
"file": {
"path": "/path/to/your/MFT"
}
},
"tags": [
"mft"
]
},
...
]$ pip install mft2esThe version compiled into a binary using Nuitka is also available for use.
$ chmod +x ./mft2es
$ ./mft2es {{options...}}> mft2es.exe {{options...}}The source code for mft2es is hosted on GitHub. You can download, fork, and review it from this repository: https://github.com/sumeshi/mft2es. Please report issues and feature requests. 🍣 🍣 🍣
- Tsurugi Linux [Lab] 2022 - 2024 - DFIR Linux distribution
Thank you for your interest in mft2es!
mft2es is released under the MIT License.
Powered by following libraries: