mirror of
https://github.com/CyberSecurityUP/NeuroSploit.git
synced 2026-06-30 07:15:30 +02:00
CyberSecurityUP
55af0d4634
Re-model the pentest agent into an autonomous, markdown-driven engine that turns a URL into a full engagement and delegates execution to a locally installed agentic CLI backend. Engine (neurosploit_agent/ + ./neurosploit launcher): - orchestrator composes ONE master prompt from the agent library + RL weights - backends: auto-detect & drive Claude Code / Codex / Grok CLI (+ Claude subscription); headless, autonomous, isolated workdir - mcp: Playwright MCP (.mcp.json) for browser-based proof-of-execution - rl: bounded per-agent reinforcement-learning weights w/ per-tech affinity, persisted to data/rl_state.json - models: latest registry incl. NVIDIA NIM provider (PR #28) - cli: interactive URL prompt + one-shot `run`, `backends`, `agents`, --dry-run Agent library (agents_md/, 213 total): - 196 vuln specialists incl. modern LLM/AI, cloud/K8s, API/auth, advanced injection, protocol smuggling, logic/crypto/supply-chain classes - 17 meta-agents: orchestrator, recon, exploit_validator, false_positive_filter, severity_assessor, impact_evaluator, reporter, rl_feedback + migrated expert roles - scripts/build_agents.py data-driven builder; REGISTRY.md index Docs: rewritten README.md, v3.3.0 RELEASE.md, .env.example (NVIDIA NIM, xAI, engine vars). Retire legacy Python orchestration (neurosploit.py + agent classes) to legacy/. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
345 lines
20 KiB
Markdown
345 lines
20 KiB
Markdown
|
||
|
||
----------
|
||
Chapter 1: Introduction
|
||
========
|
||
About Kali Linux
|
||
------------------------
|
||
|
||
> [Kali Linux](https://www.kali.org/) is a Debian-based Linux distribution aimed at advanced Penetration Testing and Security Auditing. Kali contains several hundred tools which are geared towards various information security tasks, such as Penetration Testing, Security research, Computer Forensics and Reverse Engineering. Kali Linux is developed, funded and maintained by [Offensive Security](http://www.offensive-security.com/), a leading information security training company.
|
||
|
||
Kali Linux was released on the 13th March, 2013 as a complete, top-to-bottom rebuild of [BackTrack Linux](http://www.backtrack-linux.org/), adhering completely to Debian development standards.
|
||
|
||
Linux Basics
|
||
---------------
|
||
You should aware of some basics of Linux commands which will be used and come in handy and will be lot helpful. Here only basics are covered and more detail can be found at this [link](https://www.digitalocean.com/community/tutorials/an-introduction-to-linux-i-o-redirection)
|
||
**Streams**
|
||
Input and output in the Linux environment is distributed across three streams. These streams are:
|
||
|
||
standard input (stdin) # typically carries data from a user to a program
|
||
standard output (stdout) # writes the data that is generated by a program
|
||
standard error (stderr) # writes the errors generated by a program that has failed at some point in its execution
|
||
The streams are also numbered:
|
||
|
||
stdin (0) # cat
|
||
stdout (1) # echo
|
||
stderr (2)
|
||
**Stream Redirection**
|
||
Linux includes redirection commands for each stream. These commands write standard output to a file. If a non-existent file is targetted (either by a single-bracket or double-bracket command), a new file with that name will be created prior to writing.
|
||
|
||
Commands with a single bracket overwrite the destination's existing contents.
|
||
|
||
Overwrite
|
||
|
||
> - standard output
|
||
< - standard input
|
||
2> - standard error
|
||
|
||
Commands with a double bracket do not overwrite the destination's existing contents.
|
||
|
||
Append
|
||
|
||
>> - standard output
|
||
<< - standard input
|
||
2>> - standard error
|
||
**Pipes**
|
||
Pipes (vertical bar `*|*`) are used to redirect a stream from one program to another. When a program's standard output is sent to another through a pipe, the first program's data, which is received by the second program, will not be displayed on the terminal. Only the filtered data returned by the second program will be displayed.
|
||
**Filters**
|
||
Filters are commands that alter piped redirection and output.
|
||
>filter commands are also standard Linux commands that can be used without pipes.
|
||
|
||
* `find` - returns files with filenames that match the argument passed to find.
|
||
* `grep` - returns text that matches the string pattern passed to grep.
|
||
* `tee` - redirects standard input to both standard output and one or more files. (typically used to view a program's output while simultaneously saving it to a file.)
|
||
* `tr` - finds-and-replaces one string with another.
|
||
* `wc` - counts characters, lines, and words.
|
||
|
||
About Penetration Testing
|
||
----------------------------------
|
||
**vulnerability assessment :** simply identifies and reports noted vulnerabilities
|
||
**penetration test(Pen Test)** attempts to exploit the vulnerabilities to determine whether unauthorized access or other malicious activity is possible. Penetration testing typically includes network penetration testing and application security testing as well as controls and processes around the networks and applications, and should occur from both outside the network trying to come in (external testing) and from inside the network.
|
||
|
||
an authorised simulated attack on a computer system, performed to evaluate the security of the system. The test is performed to identify both weaknesses (also referred to as vulnerabilities), including the potential for unauthorized parties to gain access to the system's features and data,as well as strengths, enabling a full risk assessment to be completed.
|
||
|
||
***Penetration testing tools*** are used as part of a penetration test(Pen Test) to automate certain tasks, improve testing efficiency and discover issues that might be difficult to find using manual analysis techniques alone. Two common penetration testing tools are static analysis tools and dynamic analysis tools.
|
||
|
||
|
||
Legal
|
||
------
|
||
> As one might expect, there are a wealth of legal issues that are associated with information security. Whether it’s a matter of preventing security breaches in order to maintain the security of your client information (or that of your organization), or simply realizing exactly how far one’s obligations go when it comes to information security, it’s important to realize exactly what your obligations are as far as the legal world goes with information security.
|
||
|
||
Because technology is ever-changing, there are always questions about what the legal protections might be when it comes to the misuse of new technology, or even what sort of jurisdiction might govern your organization or its clients. One of the biggest problems with computer crime is that laws still aren’t clear as to who polices what online, if anything. As a result, companies must protect themselves against an attack on their internal servers and other information that might be at risk.
|
||
**Major Issues**
|
||
- One of the biggest issues that organizations will face as far as maintaining your information security goes is that technology is developing so quickly that it is hard for the legal system to keep up. Even if you have taken the time to amass evidence against those who may have breached your information security system, there are no guarantees that this evidence will even be admissible in a court of law.
|
||
- Penetration testing may affect system performance, and can raise confidentiality and integrity issues; therefore, this is very important, even in an internal penetration testing, which is performed by an internal staff to get permission in writing. There should be a written agreement between a tester and the company/organization/individual to clarify all the points regarding the data security, disclosure, etc. before commencing testing.
|
||
> One consideration that pen testers should be aware of is the laws surrounding the practice of port scanning.
|
||
|
||
You need to consider exactly how tightly your pen test will need to scan the systems that you are authorized to scan. Also, ensure you have permission to conduct the scan with a legitimate reason to do so; it is far easier to ask permission in this case than to beg forgiveness.
|
||
|
||
|
||
----------
|
||
|
||
|
||
Chapter 2: The Essential Tools
|
||
========
|
||
Netcat
|
||
--------
|
||
> This simple utility reads and writes data across TCP or UDP network connections. It is designed to be a reliable back-end tool to use directly or easily drive by other programs and scripts. At the same time, it is a feature-rich network debugging and exploration tool, since it can create almost any kind of connection you would need, including port binding to accept incoming connections.
|
||
|
||
Official website: http://nc110.sourceforge.net/
|
||
### Features
|
||
The original netcat's features include:
|
||
|
||
* Outbound or inbound connections, TCP or UDP, to or from any ports
|
||
* Full DNS forward/reverse checking, with appropriate warnings
|
||
* Ability to use any local source port
|
||
* Ability to use any locally configured network source address
|
||
* Built-in port-scanning capabilities, with randomization
|
||
* Built-in loose source-routing capability
|
||
* Can read command line arguments from standard input
|
||
* Slow-send mode, one line every N seconds
|
||
* Hex dump of transmitted and received data
|
||
* Optional ability to let another program service establish connections
|
||
* Optional telnet-options responder
|
||
* Featured tunneling mode which permits user-defined tunneling, e.g., UDP or TCP, with the possibility of specifying all network parameters (source port/interface, listening port/interface, and the remote host allowed to connect to the tunnel).
|
||
|
||
#### The Basics
|
||
The most basic syntax is:
|
||
|
||
$ netcat [options] host port
|
||
This will attempt to initiate a TCP to the defined host on the port number specified. This is basically functions similarly to the old Linux telnet command. Keep in mind that your connection is entirely unencrypted.
|
||
|
||
If you would like to send a UDP packet instead of initiating a TCP connection, you can use the -u option:
|
||
|
||
$ netcat -u host port
|
||
You can specify a range of ports by placing a dash between the first and last:
|
||
|
||
$ netcat host startport-endport
|
||
### Netcat for Port Scanning
|
||
the most common uses for netcat is as a port scanner.
|
||
|
||
$ netcat -z -v domain.com 1-10000
|
||
`-z` - to perform a scan instead of attempting to initiate a connection
|
||
`-v` - provide more verbose information.
|
||
`1-10000` - scan all ports up to 10000 by issuing this command
|
||
Output:
|
||
|
||
nc: connect to domain.com port 1 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 2 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 3 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 4 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 5 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 6 (tcp) failed: Connection refused
|
||
nc: connect to domain.com port 7 (tcp) failed: Connection refused
|
||
. . .
|
||
Connection to domain.com 22 port [tcp/ssh] succeeded!
|
||
. . .
|
||
Connection to domain.com 8000 port [tcp/*] succeeded!
|
||
|
||
> scan will go much faster if you know the IP address that you need. You can then use the `-n` flag to specify that you do not need to resolve the IP address using DNS
|
||
|
||
Another example:
|
||
|
||
Checking whether UDP ports (-u) 27010-27015 are open on 209.58.178.32 using zero mode I/O (-z)
|
||
|
||
$ nc -vzu 209.58.178.32 27010-27015
|
||
Connection to 209.58.178.32 27015 port [udp/*] succeeded!
|
||
|
||
\* for education purpose only I have use ip of open server for the game counter strike
|
||
### Communicate through Netcat
|
||
|
||
Netcat can listen on a port for connections and packets. This gives us the opportunity to connect two instances of netcat in a client-server relationship.
|
||
|
||
On one machine, you can tell netcat to listen to a specific port for connections. We can do this by providing the `-l` parameter and choosing a port:
|
||
|
||
$ netcat -l 4444
|
||
|
||
As a regular (non-root) user, you will not be able to open any ports under 1000, as a security measure.
|
||
On another machine we'll connect to the first machine on the port number we choose
|
||
|
||
$ netcat domain.com 4444
|
||
|
||
### File Transfer with NetCat
|
||
Because we are establishing a regular TCP connection, we can transmit just about any kind of information over that connection. It is not limited to chat messages that are typed in by a user. We can use this knowledge to turn netcat into a file transfer program.
|
||
|
||
again, we need to choose one end of the connection to listen for connections. However, instead of printing information onto the screen, we will place all of the information straight into a file.
|
||
|
||
$ netcat -l 4444 > received_file
|
||
On other machine transfer the file as:
|
||
|
||
netcat domain.com 4444 < original_file
|
||
For instance, we can transfer the contents of an entire directory by creating an unnamed tarball on-the-fly, transferring it to the remote system, and unpacking it into the remote directory.
|
||
|
||
On the receiving end, we can anticipate a file coming over that will need to be unzipped and extracted by typing:
|
||
|
||
$ netcat -l 4444 | tar xzvf -
|
||
the ending dash (`-`) means that tar will operate on standard input, which is being piped from netcat across the network when a connection is made.
|
||
On the side with the directory contents we want to transfer, we can pack them into a tarball and then send them to the remote computer through netcat:
|
||
|
||
$ tar -czf - * | netcat domain.com 4444
|
||
This time, the dash (`-`) in the tar command means to tar and zip the contents of the current directory (as specified by the `*` wildcard), and write the result to standard output.
|
||
> use the `dd` command to image a disk on one side and transfer it to a remote computer.
|
||
|
||
### Netcat as a Simple Web Server
|
||
create a HTML `index.html` file and serve it to desire port address (as previously you can not host to port below 1000 as non root user)
|
||
|
||
printf 'HTTP/1.1 200 OK\n\n%s' "$(cat index.html)" | netcat -l 8888
|
||
This will serve the page, and then the netcat connection will close. If you attempt to refresh the page, it will be gone
|
||
We can have netcat serve the page indefinitely by wrapping the last command in an infinite loop, as:
|
||
|
||
while true; do printf 'HTTP/1.1 200 OK\n\n%s' "$(cat index.html)" | netcat -l 8888; done
|
||
----------
|
||
***Ncat***
|
||
Ncat is a feature-packed networking utility which reads and writes data across networks from the command line. Ncat was written for the Nmap Project as a much-improved reimplementation of the venerable Netcat. It uses both TCP and UDP for communication and is designed to be a reliable back-end tool to instantly provide network connectivity to other applications and users. Ncat will not only work with IPv4 and IPv6 but provides the user with a virtually limitless number of potential uses.
|
||
|
||
Among Ncat’s vast number of features there is the ability to chain Ncats together, redirect both TCP and UDP ports to other sites, SSL support, and proxy connections via SOCKS4 or HTTP (CONNECT method) proxies (with optional proxy authentication as well). Some general principles apply to most applications and thus give you the capability of instantly adding networking support to software that would normally never support it.
|
||
|
||
|
||
----------
|
||
Wireshark
|
||
-------------
|
||
> Official document: https://www.wireshark.org/docs/wsug_html_chunked/
|
||
> Other helpful link(s):
|
||
> https://www.howtogeek.com/104278/how-to-use-wireshark-to-capture-filter-and-inspect-packets/
|
||
|
||
Wireshark is a network packet analyzer. A network packet analyzer will try to capture network packets and tries to display that packet data as detailed as possible.
|
||
|
||
Wireshark is a free application that allows you to capture and view the data traveling back and forth on your network, providing the ability to drill down and read the contents of each packet – filtered to meet your specific needs. It is commonly utilized to troubleshoot network problems as well as to develop and test software. This open-source protocol analyzer is widely accepted as the industry standard, winning its fair share of awards over the years.
|
||
|
||
## Why use Wireshark?
|
||
- Network administrators use it to troubleshoot network problems
|
||
- Network security engineers use it to examine security problems
|
||
- QA engineers use it to verify network applications
|
||
- Developers use it to debug protocol implementations
|
||
- People use it to learn network protocol internals
|
||
|
||
### Features
|
||
- _Capture_ live packet data from a network interface.
|
||
- _Open_ files containing packet data captured with tcpdump/WinDump, Wireshark, and a number of other packet capture programs.
|
||
- _Import_ packets from text files containing hex dumps of packet data.
|
||
- Display packets with _very detailed protocol information_.
|
||
- _Filter packets_ on many criteria.
|
||
: i.e. IPv4 address, IPv6 address, ethernet address, port, tcp, udp etc.
|
||
- _Search_ for packets on many criteria.
|
||
- Create various _statistics_.
|
||
|
||
## Making Sense of Network Dumps
|
||
## Capture and Display Filters
|
||
Some of the filters are as below:
|
||
|
||
filter packets if ipv4 address is equal to 54.36.48.153 (using `eq` or `==`)
|
||
|
||
ip.addr eq 54.36.48.153
|
||
you can use multiple expression with `and` or `&&`
|
||
|
||
ip.addr eq 54.36.48.153 and tcp.stream eq 6
|
||
|
||
get conversation with specific ip and port
|
||
|
||
(ip.addr eq 54.36.48.153 and ip.addr eq 200.200.200.9) and (tcp.port eq 8000 and tcp.port eq 34018)
|
||
|
||
Look at below filter options in wireshark, here various available filter with example expression and as per requirement we can combine various filter with various Boolean operators
|
||
|
||
|
||
## Following TCP Streams
|
||
A good [link](https://www.youtube.com/watch?time_continue=4&v=xPgCZwj446o) to learn in detail how to follow tcp stream:
|
||
|
||
|
||
|
||
----------
|
||
|
||
|
||
Tcpdump
|
||
-----------
|
||
Official [site](https://www.tcpdump.org/tcpdump_man.html)
|
||
|
||
other references:
|
||
https://linux.die.net/man/8/tcpdump
|
||
https://danielmiessler.com/study/tcpdump/
|
||
> Tcpdump is the premier network analysis tool for information security professionals.
|
||
|
||
When using a tool that displays network traffic a more natural (raw) way the burden of analysis is placed directly on the human rather than the application. This approach cultivates continued and elevated understanding of the TCP/IP suite
|
||
### Options
|
||
|
||
- **`-i any`** : Listen on all interfaces just to see if you’re seeing any traffic.
|
||
- **`-i eth0`** : Listen on the eth0 interface.
|
||
- **`-D`** : Show the list of available interfaces
|
||
- **`-n`** : Don’t resolve hostnames.
|
||
- **`-nn`** : Don’t resolve hostnames _or_ port names.
|
||
- **`-q`** : Be less verbose (more quiet) with your output.
|
||
- **`-t`** : Give human-readable timestamp output.
|
||
- **`-tttt`** : Give maximally human-readable timestamp output.
|
||
- **`-X`** : Show the packet’s _contents_ in both [hex](https://en.wikipedia.org/wiki/Hexidecimal) and [ascii](https://en.wikipedia.org/wiki/Ascii).
|
||
- **`-XX`** : Same as **`-X`**, but also shows the ethernet header.
|
||
- **`-v, -vv, -vvv`** : Increase the amount of packet information you get back.
|
||
- **`-c`** : Only get _x_ number of packets and then stop.
|
||
- **`-s`** : Define the _snaplength_ (size) of the capture in bytes. Use `-s0` to get everything, unless you are intentionally capturing less.
|
||
- **`-S`** : Print absolute sequence numbers.
|
||
- **`-e`** : Get the ethernet header as well.
|
||
- **`-q`** : Show less protocol information.
|
||
- **`-E`** : Decrypt IPSEC traffic by providing an encryption key.
|
||
### Expressions
|
||
|
||
In `tcpdump`, _Expressions_ allow you to trim out various types of traffic and find exactly what you’re looking for. Mastering the expressions and learning to combine them creatively is what makes one truly powerful with `tcpdump`.
|
||
|
||
There are three main types of expression: `type`, `dir`, and `proto`.
|
||
|
||
- Type options are: `host`, `net`, and `port`.
|
||
- Direction lets you do `src`, `dst`, and combinations thereof.
|
||
- Proto(col) lets you designate: `tcp`, `udp`, `icmp`, `ah`, and many more.
|
||
## Filtering Traffic
|
||
**Filtering hosts:**
|
||
| | |
|
||
|--|--|
|
||
| Match any traffic involving 192.168.1.1 as destination or source | `$ tcpdump -i eth1 host 192.168.1.1` |
|
||
| As source only | `$ tcpdump -i eth1 src host 192.168.1.1` |
|
||
| As destination only | `$ tcpdump -i eth1 dst host 192.168.1.1` |
|
||
**Filtering ports :**
|
||
| | |
|
||
|--|--|
|
||
| Match any traffic involving port 25 as source or destination | `$ tcpdump -i eth1 port 25` |
|
||
| As source only | `$ tcpdump -i eth1 src port 25` |
|
||
| As destination only | `$ tcpdump -i eth1 dst port 25` |
|
||
**Network filtering :**
|
||
|
||
$ tcpdump -i eth1 net 192.168
|
||
$ tcpdump -i eth1 src net 192.168
|
||
$ tcpdump -i eth1 dst net 192.168
|
||
**Protocol filtering :**
|
||
|
||
$ tcpdump -i eth1 arp
|
||
$ tcpdump -i eth1 ip
|
||
|
||
$ tcpdump -i eth1 tcp
|
||
$ tcpdump -i eth1 udp
|
||
$ tcpdump -i eth1 icmp
|
||
***Combine expressions :***
|
||
*Negation* : `!` or `not` (without the quotes)
|
||
*Concatanate* : `&&` or `and`
|
||
*Alternate* : `||` or `or`
|
||
|
||
- This rule will match any TCP traffic on port `80` (web) with `192.168.1.254` or `192.168.1.200` as destination host
|
||
|
||
`$ tcpdump -i eth1 '((tcp) and (port 80) and ((dst host 192.168.1.254) or (dst host 192.168.1.200)))'`
|
||
|
||
- Will match any ICMP traffic involving the destination with physical/MAC address `00:01:02:03:04:05`
|
||
|
||
`$ tcpdump -i eth1 '((icmp) and ((ether dst host 00:01:02:03:04:05)))'`
|
||
|
||
- Will match any traffic for the destination network `192.168` except destination host `192.168.1.200`
|
||
|
||
`$ tcpdump -i eth1 '((tcp) and ((dst net 192.168) and (not dst host 192.168.1.200)))'`
|
||
|
||
## Advanced Header Filtering
|
||
> Helpful [link](https://www.wains.be/pub/networking/tcpdump_advanced_filters.txt)
|
||
| | |
|
||
|--|--|
|
||
| `proto[x:y]` | will start filtering from byte `x` for `y` bytes. `ip[2:2]` would filter bytes `3` and `4` (first byte begins by 0) |
|
||
| `proto[x:y] & z = 0` | will *match* bits set to `0` when applying `mask z` to `proto[x:y]`
|
||
| `proto[x:y] & z !=0` | some bits are *set* when applying `mask z` to `proto[x:y]`
|
||
| `proto[x:y] & z = z` | *every* bits are *set* to `z` when applying `mask z` to `proto[x:y]`
|
||
| `proto[x:y] = z` | `p[x:y]` has exactly the bits set to `z`
|
||
|
||
**IP header**
|
||
|