fluffy/README.md

344 lines
9.4 KiB
Markdown
Raw Normal View History

2013-07-23 16:30:38 -06:00
The Fluffy Suite
============
Fluffy was begun in April 2011 in Tennessee,
as a replacement for the aging "dirtbags.ip" codebase.
It is comprised of multiple small standalone binaries,
which are meant to be chained together,
either on the command-line or from a shell script,
to create a more powerful (and specific) piece of software.
Usually, a program expects input on stdin,
and produces output on stdout.
Flags are sparse by design.
2017-08-08 18:14:02 -06:00
Fluffy source code is purposefully spartan and easy to audit.
Forks are encouraged,
please let me know if you make one.
2013-07-23 16:30:38 -06:00
2020-11-17 15:48:04 -07:00
How To Build And Install
2018-06-11 17:57:35 -06:00
============
2017-07-09 11:21:46 -06:00
2020-11-17 15:48:04 -07:00
Ubuntu
-------
2020-11-17 15:48:04 -07:00
sudo apt install build-essential
2018-07-10 16:24:16 -06:00
curl -L https://github.com/dirtbags/fluffy/archive/master.tar.gz | tar xzvf -
cd fluffy-master
2020-11-17 15:48:04 -07:00
make
2020-11-17 14:56:19 -07:00
sudo make DESTDIR=/usr/local install
2020-11-17 15:48:04 -07:00
Red Hat
-------
2017-08-08 06:56:13 -06:00
2020-11-17 15:48:04 -07:00
yum groupinstall 'Development Tools'
2020-11-17 14:56:19 -07:00
curl -L https://github.com/dirtbags/fluffy/archive/master.tar.gz | tar xzvf -
cd fluffy-master
2020-11-17 15:48:04 -07:00
make
sudo make DESTDIR=/usr/local install
2017-07-09 11:21:46 -06:00
2020-11-17 14:51:26 -07:00
2020-11-17 15:48:04 -07:00
How To Uninstall
============
make DESTDIR=/usr/local uninstall
2020-11-17 14:51:26 -07:00
Forks and Packages
==================
2018-07-19 10:09:58 -06:00
## Ubuntu
pi-rho, a network archaeology instructor,
has forked these tools,
added command-line options,
manual pages,
and packaged them for Ubuntu.
This fork is mostly compatible with these tools,
but there are a few subtle differences.
If you are installing these for Cyber Fire,
you should probably stick with a source install.
[pi-rho's packages](https://launchpad.net/~pi-rho/+archive/ubuntu/security)
2018-07-19 10:09:58 -06:00
2018-06-11 17:57:35 -06:00
## Arch Linux
The AUR package [`fluffy-git`](https://aur.archlinux.org/packages/fluffy-git/)
builds against the latest revision and installs it to `/usr/bin`.
This was packaged by Cyber Fire attendee AGausmann.
Thanks!
2017-07-09 11:21:46 -06:00
Programs
2018-06-11 17:57:35 -06:00
========
2017-07-09 11:21:46 -06:00
2018-06-11 17:57:35 -06:00
## hd: Hex Dump
2013-07-23 16:30:38 -06:00
Like the normal hd,
but with unicode characters to represent all 256 octets,
instead of using "." for unprintable characters.
2018-07-10 16:24:16 -06:00
$ printf "\0\x01\x02\x03\x30\x52\x9a" | hd
2020-12-22 14:28:20 -07:00
00000000 00 01 02 03 30 52 9a ·☺☻♥0RÜ
2018-07-10 16:24:16 -06:00
00000007
2020-03-17 11:00:48 -06:00
Also like the normal hd,
2020-12-22 09:15:56 -07:00
this one will print an ellipsis if the preceding 16 octets are repeated.
2020-03-17 11:00:48 -06:00
Use the offset printed next to determine how many repeats you have.
2020-12-22 09:15:56 -07:00
$ printf '%64s' hello | hd
2020-03-17 11:00:48 -06:00
00000000 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
2020-12-22 09:15:56 -07:00
2020-03-17 11:00:48 -06:00
00000030 20 20 20 20 20 20 20 20 20 20 20 68 65 6c 6c 6f hello
00000040
2020-12-22 09:15:56 -07:00
You can disable this with `-v`
$ printf '%64s' hello | hd
00000000 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00000010 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00000020 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20 20
00000030 20 20 20 20 20 20 20 20 20 20 20 68 65 6c 6c 6f hello
00000040
2013-07-23 16:30:38 -06:00
2018-06-11 17:57:35 -06:00
## unhex: unescape hex
2017-07-09 11:21:46 -06:00
Reads ASCII hex codes on stdin,
writes those octets to stdout.
2018-07-10 16:24:16 -06:00
$ echo 68 65 6c 6c 6f 0a | unhex
hello
2017-07-09 11:21:46 -06:00
2018-06-11 17:57:35 -06:00
## xor: xor octets
2017-07-09 11:21:46 -06:00
Applies the given mask as an xor to input.
The mask will be repeated,
so for a 1-value mask, every octet is xored against that value.
For a 16-value mask, the mask is applied to 16-octet chunks at a time.
The "-x" option treats values as hex.
2018-07-10 16:24:16 -06:00
$ printf 'hello' | xor 22; echo
~szzy
$ printf 'hello' | xor 0x16; echo
~szzy
$ printf 'hello' | xor -x 16; echo
~szzy
$ printf 'bbbbbb' | xor 1 0; echo
cbcbcb
2017-07-09 11:21:46 -06:00
2018-06-11 17:57:35 -06:00
## slice: slice octet stream
2017-08-08 17:55:05 -06:00
Slices up input octet stream,
similar to Python's slice operation.
2017-08-08 17:55:05 -06:00
2020-12-22 09:15:56 -07:00
$ printf '0123456789abcdef' | slice 2; echo
2018-07-10 16:24:16 -06:00
23456789abcdef
2020-12-22 09:15:56 -07:00
$ printf '0123456789abcdef' | slice 2 6; echo
2018-07-10 16:24:16 -06:00
2345
2020-12-22 09:15:56 -07:00
$ printf '0123456789abcdef' | slice 2 6 8; echo
2018-07-10 16:24:16 -06:00
234589abcdef
2020-12-22 09:15:56 -07:00
$ printf '0123456789abcdef' | slice 2 6 8 0xa
2018-07-10 16:24:16 -06:00
234589
2017-08-08 17:55:05 -06:00
2018-06-11 17:57:35 -06:00
## pcat: print text representation of pcap file
2013-07-23 16:30:38 -06:00
Prints a (lossy) text representation of a pcap file to stdout.
2017-08-08 17:55:05 -06:00
2013-07-23 16:30:38 -06:00
This program is the keystone of the Fluffy Suite.
By representing everything as text,
programmers can use any number of standard Unix text processing tools,
such as sed, awk, cut, grep, or head.
2017-08-08 17:55:05 -06:00
Output is tab-separated, of the format:
timestamp protocol src dst options payload
2017-08-08 17:55:05 -06:00
Frequently you are only interested in the payload,
so you can run pcat like:
2020-12-22 09:15:56 -07:00
$ cat myfile.pcap | pcat | cut -f 6
2017-08-08 17:55:05 -06:00
Remember the `unhex` program,
which will convert payloads to an octet stream,
after you have done any maniuplations you want.
2013-07-23 16:30:38 -06:00
## pmerge: merge pcap files
2013-07-23 16:30:38 -06:00
Takes a list of pcap files, assuming they are sorted by time
(you would have to work hard to create any other kind),
and merges them into a single sorted output.
2018-06-11 17:57:35 -06:00
## puniq: omit repeated frames
2013-07-23 16:30:38 -06:00
Removes duplicate frames from input,
2013-07-23 16:30:38 -06:00
writing to output.
2018-06-11 17:57:35 -06:00
## hex: hex-encode input
2017-08-08 18:14:02 -06:00
The opposite of `unhex`:
encoding all input into a single output line.
2017-08-08 18:14:02 -06:00
This differs from `hexdump` in the following ways:
* All input is encoded into a single line of output
* Does not output offsets
* Does not output glyph representations of octets
In other words: you can feed `hex` output into `unhex` with no manipulations.
2018-07-10 16:24:16 -06:00
$ printf "hello\nworld\n" | hex
68 65 6c 6c 6f 0a 77 6f 72 6c 64 0a
$ printf A | hex
41
2017-08-08 17:55:05 -06:00
2018-06-11 17:57:35 -06:00
## entropy: compute shannon entropy
2018-01-09 09:42:14 -07:00
Displays the Shannon entropy of the input.
2020-12-22 09:15:56 -07:00
$ echo -n a | ./entropy
2018-07-10 16:24:16 -06:00
0.000000
2020-12-22 09:15:56 -07:00
$ echo -n aaaaaaaaa | ./entropy
2018-07-10 16:24:16 -06:00
0.000000
2020-12-22 09:15:56 -07:00
$ echo -n aaaaaaaaab | ./entropy
2018-07-10 16:24:16 -06:00
0.468996
2020-12-22 09:15:56 -07:00
$ echo -n aaaaaaaaabc | ./entropy
2018-07-10 16:24:16 -06:00
0.865857
2018-01-09 09:42:14 -07:00
2018-06-11 17:57:35 -06:00
## pyesc: python escape input
2013-07-23 16:30:38 -06:00
Escapes input octets for pasting into a python "print" statement.
Also suitable for use as a C string,
a Go string,
and many other languages' string literals.
2013-07-23 16:30:38 -06:00
2018-07-10 16:24:16 -06:00
$ printf "hello\nworld\n" | pyesc
hello\nworld\n
2017-08-13 11:52:17 -06:00
2018-06-11 17:57:35 -06:00
## octets: display all octets
2017-08-13 11:52:17 -06:00
Shows all octets from `00` to `ff` in a hex dump.
This is occasionally more helpful than `man ascii`.
$ octets
2020-12-22 14:28:20 -07:00
00000000 00 01 02 03 04 05 06 07 08 09 0a 0b 0c 0d 0e 0f ·☺☻♥♦♣♠•◘○◙♂♀♪♫☼
00000010 10 11 12 13 14 15 16 17 18 19 1a 1b 1c 1d 1e 1f ⏵⏴↕‼¶§‽↨↑↓→←∟↔⏶⏷
2018-07-10 16:24:16 -06:00
00000020 20 21 22 23 24 25 26 27 28 29 2a 2b 2c 2d 2e 2f !"#$%&'()*+,-./
00000030 30 31 32 33 34 35 36 37 38 39 3a 3b 3c 3d 3e 3f 0123456789:;<=>?
00000040 40 41 42 43 44 45 46 47 48 49 4a 4b 4c 4d 4e 4f @ABCDEFGHIJKLMNO
00000050 50 51 52 53 54 55 56 57 58 59 5a 5b 5c 5d 5e 5f PQRSTUVWXYZ[\]^_
00000060 60 61 62 63 64 65 66 67 68 69 6a 6b 6c 6d 6e 6f `abcdefghijklmno
00000070 70 71 72 73 74 75 76 77 78 79 7a 7b 7c 7d 7e 7f pqrstuvwxyz{|}~⌂
00000080 80 81 82 83 84 85 86 87 88 89 8a 8b 8c 8d 8e 8f ÇüéâäàåçêëèïîìÄÅ
00000090 90 91 92 93 94 95 96 97 98 99 9a 9b 9c 9d 9e 9f ÉæÆôöòûùÿÖÜ¢£¥₧ƒ
000000a0 a0 a1 a2 a3 a4 a5 a6 a7 a8 a9 aa ab ac ad ae af áíóúñѪº¿⌐¬½¼¡«»
000000b0 b0 b1 b2 b3 b4 b5 b6 b7 b8 b9 ba bb bc bd be bf ░▒▓│┤╡╢╖╕╣║╗╝╜╛┐
000000c0 c0 c1 c2 c3 c4 c5 c6 c7 c8 c9 ca cb cc cd ce cf └┴┬├─┼╞╟╚╔╩╦╠═╬╧
000000d0 d0 d1 d2 d3 d4 d5 d6 d7 d8 d9 da db dc dd de df ╨╤╥╙╘╒╓╫╪┘┌█▄▌▐▀
000000e0 e0 e1 e2 e3 e4 e5 e6 e7 e8 e9 ea eb ec ed ee ef αßΓπΣσµτΦΘΩδ∞φε∩
2020-12-22 14:28:20 -07:00
000000f0 f0 f1 f2 f3 f4 f5 f6 f7 f8 f9 fa fb fc fd fe ff ≡±≥≤⌠⌡÷≈°∞⊻√ⁿ²■¤
2017-08-13 11:52:17 -06:00
00000100
2018-06-11 17:57:35 -06:00
2020-12-22 09:15:56 -07:00
## freq: count octet frequencies
For all 256 octets,
show frequency of each in input.
$ printf 'hello' | freq
1 65 e
1 68 h
2 6c l
1 6f o
2020-12-22 14:29:33 -07:00
$ printf 'hello' | freq -a
2020-12-22 09:15:56 -07:00
0 00 ·
0 01 ☺
0 02 ☻
0 03 ♥
0 04 ♦
0 05 ♣
0 06 ♠
0 07 •
0 08 ◘
...
## histogram: display histogram for input
Reads the first number of each line, and prints a histogram.
`-d DIVISOR` will divide each bar's width.
$ echo 'aaaaaaaaAAAAAAAAaaaaaaaa' | freq | histogram
0a ◙ # 1
41 A ######## 8
61 a ################ 16
$ echo 'aaaaaaaaAAAAAAAAaaaaaaaa' | freq | histogram -d 4
0a ◙ 1
41 A ## 8
61 a #### 16
2018-06-11 17:57:35 -06:00
Example Recipes
===============
## Brute force single-byte xor
2018-06-12 14:25:25 -06:00
for i in $(seq 255); do cat data | xor $i; done
2018-06-11 17:57:35 -06:00
## Pretty xor brute force
For each attempt, display the value used in the xor, and hexdump the result
for i in $(seq 255); do printf "=== %02x\n" $i; cat data | xor $i | hd; done
## Brute force xor of base64-encoded data
Same pretty-print as before, and also pipe to `less` so we can page through it.
for i in $(seq 255); do
printf "=== %02x\n" $i; cat data.txt | base64 -d | xor $i | hd
done | less
## Protocol manipulation
For each ICMP packet, drop the first 5 octets, and base64-decode the remainder, preserving conversation chunks
cat input.pcap | pcat | grep ICMP | while read ts proto src dst payload; do
2018-06-11 17:57:35 -06:00
printf "%s -> %s (%s)\n" $src $dst $ts
echo $payload | unhex | slice 5 | base64 -d | hd
done
## Elementary protocol analysis framework
This merges (by time) `file1.pcap` and `file2.pcap`,
decoding payloads from each one,
hex dumping payloads,
and displaying meta information about each.
It displays information conversationally,
sort of like wireshark's "Follow TCP Stream",
but with more details about meta-information.
./pmerge file1.pcap file2.pcap | ./pcat | while read ts proto src dst payload; do
when=$(TZ=Z date -d @${ts%.*} "+%Y-%m-%d %H:%M:%S")
printf "Packet %s None: None\n" $proto
printf " %s -> %s (%s)\n" ${src%,*} ${dst%,*} "$when"
echo $payload | ./unhex | ./hd
echo
done