Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Analyze an email #30

Open
ghost opened this issue Oct 15, 2019 · 29 comments
Open

Analyze an email #30

ghost opened this issue Oct 15, 2019 · 29 comments
Assignees
Labels
User Issue Issue affecting a user

Comments

@ghost
Copy link

ghost commented Oct 15, 2019

I believe this feature works, I am just not easily understanding how to present it the header? I select option 1 and I get this:
Error Opening File

Extracting Headers...
Header Error
Traceback (most recent call last):
File "Sooty.py", line 592, in analyzePhish
print(" FROM: ", str(msg.SenderName), ", ", str(msg.SenderEmailAddress))
UnboundLocalError: local variable 'msg' referenced before assignment

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "Sooty.py", line 996, in
mainMenu()
File "Sooty.py", line 165, in mainMenu
switchMenu(input())
File "Sooty.py", line 54, in switchMenu
phishingMenu()
File "Sooty.py", line 573, in phishingMenu
phishingSwitch(input())
File "Sooty.py", line 101, in phishingSwitch
analyzePhish()
File "Sooty.py", line 603, in analyzePhish
f.close()

I realize it is an error saying that it didn't receive any input but where do I put the header?

Thanks,

@bc-jcarlson
Copy link

I also receive this exception when trying to open a .eml file exported from Gmail. I suspect it's expecting a different format.

@ghost
Copy link
Author

ghost commented Oct 15, 2019

According to the readme it only supports .msg / Outlook files. However I cannot figure out how it wants me to give it the data. All the other menus prompt for input. I have all the API keys input.

@TheresAFewConors TheresAFewConors self-assigned this Oct 15, 2019
@TheresAFewConors TheresAFewConors added the User Issue Issue affecting a user label Oct 15, 2019
@TheresAFewConors
Copy link
Owner

@dbailey-indofmd-org @bc-jcarlson Sooty at the moment only works with .msg files, however .eml analysis is coming in the next update. There is quite a large difference in each of the files' headers so they have to be added separately. The plan is to have it self-detect what the mail type is and perform the analysis regardless though. I'm hoping to have the update for .eml out in roughly Early Nov. If you want, I can update this issue to let you know when the update is out, or if you add Sooty to your watch list you will be automatically updated.

@TheresAFewConors
Copy link
Owner

modified the readme to make it clearer that .eml isn't supported at the moment.

@ghost
Copy link
Author

ghost commented Oct 16, 2019

Awesome. Thanks for the info. Curious still about how to point Sooty at the .msg file. I don't get a dialog after selecting the #1 and I have tried placing the file just before and just after the selection and it fails with the error I referenced above. I have also tried placing it in the Sooty directory.

@TheresAFewConors
Copy link
Owner

TheresAFewConors commented Oct 16, 2019

@dbailey-indofmd-org A wiki is on the way to show where all the options are, since there's a lot in there now. For now, from the main menu select 'Option 6 - Phishing Analysis' to open the phishing menu, then you have a choice of either Option 1 - analyze the headers and contents, or Option 3 - to generate out a basic template based on the same analysis in Option 1. You should get a file explorer pop-up to select the file, so it doesn't matter where it is downloaded as long as it can be read - hope this helps!

EDIT: Spelling is hard

@ghost
Copy link
Author

ghost commented Oct 16, 2019

It does. I am running this in WSL Ubuntu and was expecting different interaction. I will install on Windows and see how it goes.

@TheresAFewConors
Copy link
Owner

ah apologies, there's a known issue with *nix systems at the moment to do with the package being used for .msg analysis. I'm looking to write a different one from scratch for the same reason as the .eml mails. If you have any issue on Windows let me know and I'll take a look at it for you

@ghost
Copy link
Author

ghost commented Oct 16, 2019

Cool. Thanks for all the help.

@ghost
Copy link
Author

ghost commented Oct 16, 2019

I am getting an error when trying to install cryptography. Below you will find the error state. I have tried to install PEP 517 then the others with no luck. I had to install a bunch of old MS VS C++ files to get this far. Not sure what the issue is. Most things function ATM, just not the email analyzer. It opens the File Explorer and I can select test.msg but it fails. I assume because the crypto package isn't installed, which also breaks the openssl package. Everything else installed fine.

1

@TheresAFewConors
Copy link
Owner

TheresAFewConors commented Oct 16, 2019

There isn't a specific version required, so long as it is in your PATH. There is a guide on StackOverflow to fix issues like this here: https://stackoverflow.com/a/49986365

You are also correct that if the package is failing it will crash, although it should give you an error when you install from the requirements.py file if it can't be found. Try the link above and see if that helps at all.

@ghost
Copy link
Author

ghost commented Oct 16, 2019

Yeah, I just installed all the components one at a time and I guess I just need to keep adding VS components until it works. Ahhhh Windows. Took 4 minutes to install in Linux and the requirements script worked. LOL.

@TheresAFewConors
Copy link
Owner

She's a cruel mistress alright. The goal is to get Sooty completely independent of OS by the end of the year / early next year to avoid weird issues like this one. Let me know how you get on with it and any suggestions / issues you have.

@ghost
Copy link
Author

ghost commented Oct 17, 2019

That was painful, but I think I got "pip install cryptography" to work with this post:

https://stackoverflow.com/questions/45089805/pip-install-cryptography-in-windows/45089806

Installing openssl and changing environment variables fixed this problem for me.

Install OpenSSL by using the installer from here: https://slproweb.com/products/Win32OpenSSL.html
Use the Win32OpenSSL-1_1_0f, even if your system is 64 bit (I tried installing the 64 bit version of OpenSSL and this did not fix the problem)
The following two folders should now exist: C:\OpenSSL-Win32\include, C:\OpenSSL-Win32\lib
Open a cmd line terminal and run the following:

set INCLUDE=C:\OpenSSL-Win32\include;%INCLUDE%
set LIB=C:\OpenSSL-Win32\lib;%LIB%
pip install cryptography

@ghost
Copy link
Author

ghost commented Oct 17, 2019

Everything seems to work except the "Analyze an email" function. Any ideas now that all the requirements are installed? Test message is definitely a .msg I pulled right from Outlook to my desktop.
I have set up all APIs except HIBP.

What would you like to do?
OPTION 1: Analyze an Email
OPTION 2: Analyze an Email Address for Known Activity
OPTION 3: Generate an Email Template based on Analysis
OPTION 9: HaveIBeenPwned
OPTION 0: Exit to Main Menu
1
Error Opening File
Extracting Headers...
Header Error
Extracting Links...
Links Error
Extracting Emails Addresses...
Emails Error
Extracting IP's...
IP error

Edit:

Worth noting that I did NOT install the exact version of "cryptography" you had in the requirements list only the most current version.

@TheresAFewConors
Copy link
Owner

TheresAFewConors commented Oct 17, 2019

That is strange. If you want I can look at the specific email myself and see if there's a bug somewhere.

Basically from the error, its an issue opening the file, which triggers the rest of the errors as they have nothing to check. It may be an issue with not having read access to the email.

Edit: Just to add, the HIBP API is only used for enrichment of analysis, eg. it will check to see if a sender has been found in any breaches, and if so which ones. Without the key it will simply skip this part.

@ghost
Copy link
Author

ghost commented Oct 17, 2019

Got. That gives me a way to go. I will try some other emails.

@TheresAFewConors
Copy link
Owner

you can also try renaming the mail, or closing outlook as sometimes it holds the file open so it can't be read by other processes

@ghost
Copy link
Author

ghost commented Oct 17, 2019

Save As test.msg, closed Outlook, run tool, "Error Opening File". Terminal open as Admin, permissions on file have full admin access.

@TheresAFewConors
Copy link
Owner

I'm having an issue replicating this bug. Just to confirm - can you test with a verified and original .msg file through outlook, and confirm that the headers match. If the file is originally .eml and is saved as .msg it won't open until that functionality is added.

@ghost
Copy link
Author

ghost commented Oct 21, 2019

I open the email from Outlook, select "Save As .msg" and change the name to "test" and then save it to the Desktop. It is from here that I try to open the file and get the "Error Opening File" message.

@TheresAFewConors
Copy link
Owner

Is this error still occurring? I will be enabling logging soon to help with further issues like this.

@ghost
Copy link
Author

ghost commented Nov 27, 2019

On version 1.3 still get error trying to open email. Should I blow it away and clone it again?

@TheresAFewConors
Copy link
Owner

Could you try reinstalling with the newest version? (v1.3.1) There were some changes made so it may help with this issue.

@ghost
Copy link
Author

ghost commented Nov 27, 2019

Well, now I have installed 1.3.1 and now pip install cryptography==2.7 won't install again and now the email function does not work at all. I will revisit this after the holiday. Sorry for all the hassle.
I am starting to think it is my Python install and not your application.

@TheresAFewConors
Copy link
Owner

Its no hassle at all, let me know after the holidays when you get a chance and we'll see about getting this working for you.

@marcobelmonte
Copy link

I'm having this exact problem with opening .msg files but I also see above that you say Linux has a problem that is a known issue. I'm using the latest version - will I need to install Sooty into Windows to use it for mail based functionality?

@TheresAFewConors
Copy link
Owner

@marcobelmonte At the moment yes, the package used to import .msg files is specific to Windows. However I'm hoping to fix that issue in the near future as it also affects .eml files as well. So all mail extensions and OS issues should be fixed in 1.4.

I'm keeping this issue open until then, so if you want to watch the ticket you'll get a notification when its closed (fixed).

@ghost
Copy link
Author

ghost commented Dec 3, 2019

Here are the errors I am getting on the newest Windows version:
2

Here is the list of pip packages I have installed and the "pip check" shows no issues.
1

BTW Having the API keys in a .yaml file you can back up is really convenient.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
User Issue Issue affecting a user
Projects
Bug & Hot Fixes
  
Low priority
Development

No branches or pull requests

3 participants