Thursday, October 09, 2014

How Do You Secure The Internet?

This is a question I have been asked a few times now, and the asker doesn't typically mean the ENTIRE Internet, they just mean their own little corner of it.  You know, the one sitting on your desk.

When someone asks this, what they really mean is "How do we keep the bad guys out?"

The three major ways the bad guys get in are:  exploiting your desktop computers, exploiting your servers, and exploiting you.  You see, the bad guys exploit both computers and users alike.  Malware (a blanket term referencing adware, spyware, and viruses), service exploitation (this is what people usually think of when they hear the word "hacking"), and calling you or emailing you and straight up lying to you are ways that your data may be compromised.  You see, we're not dealing with nice people here.

Often people are just trying to get through their day with all of their work done well.  They're looking for a fast free tool because it might take too long to complete a purchase.  Free compression utilities, file viewers, and PDF printers have the potential to infect users computers and provide easy ways for the bad guys to get in.  Interestingly enough most of the adwares and spy wares I have encountered aren't normally detected by antivirus programs as they are classified as "PUPs" - Potentially Unwanted Programs.  These technically aren't viruses, but a nuisance as they may download other PUPs which download some more, and so forth.  I'm not just talking banners inside of the app, I'm talking full-on browser hijacking where you lose your homepage, search results are altered to promote certain products, and even your line-of-business application windows display ads.
Have you seen me?
Technicians can sometimes struggle for hours to remove them because these apps aren't meant to be easy to remove.  Furthermore these apps can download additional content from other bad characters who don't just stop at serving ads: they will try to convince you to directly give them money by scaring you.  They will intimidate you with frightening looking ads and ominous messages about your computer being slow or having viruses.  Some will even go so far as to tell you that illegal activity has been detected on your computer and offer, for a fee, to clean it off.  Some even lock your computer, say that illegal activity has been detected by the government, and you may pay a fine if your want to avoid jail - all from the convenience of your office and payable via Bitcoin.

These sorts of applications don't just come from misguided efforts at inexpensive office productivity, they are often secretly shoehorned into your computer by the advertising on web sites.  Not just scummy places nobody has any business visiting - we're talking major news outlets and popular web portals like the New York Times.  I am now seeing workers with a long history of being productive and not screwing around ending up with some pretty nasty bugs.

Another, more targeted method, seeks to install even nastier tools into the computers of workers handling sensitive financial.  Known as Spearphishing, the idea is to send emails which look like important communications to financial staff.  These emails have a file attached, typically a .zip or .pdf, which, when opened, installs software that watches for important financial data to be accessed.  When sensitive data is detected the malware neatly packs it up and sends it to someone who is going to steal your customers credit card data and potentially empty all of your bank accounts.
Surprised?
It turns out that, short of pulling the plug on your Internet connection, there is no one solution which will prevent this sort of abuse.  What is required is a "Defense in Depth" - layers of protection which, when takes as a whole, offer a more solid defense than any one process or product.  The entire process of retrieving information from the Internet must be questioned and examined at critical junctures.

Let's look at the process:

  • First the user makes a decisions that they want to get something done on the Internet.  This is the best time to intervene - the user needs to have knowledge to avoid danger online.  The bad guys often depend on us being either inattentive or ignorant.  Here's a good article on spotting bad links.
    Just click Download already!
  • When you request your web page your computer performs a DNS (Domain Name Service) query - this is akin to looking up a number in the phone book - the response can be controlled by both good guys and bad guys alike.  Good guys can alter these results and prevent access to undesirable web sites.  Bad guys can do likewise and funnel your users into advertising or more malware.
  • The HTTP Get is the next critical juncture - your computer has the number of the party you are trying to reach, dialed it, and the metaphorical phone is now ringing.  This is where a proxy server steps in - it intercepts the call and makes the call on your behalf.  Again, proxies can be used by both good guys and bad, and some of the bad ones insert ads, open additional pages, and even prevent you from accessing web sites and information for removing the malware.  The good guys will use it to compare the URL of the web site you're visiting to a categorical list of URLs and block web sites which fall into undesirable categories.  Administrators and mangers can also look at the proxy logs and get a good idea of how you have spent your time online.  Analysis of these logs can indicate which users are engaged in risky online behavior and give managers an opportunity to offer some advice on how to better spend their time online.
    I see you've been surfing the web...
  • Ongoing HTTP Sessions can also be intercepted by a proxy, reassembled, examined for viruses, and then forwarded to the client.  This would be a great job for a powerful router, but many businesses don't have them yet.  Interestingly, Charter Business is leading the charge in this field and is providing routers with this capability to their high speed customers.
  • Now that your computer has completed its call there's data to be opened or executed.  This is where a good antivirus product comes into play.  Many of todays malwares are "drive by downloads" which means you never actually clicked a download button, nor did you click Open or Run.  It just happens...  Unless you have a good antivirus product which does behavioral monitoring.  Antivirus products can watch what your computer is doing and prevent suspicious activity.  Check with your antivirus vendor (I know a good one) to find out if your current product does this monitoring.  Hopefully the malware packages never get a chance to misbehave as your antivirus will detect and quarantine them as soon as the download completes.
  • If, after all of this, your computer still becomes infected with malware the malware must now succeed in making it past some or all of the lower level services to succeed in communicating with its masters - savvy DNS managers maintain lists of known bad actors and can block requests to hosts based on these lists.  Though they're uncommon, an Intrusion Detection Systems (IDS) at the firewall can detect the command and control communications used by malware, then generate alerts or terminate suspicious connections.  IDS's are are prone to false positives, so be careful if you go this route because they can require a lot of care and feeding.
  • If the malware has made it past all of these defenses then the bad guys have done a good job (relatively speaking, it's awful for us) and are now winning the game.  The likelihood of this has been reduced through defense in depth, but we can never consider ourselves to have won:  we have to find every hole in our defenses and fix it, the bad guys only have to find one. 
Spearphishing attacks are incredibly crafty, and the bad guys know how to get past many of these defenses.  Users must be prepared to match their own wits against those of the bad guys.  They will receive messages crafted to make the user want to open the attachment.  Messages will claim their attachments are:
  • Requests for bid
  • IRS communications
  • postal notifications
  • shipping notifications
  • bank password reset requests
  • law enforcement notifications
  • holiday greeting cards
  • invoices
  • awards
  • banking documents
All of these are important stuff, and nobody wants to drop the ball.  The bad guys know this and will do everything in their power to prey on our sense of duty or curiosity.  Needless to say, some of our friends and coworkers might benefit from a brief orientation on these threats.  I can't emphasize enough what a vital role your gut will play in this - if you think it might be a scam it probably is.
And for good reason.
There are many opportunities to prevent spearphishing attacks.  Remember, spearphishing is sending malware via email to people with sensitive jobs.  Many email servers (Like Microsoft's Office365) include virus scanning which will remove the threat before it reaches your inbox.  Email servers may also be configured so that they may only receive mail from trusted systems or from systems that can pass a series of tests that only trustworthy servers can pass.  As if that's not enough, servers will often read the email and make some decisions based on the content.  It's not uncommon to see 2/3 of the messages received by a server silently dropped.

Controlling DNS responses can be useful here, as well - compromised servers are may be detected and listed with security services.  If you have been duped into clicking a dangerous link the attempt could be blocked by services such as OpenDNS.

Think you can spot a phishing attack?  Take the OpenDNS Phishing quiz!

I hope that I've been able to help you learn a bit more about the threats faced by modern office staff.  If you would like to learn more about how Safe Mode might be able to secure your office computers please don't hesitate to call or email!

Tuesday, September 16, 2014

Windows 7 Desktop Can't Join the Domain - Path Not Found? Blame AVG (and everything else!)

Spoiler Alert:  uninstalling AVG from the desktop fixes the problem.

A desktop is complaining that it's Trust Relationship has failed.  Normal stuff, probably went through a system restore and ended with an old SID, no biggie.  Remove it from the domain, reboot, readd to the domain, boom done, right?  Not so fast... after changing the domain name and hitting OK I'm presented with the normal domain login to which I input my domain administrator credentials.  The computer complains with an error message:
The following error occurred attempting to join the domain "somedomain.local":
The network path was not found. 
This points to a DNS issue on our SBS 2008.  Rebooting the server was my first step and it yielded no positive results.

The SBS 2008 in question seems slow and balky.  It's an HP ML110 with 8GB of RAM serving as an SBS for a group of 10 or so people using email and file storage in the server as well as its normal duties authenticating users and doling out Group Policy.

Noted error 13568 with source NtFrs in the event log which basically says that the File Replication Service is in Journal Wrap Error.  It reads kind of like:

The File Replication Service has detected that the replica set "DOMAIN SYSTEM VOLUME (SYSVOL SHARE)" is in JRNL_WRAP_ERROR.   Replica set name is    : "DOMAIN SYSTEM VOLUME (SYSVOL SHARE)"  Replica root path is   : "c:\windows\sysvol\domain"  Replica root volume is : "\\.\C:"  

A Replica set hits JRNL_WRAP_ERROR when the record that it is trying to read from the NTFS USN journal is not found.  This can occur because of one of the following reasons.   

[1] Volume "\\.\C:" has been formatted.  

[2] The NTFS USN journal on volume "\\.\C:" has been deleted.  

[3] The NTFS USN journal on volume "\\.\C:" has been truncated. Chkdsk can truncate the journal if it finds corrupt entries at the end of the journal.  

[4] File Replication Service was not running on this computer for a long time.  [5] File Replication Service could not keep up with the rate of Disk IO activity on "\\.\C:".  

Setting the "Enable Journal Wrap Automatic Restore" registry parameter to 1 will cause the following recovery steps to be taken to automatically recover from this error state.  

[1] At the first poll, which will occur in 5 minutes, this computer will be deleted from the replica set. If you do not want to wait 5 minutes, then run "net stop ntfrs" followed by "net start ntfrs" to restart the File Replication Service.  

[2] At the poll following the deletion this computer will be re-added to the replica set. The re-addition will trigger a full tree sync for the replica set.  

WARNING: During the recovery process data in the replica tree may be unavailable. You should reset the registry parameter described above to 0 to prevent automatic recovery from making the data unexpectedly unavailable if this error condition occurs again.  

To change this registry parameter, run regedit.  Click on Start, Run and type regedit.  Expand HKEY_LOCAL_MACHINE. Click down the key path:    "System\CurrentControlSet\Services\NtFrs\Parameters" Double click on the value name    "Enable Journal Wrap Automatic Restore" and update the value. 

 If the value name is not present you may add it with the New->DWORD Value function under the Edit Menu item. Type the value name exactly as shown above.

Also noted Event ID 25:

The shadow copies of volume \\?\Volume{83195036-2013-11e0-9593-3c4a92d51777} were deleted because the shadow copy storage could not grow in time.  Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being shadow copied.

It sounds like the hard disk could too busy to serve up essential functions - looking at the Resource Monitor I could see that SQL was going crazy reading itself from the hard drive.  I decide to run the SBS 2008 BPA and see if it can tell me more.  I also update the HP System Management Agents and the HP Array Configuration Utility so that I could rule out hard disk problems (which indeed were not an issue).

My BPA report showed some issues which were solved with some simple netsh commands that were detailed in the BPA.  But an outsized Sharepoint and SBSMonitoring were also an issue as was the server being in Journal Wrap condition.

The outsized databases don't seem like they'd keep desktops from joining the domain, but the journal wrap might be a different story.  I followed the link to http://support.microsoft.com/kb/292438 and said to myself, "Oh Crap, they've linked to an outdated article, this is for Win2k!  Nice job Microsoft..."  Worthless - except, it's not.  Things haven't changed much in the last 14 years of Active Directory.

Sure enough, upon reading http://blog.ronnypot.nl/?p=738 I check and find the SYSVOL share was not available.  I changed the registry value (which was what the error message directed, also) and waited a few minutes.  The SYSVOL share came available again.  BUT... still cannot connect the workstation to the domain.

I decided to pursue the other issues indicated by the BPA and fix the SBSMonitoring and Sharepoint Services databases.

First SBSMonitoring - Google yielded http://kwsupport.com/2013/05/sbsmonitoring-database-is-nearing-maximum-size/  which suggests using http://blogs.technet.com/b/sbs/archive/2011/08/22/how-to-recreate-the-sbsmonitoring-database.aspx to replace the database with a new blank one.  What are the drawbacks?  Loss of historical data - no biggie.  Downloading and running the script was a breeze, I just needed to set-executionpolicy unrestricted to get it to execute.  That article then recommended I complete the steps at http://blogs.technet.com/b/sbs/archive/2009/07/14/sbs-2008-console-may-take-too-long-to-display-alerts-and-security-statuses-display-not-available-or-crash.aspx which will shorten the amount of time which logs are kept and reduce the amount of information which is logged.

Now to deal with the overweight Sharepoint Services Database - http://support.microsoft.com/kb/2000544 seems like a good place to start and it features a convenient "Fix It For Me."  This removed the issue from the BPA, but the desktop still won't join the domain.

Others have been feeling this pain, I see posts with similar issues all over the Internet.  This one:  http://richardburley.com/windows-7-unable-to-join-domain-fix/ seems like it might finally be the one which most closely matches my situation.  On the afflicted PC I cannot browse to \\servername.  I checked this from another computer and found that \\servername worked fine - an exact fit!  This fellow fixed his issue by removing everything from the network configuration that wasn't TCP/IP v4 or v6.  I'm working remotely so this seems like a real bummer of a solution, but examining the network protocols I noted the AVG Network Filter Driver.  Perhaps this is it?  I removed AVG and rebooted the PC.

Uninstalling AVG fixed the issue - a fifteen minute fix found through four hours of work.  The server is certainly having issues, but they weren't causing THIS issue!

Wednesday, August 27, 2014

Using the Same Alias in Multiple Domains in Office365


  1. Start PowerShell as an Administrator
  2. If you haven't before issue the command issue it now: Set-ExecutionPolicy RemoteSigned
  3. Connect PowerShell to Office365 - (from http://technet.microsoft.com/en-us/library/jj984289(v=exchg.150).aspx)
    1. Issue the command and input your credentials: $UserCredential = Get-Credential
    2. Then issue this command to connect:  $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://outlook.office365.com/powershell-liveid/ -Credential $UserCredential -Authentication Basic -AllowRedirection
    3. Import the PowerShell commands from the Exchange Server by issuing:  Import-PSSession $Session
    4. Now test your connection by issuing: Get-Mailbox and making sure you get output.
  4. https://www.cogmotive.com/blog/office-365-tips/create-shared-mailboxes-with-same-alias-at-different-domains-in-office-365 gives us the following steps
    1. :  New-Mailbox -Name "Test Shared Mailbox 2" -Alias test_shared2 -Shared -PrimarySMTPAddress test_shared@cogmotivereports.com       Be certain to replace "Test Shared Mailbox 2" with the desired display name, the alias "test_shared2" with the desired alias (prefix before the @ sign) and fix the primary SMTP address with the desired address.
    2. Next correct the login name for this mailbox - set-mailbox test_shared2 -MicrosoftOnlineServicesID test_shared@cogmotivereports.com - you may receive the following error WARNING: UserPrincipalName "test_shared1@cogmotivereports" should be same as WindowsLiveID "test_shared@cogmotive.onmicrosoft.com", UserPrincipalName should remain as"test_shared1@cogmotivereports". Which may be safely ignored.
  5. Check you work by issuing Get-Mailbox verifying that the new mailbox entry appears.
  6. Close your session with Remove-PSSession $Session
This will create a new shared mailbox with the desired alias.  To make certain users send from the desired address and not the placeholder alias (test_shared2) sign into Exchange and go to the shared mailbox.  Edit the email addresses and set the desired alias as the primary address.


Wednesday, August 20, 2014

Migrating Outlook 2010 Autocomplete to Outlook 2013


  1. Run Outlook 2013, allow it to finish loading, then close it - no need to wait for it fully synchronize if you have a large mailbox.
  2. Open C:\Users\username\AppData\Local\Microsoft\Outlook\RoamCache
  3. Look for files named Stream_Autocomplete.dat  and note the latest one (this is the target) and the largest one (this is the source).
  4. Make a backup copy of both files, I placed mine in C:\temp.
  5. Go back to C:\Users\\AppData\Local\Microsoft\Outlook\RoamCache and rename the source file Stream_Autocomplete.da~ by replacing the "t" with a "~".  Before you commit the change, highlight the entire name before the period and copy it with Ctrl+C then commit the change.
  6. Rename your target file by pasting the source file name over the target file name.
  7. Run Outlook and feel the joy.
Why didn't Microsoft make this an automatic function like they did in prior versions? 0_o

Thursday, July 31, 2014

Linking a Bypass Code in Umbrella by OpenDNS

Here's a stumper:  you've created a bypass code for a user under Block Page ->Bypass Codes, and as you're admiring your shiny new bypass code entry you note that there is a yellow Hazard symbol in the column under Linked Policies next to n/a.  Furthermore the users bypass code doesn't work - it says that it must be linked to a policy, and there isn't a "link policy" button anywhere to be found.

This situation can be avoided entirely by creating each bypass code by clicking Policies in the left hand pane, selecting the relevant policy in the main pane, then jumping to Step 3. "Select Block Page Settings" and clicking Add Code.  Create and save your new code, share it with the relevant user, and your done.

But say you've gone and created your bypass code by navigating to Block Page Settings in the left hand pane, selecting Bypass Codes underneath it, and clicking "+ Create a New Bypass Code."  Don't worry, you haven't just wasted that time - click Policies in the left hand pane, select the relevant policy in the main pane, then jump to Step 3. "Select Block Page Settings," check the box next to the desired user, and then click Save.

Thursday, July 24, 2014

Assigning a Public IP using AT&T UVerse - Pace Modem

1.  Visit your gateway address using your favorite browser.

2.  Go to Settings -> Firewall

3.  Go to Application Pinholes and DMZ

4.  Select your device and then click Allow All (the last option)

5.  Click Save

This will DMZ the device and allow all traffic to all ports on that device.

Wednesday, July 02, 2014

Scheduling Periodic Bandwidth Checks Between pfSense Routers Using iPerf

I have a customer who, for years, has complained that their point to point connection (over a dedicated Point-to-Point Cable connection referred to as EoC or Ethernet over Cable) slows down every afternoon regardless of the number of users.  Initial investigations revealed nothing of importance.  iPerf tests would show periodic slowdowns but without any consistency as I could only run iPerf at the console, which required me to stand over it and initiate the tests.

What I desired was iPerf tests every 5 minutes during business hours.

To get started install the iPerf packages in both of your pfSense systems.  I am clueless why there are iPerf options in the Webmin, they seem to do nothing, please ignore them and use iPerf from the console.

iperf -c 192.168.0.1 -t 28800 -i 300

SEEMS to work, but it would need to be invoked every morning, and only outputs to the screen. Furthermore it runs the test ALL DAY LONG, not just every 5 minutes.  This would hog up the connection and prevent real work from being done quickly.  What I need is for it to be done periodically then output to a text file that I can check occasionally.  Furthermore, the office is only open 8am-5pm Monday through Friday, so why fill up my file with tests all the rest of the time?  Lastly, iPerf doesn't include dates and times in the report, so I need to add them.  I decided to haul out Crontab and do the following:

1.  Install the iPerf package in both pfSense systems.

2.  Pick a pfSense system to be ny server, log into its administrative console, and run the following command:

iperf -s -D

This runs iPerf as a daemon and allows me to close the session but keep iPerf running.

3.  Create an sh script (mine is iperftest.sh) using vi containing the following:

date
/usr/local/bin/iperf -c 192.168.0.1 -t -x CSV


The date line adds the date and time to the output file, and the -x CSV prevents showing info beyond the amount of data transferred and the speed at which it was transferred.

4.  Add a crontab job (crontab -e) for the user admin similar to the following:

00/5 8-17 * * 1-5 /root/autoiperf.sh >> iperfreport.txt

5.  Now all you need to do is cat your iperfreport.txt to see reports.

Wed Jul  2 13:25:00 CDT 2014
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.0 sec  6.00 MBytes  5.03 Mbits/sec
Wed Jul  2 13:30:00 CDT 2014
[ ID] Interval       Transfer     Bandwidth
[  3]  0.0-10.1 sec  6.00 MBytes  4.98 Mbits/sec



Wednesday, March 05, 2014

Outlook 2010 and Office 2013 - Problems Changing an Expired Password

This puzzled us for a while, while there is a hotfix available (MS KB 2687351) you have to request it.  Without the hotfix the user may be repeatedly prompted to change their password, even if they changed it in the online portal.  Go to your account settings and select "Always Prompt for Credentials" then restart Outlook, input your new password, then make sure it works.  After you have verified your account is working go back to your account settings and clear the "Always Prompt for Credentials" checkbox so you may save your password.

Tuesday, February 18, 2014

Configuring Entourage 2008 for Office365 Exchange

Mac users are a different breed - they often just want to get things done and not spend a lot of time learning a new program.  Sometimes this means spending a lot of time making our old stuff work with our new stuff.  My friend Brent recently tackled the issue of using Entourage 2008 and Office365 Exchange with some success (though there seems to be a limitation which causes Entourage to only sync the past 6-8 weeks of email and not display the balance).


First Brent had to install several updates for his Office 2008:


Next he configured his Mac and Entourage using information from the following web sites to:




Thanks for sharing this solution, Brent!