How about our private informetion?

We are under attack !!!

Note: this post is adapted from an amazing article at Veracode blog on "3 Easy Steps to Making Perfect Security Possible".

All of a sudden, we realized that our company was brutally hacked. More than that, it was embarrassing: someone put company emails and files on Pastebin and deleted data from the web servers.

So... What we did?

We spend most of the time answering to management how it happened. Explaining the same again and again: we have no idea, actually, and we are trying to start investigating the incident - as soon as we can leave the dozens calls with managers and focus on our job. Our initial reaction is to feel a huge, gaping hole in our chest. We felt stupid, utter helplessness. I hide a while in my office as I keep from crying. To everyone who asked, I told that I’ve got dust allergies.
:'(

I read online news stories and posts on social networks about what everyone's saying about our breach, and I felt sick to my stomach. I found a little comfort in security pros online saying everyone gets hacked eventually.

Soon we realized we need to do something. We drop everything we had to do to investigate how the attack happened (including writing dozens of reports, answering end users' emails, and posting cat pics on Facebook). We started looking at events and logs until our head pounds and we got a sharp pain right behind our eyes. And we kept stopping the investigation once on a while to answer questions from management. Later we gave up investigating and we started telling everyone asking that we don't really know how the attack happened.

For a moment I wish I had a Chinese IP address in my log, so I could just blame Chinese hackers and go back to home.

We were trying to shut down some affected servers and some network segments in order to confine the attack and avoid the attack to spread across entire network. Everyone were working like crazy at the data center. Our network topology is a mess, so shutting down a network segment means digging into dozens of routers and following hundreds of network cables to figure out which device belongs to which network.

All of a sudden, our CIO went to the data center. It is very likely the first time ever he put a foot on our data center ! He kept asking everyone what we were doing and how long it would take to have the network back to normal operation. No pressure, for sure !!! He got impressed with the huge mess we have in our data center and he claimed we will have budget to reorganize and to fix the infrastructure. Don't tell him it's too late - but you think it. He never went back to the DC again and we never saw such investment :(

Management them asks if we should contact the police. We don't actually know. We contact the police, but they don't really know what to do. We told management that they should contact partners and customers who might be affected. They replied telling us that we have to avoid the journalists. In addition, they asked me what they should say. I don't know, so I told them to tell everyone they should change their passwords.

If you have no idea on what to do, ask people to change their passwords and they will kept busy for the rest of the day !!!

We then focused on cleaning the suspected areas of the breach and running scans on the rest. We changed all the passwords we found, everywhere.

In the next day I went to the office early because I got called in for an emergency: our web server pages have been defaced. During investigation, we discovered the database is corrupt. We were happy, since we've recently invested in a kick-ass back-up solution for it. But soon we realized the last back-up actually recoverable is a month old and I kick myself for never testing the recovery system. I felt stupid and I punch the back-up machine. Then I felt stupider for hitting a computer screen.

The same thing happened the next day: I woke up with my phone buzzing in my bed. Then I found out that the web servers got tagged again. I went to work in the clothes I'd slept in and I realize that other employees avoid me in the hallways. A co-worker from my team told me that I looked like shit. The I remembered that I got into this field for a reason: I used to like security.

As next step, we check with our SOC and we realized our WAF and IDS let the attack through. I called the vendor to complain and I got told I'd configured them wrong. Then I had to remind them that they configured it. They deny it, for sure. As I got angry, I went to meet with management about looking for new security vendors. Then I found out from management that we still have over two years left on our contract with that vendor, and someone on our team told me later that the vendor is the cousin of the CEO.

So, lets go eat chocolate to deal with it.

SysAdmin Day

July 26 is the System Administrator Appreciation Day (SysAdmin Day, for short).

Why not to celebrate it? Why not to create a post here, almost 8 years after my last one?

After all, most of us, security analysts, are former sysadmins or we are still working as a sort of "specialized sysadmin." Or, in some cases, we are "security + system + application + dababase + anything_with_a_blinking_light" administrators.

A friend decided to spread the message across the company...

[11:35:48 AM] My_Friend: Today is SysAdmin Day :)
[11:35:48 AM] Finance-Girl: Today is Grandma Day...
[11:35:59 AM] My_Friend: http://sysadminday.com/photo-gallery/
[11:36:13 AM] My_Friend: See how easy is to keep the Internet running... :)
[11:36:29 AM] My_Friend: http://sysadminday.com/photo-gallery/?locale=en_US&wppa-album=1&wppa-cover=0&wppa-occur=1
[11:36:39 AM] Finance-Girl: Today is Grandma Day... Who is a grandma here at the company? Ahhh... sisadmindai... big sh#t...

Just as a conicidence, her computer lost the Internet connection. No filters added. He swears...

Username Policy

Last week people start complaining about our Acceptable Username Policy.

I´m tired of using any usual pattern like:

• "first name + .+ last name" (e.g. george.bush, bill.clinton) or
  
• "initial + last name" (initial letter of first name + last name, e.g. gbush, bclinton) or
  
• "first name + initial" (first name + initial letter of last name, e.g. bush.g, clinton.b)

They are very usual... boring... with no creativity at all....
People ever wants to have a closer match between a user's actual name and its username. However, this usually make us have multiple users with equal usernames, so we have to have additional rules (like including more letters, a number at end, etc).
So, I´d decided to create a new username policy for the company: the first 3 letters of first name plus the initial 3 letters from the last name. Example: geobus, bilcli .
But users never get satisfied. Some of them starting cp,plaining that they dislike their usernames. My boss asked me to change the Acceptable Username Policy, so here follows the new one:

• Account names consists by only the vowels from the first name plus the last name (e.g. eoeu, iio),
  
• if there is two users with the same result, then the username must include the first 5 odd numbers of his/her social security number.
  

From now on, users will have usernames like eoeu39751 for George Bush and eoeu99715 for Leone Blur.

NAT Gateway

My Firewall admin have just gave me an user account to connect to our internal Firewall in order to verify it´s configuration. After about 4 months, he finnaly gave me a "read-only" account.
When I first connect I was impressed: our internal firewall (who was supposed to isolate our engineer network) has 8 policy rules and around 80 NAT rules. As the last access rule (of 8) is a "default permit" rule (anything from anywhere can pass), I named it as our new "NAT Gateway" security equipment.
From now on, a "NAT Gateway" is a special-purpouse security device that contains no usefull security rules, only lots of NAT rules.

Spam at Work

The spammers are getting more and more creative erevyday. We can find SPAM messages almost everywhere.
I have just received two SPAM messages on my orkut´s scrapbook !!! They are spamming Orkut !!!
I´ve also received SPAM comments on my personal blog ! Amazing....

The BOFH-style Excuse Server

I must post here the most useful thing that we could ever find on Internet: The BOFH-style Excuse Server.
This is a useful tool that all sysadmins and IT professionals should use on a regular basis.