Archives for August 2011

For the last few years, I’ve been using the free Google apps for my email and calendar solutions. I’ve been what I would call a power user on Google Mail – falling in love with the keyboard shortcuts, and the idea of archive over delete, and the conversation view of email. That really sums up my love for Google Mail. Let me dive a little deeper into each one.

• Keyboard Shortcuts. I’m sure other web applications have this, but for me, Google Mail was the first application that I could really fully utilize with the keyboard, making my email life fast and efficient.
• Archive over delete. 100 years ago (in internet years) I used a hosted exchange account where my maximum account storage was in the meg, not gigs. This meant I had to move email out of my Exchange account, so Google Mail of 6 Gig was attractive. Keep everything, and search for what you need when you need it.
• Conversation View. Without a doubt this was the KILLER FEATURE for Google Mail for me. It made life so easy, that the conversation always let you quickly and painlessly see the entire email thread. And not just a buck full of message, the actual conversation. So when Email 2 replies to Email 1, even though the body of Email 1 was included in Email 2, Google Happily hid that for me, so I could just see/read the additive parts of Email 2. This to me is true conversation view.
• Life is GOOD GOOD Google.

And here we are today, years and years later. My Google Apps email account is 92% full. Whoa, that’s full. Delete all spam and all trash, and it dropped to 72%. That’s a lot of spam 🙂 that thankfully I never had to see. So my account is almost full. Recently I have started missing Email (from lists and colleagues that have shown me the email sent from their outbox) and meeting Requests have started giving me headaches. Not sure if this is my fault or Google Mail, but the point is, I’m having a pain, and I don’t have a resolution. I can upgrade to Google Pro (or whatever it’s called) for $50/year or look around to see what others have to offer. Let’s have a look. (this was a couple of weeks ago for me, and this post is an update of what I’ve done since).

So I started looking at Office 365. All my devices connect to Google apps as an Exchange account. Google mail acting like exchange so the folders (errrm labels) and contacts and calendars and push email all just work because every device (that I know of) has Exchange built in. So why not just use Exchange? maybe that’s an option, so I start looking at Office 365.

On a regular basis I also use GoToMeeting for screen sharing and conference calls, which I pay $49/mo for. That’s $600/year for screen sharing. Not a lot month to month, but it ads up, and has no added value, other than it’s a great screen sharing platform.

But wait … as I’m looking at Office365 I see that it includes Lync. Lync is probably a lot of things to a lot of people, but for me it’s a GoToMeeting Killer. Meaning I’m canceling my GoToMeeting account, because Lync does everything I need it to. Conference calling, VOIP, and Screen Sharing. I’m sure there are edge cases that I don’t know about, but FOR ME, Lync is magical, and just works. I’ve used it for a dozen or so meetings and it’s perfect.

Office365 also has 25 gig of email. WooHoo, my 5991Meg of email is barely a dent in this new plan.

Oh wait … Snap … Office365 also includes hosted Shapepoint. I’m not sure if we’re going to use it, but it’s there if we need it.

So ….. I’ve done it. I decided, I Signed up, and I switched DNS and as of now …. all our Mail / Calendars / Contacts are hosted on Office365. My 3 main killer Google Mail features are now included as basic features in Outlook.

• Keyboard Shortcuts: outlook has them. I can fully navigate the UI with keyboard over mouse.
• Archive over Delete. I created a “All Mail” (actually this was imported from my Google Apps and I kept it) folder, and mapped the “Done” button (CTRL+SHIFT+1 is the KB Shortcut) to mark mail as Done, and move it to this “All Mail” folder.
• Conversation View: is built into outlook. I can see Sent, All Mail, Inbox, and Even deleted messages as part of a common thread.

So I’m a week into Office365, and online (twitter) I made a joke today that Office365 is to be renamed Office364, because there is a system wide outage. What does this mean for me? Well, with no time to work on Email – I can write a blog post 🙂 in my trusty Live Writer application. Email goes down. The good part about this is I’m not frantically scouring about trying to fix servers. I know that there are very smart people, and probably a lot of them trying to get the email back up and running. And who knows … maybe this is part of a larger outage because internet just went out our office from the local Cable Internet Provider. And FWIW I had plenty of outages when hosting with Google Mail as well. So this is a minor hiccup that I’m not worried about.

In the end, this is a very long blog post saying I’m happy to be trying something new, Office 365 is (so far) doing great …. and hey what do you know. Office365 is back up and running and my inbox just filled up, so I have to get back to work.

Try Office365, I think you’ll like it.

@scottcate

Welcome to the QuickStats wiki!

I’m @ScottCate and I created and am maintaining the QuickStats applications. Here I will write a few notes regarding the code for Quick Stats. Quick Stats is a sports reporting engine built on top of the FanFeedr API. The code is used for individual apps for each team in the following leagues: NBA, NFL, and NBA. In the futures, more leagues may be considered.

Version History Follows (newest on top)

---

August 14, 2011 – I’m still waiting for FanFeedR to load the NFL preseason. Once this is done, I can test the app with live data. Today, as a test, i’m updating the MLB Arizona Diamondbacks. Currently the MLB League is set to not support Pre/Post seasons, so theoretically the app will only have minor layout differences. If this app runs, then I konw the new code is sound. You’ll know what version you have becuase post Version 1.3, the version number is supported in the top right of the home page of the application.

• Minor UI: Made team name larger on welcome screen (edges off right side – no wrap)
• Minor UI: Added ads tothe game pivot control, so there is a unique advert on each pivot page
• System: Added league supprot for Preseason
• System: Added league support for Postseason
• System: Currently only NFL is set to post season true, so NFL teams are the ony that should be affected.
• Minor UI: Change Version number (currently 1.4) to Hyperlink, linking to this page.

---

August 6, 2011 – NFL season is almost here and people are asking about the preseason schedule. Today I added support to display the preseason schedule in the application, but there are a few things you should know. FanFeedr has a distinct API call for preseason data. This means the preseason games do not come back with the regular season games. Because it’s a second call, I didn’t want to automatically make the call because after preseason, this will hardly ever be used. So I created a flag on the league to support Pre and Post season calls. If this is supported (by the league), then the screen adds a button to manually load the pre and post season games. So for now, the NFL league has Preseason turned on, but there is no data coming back from the FanFeedr API. When this data becomes available for NFL, I’ll do some more testing, and release an update for all the NFL applications. And that will be the version 1.4 (Sometime in August 2011)

The software has an update to version 1.3 with several crashing bug fixes. As of version 1.3, the version number is displayed on the home screen, when you start the software, on the very top right. If you don’t see this version number, look for an update in your Windows Phone Marketplace.

I’ve recently been considering GoDaddy.com as a DNS provider. I have 150+ domains registered at GoDaddy.com, and they offer a nice software panel for DNS entries. I’ve been using DNSMadeEasy for a very long time, on the recommendation of Tim Heuer, way back in the day. I didn’t use GoDaddy from the start, because I wanted to separate my domain registrar from the DNS provider. But I’ve been using, and happy with everything else GoDaddy for a number of years, and I’ve never had any GoDaddy or DNS issues. I pay about $150-$200 a year to DnsMadeEasy.com for all the DNS hosting they do for my domains. I know they have new pricing, but I’m grandfathered in because I’ve had my account for so long.

Anyway, I’m not moving my DNS hosting to GoDaddy. Here’s why. Today, out of the blue, I get an email (copied below) that basically says, our perfect record is now lightly tarnished. What a cool email. To my knowledge, my accounts were not affected, and if they were, none of our customers called, so I don’t think we were affected by the outage. All-in-all the fix took them 6 hours to complete. WOW. These guys are genius smart, all they do is DNS. And if it took them 6 hours, I’m a little afraid of what would happen if a DDoS attack of this scale were to be aimed their competitors, or worse, my Registrar, or other service providers. So for $200 (or less) a year, I’m happy to have the expertise of DNSMadeEasy.com on my side.

And, to boot, they’re offering a refund on all accounts, not even just accounts that can prove they were affected.

Well DNSMadeEasy.com – today you saved a customer, with this service, and you didn’t even know it.

Good things happen to good companies, who do “the right thing”!

If you’re looking for DNS Hosting – I highly recommend DNSMadeEasy.com

---

Dear DNS Made Easy Client,

On August 07, 2010 DNS Made Easy was the target of a large multi Gb/s attack against all of our name servers.   The attack started at 8:00 UTC and was fully mitigated by 14:00 UTC.  During this time period there were regional outages from some or all of our name servers.  Regional outages means that certain regions of the world were not able to resolve your DNS and other regions of the world were resolving normally.  When all name servers were not reachable a DNS query would have been lost, when some name servers were not reachable then DNS performance would have been slower than normal but still operational.
The regional downtime was in very small periods but it still did affect the overall resolution for all of our client’s DNS.  It is for this reason that we are explaining the situation in full to all of our clients now.

1) How long were the DNS outages?
In some regions there were no issues, in other regions  outages lasted a few minutes, while in other regions there were sporadic (up and down) outages for a couple of hours.  In Europe for instance there was never any downtime.  In Asia downtime continued longer than other regions. In United States the west coast was hit much harder and experienced issues longer than the central and east coast.

2) Many clients have asked us if in fact there was downtime since they did not notice issues.
Many clients did not notice any DNS downtime.   In fact many clients would not have noticed this issue if we had not sent this email.  But we feel disclosure of this issue is something that we owe our client base. If you want to see if there is a significant loss of DNS queries you can quickly compare your daily queries from this Saturday to last Saturday in the DNS Made Easy control panel.  Overall query statistics comparing this Saturday’s query load (minus attack traffic) to recent Saturdays’ query loads shows that our servers properly responded to a query total this Saturday within a 2% difference from recent Saturdays.

3) Where did the attack come from?
We believe that the DDoS came from a botnet attack originating from Asia.  Most attack traffic originated in or transited through China.  The source IPs appear to be mostly spoofed but the vast majority are assigned by APNIC to Chinese Networks and Chinese ISPs.  Traffic levels reported to us by our bandwidth providers regarding their connections through which this traffic entered their networks also points to origins in Asia.

4) How large of an attack was this?
This attack hit levels that were so high that our Tier1 upstreams were suffering latency and network issues for other clients at many of their locations due to this attack.  This caused some of our Tier1 bandwidth providers to use their last resort response of null routing traffic to some of our IPs from some networks to prevent major service degradation to their core networks. Measuring the exact size of this attack is rather difficult.  However, discussions with our Tier1 bandwidth providers during the attack led to an estimate of 50 Gb/s in size.  This was based on reports of multiple 10Gb/s lines being saturated at multiple different providers in different geographic regions.
During our after-action discussions internally and with our providers after the attack was mitigated we analyzed all information available to us through monitoring systems and traffic reports and we revised our estimate of the attack size to be fluctuating between 20Gb/s and 40Gb/s during the attack.  We will never know the true size of this attack as we actively moved traffic around to different locations throughout the attack and IPs were temporarily null routed into and through various networks, and some traffic was blocked from provider to provider in response to the attack. We do know that due to the service implication to the Tier1 providers, networking teams from China Netcom, China Telecom,  Level3, GlobalCrossing, Tiscali, and Arbinet were involved to stop the attacks.  Level3 and Arbinet both played special heroic roles in facilitating that the correct people were involved from all networks to make sure that the attack was stopped as quickly as possible.

5) How was this attack stopped?
Fighting attacks of this magnitude is very complex and a full answer involves much information that we do not want these criminals to know.  What we can say is that that we used a combination of routing techniques, DDoS mitigation tools, customized firewalls, and high level inter-provider negotiations.
China Netcom and China Telecom had to null route the name servers from their networks in order for the attack to not impact other traffic they had going to the United States.

6) Will an SLA credit be issued?
Yes it will be.  With thousands paying companies we obviously do not want every organization to submit an SLA form.  Even though not all clients noticed the attack, we plan on issuing an SLA to every single paying DNS account.
You will be receiving an email about the SLA credit to your account in the next few days.

7) Does this affect your 100% uptime history?
Yes, any service outage would result in loss of uptime.  We had a history leading uptime of over 8 years of 100% uptime.  With a calculated two hour outage (which is probably longer than we were actually down for anyone) this DDOS attack put our overall uptime history at a calculated 99.9999%.  This is still an excellent uptime history.

8) What would it take to get your 100% uptime history back?
That is mathematically impossible.  But we can work on increasing our 99.9999% uptime history and we will work hard on building another run of more than 8 years of 100% uptime.  We are confident that we can do it and we look forward to the challenge.

9) Would another DNS provider have been able to stop this attack?
We are sure that our competitors will claim that the answer is yes.  In fact we have been called by several of our competitors with very amusing phone calls during and after the attack asking us to update our website to say that we no longer have a 100% uptime history (which we have started and will complete soon).  This was a very large attack, so we do not believe that other DNS services could have stopped it either.  If any of our customers are considering leaving our services based on this issue, then we would recommend highly that you request a detailed report for how any new potential DNS provider would deal with an attack of this magnitude.  Please note that this was our first issue of downtime over our 8+ years of providing enterprise managed DNS services.

10) What is the next step?
At this time all DNS resolution is functioning as intended from all of our global locations.
In our 8+ year history, we have had numerous attacks against our services.  Historically we have been able to mitigate these attacks without any service degradation. One thing we have always taken away from every attack is a deeper understanding of what we need to do to make our network and services stronger and more reliable.
This DDoS attack against us was different from others in that the size was massive enough that our standard mitigation strategies were not sufficient to prevent several network nodes from being flooded.  We now have a deeper understanding of what happened during the attack and have started planning network upgrades and mitigation strategies to help fight these criminals in the future.  It is, and always has been, our commitment to make the DNS Made Easy network the strongest and most reliable DNS network in the world.

11) Can I pay more for a higher level of service with DNS Made Easy?
We believe that we provide more service per dollar than any competitor in the DNS industry.  This is why we have the best ROI in the industry.  We do not do this by cutting networking cost.   As many of you aware DNS Made Easy feels we can cut costs by eliminating a lot of the sales (including commissions), presales, and unnecessary marketing expenditures.
Everyone at DNS Made Easy feels that our network is as strong as or stronger than any competitor in the United States and Europe and you can verify this with speed tests and our highest industry uptime.   As all DNS Made Easy customers know, as our customer base grows, so does our network.  This is how we can continually keep adding to our network and always remain a fraction of the price of our competition.
You will hear more from our network team as we plan on adding additional precautions to keep everything running smoothly during attacks in the future.
One thing that I want to say is that we sincerely apologize that this happened to your DNS service.  We understand that hundreds of thousands of domains rely on our DNS services each day to keep their businesses running smoothly.  This is not something that we treat lightly and this is not something that we are going to just let slip away.  We have already started to plan on building a network to focus on preventing attacks like this from causing any service disruption in the future.
Everyone here at DNS Made Easy would like to thank you for your continued loyalty and kind words during this time.  We can easily say the DNS Made Easy customers are the best in the business.

Question, comments, concerns?
Please let us know.  I personally will be answering as many tickets and questions as possible in the following weeks.  Our full DNS Made Easy staff is dedicated to answering your questions and easing any concerns that you have.

Regards,
-Steven Job
President and Founder of DNS Made Easy