If companies wish to prove they comply with laws governing the handling of information, they must prove they can show when something happened, not simply that it happened — such as when someone accessed a file, created a document, sent an email or logged onto a system. That’s why laws like the Sarbanes-Oxley Act, the Health Insurance Portability and Accountability Act (HIPAA), FINRA’s Order Audit Trail System (OATS), the FDA’s Code of Federal Regulations (CFR), and the Payment Card Industry – Data Security Standards (PCI-DSS) are rife with regulations concerning the accuracy and trustworthiness of time stamps.

Sarbanes-Oxley requires public companies to assess the accuracy and reliability of systems in order to show who accessed what system logs, when and for how long. HIPPA protects patient privacy in part by regulating how hospitals, medical practices and payers use time stamps to control and audit system access. OATS requires that the time stamps on specific data elements related to the handling or execution of orders be within one second of the National Institute of Standards and Technology (NIST) atomic clock. CFR-21, Part 11 requires pharmaceutical manufacturers to employ procedures and controls to ensure the authenticity, integrity and confidentiality of electronic records. Those include measures to ensure the accuracy of computer generated time stamps.

Timekeeping and Credit Card Security
The most far-reaching regulation, however, may be PCI-DSS in that it impacts every merchant that signs an agreement to accept credit or debit cards. PCI-DSS is the creation of the Payment Card Industry Security Standards Council, an organization made up of payment card providers that sets the industry’s security requirements. As of December 31, 2007, all merchants as well as credit and debit card payment processors must adhere to PCI Data Security Standards or face substantial fees, fines, and penalties. The amounts are very high and can be especially damaging for smaller merchants.

Requirement 10 — which mandates how companies should “track and monitor all access to network resources and cardholder data” — requires that whenever cardholder data or a system object is accessed that there is an audit trail for each such event that includes:

• User ID
• Event type
• Date and time
• Success or failure
• Origination of the event
• Identity or name of affect data, system component or resource

Requirement 10 also mandates specific steps companies must take to ensure time stamp accuracy — for example, that the network time server is running the most recent version of NTP and that links to external NTP services are protected from hacker exploit.

The common thread that runs through all these regulations is that it’s much harder to create a compliant environment — regardless of industry — without a modern, secure timekeeping infrastructure on which to build.

If your network falls under any of these compliance requirements, consider adding a Symmetricom SyncServer® to precisely time your network infrastructure. Learn more about how network time servers enable compliant timekeeping infrastructures.

This post was published in Symmetricom’s ThinkSync newsletter, January 2012.
For more information, see our NTP Servers page.

Regular readers of Naked Security will know that I have some strong feelings about timestamps in logfiles.

In particular, the ambiguities created by logfiles based on local time – which is subject to local timezone regulations and changes – can work against your security interests.

Here’s one reason why:

"..Don't let year-ends, timezones, daylight saving changes or varying local conventions confuse your logs. If you suffer a breach, you will almost certainly want to put together an irrefutable historical sequence of events, based on your system logs, possibly from many systems and many locations.."

Local time can confuse even local residents, let alone outsiders trying to make sense of unqualified timestamps in logfiles some time after the event.

For example, in New South Wales, Australia, there are three official timezones: one for the far west of the state, 1000km inland; one for the bulk of the mainland; and a third for Lord Howe Island, 700km to the east of Sydney. And there are two different increments for daylight savings: one hour for most of us; but just half an hour for Lord Howe.

Furthermore, the Government of New South Wales has made three legislative tweaks to local time in the past six years: a switch from GMT to UTC in 2005; a temporary change to daylight savings for the Commonwealth Games in 2006; and a long-term change to prolong daylight savings in 2007.

Confused?

Don’t be, because all of this is as nothing compared to what is going to happen in Samoa and the nearby New Zealand dependency of Tokelau tomorrow.

Or, to put it another way, the day after tomorrow.

It’s currently 1pm on Friday 30 December 2011 in Sydney. It’s 2pm in Samoa, and 3pm in New Zealand. That sounds pretty convenient, considering that the majority of Samoan expatriates live in New Zealand and Australia, and that the three countries have strong business and sporting ties.

Except that it’s only Thursday 29 December in Samoa.

Back in 1892, Samoa did quite a bit of trade with Hawaii and California, so it made sense to decide to be twelve hours behind Greenwich, rather than 12 hours ahead. (Hawaii is UTC-10; California is UTC-8 or UTC-7.)

But in the 21st century, being the most westerly country in the world has become a huge business pain to Samoans when it comes to dealing with Australia and New Zealand, since our weekends don’t line up.

By Friday, Samoans trying to wrap up the week’s business can no longer get hold of their counterparts across the South Pacific – we’re all at the beach, at the shopping mall, or in the pub. And to contact us early on Monday to catch up, the Samoans have to work on their Sunday.

So the Samoan legislature has taken a surprisingly simple, but astonishingly bold, step. At midnight tonight, the country will make a timezone adjustment, switching from UTC-12 to UTC+12. Figuratively, at least, Samoa will jump from one side of the world to the other.

Clocks won’t change at all. Just the calendar will.

Simply put, there will be no Friday 30 December 2011 in Samoa.

How funky is that?

(Samoa is surprisingly good at low-fuss but potentially high-impact bureaucratic change. In 1997, the country changed its name from Western Samoa; in 2009, it switched from driving on the right to driving on the left, as does most of the South Pacific; and in 2011, it will calmly skip an entire day.)

Let this remind you once again why standardised and unambiguous timestamps are vital in logfiles, and take a moment to revisit RFC3339: Date and Time on the Internet: Timestamps.

As I wrote back at the start of 2010:

"..Without reliable logs, you are unlikely to understand [a security] breach, which makes it harder to prevent it happening again. Without reliable logs, you are unlikely to be able to prove your case against the perpetrator, if you are even able to get anyone in your sights. And without reliable and consistent logs, you might not even spot breaches in the first place.."

To everyone in Samoa – Happy New Year! This time, we can celebrate together. And remember this: tomorrow, footy season will be two days closer, not one.

Think about it. Cloud computing is based on the idea of multiple users accessing shared computing resources (servers, applications, storage, networks, etc.) without needing to know where these resources are physically located or even how much of a resource there is. Just like in the electric grid; as demand grows, more capacity comes online without users needing to request it. Theoretically, there could be an unlimited number of machines and software instances interoperating with each other, sharing data and computational tasks dynamically across wide geographic areas. Imagine what could happen if all these players aren’t kept precisely and reliably in sync.

Consider fault diagnosis. If a system crashes, or even if there’s an error that doesn’t bring a machine or application entirely down, technicians need to track down what went wrong. That means creating a timeline of events that transpired over time across many different systems — to see how each event impacted events that followed. Only by “rewinding” the states of the various systems on the same timeline and then “playing” them forward again can technicians get a clear picture of exactly what went wrong and how to prevent it from happening again in the future. However, if the various system clocks are set to even slightly different times it may take investigators much longer to piece together the puzzle, if they ever can.

Or consider data integrity. To operate correctly, distributed applications — such as financial accounting, manufacturing, inventory tracking and many others — rely on time stamps on the data they receive from other locations in order to accept and process that data. These applications must “make sense” out of events occurring almost simultaneously — such as when sales orders received from stores trigger inventory updates at regional warehouses, which in turn are synced with suppliers. Out-of-sync time stamps can cause applications to tag some received data as invalid. In a supply chain scenario it can also cause supply chain irregularity — overstocks on some items accumulating in some places, with shortages elsewhere.

More Challenging to Sync

And just as clouds are more sensitive to time, they are also more challenging to sync. If computers are not all in the same location they may not all be connected to the same time server. Even so, they can still all reference the same “clock in the sky,” i.e., GPS. Having a highly stable local clock for the time server to reference is also key. Ideally that local clock is a rubidium oscillator. If the GPS signal is ever lost (for example, if an antenna is broken or the antenna cable is cut) the Rubidium oscillator will only drift a few milliseconds per day versus a crystal oscillator, which may drift hundreds of milliseconds.

That’s good news for IT managers looking to precisely sync their cloud computing infrastructures. Click here for details.

One such step is development of standards, most notably the Payment Card Industry (PCI) Data Security Standard (DSS). Published last year by the PCI Security Standards Council, the standard includes stringent rules on how systems keep accurate time. Presented as a worldwide standard, the PCI DSS applies to all entities involved in payment card processing — including merchants, processors, acquirers, issuers, and service providers — as well as all other entities that store, process or transmit cardholder data.

With specific reference to timekeeping, the standard requires companies to:

• Have a process for acquiring, distributing and storing the correct time within the organization
• Verify that time-synchronization technology is implemented and kept current.
• Verify that only designated central time servers receive time signals from external sources that are based on UTC (Universal Coordinated Time) or International Atomic Time.
• Verify that the time servers peer with each other to keep accurate time
• Verify that access to time data is restricted to only personnel with a business need to access time data.
• Verify that the time servers accept time updates from specific, industry-accepted external sources (to prevent a malicious individual from changing the clock).

The standard addresses so many timekeeping issues at once that one big compliance challenge will simply be finding solutions comprehensive enough to meet all its requirements. Feature checklists will be long, and the systems providing those features will need to work tightly together — especially time servers, management consoles and auditing software.

Symmetricom’s SyncServer time servers — which interoperate with Domain Time II, Symmetricom’s Windows-based timing infrastructure manager — and which in turn runs Audit Server meet all of the PCI DSS requirements. Aside from synchronizing the time with the SyncServer and Domain Time II, the Audit Server software allows auditors to verify (among other things) when a machine was last synchronized, with what time source, and with what variance from the time source. This end-to-end approach not only ensures compliance virtually out of the box, it also protects cardholder information better than would a collection of individual point solutions cobbled together.

This post was published in Symmetricom’s ThinkSync newsletter, August 2011.
For more information, see our NTP Servers page.

Here’s the lesson: if it can happen to trusted Internet time providers, it can happen anywhere — putting all organizations that rely on an Internet time source at risk.

Today most IT infrastructures require highly precise and reliable time. Accurately synchronized clocks are critical for network log file accuracy, security billing systems, electronic transactions, database integrity and many other essential applications. That’s why outages like the one in May are so troubling.

Organizations can easily avoid this risk, however, if they implement their own time sync service. They can, for example, install highly accurate Symmetricom SyncServer NTP servers with internal with internal atomic oscillators that ensure that accurate time is served, even when the server’s external reference is lost.

Symmetricom’s Network Time Servers SyncServers sync to a GPS satellite receiver or another redundant time server, a best practice that further mitigates risk. Server management software with intuitive web-based interfaces can help IT staff identify timing issues, such as loss of a time reference or an equipment problem — before an outright failure occurs. Such software makes it easy for organizations to set up and manage their own internal time sync service. And once they do, they’ll have peace of mind knowing that their critical applications no longer rely on an Internet time service that is outside their control and which can fail at any time.

This post was published in Symmetricom’s ThinkSync newsletter, June 2011.
For more information, see our NTP Servers page.

SynCan 2011

A warm thank you to all who invested their valuable time to join us, and make our 9th annual SynCan event the great success that it was.

NB. To access the session notes and more distributed to attendees, please register here.

SynCan Background

SynCan is the Canadian Synchronization Users Meeting, hosted by Symmetricom and Empowered Networks.  Each year, synchronization specialists from Canada’s leading players in telecom, wireless, cable and utilities companies are drawn to SynCan, as are a number of key network equipment manufacturers in Canada.

Attendees come to learn – SynCan is a unique learning opportunity for synchronization specialists in Canada.  The educational sessions in key timing and synchronization issues and developments, updates on new and evolving standards and products, and the sharing of “lessons learned” from active deployment of new synchronization technology are an opportunity to learn that is unparalleled in Canada, or anywhere else.

Networking is also an important part of SynCan.  Sync specialists from across the country gain the opportunity to meet with peers, both within their own company, other companies, and with Symmetricom key engineers, developers and product managers – not to mention, Empowered’s sales, service and support team, that stand behind their sync deployments.

2011 Highlights

Here’s selected highlights from SynCan 2011, and attendee feedback:

• Empowered’s Glen Emo opened up SynCan with a few key points:  Synchronization touches virtually all of the infrastructure in the Carrier Central Office, even though it’s often the oldest technology deployed – Modernization is key to mitigating risk, and as a platform supporting growth, and new technology.
• The reality of IEEE 1588v2, aka. Precision Time Protocol or PTP, was evident. Trials are wrapping up, test results are in, demonstrating the value of PTP in distributing synchronization over Ethernet networks.  Moreso, field deployments are advancing rapidly to support new LTE deployments.  If it’s not done, it’s the key item on the planning horizon.
• Symmetricom’s new “Packet PRS” solution, the TimeProvider 1500, combines 1588 PTP with a highly stable rubidium atomic oscillator to deliver PRS performance and holdover at remote sites – without requiring GPS.
• A key focus in the standards bodies (ITU-T, IEEE, IETF, MEF and others) is the “performance” of packet timing – distributing frequency sync over packet networks – and how it is affected by packet delay variation, or PDV.
• An excellent customer presentation from Nima Salehi of TELUS drove home the point that PDV is the key variable that network operators can control.
• Key priorities in Sync for SynCan attendees centered around deploying PTP, with specific reference to leveraging new features and capabilities in the SSU 2000 and TimeProvider 5000 platforms.  Other priorities included improving redundancy (eg. deploying GPS Backup), and refining operational processes.
• Feedback confirmed that SynCan was an excellent learning opportunity.
• The less formal discussions over Wednesday’s group dinner (and after!) somehow took the top rankings in attendee feedback.  Perhaps that’s unfair to our speakers, who did an excellent job on all accounts.
• Our sincere thanks to Symmetricom for their generous support of SynCan 2011

Going Forward

To access the session notes and information distributed to SynCan 2011 attendees, please register here. We’ll keep you in the loop on Timing & Synchronization, going forward.

If there’s something else you need, please contact your Empowered representative.
We’re here to help!

On behalf of Symmetricom and Empowered Networks,
thank you for making SynCan 2011 another great success!

The problem: an intense period of solar flares that occurs roughly every 11 years, called the solar maximum. Charged particles from flares produce intense bursts of radio noise that peak in the 1.2 and 1.6 gigahertz bands used by GPS. Usually radio noise in GPS bands is very low so receivers can pick up signals from orbiting satellites, even though GPS signals themselves are very weak. Charged particles trapped in the ionosphere can cause additional disruption. The trapped particles create wide fluctuations in the time GPS signals take to traverse the ionosphere, causing significant timing calculation errors.

Since the last solar maximum occurred roughly a decade ago, before GPS “came of age,” society has learned to rely on GPS during a period of relative solar calm. That means any disruptions that do occur over the next two years could very much be unanticipated — and unplanned for.

Users of GPS applications should ask themselves now: “What would it mean if my applications lost GPS for ten minutes, an hour, or more several times a year? In particular, what would it mean to my ability to keep precise time?

There is, after all, no backup to satellite-based time and navigation systems. If GPS receivers “go blind,” as predicted, they will rely on their holdover clocks — internal crystal or rubidium oscillators or external cesium oscillators — until GPS signals are reacquired. The question then becomes: “Will my current holdover clock maintain sufficient precision throughout the period of GPS signal loss?”

The answer depends on the amount of time that the holdover clock will drift over that period. The more the drift, the faster time goes “out of spec.” Here, for example, is the amount of time in seconds that various types of oscillators in GPS time and frequency instruments can be expected to drift over a one day period. Cesium: <100 nanoseconds; Rubidium: <5 microseconds; OCXO: <1 millisecond;  TCXO:  <18 milliseconds.

As the solar maximum approaches, now might be a good time to reevaluate your GPS receivers’ holdover clocks. Your time can still be reliable, even when your GPS signal is not.

See Symmetricom’s line of NTP time servers.

This entry was first published in Symmetricom’s “Just A Second” newsletter, September 2010.

Overview
SynCan 2010 was the 8th annual Canadian Users Meeting, hosted by Symmetricom and Empowered Networks.  As in previous years, attendees included key representatives of the telecom, wireless and utility telecom marketplace in Canada.

Highlights
Some highlights from SynCan 2010, and attendee feedback:

• Empowered’s Glen Emo opened up SynCan with lessons learned from Aloha Flight 243, drawing a parallel between the life expectancy of short-hop aircraft, and the legacy sync gear deployed in networks across Canada, now at or beyond End-of-Life
• Material progress of standards around Telecom Synchronization, especially the IEEE 1588-2008 standard (aka. Precision Time Protocol or PTP), and Synchronous Ethernet (aka. SyncE)
• Peter Roberts of Alcatel-Lucent highlighted the importance of PTP and SyncE to equipment manufacturers, and their solutions for Carrier Networks, and Services
• Legacy Upgrade Programs generated lots of discussion around both the challenges highlighted by Rob Hockin, and the help our team can provide.  Some attendees have programs in place – other recognize the urgent need to get such programs underway.
• The importance of sync to Power Utilities was an eye-opener for many attendees.  The developing Smart Grid will make sync even more important.
• Attendees said their key priorities in Sync for 2010 ranged from modernizing existing Sync infrastructure, to testing and deploying PTP solutions.
• There’s an evident need to concisely explain the importance of Sync to others. We’re presently working on a related initiative, and will share the results on this soon.
• Attendee feedback strongly reflected that SynCan was an excellent learning opportunity
• The venue for the group dinner was excellent – The views from the 38th floor, overlooking the islands of Toronto Harbour, the landing path for Island Airport, and as night fell, the Toronto skyline was a highlight.
• Our sincere thanks to Symmetricom for their generous support of SynCan 2010

Going Forward
To access the session notes and information distributed to those who attended SynCan 2010, please register here. We’ll keep you in the loop on Timing & Synchronization, going forward.  If there’s something else you need, please contact us directly. We’re here to help!

On behalf of Symmetricom and Empowered Networks, thank you for making SynCan 2010 another great success!

We hope to see you at SynCan 2011!