[time-nuts] NTP stratum 1 appliances with different (GPS, etc) cores...

[Robert Seastrom]

Hi folks,

Had a recent requirement at $DAYJOB to give our NTP architecture a good swift kick in the pants. High precision is not a particular requirement (1 ms is more than adequate) but high reliability is. I'm about done with explaining the difference between the two to people internally who don't get it. :-(

Unfortunately, these days it's impossible to get sufficient out-of-band sources for stratum 1 clocks unless we're building our own constellation of primary frequency standards (Not Gonna Happen). So I definitely intend to have our stratum 2 clocks (which customer and production devices will be pointing at) chiming off of plenty of off-site stuff too, hopefully things that are connected physically to a diverse set of primary sources (i.e. not gps or wwvb)

This approach would probably be fine for our internal consumption, frankly, considering our need for reliability but not supreme accuracy. But there's still a desire internally to run a set of stratum 1 radio clocks. In the course of this discussion, I invoked a particularly nasty "conspiracy" that happened back in 2002 wherein every TrueTime NTS-100 (which were extremely popular in those days) simultaneously decided that it was 20 years in the future and warped the time. That's not the only case of devices losing their cookies; I seem to recall a problem with certain Motorola cores having an issue with leap seconds.

Anyway, I couldn't keep my mouth shut and suggested that at the very least, trying for multiple vendors (we have a strong preference for appliancey stuff here due to internal fiefdoms, ownership issues, and balkanization typical of a Fortune 200 company) with different GPS cores would be a really fine plan.

So of course, now it's my job to come up with proper nominees for this list. Any suggestions would be greatly appreciated. Hopefully someone's already done the legwork for this. I will likely send this same request to the ntp dev list...

Thanks,

-r


_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Neil Schroeder]

Ones ive deployed:

End run. BSD
Microsemi S350 LINUX
Meinberg MRS LINUX

Others:
Spectra com
Brandywine

The three i used were all very good. You want more re details I will put
some effort into it.
_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Bob Darlington]

I run a Symmetricom Syncserver S300 at home and am trying to convince the
fine folks at work that running "ntpdate" every 15 minutes is about the
worst possible way to set time. I'm in the NTP.org pool but am open to
direct peering if you ever get something setup:

http://www.pool.ntp.org/scores/23.31.117.13

This server hardware can support an orginization at least 10x the size of
mine (Los Alamos National Laboratory).

-Bob

Post by Robert Seastrom
Hi folks,
Had a recent requirement at $DAYJOB to give our NTP architecture a good
swift kick in the pants. High precision is not a particular requirement (1
ms is more than adequate) but high reliability is. I'm about done with
explaining the difference between the two to people internally who don't
get it. :-(
Unfortunately, these days it's impossible to get sufficient out-of-band
sources for stratum 1 clocks unless we're building our own constellation of
primary frequency standards (Not Gonna Happen). So I definitely intend to
have our stratum 2 clocks (which customer and production devices will be
pointing at) chiming off of plenty of off-site stuff too, hopefully things
that are connected physically to a diverse set of primary sources (i.e. not
gps or wwvb)
This approach would probably be fine for our internal consumption,
frankly, considering our need for reliability but not supreme accuracy.
But there's still a desire internally to run a set of stratum 1 radio
clocks. In the course of this discussion, I invoked a particularly nasty
"conspiracy" that happened back in 2002 wherein every TrueTime NTS-100
(which were extremely popular in those days) simultaneously decided that it
was 20 years in the future and warped the time. That's not the only case
of devices losing their cookies; I seem to recall a problem with certain
Motorola cores having an issue with leap seconds.
Anyway, I couldn't keep my mouth shut and suggested that at the very
least, trying for multiple vendors (we have a strong preference for
appliancey stuff here due to internal fiefdoms, ownership issues, and
balkanization typical of a Fortune 200 company) with different GPS cores
would be a really fine plan.
So of course, now it's my job to come up with proper nominees for this
list. Any suggestions would be greatly appreciated. Hopefully someone's
already done the legwork for this. I will likely send this same request to
the ntp dev list...
Thanks,
-r
_______________________________________________
To unsubscribe, go to
https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Chris Albertson]

Post by Bob Darlington
I run a Symmetricom Syncserver S300 at home and am trying to convince the
fine folks at work that running "ntpdate" every 15 minutes is about the
worst possible way to set time.

Best way to explain the difference is to use a real spring driven clock as
an example. ntpdate is like seeing the hands to the correct time while
the normal mode (running ntpd) is like adjusting the fast/slow lever such
that you never have to set the hands.

Also ask some one to compute a time interval on a spring driven clock,
while they are doing it adjust the hands on the clock several times.

[Chris Albertson]

The exact architecture you need depends a lot on how your company is
geographically distributed and the network that connects those parts.

If you have decided you want to run your own stratum 1 NTP servers
then you'd place a few of those in each plant or in each city. Then your
next level servers, one server per local network can look at ALL the
stratum 1 servers plus some others out on the Internet.

The way to keep this RELIABLE is through e config files. Make sure that
EVERY computer that uses NTP has at least FIVE reference clocks. Some
company servers then some Internet pool servers.

Make sure to use authentication. This will keep spoofers out

The other thing that can help reliability is use of orphan mode. This is a
fall back mode where a group of NTP servers gets disconnected from the
outside world. Then they can use each other as a reference clock and keep
and kind of consensus going.

Also remember when I say "NTP Servers" I mean anything that runs NTP which
should be just about anything including routers and file servers. All of
these need at least five reference clocks that are widely distributed and
to be configured to fall back on Orphan Mode..

Reliability starts near the end users, right on their computers. Configure
each one to look at many other clocks and those clocks to look at many
other. You want the fan out be very wide from the start. Orphan networks
can work well if part of your network is isolated from the rest of the
world if there are many computers. The ones with good internal clocks will
all tend to agree with each other and drive the others. Then when Internet
service is restored to the building or lab better clocks will become
available.

Post by Robert Seastrom
Hi folks,
Had a recent requirement at $DAYJOB to give our NTP architecture a good
swift kick in the pants. High precision is not a particular requirement (1
ms is more than adequate) but high reliability is. I'm about done with
explaining the difference between the two to people internally who don't
get it. :-(
Unfortunately, these days it's impossible to get sufficient out-of-band
sources for stratum 1 clocks unless we're building our own constellation of
primary frequency standards (Not Gonna Happen). So I definitely intend to
have our stratum 2 clocks (which customer and production devices will be
pointing at) chiming off of plenty of off-site stuff too, hopefully things
that are connected physically to a diverse set of primary sources (i.e. not
gps or wwvb)
This approach would probably be fine for our internal consumption,
frankly, considering our need for reliability but not supreme accuracy.
But there's still a desire internally to run a set of stratum 1 radio
clocks. In the course of this discussion, I invoked a particularly nasty
"conspiracy" that happened back in 2002 wherein every TrueTime NTS-100
(which were extremely popular in those days) simultaneously decided that it
was 20 years in the future and warped the time. That's not the only case
of devices losing their cookies; I seem to recall a problem with certain
Motorola cores having an issue with leap seconds.
Anyway, I couldn't keep my mouth shut and suggested that at the very
least, trying for multiple vendors (we have a strong preference for
appliancey stuff here due to internal fiefdoms, ownership issues, and
balkanization typical of a Fortune 200 company) with different GPS cores
would be a really fine plan.
So of course, now it's my job to come up with proper nominees for this
list. Any suggestions would be greatly appreciated. Hopefully someone's
already done the legwork for this. I will likely send this same request to
the ntp dev list...
Thanks,
-r
_______________________________________________
To unsubscribe, go to
https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Robert Seastrom]

Post by Robert Seastrom
Anyway, I couldn't keep my mouth shut and suggested that at the very least, trying for multiple vendors (we have a strong preference for appliancey stuff here due to internal fiefdoms, ownership issues, and balkanization typical of a Fortune 200 company) with different GPS cores would be a really fine plan.
So of course, now it's my job to come up with proper nominees for this list. Any suggestions would be greatly appreciated. Hopefully someone's already done the legwork for this. I will likely send this same request to the ntp dev list...

Appreciate the feedback I've gotten on and off list so far. It occurs to me that I may not have made it completely clear what I'm looking for.

Specifically, I'm looking for information of the form "We are using X radio clock appliance which is still available/not EOL and it uses Y core and has a source of Z (likely limited to GPS and WWVB)".

Or better yet, "Last year we deployed one each of A with A' core, B with B' core, C with C' core, D with D' core, and E which unfortunately has the popular A' core but those are the breaks, and unfortunately ".

Roof access is not a problem. We own the datacenters in question and the land underneath them; I just write up engineering docs and a MOP and open a ticket with Critical Infrastructure and voila! antenna on roof, properly mounted and grounded, with cabling that comes to the rack where the radio clocks live. I don't even have to climb ladders with a tool satchel. Whenever I am annoyed by certain aspects of working for a big company, I remember things like this.

Besides GPS, we have a datacenter around 115 km from WWV and WWVB, so those are viable choices too. Alas, no more GOES, LORAN, etc. If anyone knows of over-the-air solutions or a pure GLONASS solution that unwinds cleanly to UTC within the tolearances I outlined in the original post so much the better.

Money is not a problem, though it would be a mistake to regard us as an organization with more money than clue.

Unfortunately, lovingly hand-built solutions, even OS-on-commodity-server-with-board, aren't going to fly due to the aforementioned ownership issues. I'd love to be assembling stuff myself, but then I'd be stuck with supporting it. I suffer from both the "hit by a bus" problem and the fact that getting to Operations from where I sit in the org chart involves traversing a pair of senior vice presidents. When one is purely on the R&D/Architecture side, it won't do to deliver a solution to Operations that they can't support... so commercial product is 100% a must.

Ditto chiming to anyone as a source who we don't have a contractual relationship with (and probably a direct connection to). Our change control process makes this unpalatable.

I also get it as far as network time infrastructure goes and how to put it together and monitor it. Still grateful to Dr. Mills for failing to chase me away from his lab when I was an ankle-biting high school student.

If anyone's done FMECA for time infrastructure and can share the document, it would save me a fair amount of work and get me additional traction internally.

Thanks,

-r


_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.