[time-nuts] A (slightly) different apu2 question

[Jay Grizzard]

So there's been a lot of discussion going around on how to do GPS foo on
pcengines.ch's apu2 hardware, but there's one question I haven't seen
discussed ... which I'm now going to discuss. Or at least ask about.

I can't find a public datasheet for the actual processor in these (a AMD
GX-412TC SOC), but looking at datasheets for similar AMD chips, this SOC
seems to use a single 48MHz external crystal from which all the other
system clocks are derived (save for the 32.768kHz RTC).

On the apu2, this crystal is easily accessible (at least as easy as
anything SMD is). Can anyone think of a reason that it wouldn't be
feasible to replace this crystal with an external reference, à la the
widely known clockblock + Soekris net4501 hack (but with 64x the RAM)? I
figure the higher frequency might make it a bit trickier to get the
signal to the board intact, but is there any other good reason this
wouldn't work?

The CPU itself is four cores (no hyperthreading), so I'm figuring
dedicate one core to PPS handling (should give really low-jitter
interrupt handling), maybe one to ntpd, and combined with that precision
reference, a pretty nice NTP/PTP server should pop out the other side.
The ethernet on the apu2 even does hardware timestamping.

Can anyone think of a reason this wouldn't work, before I break out the
rework gear?

-j

_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Mike Cook]

Sorry if this is a dup. I had accidentally left the « SPAM » prefix on my first reply.

So there's been a lot of discussion going around on how to do GPS foo on pcengines.ch's apu2 hardware, but there's one question I haven't seen discussed ... which I'm now going to discuss. Or at least ask about.
I can't find a public datasheet for the actual processor in these (a AMD GX-412TC SOC), but looking at datasheets for similar AMD chips, this SOC seems to use a single 48MHz external crystal from which all the other system clocks are derived (save for the 32.768kHz RTC).

The principle should be applicable…. BUT you had better look closely at the schematics. There are some very low voltages in there.

On the apu2, this crystal is easily accessible (at least as easy as anything SMD is). Can anyone think of a reason that it wouldn't be feasible to replace this crystal with an external reference, à la the widely known clockblock + Soekris net4501 hack (but with 64x the RAM)? I figure the higher frequency might make it a bit trickier to get the signal to the board intact, but is there any other good reason this wouldn't work?
The CPU itself is four cores (no hyperthreading), so I'm figuring dedicate one core to PPS handling (should give really low-jitter interrupt handling), maybe one to ntpd, and combined with that precision reference, a pretty nice NTP/PTP server should pop out the other side. The ethernet on the apu2 even does hardware timestamping.
Can anyone think of a reason this wouldn't work, before I break out the rework gear?
-j
_______________________________________________
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

"The power of accurate observation is commonly called cynicism by those who have not got it. »
George Bernard Shaw

_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Attila Kinali]

On Wed, 16 Nov 2016 15:24:06 -0800

Post by Jay Grizzard
On the apu2, this crystal is easily accessible (at least as easy as
anything SMD is). Can anyone think of a reason that it wouldn't be
feasible to replace this crystal with an external reference, à la the
widely known clockblock + Soekris net4501 hack (but with 64x the RAM)? I
figure the higher frequency might make it a bit trickier to get the
signal to the board intact, but is there any other good reason this
wouldn't work?

You should check with Pascal Dornier (the guy behind pc-engines).
He can exactly tell you what would work and what wouldn't.
And he is generally very very helpful.

Attila Kinali

[Wojciech Owczarek]

...do excuse the slightly off-topic 5(insert your preferred sub-currency
here).

I think many people put a significant effort into getting a precise pulse
into a board like the APU2, losing a lot of precision and accuracy by
getting it into the O/S software clock, and then perfectly waste what's
left by then retransmitting it via NTP with software timestamping. I am not
saying that there is anything wrong with NTP, just that software packet
transmission and receipt timestamps ruin both precision and accuracy.

The APU2 is a real gem for many reasons, but one is of significance for
time sync. The board uses discrete i210 PHY/MAC chips which support
hardware timestamping (of all packets, not just PTP) - rather than using an
in-SOC Intel MAC paired with a different PHY like Marvell or, diety forbid,
Realtek. When designing the board, Pascal did the time nuts a huge favour
by presenting pins 60-63 of each i210 as test pads on the board, four per
chip. These pins are software-defined I/O pins which can be used for 1PPS
output, arbitrary square wave output, and input event timestamping - all
tied to the i210 hardware clocks.

I have exchanged a few e-mails with Pascal and he hinted that he will
consider breaking those i210 pads either as a pin header or possibly as
u.FL connectors - maybe not all, but say two per NIC. The only thing that
could make this board better (but then it would be a different board) is if
it used the latest generation of Intel Atom, where they implemented counter
correlation between the CPU and NICs, allowing for very tight O/S clock to
NIC clock sync. But that Atom was only recently announced (Skylake CPUs do
this already), and an AMD CPU was selected for the APU2 for specific
reasons.

Another small board with similar potential is the upcoming Minnowboard
Turbot Dual-E (
http://www.adiengineering.com/products/minnowboard-turbot-duale/):
Atom-based, small, with two i211 right on the board.

Cheers,
Wojciech

Post by Attila Kinali
On Wed, 16 Nov 2016 15:24:06 -0800

Post by Jay Grizzard
On the apu2, this crystal is easily accessible (at least as easy as
anything SMD is). Can anyone think of a reason that it wouldn't be
feasible to replace this crystal with an external reference, à la the
widely known clockblock + Soekris net4501 hack (but with 64x the RAM)? I
figure the higher frequency might make it a bit trickier to get the
signal to the board intact, but is there any other good reason this
wouldn't work?

You should check with Pascal Dornier (the guy behind pc-engines).
He can exactly tell you what would work and what wouldn't.
And he is generally very very helpful.
Attila Kinali
--
It is upon moral qualities that a society is ultimately founded. All
the prosperity and technological sophistication in the world is of no
use without that foundation.
-- Miss Matheson, The Diamond Age, Neil Stephenson
_______________________________________________
To unsubscribe, go to https://www.febo.com/cgi-bin/
mailman/listinfo/time-nuts
and follow the instructions there.

[Paul]

... waste what's
left by then retransmitting it via NTP with software timestamping. I am not
saying that there is anything wrong with NTP, just that software packet
transmission and receipt timestamps ruin both precision and accuracy.

NTP does not require softstamps. NTP can be (and I believe it has been in
a product) modified to use "PTP" hardware (hardstamps) and reasonably
current releases can run with "drivestamps" (sampled in the NIC driver)
between cooperating endpoints.
_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Wojciech Owczarek]

Post by Paul
NTP does not require softstamps. NTP can be (and I believe it has been in
a product) modified to use "PTP" hardware (hardstamps) and reasonably
current releases can run with "drivestamps" (sampled in the NIC driver)
between cooperating endpoints.

Of course it does not *require* software timestamps, I never said that, it
is just the fact that most people use it with software timestamps, even if
the NIC permits hardware timestamps. You need a secondary servo to sync the
NIC to O/S clock if you want to serve that (or consume that) - or sync the
NIC to external 1PPS. What you call a "drivestamp", I call a software
timestamp. If it's not a function of the silicon, it's software. All I
meant was that there is some unexploited potential.

[Martin Burnicki]

Post by Wojciech Owczarek

Post by Paul
NTP does not require softstamps. NTP can be (and I believe it has been in
a product) modified to use "PTP" hardware (hardstamps) and reasonably
current releases can run with "drivestamps" (sampled in the NIC driver)
between cooperating endpoints.

Of course it does not *require* software timestamps, I never said that, it
is just the fact that most people use it with software timestamps, even if
the NIC permits hardware timestamps. You need a secondary servo to sync the
NIC to O/S clock if you want to serve that (or consume that) - or sync the
NIC to external 1PPS. What you call a "drivestamp", I call a software
timestamp. If it's not a function of the silicon, it's software. All I
meant was that there is some unexploited potential.

Just a few thoughts regarding NTP vs. PTP:

IMO NTP can yield pretty good results even over WAN connections
*without* the requirement of special hardware which supports
timestamping of network packets, including special switches and routers,
even over WAN connections, and in the presence of network jitter. For
example from one of my workstation running Linux, here in Germany:

remote refid st t when poll reach delay offset jitter
========================================================================
*SHM(0) .shm0. 0 l 4 8 377 0.000 -0.001 0.000
lt-martin.py.me .MRS. 1 u 9 64 377 0.095 -0.005 0.020
ntp-master-1.py .PPS. 1 u 58 64 377 0.191 -0.019 0.013
ptbtime1.ptb.de .PTB. 1 u 11 128 377 11.927 0.256 74.533
ptbtime2.ptb.de .PTB. 1 u 16 128 377 11.706 0.085 79.565
ptbtime3.ptb.de .PTB. 1 u 35 128 377 11.590 0.058 74.049
time-b.timefreq .ACTS. 1 u 103 128 377 198.358 -1.377 91.992

where

- SHM is a local GPS PCI card

- lt-martin and ntp-master-1 are GPS controlled NTP servers on the local
network

- ptbtime{1,2,3} are the public NTP servers operated by the German
metrology institute PTB, reached via 7 network hops

- time-b.timefreq is a public server in the U.S., reached via 9 network
hops, including a trans-atlantic connection.


Of course, PTP can do *much* better if special hardware is being used.

We have made tests here at Meinberg where we could demonstrate that NTP
yields the same level of accuracy as PTP with a patched ntpd which
supports hardware time stamping of NTP packets. We used an own time
stamp unit which can also time stamp NTP packets.

There are 2 problems with this, though:

1.) While the PTP folks have been requesting support for time stamping
PTP packets in NIC chips as well as specific PTP support by switches and
routers for a long time, this hasn't happened for NTP.

So today we have indeed NIC chips which time stamp PTP packets, and
switches which are PTP-aware and measure the propagation delay of PTP
packets (in transparent clock mode) so that the PTP daemon can
compensate this delay and eliminate the network jitter.

There's no such thing for NTP.

2.) In the original approach introduced by PTP an outgoing packet is
time stamped by the NIC, and then a "followup" packet is sent which
contains that time stamp ("twostep" mode). This eliminates the
processing delay between the time a packet is sent out by the daemon
until it goes onto the network cable.

NTP doesn't support this feature, and introduction of this feature would
either break compatibility of the existing NTP protocol, or would
require the definition of a new protocol version, e.g. NTPv5.
(BTW, I'm aware of ntpd's "interleave" mode, but IMO that's just a hack,
and doesn't even work with normal client/server packet exchanges).

Of course PTP has a "onestep" mode today where a time stamp is taken as
soon as the packet goes out on the wire, and that time stamp is inserted
into the same packet on the fly, so no followup packet is required.

However, this requires even more sophisticated hardware which explicitly
supports this, and even if this was available for NTP this still
wouldn't solve problem of NTP time stamping support missing in NICs and
switches.

So I think NTP still meets the requirements of many users, without
requirement for specific hardware, while PTP can be used to yield a much
higher level of accuracy if you have dedicated hardware available.

Martin

_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.

[Wojciech Owczarek]

Oh dear... OK, before this snowballs into a bigger discussion this was not
meant to be:

My original post was _not_ about PTP vs. NTP, it was about the APU2 boards
and hardware timestamping.

My only point was to highlight the following:

1 This board has got exposed test pads for lots of 1PPS / frequency in /
out pins tied to timestamping hardware making it very easy to tap into
them. This is a great opportunity which begs to be exploited.

2. With a stable 1PPS from GPS or another source, this board can serve
nanosecond time, and that includes NTP in hardware, because the Intel NICs
can timestamp every packet, not just PTP. And yes, this mostly matters for
LAN.

I am fully aware of what you listed. NTP is just a different beast and
silicon vendors jumped onto PTP, and PTP inherently lacks the sophisticated
quorum and trust mechanisms NTP has, and yes, those are key items on the
P1588 WG agenda.

Thanks,
Wojciech

[Jay Grizzard]

Post by Wojciech Owczarek
1 This board has got exposed test pads for lots of 1PPS / frequency in /
out pins tied to timestamping hardware making it very easy to tap into
them. This is a great opportunity which begs to be exploited.

I was actually unaware that you could discipline these ports like this!
This is actually the first system I'm getting to play with PTP on (to any
serious degree), so I'm afraid I'm missing a lot of desirable background.

RESEARCH TIME!

I got this apu2 specifically to hack the hardware of... I'll report back
once I play with this.

-j
_______________________________________________
time-nuts mailing list -- time-***@febo.com
To unsubscribe, go to https://www.febo.com/cgi-bin/mailman/listinfo/time-nuts
and follow the instructions there.