Roger Rehr W3SZ w3sz73 at gmail.com
Fri Nov 4 13:42:24 CET 2016

Hi All,

I second everything that Alex sent.  Two of his points bear re-emphasis:

1.  If you are not using identical hardware feeding identical data to
both programs when comparing MAP65 with WSJT, then you are NOT really
comparing MAP65 with WSJT.  You are comparing (hardware-1 and MAP65)
with (hardware-2 and WSJT), and may get different results than if you
were truly comparing MAP65 with WSJT.

2.  If you don't have your hardware optimized then your comparison won't
reflect what the system in question is really capable of.  Specifically,
if you don't feed the IQ+ with the signal levels it was designed to use,
it won't perform optimally.  Running it with inadequate signal levels
and then being disappointed in the result is like buying a Lamborghini
Aventador and then running it on diesel fuel and wondering why it
doesn't perform properly.

I would also add a third comment:
3.  I am not aware of anyone who has performed a comparison between
MAP65 and WSJT using identical hardware feeding identical data to both
programs who has published data showing a significant difference between
the MAP65 and WSJT decoding results.  That is not to say that at
individual stations differences may not be seen between results with
these two programs.  But it suggests that the differences that these
stations see is not necessarily due to differences between MAP65 and WSJT.

As most of you know, I did a study using data from the 2012 ARRL EME
contest which I published online in 2013 including a total of 8805
decodes that showed what Alex described, namely that the MAP65 and WSJT
decoders performed similarly.  I went into some detail, and the results
are still online at:

Alex was too modest to post the link to his excellent paper from this
year's EME contest, but the slides are online here:
and the paper is in here, on page 39:
I encourage all to read both of these excellent documents.

My results from 2012 are now obsolete because they used WSJT with the
old KV decoder.  I have started a new analysis using data from this
year's ARRL EME contest, where I used WSJT-X with the new Franke-Taylor
decoder instead of WSJT.  Because MAP65 still has the old KV decoder,
one would expect that there might be a different result from this
analysis than the result I obtained with the 2012 data.  We shall see :)

This new comparison is of course in one respect comparing apples (KV
decoder) to oranges (Franke-Taylor decoder). But the comparison is
really of what the available version of MAP65 will be able to do for a
user compared to what WSJT-X will be able do for a user.  In that
respect the comparison is of apples to apples.

Of course, this new analysis will also become obsolete when the
Franke-Taylor detector is added to MAP65 or its updated replacement, and
then another analysis will need to be done.  I hope that the need for
that analysis will be very soon!


Roger Rehr W3SZ

On 11/4/2016 12:42 AM, Alex Artieda wrote:
> Dear Friends
> With great interest I'm following this post and with special attention the
> observations done by Serge, Les and both Rogers, this topic is not new, for
> that reason during the las EME Conference in Venice I present a properly
> setup to compare WSJT vs MAP65 or any other software.
> I will not re-write what is already printed in the proceeds of the
> Conference but I will point to some key elements:
> 1) Again "IQ+ DONT DECODE" is just the receiver presenting the Analog signal
> properly amplified to your ADC, means your audio card.
> 2) The IQ+ is designed to lift the noise floor 16dB when a preamp with 26dB
> gain is in front, the properly test is remove the antenna from the preamp
> input and place a 50ohm dummy load on the preamp input, if you power on/off
> the preamp under this condition the noise floor MUST lift 16dB, if not: your
> preamp is defective or has not enough gain, your attenuation after the
> preamp is too high, your audio levels in your audio card are not properly
> adjusted or your internal preamp in the IQ+ is defective. Could be just one
> or more or all this options together.
> 3) Splitting your RX line can be done always with a CERO DEGREE 2 way
> splitter, you must split the signal preserving phase and resulting with same
> amplitude on both output ports, normally 3dB less. A simple BNC TEE; IS THE
> 4) The properly setup is if you use Linrad as input and then you send the
> digital signal to MAP65 and WSJT, then your comparison will be valid because
> you are presenting the same signal coming from the same receiver.
> 5) Here are my conclusions presented in the EME Conference in Venice after
> many years of observations and thousands of QSO in 144Mhz with both systems
> running in parallel n HB9Q:
> ___________________
> I don't' think so, what I believe is both programs behave (in terms of
> decodes) pretty much the same and if exist a difference is not more than1%.
> How both software can decode "the same" when WSJT is looking just for 4KHz
> of bandwidth and MAP65 90 KHz.? I think exist a trade-off compromise about
> Wide-decoding, sensitivity and confidence to avoid false decodes.
> Maybe is a matter of interpretation and your goals you establish about
> performance. What is clear is sometimes WSJT decodes when MAP65 doesn't but
> I see many times the inverse relation when MAP65 beat WSJT in very low SNR
> ratios, is very difficult (and non-professional) to just assume WSJT is
> better than MAP65.
> I think, and this needs to be confirmed in the next years (when enough data
> is collected and analyzed properly with the correct Methodology and
> Metrology), in some specific cases WSJT, under Multi-polarization
> configuration, decodes when MAP65 not. These specific cases are
> statistically very small (less than 1% for SNR -27 to -30dB) but if those
> particular cases are in relation with a new DXCC or rare grid then the value
> to implement such a complex system is tremendous and corresponds to each
> individual to evaluate and balance the cost/benefit of this system. Not
> because the differences are less than 1% we will neglect that differences
> exist, especially when pushing to the limits our software and Hardware could
> be translated in just 1 DXCC more in your list, is a matter of individual
> decision I guess.
> __________________
> 73 de Alex, ZS6EME

More information about the Moon-net mailing list