Edward R Cole kl7uw at acsalaska.net
Fri Nov 4 18:49:48 CET 2016

A couple coments (inserted):

At 04:42 AM 11/4/2016, Roger Rehr W3SZ wrote:
>Hi All,
>I second everything that Alex sent.  Two of his points bear re-emphasis:
>1.  If you are not using identical hardware feeding identical data to
>both programs when comparing MAP65 with WSJT, then you are NOT really
>comparing MAP65 with WSJT.  You are comparing (hardware-1 and MAP65)
>with (hardware-2 and WSJT), and may get different results than if you
>were truly comparing MAP65 with WSJT.

Its my understanding that one can record the soundcard audio stream 
as a .wav file which you could then run on WSJT or MAP65, and compare 
how well they decode.  I am less concerned that the signal level 
reported is the same, than that the decoding thresholds are 
equal.  (actually I am not that concerned as I have excellent 
performance from MAP65 and WSJT10, both using the KV decoder.

>2.  If you don't have your hardware optimized then your comparison won't
>reflect what the system in question is really capable of.  Specifically,
>if you don't feed the IQ+ with the signal levels it was designed to use,
>it won't perform optimally.  Running it with inadequate signal levels
>and then being disappointed in the result is like buying a Lamborghini
>Aventador and then running it on diesel fuel and wondering why it
>doesn't perform properly.

Very much agree about this point.  One must measure not use 
assumptions, to know if they have set up with required levels for 
their equipment.  If you have not done this, then your observation 
are intuitive (guesses).  JT65 works using statistical processing so 
one cannot "really" know from on-air use if one sw is working better.

>I would also add a third comment:
>3.  I am not aware of anyone who has performed a comparison between
>MAP65 and WSJT using identical hardware feeding identical data to both
>programs who has published data showing a significant difference between
>the MAP65 and WSJT decoding results.  That is not to say that at
>individual stations differences may not be seen between results with
>these two programs.  But it suggests that the differences that these
>stations see is not necessarily due to differences between MAP65 and WSJT.

This is a Biggy.  Serge's comparison is with two different hardware 
and sw.  That is not going to prove how well just the sw works. One 
test I would suggest is swaping the WSJT and MAP65 to compare how 
well they run on the other hardware.  Still subjective comparison but 
might indicate if one of the hardware systems is not up to par.

>As most of you know, I did a study using data from the 2012 ARRL EME
>contest which I published online in 2013 including a total of 8805
>decodes that showed what Alex described, namely that the MAP65 and WSJT
>decoders performed similarly.  I went into some detail, and the results
>are still online at:
>Alex was too modest to post the link to his excellent paper from this
>year's EME contest, but the slides are online here:
>and the paper is in here, on page 39:
>I encourage all to read both of these excellent documents.

I'll take a look at all.  I did save all the presentations from 
EME-2016 for reading.

>My results from 2012 are now obsolete because they used WSJT with the
>old KV decoder.  I have started a new analysis using data from this
>year's ARRL EME contest, where I used WSJT-X with the new Franke-Taylor
>decoder instead of WSJT.  Because MAP65 still has the old KV decoder,
>one would expect that there might be a different result from this
>analysis than the result I obtained with the 2012 data.  We shall see :)

I am a little surprised that WSJT-X is usable for eme; my take was 
that it was designed for HF?
I hope no one is writing WSJT-X when they mean WSJT10.

I have not had time to try the new decoders; my understanding was 
that the new QRA would not be compatible with JT65?  But I have 
hardware to fix before bothering to get into this.  Winter is 
starting so "its definitely burning daylight" as our days shorten.

>This new comparison is of course in one respect comparing apples (KV
>decoder) to oranges (Franke-Taylor decoder). But the comparison is
>really of what the available version of MAP65 will be able to do for a
>user compared to what WSJT-X will be able do for a user.  In that
>respect the comparison is of apples to apples.

Yes, keep comparisons for same decoding sw.

>Of course, this new analysis will also become obsolete when the
>Franke-Taylor detector is added to MAP65 or its updated replacement, and
>then another analysis will need to be done.  I hope that the need for
>that analysis will be very soon!

That will be another topic.  Be interested to read about and try using.

>Roger Rehr W3SZ

Finally comments on my hardware:
I run WSJT10 on 6m-eme (JT65a) with 
antenna+preamp+K3+emu0202=>computer 2.3G duo-core 2GB running XP32-SP3
I run WSJT10 on 23cm-eme (JT65c) with ant+preamp+demi 
xvtr+K3+emu0202=>same computer
I run MAP65* on 2m-eme (JT65b) with ant+two preamps+dual Rx demi 
xvtr+dual Rx K3+two SDR+delta44=>same computer

* I can also run 2m-eme with emu0202 with WSJT10 on a single polarity 
but it provides no advantage over the dual-pol adaptive Rx using MAP65.

This not comparing MAP65 to WSJT10 decoders, just that adaptive pol 
Rx has considerable operating advantage over single pol Rx.  I have 
not successfully gotten MAP65 running for 23cm but that may yet 
happen.  Instead, I have a waterfall watching 100-KHz bandwidth using 
my SDR-IQ in parallel with the K3 (lazy man's solution).  The SDR-IQ 
does allow me to track system noise to detect trees in front of the 
dish (also used for sun noise msmts).  SDR-IQ=>Laptop running 
win10pro with separate monitor.  I am now converted e-mail/internet 
to the Laptop so viewing loggers is on a separate monitor from my eme 
sw.  Eventually the XP32 computer will go off-line and use GPS-time 
instead of D4.

Note: none of my operating configurations will provide decoding 
performance evaluation as different hardware are used.

I might be able to run WSJT10 and MAP65 in parallel from the delta44 
data stream.  Haven't been interested enough to try that.

73, Ed - KL7UW

>On 11/4/2016 12:42 AM, Alex Artieda wrote:
> > Dear Friends
> >
> > With great interest I'm following this post and with special attention the
> > observations done by Serge, Les and both Rogers, this topic is not new, for
> > that reason during the las EME Conference in Venice I present a properly
> > setup to compare WSJT vs MAP65 or any other software.
> >
> > I will not re-write what is already printed in the proceeds of the
> > Conference but I will point to some key elements:
> >
> > 1) Again "IQ+ DONT DECODE" is just the receiver presenting the 
> Analog signal
> > properly amplified to your ADC, means your audio card.
> > 2) The IQ+ is designed to lift the noise floor 16dB when a preamp with 26dB
> > gain is in front, the properly test is remove the antenna from the preamp
> > input and place a 50ohm dummy load on the preamp input, if you power on/off
> > the preamp under this condition the noise floor MUST lift 16dB, 
> if not: your
> > preamp is defective or has not enough gain, your attenuation after the
> > preamp is too high, your audio levels in your audio card are not properly
> > adjusted or your internal preamp in the IQ+ is defective. Could be just one
> > or more or all this options together.
> > 3) Splitting your RX line can be done always with a CERO DEGREE 2 way
> > splitter, you must split the signal preserving phase and 
> resulting with same
> > amplitude on both output ports, normally 3dB less. A simple BNC TEE; IS THE
> > 4) The properly setup is if you use Linrad as input and then you send the
> > digital signal to MAP65 and WSJT, then your comparison will be 
> valid because
> > you are presenting the same signal coming from the same receiver.
> > 5) Here are my conclusions presented in the EME Conference in Venice after
> > many years of observations and thousands of QSO in 144Mhz with both systems
> > running in parallel n HB9Q:
> > ___________________
> >
> > I don't' think so, what I believe is both programs behave (in terms of
> > decodes) pretty much the same and if exist a difference is not more than1%.
> > How both software can decode "the same" when WSJT is looking just for 4KHz
> > of bandwidth and MAP65 90 KHz.? I think exist a trade-off compromise about
> > Wide-decoding, sensitivity and confidence to avoid false decodes.
> > Maybe is a matter of interpretation and your goals you establish about
> > performance. What is clear is sometimes WSJT decodes when MAP65 doesn't but
> > I see many times the inverse relation when MAP65 beat WSJT in very low SNR
> > ratios, is very difficult (and non-professional) to just assume WSJT is
> > better than MAP65.
> > I think, and this needs to be confirmed in the next years (when enough data
> > is collected and analyzed properly with the correct Methodology and
> > Metrology), in some specific cases WSJT, under Multi-polarization
> > configuration, decodes when MAP65 not. These specific cases are
> > statistically very small (less than 1% for SNR -27 to -30dB) but if those
> > particular cases are in relation with a new DXCC or rare grid 
> then the value
> > to implement such a complex system is tremendous and corresponds to each
> > individual to evaluate and balance the cost/benefit of this system. Not
> > because the differences are less than 1% we will neglect that differences
> > exist, especially when pushing to the limits our software and 
> Hardware could
> > be translated in just 1 DXCC more in your list, is a matter of individual
> > decision I guess.
> > __________________
> >
> > 73 de Alex, ZS6EME
> >
>Moon-Net posting and subscription instructions are at 
>No virus found in this message.
>Checked by AVG - www.avg.com
>Version: 2015.0.6201 / Virus Database: 4664/13344 - Release Date: 11/04/16

73, Ed - KL7UW
     "Kits made by KL7UW"
Dubus Mag Business e-mail:
     dubususa at gmail.com 

More information about the Moon-net mailing list