Planetary impact flashes detection with DTC software

The International Outer Planets Watch team has been offering since October 2012 two softwares which help testing planetary (jovian) acquisitions a posteriori looking for impacts.
The first one, "JID" is a windows software (with a graphical interface) offering to work on acquisitions while changing detection parameters.
The second one, "DeTeCt", is usable in command mode, which permits to link analysis of several acquisitions in a row without user interaction.
It supports SER, FITS, avi, wmv, bmp, jpg, tif, ... formats. It implements two detection algorithms de détection performed simultaneously:
- the first "classic" one analyses after images alignment every zone in every frame in order to check if a suspect brightness appareance occurs. It then displays a frames' interval when an impact moght have been detected.
- the second one analyses globaly the movie and generate a simple image to easily check visually if there is an impact candidate in this acquisition.

We will see here that second "DeTeCt" software, for several reasons:
    - it allows analysis of all old acquisitions in a row without user interaction (for example I kept about 1000 Jupiter acquisitions since I do planetary imaging - that is impossible to process easily with "JID", which is interactive with a graphical user interface)
    - it implements, additionally to the classic algorithm which generate false positives which have to be studied closely after processing, my own algorithm which sound to me simpler, faster and more efficient: after images alignement, it generated one image representing maximum value for each aligned pixel in the image, minus the average value of each aligned pixel. This allows after a visual inspection for a couple of seconds to know if a temporary flash occured during acquisition. This method also has a better sensitivity permiting to potentialy detect fainter flashes.
    - once the first version out (by Luis Calderon), I pursued its development to correct bugs, improve robustness, add features which sounded interesting and very amateur oriented, and allowing to launch more easily a real impact frequency statistical analysis project with easy participation for any interested planetary amateur astronomer
    - a bonus in this tool is that in its log file, for each acquisition it indicated the middle acquisition time in winjupos format, for easy usage by copy-paste in measure function for example in order to use WinJupos derotation feature

I hence developed a new DeTeCt version, tested successfully on thousands of acquisitions (mine and other amateurs').
If you want to test it on your own acquisitions download latest DeTeCt version.

Please send your results (DeTeCt.log file) to delcroix.marc@free.fr.

Feel free to contact me for any usage question, any error, any modification suggestion, request for supporting a new log version of your favorite acquisition softwares if they are not already supported (after checking your DeTeCt.log - see below)

Results

Processing results are the following:

- in the "Impact_detection" directory, you will find for each acquisition, with an automated generated filename from the video filename "filename_dtc_max.jpg", which is the image consisting of the max minus mean value for each pixel. This is that image which is used to detect visually if there was an impact flash candidate, see "how to analyse the images"

- in the "Impact_detection\details" directory, you will find for each acquisition, with an automated generated filename from the video filename:
1. "filename.log", consisting of the DeTeCt run output. You will find in this file the acquisition full path and name, individual frames information if you used -v option, or text indicating if an impact was found by the classic algorithm.
2. a copy of "filename_dtc_max.jpg" (see above)
3. "filename.jpg", same image but without an histogram stretch (it is darker and less "readable")
4. "filename_mean.jpg", average of all aligned images - which is used to check the "normal" aspect of the acquisition, for example the existence of satellites or dust on the sensor which can lead to false-positive detections.

- in the same directory as DeTeCt.exe file, a DeTeCt.log file which for each processed video will add a line with date, duration, ... information from video file information or from an existing acquisition log file from Lucam Recorder, Genika, Firecapture, PLxCapture, Avi felopaul or Genicap. Please send this DeTeCt.log file to delcroix.marc@free.fr for a work on impact frequenvy.

Let's see now this log file in detail:

Log file with datation, usage in WinJupos

Each time DeTeCt.exe is launched (then also each time detect_batch.bat is launched), a log file called "DeTeCt.log" saved in the same directory as DeTeCt.exe will be appended with a new line per video processed, with the following information:

DeTeCt; jovian impact detection software
PLEASE SEND THIS FILE to Marc Delcroix - delcroix.marc@free.fr - for work on impact frequency (participants will be named if work is published) - NO DETECTION MATTERS!
Rating;    Start;                      End;                        Mid;                       Duration (s); fps (fr/s); File;                        DeTeCt version and comment
0       ; 2014/12/03 11:41,071083 UT; 2014/12/03 11:41,903900 UT; 2014/12/03 11:41,487491 UT;   49.9690 s; 23.000 fr/s; 031214_114104.avi; detect v2.0(Dec.1,2014) (Firecapture 2.3)
0       ; 2014/12/03 11:41,907800 UT; 2014/12/03 11:42,741667 UT; 2014/12/03 11:42,324733 UT;   50.0320 s; 23.485 fr/s; 031214_114154.avi; detect v2.0(Dec.1,2014) (Firecapture 2.3, fps calculated)
0       ; 2014/12/03 11:42,742967 UT; 2014/12/03 11:43,063800 UT; 2014/12/03 11:42,903384 UT;   19.2500 s; 23.000 fr/s; 031214_114244.avi; detect v2.0(Dec.1,2014) (Firecapture 2.3)

For each video, there are column per column:
- Rating:                                "0"; if -ADUdtconly option is not used : Number of candidates detected by the classic algorithm
- Start:                                  acquisition start time, with "LT" if it is in local time, "UT" in universal time or "xx" if unknown
- End:                                    acquisition end time, with "LT" if it is in local time, "UT" in universal time or "xx" if unknown
- Mid:                                    calculated acquisition mid time useable easily with copy-paste in the fields "date" and "UT" in the WinJupos measure function (beware it must be in universal time "UT")
- Duration:                             acquisition duration
- fps:                                 frames per second
- File:                                   full path and filename of the processed video
- DeTeCt version and comment:    used DeTeCt version, and a comment on how were datation and duration information estimated:
        - with file system information ("Date from file info")
        - with information embeded in SER file ("Date from ser file info")
        - with information in acquisition software logfile (for example "Date from Lucam Recorder log"); supported software are Lucam Recorder, Genika, Firecapture, PLxCapture, Avi felopaul, Genicap. For any other log file type support (other software, or a non supported version of supported acquisition software - unfortunately formats change with versions), please send me the acquisition software log file and I will implement it quickly.
        - with date information and possibly the fps, or if duration could only be estimated (not coming directly from a log file) ("Duration estimated")
        - if a date was estimated from another one and acquisition duration ("start date estimated" or "end date estimated")

How to analyze the results : images generated in "Impact_detection" directory

positive results
Interpreting result images by example	filename_dtc_max.jpg (in "Impact_detection") (image with maximum value minus mean value for each aligned pixel)	filename_mean.jpg (in "Impact_detection\details") (average of aligned images)
Acquisition from Anthony Wesley June 3rd 2010 (c) Anthony Wesley (red filter, processing on a 8s extract) Details on planet disappear totaly in the dtc_max image, showing only the impact flash, very bright.
Acquisition from Christopher Go June 3rd 2010 (c) Christopher Go (blue filter, processing on a 2s extract)
Acquisition from Kazuo Aoki August 20th 2010 (c) Kazuo Aoki (processing on a 2.7s extract)
Acquisition from Mazayuki Ichimaru August 20th 2010 (c) Mazayuki Ichimaru (processing on a 3.3s extract) Flash is obvious on detection image dtc_max, whereas hardly visible on mean image, despite it resulting not from the whole video but from a mere 3 seconds extract centered on the flash (on a one minute movie it would be totally invisible). Acquisition was made with a small 6" télescope. This algorithm shows here it's efficiency.
Acquisition from Masayuki Tachikawa August 20th 2010 (c) Masayuki Tachikawa (processing on a 3.2s extract)
Acquisition from George Hall September 10th 2012 (c) George Hall (luminance filter, processing on a 4s extract)
negative results
Acquisition from Marc Delcroix September 8th 2012, IR>685nm filter, 50s duration (c) Marc Delcroix Note how details are faint, nothing pops up in dtc_max image. The limb is just slightly bright (due to turbulence or alignment being not perfect).
Acquisition from Marc Delcroix May 28th 2006 (c) Marc Delcroix No impact detected: after checking carefully the original video, this is a hot pixel which pops up in dtc_max image. The aspect of one single pixel brught was suspect ...
Acquisition from Marc Delcroix May 6th 2006 (c) Marc Delcroix No impact detected: the bright zone on the limb on the right à droite is recognized as a satellite after checking ephemeris (or on other better acquisitions directly on the mean image): the satellite moving over Jupiter during acquisition, its contrast is reinforced compared to the planet on the "dtc_max" image. The same effect can be seen if a piece of dust (dark) is on the sensor and if the planet during acquistion moves a bit: the zone on the planet where the dust was will be very dark on the mean image, but if on a few images that zone was not hidden by the dust it will appear very contrasted - bright on the dtc_max image. A satellite shadow transiting during acquisition will provoke the same effect but will be easily identified as suchye on the "mean" image.
Acquisition from Marc Delcroix July 23rd 2006 (c) Marc Delcroix Ah, good old times of webcams ... we used the "raw" mode at these times to improve the images quality a bit. Problem was, sometimes the webcam would do an ugly "glitch" with many bright pixels paquets. Not a problem when you use an average image, but on a dtc_max image like showed on the right it is difficult to see anything behind this pixellized band...
false positive result
Acquisition from Marc Delcroix July 30th 2011 green filter (c) Marc Delcroix Not to hide you anything, I believed I got one ... a small zone a little bit brighter, rather rounded shape, it lookes as one .... But after checking frame by frame the original ser video (in Genika's SER toolbox), it revealed that a hot pixel appeared during a hundred of frames only, while the planet was turning a bit around the same pixel on the sensor. Result is a bright pixel still on a hundred of frames (the mean image of thousands of frames does not show anything), and on the dtc_max image a nice impact flash aspect ... a flase positive one! The only one in more than one thousand analyzed videos .... Please note that the classic algorithm, as well as JID did not detect anything as their algorithm study each individual frame, contrarly to the more global algorithm.
In any case, if you want to check if there was really a flash detected, use these 3 methods: - inspect visually the vidéo (with Registax, Genika's SER toolbox, Autostakkert, ...) - check carefully if and what the classic algorithm has detected (lines like "Impact detected: frame 89 (0h 00min 03s) to 100 (0h 100min 04s). Max lum 92 at frame 90") - check what the windows graphical software JID detects

Communicate your positive as well as negative results !!!

Please send me your DeTeCt.log files (my e-mail adress delcroix.marc@free.fr is in the header of the file) : even if you do not have detected anything on your acquisitions, this information is interesting to evaluate the impact frequency on Jupiter. I will work on this topic collecting as much information possible from as many amateur astronomers as possible, this work will be presented/realized in collaboration with professionals - all amateurs participants will be duly acknowledeged and cited.

If you are lucky enoough to detect an impact, please inform me. Send your files generated by DeTeCt, and prepare to make your acquisition file available. Prepare also for a temporary celibrity, once announced to all international amateur community, medias, web sites, the pros will contact your to get more info, this is everything I wish for you!

Good luck in your search for impact flashes !!!