15th Feb 2024: Astro Pixel Processor 2.0.0-beta29 released - macOS native File Chooser, macOS CMD-Q fixed, read-only Fits on network fixed and other bug fixes
7th December 2023: added payment option Alipay to purchase Astro Pixel Processor from China, Hong Kong, Macau, Taiwan, Korea, Japan and other countries where Alipay is used.
I supplied files for flats and matching darkflats, light frames and a master dark. So how can it be missing. Wouter did you load the master dark when you had this error can you contact madula
More important is i don't see any errors my end and i still don't get calibrated images.
Wouter seems to think there is a bug when i load all my lights and process them all in one go so asked me to process them one at a time but i can't get it to work.
Could you talk to wouter not sure what going on and try and figure out the bug and also why i can't reproduce the method of doing one colour at a time as i tried L but get same uncalibrated image.
having just spent money out for a owners version all i want to do is process my images and get a good result
Mike
Hi Michael, in the post of Wouter where he got that warning, he did not load the MasterDark 😉 If the MasterDark is loaded, the warning will not show. So this is normal and correct.
In any case, it looks like you have discovered a bug in APP. When I only load the lights, flats and dark flats for L and the master dark and then integrate those I get a nearly perfectly corrected image:
The same for, respectively, Blue, Green and Red:
The final artifacts you probably can eliminate by making sure that the peak of the histogram of the flats is much further to the right. For some reason people think that the peak should be in the middle. And apparently this is what NINA tries to aim for as well. However, when the peak is much further to the right (without saturating any pixels) then a larger dynamic range of the sensor is used leading to much better correction.
I suspect that the bug is related to the fact that you have dark flats with different exposure times and it look like those do not get applied correctly to the flats. But I may be wrong and this is for Mabula to investigate. Thanks very much for your patience and for answering all of our questions!
On my computer monitor I still see the ringing in the L-data, the images posted are also weakly stretched so it is hard to see, but I see it, so I expect the results are the same and not good. Like I mentioned, If I process the L data only, i get the same result as processing L,R,G,B combined, so I don't see a bug and have no clue how you could get a different result.
Mabula
This post was modified 3 years ago by Mabula-Admin
Hi I made my masterdark with APP back when i took them, my master dark i have upload for Wouter with my files it looks to be 50mb in size so may be a 16 bit file as the final integrated images are 32 bit and 125mb in size. I can try and make a new master dark.
I don't think from the test i have done and i have used different flat method there is a ligh leak.
Hi I made my masterdark with APP back when i took them, my master dark i have upload for Wouter with my files it looks to be 50mb in size so may be a 16 bit file as the final integrated images are 32 bit and 125mb in size. I can try and make a new master dark.
I don't think from the test i have done and i have used different flat method there is a ligh leak.
Okay, let us first rule out that the masterdark is in fact the issue. So please make just a couple of darks (8 would be enough) with the same temperature, exposure time, and sensor gain + offset. Then create a new MasterDark of 32bits 😉 and reprocess with this new MasterDark 😉
Chances are good that this is the issue. I have seen from another APP user that this caused the ringing issue on these new camera's. 16bits masters are not good enough it seems for these sensor's to get optimal calibration.
Ok made a new masterdark i have uploaded to the folder if you want to try i will run it too. By luck i keep all my camera images so it was just a case of re processing them to 32 bit and not having to retake any
Indeed, I see no difference with the 32bits MasterDark. 16bits processing will never make it better for sure.
This leaves me to conclude that the issue is in fact in the flats themselves when compared to the light frames... somehow the illumination profile in the flats don't match with lights then and this can have many causes. I will study the frames in a bit more detail if I can see what is going on...
Okay, I have now studied the flats in more detail to find out of the illumination profiles are logical between the different filters. If they are logical, you expect that if you, for instance divide the Red MasterFlat by the Luminance MasterFlat, that you get a flat surface showing only the dust spots difference between the 2 filters. If they are not logical, it means that there is a problem in the flats themselves.
The illumination profiles between the L, R,G,B filter data should almost be identical leading to flat fields when you load a certain masterflat as light and apply another masterflat as masterflat in calibration.
Now if there is a problem in
how the flats are created or
light leakage in the optical train
this test should show it normally.
This is what I get when I load the Red masterflat as a light and use the luminance masterflat as a masterflat. First the masterflat is shown, then the division by the other MasterFlat. Please note, the multi-channel and multi-session functionality is disabled to perform this test. auto-detect Masters and Integration in 1)Load is also disabled.
This is what I get when I load the Red masterflat as a light and use the luminance masterflat as a masterflat.First the masterflat is shown, then the division by the other MasterFlat.
This is what I get when I load the Blue masterflat as a light and use the luminance masterflat as a masterflat.First the masterflat is shown, then the division by the other MasterFlat.
You can clearly see that there is no consistency between the divisions. The R/L results is opposite to B/L which is an idication of a tilting filterwheel perhaps. See depening on how the telescope is aimed, the filters are never orthogonal to the direction of incoming light and always off with a certain angle.
Final 3 checks:
R divided by R, should give perfect flat result which it does:
G divided by R:
B divided by R:
We clearly see not-logical illumination profiles between the masterflats, which for me is hard evidence that the issues are caused by
a problem in how the flats are created
or a problem with the optical train like a light leakage or a tilting filterwheel.
I would not be surprised now if the issue is in fact the filterwheel. Can you check internally if the filterwheel will wobble depending on how the telescope is aimed?
Mabula
This post was modified 3 years ago 2 times by Mabula-Admin
If you can it will help i have tried both my light box flats and also sky flats both with same results. I tried Wouter checks for any IR light and used a 4 watt IR torch and can find no light leak for IR and also my dark flats even the 40 second exposure of narrow band filters taken during the day show no hint of a light leak so am at a loss of what the issue could be.
Strange then, the optical connection from camera are fix to the filter wheel the camera directly bolts to the filter wheel via a tilt adaptor so no flex there. The camera has a tilt adaptor before the filter wheel but it is adjusted flat but need to be there to get the correct focal distance for my OAG fitted after filterwheel. From OAG its to reducer then it goes in tot a moonlite focusser.
Like Wouter indicated, please also check the actual wheel in the filterwheel 😉 that wheel that is rotating in the housing could not be that secure or robust 😉 I suffered from this myself with a bad quality filterwheel from StarLight Express (many years ago, but it was badly constructed). So whenever it rotated a bit, the plane in which the filters reside, always had a different angle to the optical axis... I never could get good flats with the filterwheel as a consquence.
If there indeed is a light leakage, please keep us updated 😉
I am 3D printing some plates to make a felt bellows it seems at some angles light is getting up the moonlite barrel has to be moving light block so it can still focus.
I will do this first before i go down the filter wheel check as i would need to take the camera in to work as we have a massive optical clean room how lucky i am working for Zeiss i where i can take it apart with out a worry of any dust getting in as it currently clean and sealed by the reducer.
I am 3D printing some plates to make a felt bellows it seems at some angles light is getting up the moonlite barrel has to be moving light block so it can still focus.
I will do this first before i go down the filter wheel check as i would need to take the camera in to work as we have a massive optical clean room how lucky i am working for Zeiss i where i can take it apart with out a worry of any dust getting in as it currently clean and sealed by the reducer.
Mike
Okay Michael @moviecells, please keep us updated 😉
We know now that for the new cmos camera's 16bits masters can be insufficient leading to problems in data calibration. So with your camera model, yes, I think it is wise to recreate the masterdarks in 32bits.
Well spent most of the weekend making some parts to get rid of any light leak and taking some more flats.
First results don't look good, Wouter suggested to increase the ADU range of flat so i set them high just below clip around a mean of 60000 but i am not sure if this is the issue so i am taking new ones now at about 50% so around 32000.
@moviecells A mean of 60000 is too high since this will lead to saturated pixels. I usually try to aim at 55000 and that works much better for me than aiming at 32000. My setup introduces considerable vignetting since I am using full frame sensor and a higher ADU will compensate better in those regions.
Also, most forums suggest this because those values are copied from the days when modern low-noise CMOS sensors weren't used as much as today. It is very difficult to get rid of vintage knowledge. Anyway, if lower ADU values work for you then by all means go for them.
I am trying to follow what you did by taking one master flat away from another to see if my new flats are better. I have solved all the light leaks and also took the filter wheel apart and it all seem to rotate ok no flex i can see side to side. I thought it would be a good idea to repeat your test but i am finding it hard to follow if i upload some files could you check is it just flats that are needed ?
@moviecells I have shot darks and dark flats for my cameras at the same temperature, gain and offset that I always shoot with. I am using an ASI6200MM and an ASI2600MC. The darks I shot at 60 seconds, the dark flats at 2 seconds. That way, I need a master bias to scale the master dark and/or master dark flat up or down. For instance, I took SII images at 120 sec the other day so when I wanted to use the 60 sec master dark, a master bias was needed.
If you use the same exposure time for the darks as for that lights and the same exposure time for the dark flats as for the flats, then you do not need a master bias. But since I do not use the same exposure times, I do need them. This way, I only need to take darks and dark flats once a year instead of with every imaging session.
Did you do a new integration with the new flats? Do you still see the rings?
So it strange with flats at around 55000 i get this result i can see those strange rings again and correction does not go right to corner of frames this is red below
Now with flats around 32000 i get no strange rings and this
Not sure what going on.
Going to close APP and try again
One thing does not look right but not sure is the histogram of one of the flat images below not sure what the peak is just before 36408 up to that point is look like normal light loss to the edges.
This is what it looks like without stretch very close to one of my light frames some drop off of light in the corners
This is the red light frame
Mike
This post was modified 3 years ago 4 times by Michael Purver
Well from all my work at the weekend with the light leak fix the final results look worse than before. hitting my head on the wall as i write this.
First thing that i need to confirm is if my flats now look better using @mabula-admin tests taking one from another. there was a light leak there on those flats and thats now fix 100%.
I will uplaod to my folder in folder called new flats my new flats so far could @mabula-admin check them i tried to do his test but got lost on all the settings that need changing.
I am repeating my flats again tonight with a tee shirt over top between the light box to increase my LRGB exposures in to the 5-10 second range just to make sure my light box is not to bright and causing a reflection somewhere.
Now with flats around 32000 i get no strange rings and this
Yes there are rings and the corners are not corrected at all. The pic resulting from the 55000 flat has better corrected corners and also rings. Both flats still are not correct yet. I am really not sure what's going on, sorry.
Hi yes not sure what to try next as there is no light leak and filter wheel seems fine, may be mabula can help can you ask him if he can take a look uploaded some new flats.
I have analysed the new flats again by creating masterflats per filter where the old darkflats were used.
I simply created 1 MasterDarkFlat of the old 20 darkflats. That should be perfectly okay. They have the same gain (100) and offset (50) as your new flats.
So the MasterDarkFlat is used to subtract the sensor offset from the flats to create proper MasterFlats.
Then I calibrate the Red MasterFlat (loaded as a light) with the Luminance MasterFlat (loaded as MasterFlat):
That looks rather flat so quite okay actually.
I calibrate the Green MasterFlat (loaded as a light) with the Luminance MasterFlat (loaded as MasterFlat):
That clearly does not look okay, we see tilt again and we even see a rinigng/vignetting profile here.. not good. So that means that the illumination profile between the green flats and luminance flats is not consistent at all, meaning a problem with either the optical train or in the actual method of creating the flats.
Next, I calibrate the Blue MasterFlat (loaded as a light) with the Luminance MasterFlat (loaded as MasterFlat):
That also shows a tilted ringing/vignetting profile thus not okay.
Mike, so you have already spent considerable time in checking problems in the optical train, perhaps you should re-evaluate the way in which you actually shoot the flats? What exactly do you do to shoot the flats? Are you using a flat-panel? Are you creating the flats in daylight or at night?
1. a light panel made of a drawing pad, over the top is a ND filter and then a acrylic opaque sheet. I have also tried a white tee shirt between too to lower light even more. I get 2-6 second LRGB exposures. Tried both day and night last set you have is from this methos taken during the day as i know the light leak is now fixed
2. a electro luminescence panel have tried this but will try again problem is with this panel it give very little light for HA,O,S narrow band and i send up with 90-120 second exposure with those filter hence why i got the other panel above.
3. Sky flats using two tee shirts late in day pointing to sky about 60 deg point north clear sky.
both 1 and 3 i tried in the last week got very must the same results.
I will try the electro luminescence panel again just in LRGB would you be able to check those ? i will try and take some now.
I did take the same files and put them through DeepSkyStacker and got not bad results see very stretched image below it very soft so stacking not a good but looks a lot more flat.
