2023-09-16: APP 2.0.0-beta23 has been released !
Improved performance again, CMD-A now works in macOS File Chooser, big improvement for bad column cosmetic correction, solved several bugs
We are very close now to releasing APP 2.0.0 stable with a complete printable manual...
Analyze stars creating extra Master files
I am doing a 3 session integration: I can get through Calibration fine. After the Calibration step I end up with a SINGLE MF and MDF for each of the three sessions and a single MD. Before I start the analyze stars I double check that I don't have extra master frames. About 90% of the way through Analyze stars I get this warning, which I have had in the past. When I get this warning I will then have an extra MF and MDF for each session. See first Image below. No exaggeration, I have closed APP and then reloaded all of my frames probably close to a dozen times. I assumed I was loading my frames incorrectly, which I have done before.
So what I did when I get this warning was to click okay for all 450 subframes; then at 100% it started all over and goes through the whole process fine.
P.S. If I load just 5 lights for each session and 5 calibration frames for each session it doesn't do this. Also, One thing I have started to do differently because of issues I have had in the past with APP. All of my flats and DF are now set at 3.5sec. Before I had a range that they could fall in. The flats and darkfats were always the same exposure time for that particular session but the next session it may be a different exposure time.
Just to confirm, but are you adding the flats to their own session as well?
edit: Ah, I see in the MF overview. Usually this would happen if there are differences in some of the flat frames, where APP then thinks it's a different flat integration. Are you sure all flat frames are exactly the same? Perhaps try with a small subset of like 5 flats per session just to see what happens..
Thanks for the reply.
Why does the Analyze Stars step create new master files? That seems so odd.
I was able to get it to work this morning and it makes no sense why this worked; here's what I did. I exited out of APP, I deleted the extra master files that were created in the analyze stars step; I went back into APP; instead of loading the individual subs I loaded the master files (up to this point I would delete all masters and start from scratch to try and get it to work); I skipped the calibration step and went right to analyze stars (Can I do this if I already have master frames?). When I did this I did not get any errors. It's all the exact same data, the only difference is I closed out APP and only loaded the master files.
All of my flats and DF are 3.5 seconds, 121 gain, offset 30. I make my flats using Flat Wizard in NINA which is the way I always do, so even if I loaded a flat frame with the wrong DF session it should still work; however, I know I did not do this. I spent several hours last night trying to get this to work. After the second time I made a checklist to ensure I did not incorrectly load files. I have checked and rechecked every time that the data was loaded correctly.
I did do a smaller subset and it was fine; that's what is especially frustrating (I'll include a screen shot of that data to show the flats and DF are the same.). What is even crazier is, just last week, I finished an 8 session on the heart nebula; and with that many sessions, 800 calibration frames, 25 hours of data, I had zero issues. The only thing I did since last week was redo my Dark library because it was a few months old.
Thanks for the help, much appreciated.
One thing I forgot to mention was a couple of times I tried unchecking the Auto Detection of masters in the "Load" step. I thought maybe that was causing an issue; however, I had the problem whether it was checked or not.
That it would be added during the analyze stars step seems odd yes. Let me ask Mabula if that would make sense. My feeling is it might still be a workflow issue you're experiencing somehow (it happens).
Having just the masters, but skipping the calibration step seems like you're then not calibrating your lights right? What happens if you only use those masters and still calibrate the lights with them?
I do seem to see a $ sign in some of your files, any idea what that would be? The latest APP version should have no issue with those, are you using the latest?
Good catch on the $ sign. In Nina I changed the naming scheme, when I did that a $ symbol was left behind. I caught it and changed it after one of my sessions. I saw the update had to do with naming issues, so, yes I did download the newest update after the first attempt I tried last night failed.
I will try the calibration step with the master files and see what happens. Maybe I'll go through those files and remove the dollar sign as well.
I can absolutely verify that, after calibration, there was only one set of Master Files and the second set was created at some point during analyze stars. I think I took a screen shot at one point so I had proof.
As far as it being a workflow issue, I'm just not sure about that. For the projects I do get to work, I am doing nothing different than I do with this project. To have an error at Analyze Stars, there's only so many things I can mess up by the time I get to that step. I keep every setting stock/automatic; no drizzle.
edit: I'm running a test now with the same master files as before; however, this time I did use the calibration step. I'll update how it goes.
Yes, of course I don't know if that would be the case, it's just something that might happen with a lot of data and something we always have to keep in mind. I forwarded this to Mabula, so when he has time he'll definitely also have a look. Thanks for sharing!
@vincent-mod Thanks again, much appreciated. Mabula is going to be happy to see my name again... 😀
So far it seems fine. I'm halfway through Analyze stars and no issues. I'll update if I can get through the entire process with out issues.
It worked and these are the steps I had to take; posting this more for myself so I have record of it.
1. Load all light subframes and calibration subframes.
2. Calibrate-one set of master calibration files is created.
3. Analyze stars - extra masters are created in this step and they are created within a few seconds of starting this step then the error described above is generated.
4. Exit APP
5. Delete master files created during Analyze stars (they are easy to identify as they are labled with _1 at the end of the file name.
6. Launch APP and load all Lights and load only the master files created during the initial calibration step.
7. Run Calibration.
8. Run remaining steps without issue.
During step 3, APP will very quickly get to 90% (within seconds) and then I get the error of having to select the correct MasterFlat. If I click okay for all of the subframes, it will then rapidly get to 100% completion (maybe 1 second) then go back to 0% and slowly (normal amount of time for analyze stars) go through the process fine.
During step 1, if I only load the first 5 subs for lights and calibration, I had no problems. When I look at the calibration files, they all are 3.5 seconds and correct gain.
Very interesting still, glad it worked for you but it shouldn't be like that. Let's see what Mabula has to say. 🙂