added a workaround for issue 4604

[guj]

Guard against BTD case:

• 0 particles are written at the last flush step (due to fraction = 0)
• offset > 0 (particles do exist although not selected)

See issue #4604

[ax3l]

I am not sure this should go exactly like this here.

What do you think about this?

Suggested change

	if (is_last_flush_to_step) {
	SetupPos(currSpecies, NewParticleVectorSize, isBTD); // workaround for issue 4604
	SetConstParticleRecordsEDPIC(currSpecies, NewParticleVectorSize, charge, mass);
	}
	if (is_last_flush_and_never_particles) {
	SetupPos(currSpecies, NewParticleVectorSize, isBTD);
	}
	if (is_last_flush_to_step) {
	SetConstParticleRecordsEDPIC(currSpecies, NewParticleVectorSize, charge, mass);
	}

[ax3l]

Admittedly, this is exactly what is done a few lines earlier, isn't it? Why is that not called, I wonder.

[ax3l]

What are the values of num_dump_particles and ParticleFlushOffset for this problem?

[guj]

num_dump_particles=0
ParticleFlushOffset=1000

so is_last_flush_and_never_particles (= is_last_flush_to_step && num_dump_particles == 0 && ParticleFlushOffset == 0) will be false.

[ax3l]

Thanks, got it.

If ParticleFlushOffset > 0 that means at some point, at least one rank had particles to write.

We need to check if maybe in the call where we write particles (earlier), we have a logic mistake that forgets to call SetupPos collectively.

[ax3l]

Hm, maybe we filter twice and forget num_fraction in one case? Let's take a look where num_fraction is applied and if the counter is done correctly afterwards.

[guj]

My understanding is that the offsets are set in the BTD object, and when it calls flush(), the filters are then applied in the underlying flush formats during copyParticles(). Unlike other filters, the random filter affected the offsets of the caller BTD object.
I think the workaround is the easy and harmless. Otherwise, have to collect the particles written from all processes and update the offset in BTD.

[ax3l]

I think the workaround is the easy and harmless.

I would rather like to fix the underlying issue in the BTD offset counter. We need consistent meta data.

cc @RevathiJambunathan :)

[guj]

If it makes sense, can apply the random filter before BTD calls flush(). It will make things cleaning and easier.

[ax3l]

Yes, we probably need to add this to:

• full diags
• BTD
• scraping

[guj]

Not sure why it fails during "copying files to run directory..."

[ax3l]

Fixing the broken meta-data in #4657 :)