A loophole

[Yida-Lin]

@cbrnr Just noticed a loophole when debugging Stefan's file:

Before I thought string streams are always irregular streams, but turns out Stefan's file has a stream that is 5000Hz but is string only.

Could also be why that file doesn't load.

I am debugging more into this.

Sorry @stfnrpplngr it might take a while.

[Yida-Lin]

@cbrnr @stfnrpplngr I actually successfully opened this file, though it took quite long loading the file because it has 170,000 events...

I have pushed new commits into SigViewer and Libxdf fixing the loopholes. (in a Pull Requested called fixEventTS, which hasn't been merged into master yet). Now the smaller file that Stefan sent earlier can be successfully opened. The 1.09GB one still can't due to allocation failure (the shortcoming of a 32bit build).

@cbrnr Clemens do you have anything to comment here?

[cbrnr]

Is this fix in #71?

[Yida-Lin]

@cbrnr Hi Clemens, yes, together with the new commit in libxdf

[cbrnr]

I still think that converting the original event stream from numeric to string isn't a great idea. However, maybe the easiest solution for us is to treat regular string streams with type sampledMarkers (or whatever the exact name is) as a special case. Specifically, we should remove all empty samples from such a stream and convert only non-empty strings to events.

[Yida-Lin]

@cbrnr Hmm I am not sure I exactly understand. Are you saying if the string is empty we should ignore it?

[cbrnr]

Yes, if the stream type is sampledMarker.

[Yida-Lin]

@cbrnr wait, isn't the stream name or type arbitrarily entered in any way the user wants?

Also, if we filter those empty strings out, we won't see those events on the signals and it becomes an empty stream. Is that OK?

[cbrnr]

Yes, you can choose the type but there are standards for meta data. I'm not sure if this type is in there yet.

This stream should contain some non-empty strings, otherwise what is the point of this signal? So if you discard all empty strings you should be left with the useful ones.

[Yida-Lin]

@cbrnr seems the sampledMarker meta standard isn't there yet.

I see. I mean at least in Stefan's test file, it contains only empty strings, and if filtered out, there are no events left at all.

I can go ahead and make the changes, but might need another test file to better test the results.

[cbrnr]

@stfnrpplngr is this intended? Shouldn't there be at least some non-empty strings in the marker stream?

[cbrnr]

@stfnrpplngr @Yida-Lin any progress on this one?