Example bump #32

gerbaudo · 2013-10-07T09:18:13Z

This is still a work in progress, but I need some input, so I am using a pull request to discuss the code.
I do not understand how to use the new setup: I thought that the 'model' and template were not needed anymore, but it complains with an AssertionError: A template is required.
Do you have a working example?

Also, I have two comments/suggestions for pyFBU:

we should specify the input data, not the json file (the input can be either from json or from another source, we don't care)
one of the input parameters is the bkg. This is not always well defined (the bump case is one example where we don't need to differentiate between bkg spectrum and signal spectrum, b/c we're unfolding one spectrum). I think we should have by default a null background, i.e. nreco*[0.] and ntruth*[0.]
Please let me know what you think.

Thanks

Davide

Details: - factorize the code in well-defined functions - add docstrings explaining what I am doing - rewrite most of this script. Now plotting and using the resulting histograms as bins/counts that can be used for the unfolding. - compute normalized response matrix - add plotting functions

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

francescorubbo · 2013-10-07T12:27:29Z

I think the problem is that your branch is not in sync with the master. Could you try to fast-forward it to get that latest developments?

A working example is tests/priors/runPriors.py

Details: - factorize the code in well-defined functions - add docstrings explaining what I am doing - rewrite most of this script. Now plotting and using the resulting histograms as bins/counts that can be used for the unfolding. - compute normalized response matrix - add plotting functions

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

Details: - factorize the code in well-defined functions - add docstrings explaining what I am doing - rewrite most of this script. Now plotting and using the resulting histograms as bins/counts that can be used for the unfolding. - compute normalized response matrix - add plotting functions

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

Details: the vertical axis of the histo provided by histogram2d is reversed (see http://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram2d.html)

Conflicts: python/generate_bump.py python/utils.py

gerbaudo · 2013-10-10T04:45:53Z

Hi,
Thanks for the suggestion. After pulling in the changes from the master, and some fixes see #34 , now it works, almost. I see the warning below

$ python/generate_bump.py
Warning: Stochastic truth's value is neither numerical nor array with floating-point dtype. Recommend fitting method fmin (default).

Have you seen it before? it also seems to be quite slow...can you please try to run my example and tell me whether you see anything obviously wrong?

Thanks!

francescorubbo · 2013-10-10T14:55:15Z

Ciao,

I'm not surprised it takes so long, you are doing a 50-dimensional sampling! You might want to try a coarser binning.
About the Warning, I understand from Clement that this is coming from using MAP.

Unrelated, would it be possible grouping all the test and example files in the test directory (or in a "example" directory). I would prefer to have only FBU code in the main directory (whatever we will call it).

clementhelsens · 2013-10-10T16:39:54Z

Hi Davide,

I see you want to merge this branch with the master, but now your "cleanup" branch has been merged to master, I'm afraid you will have some conflicts to solve first

gerbaudo · 2013-10-13T11:44:05Z

Hi,

I made some progress (now it runs), but I am getting nonsense.
Can you please try to run this example and tell me if you see anything obviously wrong?
It's probably some stupid but on my side...for the response matrix are we using [t][r] or [r][t]?
Thanks.

…xample-bump

clementhelsens · 2013-10-22T07:15:12Z

Hi Davide,

sorry for the late answer.
For the response matrix we use [r][t]
I'll try your example now and let you know.

Details: when computing p(r|t), there can be bins without any reconstructed events. Protect against divide by zero.

gerbaudo added 5 commits September 24, 2013 23:23

initial draft (spectrum similar to fbu preprint)

d85bff0

utils to write/read np.array from/to json

49722d7

separate the generation from the unfolding

72d42d2

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

almost there, but pyfbu does not work

e59f632

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

gerbaudo added 18 commits October 9, 2013 19:19

initial draft (spectrum similar to fbu preprint)

99ecb5c

utils to write/read np.array from/to json

c2a1f51

separate the generation from the unfolding

02fa5f4

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

almost there, but pyfbu does not work

ba2a605

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

Merge branch 'example-bump' of github.com:gerbaudo/fbu into example-bump

2d0d871

initial draft (spectrum similar to fbu preprint)

153f490

utils to write/read np.array from/to json

168ea4e

separate the generation from the unfolding

3873eed

Details: two separate logic steps: generate the pseudo-data, and then unfold. The pseudo-data can come either from the json file, or be generated.

almost there, but pyfbu does not work

5a5dfab

Details: complaining about template. I thought it wasn't needed anymore. Need to ask clement and Francesco about it.

flip up-down the response matrix

580ed11

Details: the vertical axis of the histo provided by histogram2d is reversed (see http://docs.scipy.org/doc/numpy/reference/generated/numpy.histogram2d.html)

delete assignement of obsolete parameters

001ecff

reduce the number of samplings to test things out faster

9710442

typo

4106129

regenerate when json is not there

69b0b6b

try to adjust lower/upper

ab547b2

Merge branch 'example-bump' of github.com:gerbaudo/fbu into example-bump

aac8525

Conflicts: python/generate_bump.py python/utils.py

still work in progress, getting there

dbe9c31

Merge branch 'example-bump' of https://github.com/gerbaudo/fbu into e…

3bbce37

…xample-bump

gerbaudo added 2 commits November 18, 2013 10:41

protect against divide-by-zero

0587d84

Details: when computing p(r|t), there can be bins without any reconstructed events. Protect against divide by zero.

Merge branch 'example-bump' of github.com:gerbaudo/fbu into example-bump

9434394

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Example bump #32

Example bump #32

gerbaudo commented Oct 7, 2013

francescorubbo commented Oct 7, 2013

gerbaudo commented Oct 10, 2013

francescorubbo commented Oct 10, 2013

clementhelsens commented Oct 10, 2013

gerbaudo commented Oct 13, 2013

clementhelsens commented Oct 22, 2013

Example bump #32

Are you sure you want to change the base?

Example bump #32

Conversation

gerbaudo commented Oct 7, 2013

francescorubbo commented Oct 7, 2013

gerbaudo commented Oct 10, 2013

francescorubbo commented Oct 10, 2013

clementhelsens commented Oct 10, 2013

gerbaudo commented Oct 13, 2013

clementhelsens commented Oct 22, 2013