Pinned post
Welcome everyone to #HumEval2024 The first session has started and our schedule can be found here. humeval.github.io/2024/programme/
humeval.github.io Workshop Programme Workshop on Human Evaluation of NLP Systems Craig Thomson
@ThomsonSoftware
Research Fellow at Dublin City University.
150 Following 185 Followers 461 Posts
Latest posts 5
humeval.github.io Workshop Programme Workshop on Human Evaluation of NLP Systems Dear @SkyUK, I am a little worried about your staff. This Virtual Assistant seems to be quite possessive. Can you please confirm that your premises have not been taken over by an unruly AI.
Please consider doing our new survey on reproducibility. This is for everyone in #NLProc and #ML. If you completed a similar survey in 2022 then you can still complete this one, we are interested in the difference in your experience and views between then and now.
I recently reviewed some #NLProc papers and several had very small figures. One had text that was < 1mm high when printed and had to be zoomed to 250% on a 25” monitor to read. I print papers to reduce eye strain. Many venues, including ACL state that papers should be printable.
#NLProc researchers, maybe it is not that crowdsource workers are unreliable for human evaluation of system output. Maybe the questions asked are confusing, and their task poorly defined.