@is_eqv
Pinned post
reminder that the bcrypt hash function ignores input above a certain length! so if you do bcrypt(username || password) for some reason, a sufficiently long username will make it accept any password. to fix this you can sha256 the input first.View quoted post on X
Thanks @is_eqv for raising such an important question for the fuzzing research. Open questions and discussion are the keys to the advancement of fuzzing research. @laosong, @andreafioraldi, I fully agree that results from the third-party fuzzbench infra are more reliable and convincing. Thankfully, we do have results from fuzzbench infra that are consistent with our paper. fuzzbench.com/reports/experi…
Your question: Why not use FuzzBench results in the paper? Are there any hidden dirty tricks/hacks/unfair engineering tricks? How can we trust the results?
Our answer: It takes some engineering effort to build all programs automatically under fuzzbench infra. When building the FOX prototype, we have to fix many compilation issues locally. Due to the time constraint before the paper submission deadline, we locally set up the fuzzing campaign to avoid potential building failure (all code and replication results are open-sourced).
We evaluate FOX under fuzzbench infra after the paper submission. The breakdown performance of FOX vs. AFL++ is consistent with our paper results. Please see the attached plot from the above experimental results under fuzzbench infra. Note that there is still some compilation issue on "proj4" under fuzzbench infra, so the result of FOX is "nan". This compilation failure issue also influences the average ranks of FOX due to the "nan" result on "proj4".
Let me know if you have any questions or concerns about FOX. I am happy to discuss further. We found several fatal bugs regarding the MLFuzz evaluation issue and confirmed with MLFuzz's authors. We are now discussing with MLFuzz's authors and would like to collaborate on the errata of MLFuzz and future adversarial collaboration. We plan to share the experience and lessons of this incident with the community later. Stay tuned.
x.com/is_eqv/status/…View quoted post on X
github.com Questions regarding your paper's evalutation · Issue #2 · FOX-Fuzz/FOX Hello, congratulations on your paper -- nice work, interesting idea! I was interested in reproducing your results and was looking into your artifact for that. During this, I noticed a few things th... Honestly, I'm pretty excited about this. Both (a) frontier scheduling and (b) branch distance are ideas I've wanted to see implemented robustly for a while, and (c) using a newton-like mutation approach is really clever!View quoted post on X
