“A core selling point of machine learning is discovery without understanding, which is why errors are particularly common in machine-learning-based science.” I could not resist the temptation to start this article with a quote by AI as Normal Technology – it captures the current state of AI-everything perfectly. Should investigators really trust black boxes running a set of non-deterministic algorithms and providing different results on every reroll? And can we still use such black boxes to automate routine operations? Let’s try to find out.