A good starting point would be OPAR ch. 1, which says “Science is systematic knowledge gained by the use of reason based on observation.” Science thus includes “specialized science” and philosophy. It differs from mere observation, which is not systematic. It differs from religion and emotion, which are not based on reason or observation. Philosophy (actual philosophy, not purported philosophy) is a science: again, OPAR ch. 1 “philosophy is a system of ideas. By its nature as an integrating science…”, Peikoff in “The analytic-synthetic dichotomy”: “Epistemology, the theory of knowledge, the science that defines the rules by which man is to acquire knowledge of facts…”. Rand says (“Philosophy: who needs it?”) that “Philosophy studies the fundamental nature of existence, of man, and of man’s relationship to existence. As against the special sciences, which deal only with particular aspects, philosophy deals with those aspects of the universe which pertain to everything that exists. In the realm of cognition, the special sciences are the trees, but philosophy is the soil which makes the forest possible”.
In the broader context, “science” refers to systematic knowledge gained by the use of reason based on observation, but in the narrower context where philosophy is distinguished, we would contrast philosophy and special sciences. In the appendix to ITOE, “Philosophic vs. Scientific issues”, Rand begins by noting “Philosophy by its nature has to be based only on that which is available to the knowledge of any man with a normal mental equipment. Philosophy is not dependent on the discoveries of science; the reverse is true”. Philosophy is not “the art of just making crap up”. In this context (which presupposes the distinction between science and philosophy), the simple term “science” is used where elsewhere “special science” might be used. This second sense of “science” as special science, specialized knowledge, is what is ordinarily called “science” especially by people who haven’t read OPAR and ITOE. Philosophy is science, in the broader sense, but not in the narrower sense.
“Evidence” is not, as far as I know, defined in Objectivism, but observation of how the word is used shows that it refers to knowledge in relation to a proposition – a fact supports a proposition, or it contradicts a proposition. A bit of knowledge can depend heavily on an immediate observation – “I just saw an eagle!” – or it can depend heavily on applying knowledge to previously gained knowledge (insert your favorite mathematical proof here). When people speak of “empirical evidence”, they mean knowledge that depends heavily on immediate observation. “Empirical evidence” brings us back to the axiomatic, because the distance from the axiomatic to the conclusion is shortened. All knowledge rests on observation, but some knowledge is separated by quite a distance from observation. It is true that some people treat philosophy as non-empirical, which allows patent nonsense to be promulgated as “philosophy”.
You have to consider the concept “evidence” from two perspectives as well, depending on whether it has been evaluated. People often look at the observation as being the “evidence”, in which case since you can’t deny the axiomatic, you end up with a very goofy notion of “balancing” evidence, and seeing truth as scalar. Which, b.t.w., is poppycock. This notion that evidence is the raw observation is wrong. An observation has to be logically evaluated and integrated with all of your knowledge, before it becomes “evidence” for or against anything. “Uncontrolled observations” then are not evidence, because there has been no validation of the relation between the observation and the proposition that the observation stands in a supposed evidentiary relation to. How does that observation integrate with other observations (all other observations, not just the ones of interest to the advocate of the position)?
The specific form of stupidity that you’ve identified is failing to consider alternative. There are alternative propositions that are consistent with the observation, and those alternatives are arbitrarily rejected. That means that the resulting emotion of “certainty” is achieved at the expense of acquiring knowledge.