On September 9, 1947, the operators of the Harvard Mark II computer found a moth shorting out relay 70 in Panel F. They taped the insect into the machine's logbook with a handwritten note: "First actual case of bug being found." The logbook is now at the Smithsonian's National Museum of American History.
The joke in the wording is the point of the story. "Bug" had been engineers' slang for a defect in a machine since at least the 1870s, and Thomas Edison used it in his notebooks in 1878 when he was having trouble with his telephone and phonograph designs, writing that such faults had been "bugs" in the apparatus for years. What the Harvard operators were noting was not that a fault existed but that for once the fault was an actual insect, and the pun was apparently deliberate.
The Mark II was one of the large electromechanical computers built for the US Navy at Harvard, and it was the machine's relays that made the moth possible. A relay is a switch with moving contacts. There were thousands of them. Insects got in. The electronic machines that replaced it had no moving contacts and no room for moths. They failed for other reasons.
The people working on the Mark II included Grace Hopper, a Navy reserve officer and mathematician who had worked on the Mark I during the war and who is the name most often attached to the logbook entry. She was not the only one in the room, and she did not coin the word; what she did later was popularize the idea of the compiler, the program that translates human-readable instructions into machine code, and the story of the moth followed her around for the rest of her career in a way she found mildly annoying.
Debugging, in the sense of going through a system looking for the cause of a fault, is still the word. The entry in the logbook is dated September 9, 1947, and the time recorded is 3:45 in the afternoon.