The story you are thinking of is “Transfusion” by Chad Oliver.
• first published in The Magazine of Fantasy & Science Fiction, January 1959
• very widely re-printed – for example in Robert Silverberg’s anthology “Worlds of Maybe” (1968) and in Isaac Asimov & Martin H. Greenberg’s “The Time-Travelers” (1986)

Outline that matches your memories

• The viewpoint character is a twentieth-/twenty-first-century scientist (an anthropologist in Oliver’s story) whose own future world has been blighted by war and mis-used technology. Convinced that the only way to save mankind is to keep it “pure” and ignorant, he volunteers to go back about 12,000 years and live out his life with one of the first bands of Homo sapiens.

• In the Late-Pleistocene world he becomes the tribe’s resident wise-man / shaman. His real purpose is not to help them progress but to hold them back—quietly ridiculing any new idea, mis-directing experiments, even brooding over whether he ought to kill the especially clever ones before they can change things.

• His problem is a young hunter–tinkerer (named K’ta in most reprints): a natural scientist who keeps experimenting with charcoal, odd crystals scraped from cave walls, coloured stones, etc. The time-traveller sees, step by step, that the boy is edging toward chemical discovery—what we would recognise as the makings of gun-powder.

• Climax: a runner arrives in camp to say that K’ta has found “a strange yellow stone that burns with a sour smoke” and wonders what to do with it. The traveller finally understands that curiosity is in humanity’s bones—if he stops this boy, there will simply be another, and another after that. He tells the runner:

“Tell him to grind the yellow stone very fine … then add it to the rest of his mixture.”

(The “yellow stone” is, of course, sulphur—the last ingredient the boy needs.)

• Resolution: having just supplied the final key to gun-powder, the narrator reflects that all he has really done is delay the inevitable. Human beings are, by their nature, discoverers; the advance of knowledge can be postponed, but never prevented.

Every element you listed—the flight to man’s earliest period, the masquerade as a village wise-man, the urge to hold civilisation back, the gifted experimenter he considers killing, and the final fatal help with “grinding the yellow stone”—comes straight from Chad Oliver’s “Transfusion.”