Honestly the inclusion of Yoda in the prequels undermine his later appearance in The Empire Strikes Back.
They could've just left him alone, referred to him in passing, said he was on a vision quest or a mission. And this is all to say nothing of how he was actually handled - a martial artist entirely in tune with the universe who has had hundreds of years to hone his craft, and what we get is a cricket on meth with a glow stick.
Well, everything in the prequels undermines the originals so it just kind of goes with the territory, but you're absolutely correct.
In general, the prequels fetishized light saber choreography in the apparent belief that light sabers were what made Star Wars great. So more light sabers = better, right? And more light saber geometry = even better, right? But yes, Yoda was a mystic in the originals and not a warrior. In fact, doesn't he even make fun of warriors?
Here's a fun story for contrast, that I think is correct but not sure. In Crouching Tiger, Hidden Dragon, the lead role of Bai was written for Jet Li. The character was supposed to have lots of intense fighting scenes -- you know, like the hero in every kung fu movie. But Jet Li pulled out, and they replaced him with Chow Yun Fat for the star power. Except Chow Yun Fat doesn't do kung fu. That's why we never see Bai do much fighting. When he does fight, he swings his sword three times and then just wins.
And that accidental development is actually what makes the film work. Bai is the undisputed master. He's above fighting. Nobody can even hang with him, because he can win the fight in a few seconds. It creates the contrast with Zhang Yiyi, whose character is extremely powerful but obviously undisciplined compared to Bai. Eventually she realizes that she needs more understanding of the spiritual side of kung fu and not just the mechanical, which is what is depicted, I think, in the final scene (I'm of the opinion that she doesn't die).
In any event, I think the film is stronger because Chow Yun Fat couldn't fight. Lucas did not learn that lesson.