Texas is now poised to join seven other states that passed similar laws this year, part of a national campaign in which Republican legislators introduced such bills in 32 states.
Conservatives say the law, which applies to public school teams through high school, are protecting fair competition.
"We need a statewide level playing field," bill sponsor Representative Valoree Swanson said during the debate.
Equal rights activists have said there is no evidence that trans women and girls are dominating sports.
Ricardo Martinez, chief executive of the LGBTQ rights group Equality Texas, called passage of the bill a "hateful, targeted attack on transgender people."
Political analysts say the campaign is meant to animate hard-core Republican supporters.
"There's no evidence that there's a problem. This is red meat for the base," said Robert Stein, a political science professor at Rice University in Houston.
While the Texas Senate passed a companion bill, three previous House versions of the legislation stalled in the public education committee, which has a Democratic chairman. Republicans then created a new version of the bill and sent it through a select committee they control, enabling it to pass the full House late Thursday.
The bill has gone back to the Senate for procedural approval and is expected to reach Abbott's desk.
Texas Republicans have passed a very conservative agenda this year, including new laws that make it more difficult to vote, all but ban abortion, and do away with the need for a permit to carry a concealed handgun.
"Like a lot of other things in Texas politics right now, this is selling mainly to very ideologically driven voters in the Republican Party. These are the voters that show up for Republican primaries," said James Henson, director of the Texas Politics Project at the University of Texas.
Alabama, Arkansas, Florida, Mississippi, Montana, Tennessee and West Virginia have passed similar transgender sports legislation, and South Dakota's governor signed an executive order. Some of these face legal challenges.
Idaho passed a similar law last year that has been blocked by a federal court, and a federal court in July ruled that an 11-year-old West Virginia trans girl must be allowed to try out for the girls' track and cross-country teams at her school.