Horowitz, who has written a new James Bond book, made his comments about Elba – best known for his roles in Luther and The Wire – to the Mail on Sunday.
He told the paper it was “not a colour issue” – but some fans accused him of making a veiled racial remark.
Horowitz has now said the word “street” was “a poor choice of word” and he was “mortified to have caused offence”.
Elba has long been touted as a possible successor to Daniel Craig as the fictional secret service agent.
Horowitz is currently promoting Trigger Mortis, a novel featuring Bond, which has been authorised by the estate of original 007 author Ian Fleming.
In the original interview, the author said he thought Elba was not right for the part.
“Idris Elba is a terrific actor, but I can think of other black actors who would do it better,” he said, suggesting Hustle’s Adrian Lester as a possible alternative.
“For me, Idris Elba is a bit too rough to play the part. It’s not a colour issue. I think he is probably a bit too ‘street’ for Bond. Is it a question of being suave? Yeah.”
Despite his insistence that his opinion had nothing to do with skin colour, some Twitter users accused him of racism.
“I’m really sorry my comments about Idris Elba have caused offence,” Horowitz responded in a statement.
“That wasn’t my intention. I was asked in my interview if Idris Elba would make a good James Bond. In the article I expressed the opinion that to my mind Adrian Lester would be a better choice but I’m a writer not a casting director so what do I know?
“Clumsily, I chose the word ‘street’ as Elba’s gritty portrayal of DCI John Luther was in my mind but I admit it was a poor choice of word. I am mortified to have caused offence.”