I've been able to track this obscure "quote" to an autobiography entitled The Life and Letters of Charles Darwin, and I've been able to verify that it is indeed a textually and contextually inaccurate quote mine. Here's the full statement for context.
"P.S. In fact, the belief in Natural Selection must at present be grounded entirely on general considerations. [He lists the 3 general considerations.] When we descend to details, we can prove that no one species has changed (i.e. we cannot prove that a single species has changed) nor can we prove that the supposed changes are beneficial, which is the groundwork of the theory."
Darwin is saying (albeit not very well, given that this is an afterthought to a letter as opposed to an edited work) that as of 1863 (only 4 years after publication of Origin of Species), we didn't have the kind of details that would show the evolutionary change of any one specific species. (He had no way of knowing that the future discovery of many fossils and of DNA, and dozens of other discoveries, would provide this detailed evidence in spades.) Instead (as he he describes in the sentence leading up to this) "In fact, the belief in Natural Selection must at present be grounded entirely on general considerations." (Note the words "at present" --this was written in 1863.)
Again, Darwin doesn't assert that no physical evidence exists to corroborate his theory, but rather that he was at the time unable to prove the mechanisms by which specific organisms evolve -- hence his statement concerning "general considerations." If Darwin had believed that no evidence at the time could substantiate his theory, he wouldn't have published and supported it. In essence, Darwin is merely admitting the extent to which science was at the time capable of substantiating his theory to the point of a reliable scientific assertion of proof. Ultimately, no specific mention of a lack of physical evidence appears to exist in any of Darwin's publications or referenced quotes, this one included. Rather, Darwin is admitting that the finer points of his theory were yet to be scientifically substantiated to the point that they could be referenced as factual.