As I ponder'd in silence,
Returning upon my poems, considering, lingering long,
A Phantom arose before me with distrustful aspect,
Terrible in beauty, age, and power,
The genius of poets of old lands,
As to me directing like flame its eyes,
With finger pointing to many immortal songs,
And menacing voice, What singest thou? it said,
The Ghost of Poets' Past 
 
 It seems that he was deeply within' his own mind or was hallucinating. An ancient ghost has appeared and he claims it’s the  genius  of past poets. This apparition is the fusion of the many writers of Whitman’s time.  “Terrible”  in this sense means intense in appearance and reputation. Assuming that the ghost is an amalgamation of them all, it is most likely Whitman is expressing intimidation by giving them the form of a  However, assuming that the ghost is an amalgamation of them all. Not so much the poets but rather their poetry in their  “beauty, age, and power” . 
 The first line lets you know that he felt like his senses were betraying him at first.
As I Ponderd in Silence