In United States v. O'Brien, why does the Court express the view that they don't think the First Amendment applies to all cases, but proceeds to analyze the case on the assumption that O'Brien's conduct of burning a draft card is speech?

From text:

"We cannot accept the view that an apparently limitless variety of conduct can be labeled "speech" whenever the person engaging in the conduct intends thereby to express an idea. However, even on the assumption that the alleged communicative element in O'Brien's conduct is sufficient to bring into play the First Amendment, it does not necessarily follow that the destruction of a registration certificate in a constitutionally protected activity."