John
15:1 I am the true vine, and my Father is the husbandman.
15:2 Every branch in me that beareth not fruit he taketh away: and every [branch] that beareth fruit, he purgeth it, that it may bring forth more fruit.
15:3 Now ye are clean through the word which I have spoken unto you.
15:4 Abide in me, and I in you. As the branch cannot bear fruit of itself, except it abide in the vine; no more can ye, except ye abide in me.
15:5 I am the vine, ye [are] the branches: He that abideth in me, and I in him, the same bringeth forth much fruit: for without me ye can do nothing.
15:6 If a man abide not in me, he is cast forth as a branch, and is withered; and men gather them, and cast [them] into the fire, and they are burned.
You are correct in your assertion that vv.2-4 speak of fruit. I agree with that, but v.6 speaks of the end of the man himself, not just his fruit.