What productivity does it destroy? What were you producing otherwise
		
		
	 
Actually good question. I view as "productive" any activity that would improve or develop me as a person, even if ever so slightly. Reading (something that is worth reading) is productive, writing is productive, coding is productive, working out is productive, and even watching anime and playing vidya can be productive if it manages to change me in some way. 
When I'm daydreaming, I'm not taking any input from the outside, as if content with what I already have. This leads me to stagnate as a person. Even worse still, daydreaming has a similar effect on me to drugs since I only use it as a cheap source of transient dopamine. Once the effects wear off, and I'm back to the real world, it can get extremely depressing.
Of course, I wasn't talking about actually producing things. I'm a NEET.