It’s been a while since I’ve seen Cool Hand Luke, so I couldn’t tell you what it is about this line that is (if it is) central to the film, nor why it assumed greater cultural significance.
Guns n’ Roses used it at the beginning of one of their songs.
Recently, a certain Fred Shapiro collected 25 quotes of famous Harvard alumni. Among gems by Gertrude Stein, Robert Frost and Henry David Thoreau, is a line from the screenplay of Cool Hand Luke by Frank Pierson, class of 1946:
What we’ve got here is failure to communicate.