Question on doing Linear Regression:

I am wondering whether we drop certain data when we are doing a linear regression. For example, if we do a linear regression on (X1, X2) and get a prediction function Y = 0 + 1X1 + 2X2. If 2 is very close to zero, indicating there is no correlation between Y and X2, should we keep this function as it is or do another regression only depending on X1? Will dropping the variable change anything?