I think that what most people are most concerned about with respect to evolutionary psychology, neuroscience and related topics is that they seem to suggest that we do not have freewill and hence moral responsibility is meaningless. No one would deny that we are in some ways influenced by our environment: for example brutalised children are more likely to end up as brutal and people who have excellent educations are generally cleverer. This is partly because the brain continues to develop as we grow and so our lives directly effect how it develops. However, there is nothing inevitable about this and people often surmount serious disadvantages through their own choice and determination. So, I don't think anyone could say that the fact we are effected by our surroundings means we have no freedom at all.
I think it is best to see our genes as part of our environment and thus they give us certain predispositions without the inevitability that would preclude our freedom of choice. A very similar argument raged in the Middle Ages over the extent to which behavior was determined by the stars. On one side were the astrological extremists like Cecco D'Ascoli who claimed that our lives were determined in advance by the stars under which we were born. Freewill did not exist and with it went moral responsibility. Opponents often admitted that there was something to astrology but said that the stars could only give rise to predispositions and could never entirely negate freedom and morality. The Church, of course, aided by thinkers such as St Thomas Aquinas, insisted that moral choice did exist and those who claimed our personalities are determined by the stars are wrong. St Augustine rejected the efficacy of astrology all together, raising the argument that twins born under the same stars could have very different fates.
If genetics and neuroscience do not logically have to conflict with the doctrine of freewill, many people believe that the associated ideas about the mind being a purely physical phenomenon do make freewill impossible. But this, I suggest is an entirely separate question. We can accept that things can influence us without it leading to the lose of freedom. My thoughts on the much more serious difficulty of what the mind actually is, I will come to in a later post.