This paper deals with various ways of criticizing arguments. In particular, I focus on John Pollock’s theory of defeasible reasoning and show that his conception of rebutting and undercutting defeaters omits some important ways of attacking arguments. I discuss not only the tenets of Pollock’s theory but I compare it also to some traditional ways of criticizing arguments occurring in philosophical logic, informal logic and argumentation theory. The paper provides some interesting kinds of counterexamples to deductive arguments which play a similar role to those of the undercutting and the rebutting defeaters occurring in case of non-deductive arguments. Finally, I point to a dilemma emerging from the analysis of Pollock’s conception: Either we acknowledge that Pollock’s theory is applicable only to a limited scope of defeasible arguments or we admit that we have to broaden the definition of defeasible arguments in such a way that it covers also deductive arguments. However, the latter option can lead to a conclusion that there are no nondefeasible arguments.