Speaker meaning is generally defined in pragmatics in terms of the speaker's intentions. The received view is that a speaker means something by intending that the hearer recognise what is meant as intended by the speaker, thereby grounding speaker meaning in a presumed cognitive reality. In this paper it is proposed that speaker meaning can also be conceptualised from a social, deontological perspective where the speaker is held accountable to the moral order for what he or she is taken to mean in interaction. Speaker meaning in this sense encompasses moral or ethical concerns such as rights, obligations, responsibilities, permissibility, and thus is a real-world, consequential concept for participants in interaction. One result of this real-world consequentiality is that the degree of accountability for speaker meanings can be observed to be disputed by participants in both institutional and everyday talk. A second consequence is that the degree of accountability for speaker meanings can be modulated through various meaning-actions that either increase or decrease a speaker's level of accountability for particular meanings. The practice of not-saying is argued to be one relatively neglected meaning-action through which speakers may decrease their level of accountability in interaction. It is concluded that work remains to investigate whether a deontic conceptualisation of speaker meaning can be reconciled with the received view.