Sustainable management of fisheries is often compromised by management delaying implementation of regulations that reduce harvest, in order to maintain higher catches in the short term. Decreases or increases in fish population growth rate driven by environmental change, including ecosystem and climate change, affect the harvest that can be taken sustainably. If not acted on rapidly, environmental change could result in unsustainable fishing or missed opportunity for higher catches. Using simulation models of harvested fish populations influenced by environmental change, we explore how long fisheries managers can afford to wait before changing harvest regulations in response to changes in population growth. If environmental change causes population declines, delays greater than five years increase the probability of population collapse. Species with fast and highly variable population growth rates are more susceptible to collapse under delays and should be a priority for revised management where delays occur. Generally, the long-term cost of delay, in terms of lost fishing opportunity, exceeds the short-term benefits of overfishing. Lowering harvest limits and monitoring for environmental change can alleviate the impact of delays; however, these measures may be more costly than reducing delays. We recommend that management systems that allow rapid responses to population growth changes be enacted for fisheries management to adapt to ecosystem and climate change.