Solving a large‐scale optimization problem with nonlinear state constraints is challenging when adjoint gradients are not available for computing the derivatives needed in the basic optimization algorithm used. Here, we present a methodology for the solution of an optimization problem with nonlinear and linear constraints, where the true gradients that cannot be computed analytically are approximated by ensemble‐based stochastic gradients using an improved stochastic simplex approximate gradient (StoSAG). Our discussion is focused on the application of our procedure to waterflooding optimization where the optimization variables are the well controls and the cost function is the life‐cycle net present value (NPV) of production. The optimization algorithm used for solving the constrained‐optimization problem is sequential quadratic programming (SQP) with constraints enforced using the filter method. We introduce modifications to StoSAG that improve its fidelity [i.e., the improvements give a more accurate approximation to the true gradient (assumed here to equal the gradient computed with the adjoint method) than the approximation obtained using the original StoSAG algorithm]. The modifications to StoSAG vastly improve the performance of the optimization algorithm; in fact, we show that if the basic StoSAG is applied without the improvements, then the SQP might yield a highly suboptimal result for optimization problems with nonlinear state constraints. For robust optimization, each constraint should be satisfied for every reservoir model, which is highly computationally intensive. However, the computationally viable alternative of letting the reservoir simulation enforce the nonlinear state constraints using its internal heuristics yields significantly inferior results. Thus, we develop an alternative procedure for handling nonlinear state constraints, which avoids explicit enforcement of nonlinear constraints for each reservoir model yet yields results where any constraint violation for any model is extremely small.