Modeling hardening and softening due to high-angle grain boundaries in crystalline solids