The closure problem in fluid modeling is a well-known challenge to modelers aiming to accurately describe their systems of interest. Over many years, analytic formulations in a wide range of regimes have been presented but a practical, generalized fluid closure for magnetized plasmas remains an elusive goal. In this study, as a first step toward constructing a novel data-based approach to this problem, we apply ever-maturing machine learning methods to assess the capability of neural network architectures to reproduce crucial physics inherent in known magnetized plasma closures. We find encouraging results, indicating the applicability of neural networks to closure physics, but also arrive at recommendations on how one should choose appropriate network architectures for the given locality properties dictated by the underlying physics of the plasma.