Aging is associated with microstructural changes in brain tissue that can be visualized using diffusion tensor imaging (DTI). While previous studies have established age-related changes in white matter (WM) diffusion using DTI, the impact of age on gray matter (GM) diffusion remains unclear. The present study utilized DTI metrics of mean diffusivity (MD) to identify age differences in GM/WM microstructure in a sample of healthy older adults (N = 60). A secondary aim was to determine the functional significance of whole-brain GM/WM MD on global cognitive function using the Repeatable Battery for the Assessment of Neuropsychological Status (RBANS). Participants were divided into three age brackets (ages 50-59, 60-69, and 70+) to examine differences in MD and cognition by decade. MD was examined bilaterally in the frontal, temporal, parietal, and occipital lobes for the primary analyses and an aggregate measure of whole-brain MD was used to test relationships with cognition. Significantly higher MD was observed in bilateral GM of the temporal and parietal lobes, and in right hemisphere WM of the frontal and temporal lobes of older individuals. The most robust differences in MD were between the 50-59 and 70+ age groups. Higher whole-brain GM MD was associated with poorer RBANS performance in the 60-69 age group. Results suggest that aging has a significant and differential impact on GM/WM diffusion in healthy older adults, which may explain a modest degree of cognitive variability at specific time points during older adulthood.