If your number of members is small, and you'd rather hide the total (it automatically displays at the top of the Member Directory page), add this snippet to your Display Settings > Custom CSS:

.member-count-container {display: none;}
.h-member-count {display: none;}