The hack of a technology vendor for real estate financiers left major American banks and mortgage lenders working Sunday to assess whether they were affected by the data breach.