When asked whether he is concerned about his grip on the job, though, Harbaugh brushed off the threat of a firing.