Swinson has a hand injury that requires further specialist assessment while Harley sustained a knee injury in training last week and has returned to the care of his Glasgow's medical team.
Willem Nel also drops out after fracturing an arm in Saturday's win over Samoa. Nel is replaced by Newcastle's Jon Welsh.
Wing Tommy Seymour injured his toe on Saturday a scan confirmed he'll require ongoing treatment by the Scotland medical team before returning to training this week.