Alon Bar-Lev has posted comments on this change. Change subject: aaa: Fix sync ......................................................................
Patch Set 13: (13 comments) thank you for such quick round! please see the notes, I think that we should completely detach from the db entities in aaa, and move to a very simple signature for the fetch principals function... as the Authz entities are sufficient for all you need. Maybe I am wrong, be happy to understand why. http://gerrit.ovirt.org/#/c/28561/13/backend/manager/modules/aaa/src/main/java/org/ovirt/engine/core/aaa/sync/SyncAAAEntries.java File backend/manager/modules/aaa/src/main/java/org/ovirt/engine/core/aaa/sync/SyncAAAEntries.java: Line 19: import org.ovirt.engine.core.utils.extensionsmgr.EngineExtensionsManager; Line 20: import org.ovirt.engine.core.utils.log.Log; Line 21: import org.ovirt.engine.core.utils.log.LogFactory; Line 22: Line 23: public class SyncAAAEntries { I know I am peaky... but this is no sync, but fetch... :)) the fact that we use this fetch to sync (fetch from database, fetch from authz, update database) is for bll to mind. Line 24: Line 25: private static final Log log = LogFactory.getLog(SyncAAAEntries.class); Line 26: Line 27: // Load all the users from the database and refresh them. The refresh logic is as follows : Line 30: // 3. Fetch groups from DB Line 31: // 4. Resolve the fetched groups (for each group that was not already resolved) Line 32: // 5. Resolve recursively all groups Line 33: // 6. Compare the user from db and the fetched user from the directory + store in DB if there is a change Line 34: public void sync( fetchUsers? Line 35: Map<String, Map<String, Set<String>>> userIdsToFetch, Line 36: Set<String> activeGroupIds, Line 37: Map<String, DirectoryGroup> outDirectoryGroupsByIds, Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, Line 31: // 4. Resolve the fetched groups (for each group that was not already resolved) Line 32: // 5. Resolve recursively all groups Line 33: // 6. Compare the user from db and the fetched user from the directory + store in DB if there is a change Line 34: public void sync( Line 35: Map<String, Map<String, Set<String>>> userIdsToFetch, userIdsToFetchPerAuthz? to make it clear what is the 2nd dimension. Line 36: Set<String> activeGroupIds, Line 37: Map<String, DirectoryGroup> outDirectoryGroupsByIds, Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, Line 39: List<String> outNonActiveUserIds, Line 32: // 5. Resolve recursively all groups Line 33: // 6. Compare the user from db and the fetched user from the directory + store in DB if there is a change Line 34: public void sync( Line 35: Map<String, Map<String, Set<String>>> userIdsToFetch, Line 36: Set<String> activeGroupIds, this is not required as it can be concluded from the output, no? Line 37: Map<String, DirectoryGroup> outDirectoryGroupsByIds, Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, Line 39: List<String> outNonActiveUserIds, Line 40: List<String> outNonActiveGroupIds) { Line 34: public void sync( Line 35: Map<String, Map<String, Set<String>>> userIdsToFetch, Line 36: Set<String> activeGroupIds, Line 37: Map<String, DirectoryGroup> outDirectoryGroupsByIds, Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, I would like to avoid database entities as much as possible during this logic. Line 39: List<String> outNonActiveUserIds, Line 40: List<String> outNonActiveGroupIds) { Line 41: Map<String, DirectoryGroup> fetchedGroupsCache = new HashMap<>(); Line 42: Set<String> authzEntries = userIdsToFetch.keySet(); Line 36: Set<String> activeGroupIds, Line 37: Map<String, DirectoryGroup> outDirectoryGroupsByIds, Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, Line 39: List<String> outNonActiveUserIds, Line 40: List<String> outNonActiveGroupIds) { these should be concluded by the caller based on all above data. non active users = user ids to fetch - outDirectoryUserByIds.keySet() for the non active group ids - I thought there is no need... as we always go via users, never fetch group, so we do not care if group exists in directory or not. Line 41: Map<String, DirectoryGroup> fetchedGroupsCache = new HashMap<>(); Line 42: Set<String> authzEntries = userIdsToFetch.keySet(); Line 43: for (String authz : authzEntries) { Line 44: ExtensionProxy authzExtension = null; Line 38: Map<String, DirectoryUser> outDirectoryUserByIds, Line 39: List<String> outNonActiveUserIds, Line 40: List<String> outNonActiveGroupIds) { Line 41: Map<String, DirectoryGroup> fetchedGroupsCache = new HashMap<>(); Line 42: Set<String> authzEntries = userIdsToFetch.keySet(); temp variable not needed Line 43: for (String authz : authzEntries) { Line 44: ExtensionProxy authzExtension = null; Line 45: try { Line 46: authzExtension = EngineExtensionsManager.getInstance().getExtensionByName(authz); Line 46: authzExtension = EngineExtensionsManager.getInstance().getExtensionByName(authz); Line 47: } catch (ConfigurationException ex) { Line 48: log.warn(String.format("The extension %1$s could not be found. Users associated with this authz extension will not be refreshed.", Line 49: authz)); Line 50: continue; please do not use continue within catch.. for () { try { } catch() { continue; } logic } == for () { try { logic } catch() { } } which is exceptional pattern valid without spagetti goto. or if you really like to not nest within try, nest within if: for () { boolean f = false; try { xxx; f = true; } catch() { } if (f) { logic } } Line 51: } Line 52: Line 53: boolean supportRecursiveGroupResolution = Line 54: (authzExtension.getContext().<Long> get(Authz.ContextKeys.CAPABILITIES, 0L) & Authz.Capabilities.RECURSIVE_GROUP_RESOLUTION) != 0L; Line 53: boolean supportRecursiveGroupResolution = Line 54: (authzExtension.getContext().<Long> get(Authz.ContextKeys.CAPABILITIES, 0L) & Authz.Capabilities.RECURSIVE_GROUP_RESOLUTION) != 0L; Line 55: Line 56: Map<String, Set<String>> groupIdsToFetch = Collections.emptyMap(); Line 57: Set<DirectoryGroup> groupsToFetch = new HashSet<>(); I stopped here... I truly want to see you are using PrincipalRecord and GroupRecord instead of using these db entities. Line 58: Map<String, Set<String>> usersToFetchPerNamespace = userIdsToFetch.get(authz); Line 59: for (String namespace : usersToFetchPerNamespace.keySet()) { Line 60: List<DirectoryUser> fetchedUsers = AuthzUtils.findPrincipalsByIds( Line 61: authzExtension, Line 103: } Line 104: groupIdsToFetch = getIdsFromGroups(groupsToFetch, fetchedGroupsCache); Line 105: } Line 106: } Line 107: outDirectoryGroupsByIds.putAll(fetchedGroupsCache); I am unsure why you need both. Line 108: } Line 109: Line 110: Line 111: private void addGroupsToFetch(Set<String> activeGroupIds, http://gerrit.ovirt.org/#/c/28561/13/backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/aaa/DirectoryUtils.java File backend/manager/modules/bll/src/main/java/org/ovirt/engine/core/aaa/DirectoryUtils.java: Line 59: dbGroupsPerExternalId.put(dbGroup.getExternalId(), dbGroup); Line 60: if (dbGroup.isActive()) { Line 61: activeGroupExternalIds.add(dbGroup.getExternalId()); Line 62: } Line 63: } ok, so now I am confused... why do we need to populate groups? can we read the result? as this is void input to our algorithm. Line 64: Line 65: Map<String, DirectoryGroup> outDirectoryGroupsByIds = new HashMap<>(); Line 66: Map<String, DirectoryUser> outDirectoryUserByIds = new HashMap<>(); Line 67: List<String> outNonActiveUserExternalIds = new ArrayList<>(); Line 80: Line 81: for (String id : outNonActiveGroupExternalIds) { Line 82: dbGroupsPerExternalId.get(id).setActive(false); Line 83: DbFacade.getInstance().getDbGroupDao().update(dbGroupsPerExternalId.get(id)); Line 84: } both of the above should be concluded by the output of the outDirectory*** Line 85: Line 86: for (DbUser dbUser : dbUsers) { Line 87: if (dbUser.isActive()) { Line 88: DirectoryUser user = outDirectoryUserByIds.get(dbUser.getExternalId()); Line 86: for (DbUser dbUser : dbUsers) { Line 87: if (dbUser.isActive()) { Line 88: DirectoryUser user = outDirectoryUserByIds.get(dbUser.getExternalId()); Line 89: Set<DirectoryGroup> accumulator = new HashSet<>(); Line 90: DirectoryUtils.flatGroups(accumulator, user.getGroups(), outDirectoryGroupsByIds); I almost sure you have double flat here and within the aaa sync... I would like to believe that we do not need to flat there. If you use PrincipalRecord and GroupRecord, you can return a full scale list of PrincipalRecords as: // input is Map<Authz, Set<ids>> // output is Set<PrincipalRecord> Set<ExtMap> fetchPrincipals(Map<String, Set<String>> principalIds) { } This should be sufficient signature, why? Each PrincipalRecord has List<GroupRecord>, each GroupRecord has List<GroupRecord>, so there should be no problem for you to build standard structure of documented entities. Then from this List<PrincipalRecord> you can derive all: 1. inactive users = db_users - Set<PrincipalRecord> 2. groups for user = flat(PrincipalRecord.Groups) Line 91: user.setGroups(new ArrayList<DirectoryGroup>(accumulator)); Line 92: DbUser userFromDirectory = new DbUser(user); Line 93: userFromDirectory.setGroupIds(DirectoryUtils.getGroupIdsFromUser(user, outDirectoryGroupsByIds)); Line 94: if (!userFromDirectory.equals(dbUser)) { -- To view, visit http://gerrit.ovirt.org/28561 To unsubscribe, visit http://gerrit.ovirt.org/settings Gerrit-MessageType: comment Gerrit-Change-Id: Id49b51517a967c7a83e8e73f52181673baa31700 Gerrit-PatchSet: 13 Gerrit-Project: ovirt-engine Gerrit-Branch: master Gerrit-Owner: Yair Zaslavsky <yzasl...@redhat.com> Gerrit-Reviewer: Alon Bar-Lev <alo...@redhat.com> Gerrit-Reviewer: Yair Zaslavsky <yzasl...@redhat.com> Gerrit-Reviewer: automat...@ovirt.org Gerrit-Reviewer: oVirt Jenkins CI Server Gerrit-HasComments: Yes _______________________________________________ Engine-patches mailing list Engine-patches@ovirt.org http://lists.ovirt.org/mailman/listinfo/engine-patches
