这是indexloc提供的服务，不要输入任何密码
Skip to content

feat: Add new Appium setting for in-depth snapshot in JSON page source #997

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

feat: Add new Appium setting for in-depth snapshot in JSON page source #997

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion WebDriverAgentLib/Categories/XCUIApplication+FBHelpers.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -37,6 +37,7 @@
#import "XCUIElement+FBUtilities.h"
#import "XCUIElement+FBWebDriverAttributes.h"
#import "XCUIElementQuery.h"
#import "FBConfiguration.h"

static NSString* const FBUnknownBundleId = @"unknown";

Expand Down Expand Up @@ -176,7 +177,7 @@ - (NSDictionary *)fb_tree

- (NSDictionary *)fb_tree:(nullable NSSet<NSString *> *)excludedAttributes
{
id<FBXCElementSnapshot> snapshot = [self fb_takeSnapshot:YES];
id<FBXCElementSnapshot> snapshot = [self fb_takeSnapshot:[FBConfiguration inDepthSnapshot]];
Copy link

@mykola-mokhnach mykola-mokhnach Mar 31, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure how this is supposed to work. setting this argument to NO makes the snapshotting function to not return the children property, which renders the recursive tree fetching impossible

Copy link
Author

@vinaykumar0339 vinaykumar0339 Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mykola-mokhnach
The fb_takeSnapshot function allows passing inDepth: NO.

[self snapshotWithError:&error]

According to the official documentation, this method retrieves the entire hierarchy, including child elements.

I compared the results of passing YES and NO to inDepth, and both responses appear identical.

with YES parameter:
image
with No Parameter:
image

I also ran a diff check, and it showed no differences:
image

Copy link

@mykola-mokhnach mykola-mokhnach Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

perhaps this trick only works for root/application elements. Let me test it locally.
Could you also provide performance comparison results?

Copy link
Author

@vinaykumar0339 vinaykumar0339 Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mykola-mokhnach
Is time comparison metrics enough?

Copy link
Author

@vinaykumar0339 vinaykumar0339 Apr 1, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mykola-mokhnach
💡 Performance Summary:
With visible attribute (default):

inDepth=true: ~15s
inDepth=false: ~11s
With visible attribute excluded (appiumpageSourceExcludedAttributes: "visible"):
inDepth=true: ~3s (5× faster)
inDepth=false: ~0.5s (22× faster)

Yes, performance varies depending on the screen complexity. Using inDepth=false with excluded visible attributes can bring the page source API response time down to under 1 second in most cases.
The main motivation for this optimization in WebDriverAgent is to support a new library for the Appium ecosystem that will provide more generic element finding capabilities. This library will use element coordinate information (x,y bounds) to implement relative positioning strategies like "left of text", "right of text", "below", "above", etc.
We can make these spatial element-finding strategies much more efficient by improving the page source retrieval performance.

Copy link

@mykola-mokhnach mykola-mokhnach Apr 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would say in case this snapshotting strategy really improves the performance then there is no point in putting it under a flag. Lets make it the default setting instead. @vinaykumar0339 Please check if #998 works for you

Copy link
Author

@vinaykumar0339 vinaykumar0339 Apr 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@mykola-mokhnach Yes, this change (#998) works for me as well since it uses:

[self snapshotWithError:&error]

May I ask when this change could be pushed along with Appium? (https://github.com/appium/appium-xcuitest-driver/)

Copy link

@mykola-mokhnach mykola-mokhnach Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Published in the xcuitest driver 9.1.3+

Copy link
Author

@vinaykumar0339 vinaykumar0339 Apr 2, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mykola-mokhnach for the quick response and for considering my suggestion! Appreciate the effort in creating a new version based on it. 🚀

return [self.class dictionaryForElement:snapshot
recursive:YES
excludedAttributes:excludedAttributes];
Expand Down
4 changes: 4 additions & 0 deletions WebDriverAgentLib/Commands/FBSessionCommands.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -337,6 +337,7 @@ + (NSArray *)routes
FB_SETTING_KEYBOARD_AUTOCORRECTION: @([FBConfiguration keyboardAutocorrection]),
FB_SETTING_KEYBOARD_PREDICTION: @([FBConfiguration keyboardPrediction]),
FB_SETTING_SNAPSHOT_MAX_DEPTH: @([FBConfiguration snapshotMaxDepth]),
FB_SETTING_IN_DEPTH_SNAPSHOT: @([FBConfiguration inDepthSnapshot]),
FB_SETTING_USE_FIRST_MATCH: @([FBConfiguration useFirstMatch]),
FB_SETTING_WAIT_FOR_IDLE_TIMEOUT: @([FBConfiguration waitForIdleTimeout]),
FB_SETTING_ANIMATION_COOL_OFF_TIMEOUT: @([FBConfiguration animationCoolOffTimeout]),
Expand Down Expand Up @@ -398,6 +399,9 @@ + (NSArray *)routes
if (nil != [settings objectForKey:FB_SETTING_SNAPSHOT_MAX_DEPTH]) {
[FBConfiguration setSnapshotMaxDepth:[[settings objectForKey:FB_SETTING_SNAPSHOT_MAX_DEPTH] intValue]];
}
if (nil != [settings objectForKey:FB_SETTING_IN_DEPTH_SNAPSHOT]) {
[FBConfiguration setInDepthSnapshot:[[settings objectForKey:FB_SETTING_IN_DEPTH_SNAPSHOT] boolValue]];
}
if (nil != [settings objectForKey:FB_SETTING_USE_FIRST_MATCH]) {
[FBConfiguration setUseFirstMatch:[[settings objectForKey:FB_SETTING_USE_FIRST_MATCH] boolValue]];
}
Expand Down
18 changes: 18 additions & 0 deletions WebDriverAgentLib/Utilities/FBConfiguration.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -201,6 +201,24 @@ typedef NS_ENUM(NSInteger, FBConfigurationKeyboardPreference) {
*/
+ (int)snapshotMaxDepth;

/**
* Sets whether to take a deep snapshot of the application elements.
* When enabled, the snapshot captures more detailed information about the UI hierarchy.
* This may be useful for complex views but can impact performance.
*
* @param value A boolean value indicating whether to enable deep snapshots.
* Pass YES for a detailed snapshot, NO for a faster but less detailed one.
*/
+ (void)setInDepthSnapshot:(BOOL)value;

/**
* Gets the current setting for in-depth snapshot capturing.
* If enabled, snapshots will include more detailed UI information.
*
* @return A boolean indicating whether in-depth snapshots are enabled.
*/
+ (BOOL)inDepthSnapshot;

/**
* Whether to use fast search result matching while searching for elements.
* By default this is disabled due to https://github.com/appium/appium/issues/10101
Expand Down
9 changes: 9 additions & 0 deletions WebDriverAgentLib/Utilities/FBConfiguration.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,7 @@
static BOOL FBShouldUseTestManagerForVisibilityDetection = NO;
static BOOL FBShouldUseSingletonTestManager = YES;
static BOOL FBShouldRespectSystemAlerts = NO;
static BOOL FBInDepthSnapshot = YES;

static NSUInteger FBMjpegScalingFactor = 100;
static BOOL FBMjpegShouldFixOrientation = NO;
Expand Down Expand Up @@ -369,6 +370,14 @@ + (int)snapshotMaxDepth
return [FBGetCustomParameterForElementSnapshot(FBSnapshotMaxDepthKey) intValue];
}

+ (void)setInDepthSnapshot:(BOOL)value {
FBInDepthSnapshot = value;
}

+ (BOOL)inDepthSnapshot {
return FBInDepthSnapshot;
}

+ (void)setShouldRespectSystemAlerts:(BOOL)value
{
FBShouldRespectSystemAlerts = value;
Expand Down
1 change: 1 addition & 0 deletions WebDriverAgentLib/Utilities/FBSettings.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ extern NSString* const FB_SETTING_SCREENSHOT_QUALITY;
extern NSString* const FB_SETTING_KEYBOARD_AUTOCORRECTION;
extern NSString* const FB_SETTING_KEYBOARD_PREDICTION;
extern NSString* const FB_SETTING_SNAPSHOT_MAX_DEPTH;
extern NSString* const FB_SETTING_IN_DEPTH_SNAPSHOT;
extern NSString* const FB_SETTING_USE_FIRST_MATCH;
extern NSString* const FB_SETTING_BOUND_ELEMENTS_BY_INDEX;
extern NSString* const FB_SETTING_REDUCE_MOTION;
Expand Down
1 change: 1 addition & 0 deletions WebDriverAgentLib/Utilities/FBSettings.m
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,7 @@
NSString* const FB_SETTING_KEYBOARD_AUTOCORRECTION = @"keyboardAutocorrection";
NSString* const FB_SETTING_KEYBOARD_PREDICTION = @"keyboardPrediction";
NSString* const FB_SETTING_SNAPSHOT_MAX_DEPTH = @"snapshotMaxDepth";
NSString* const FB_SETTING_IN_DEPTH_SNAPSHOT = @"inDepthSnapshot";
NSString* const FB_SETTING_USE_FIRST_MATCH = @"useFirstMatch";
NSString* const FB_SETTING_BOUND_ELEMENTS_BY_INDEX = @"boundElementsByIndex";
NSString* const FB_SETTING_REDUCE_MOTION = @"reduceMotion";
Expand Down