Making Talk Cheap:
Generative AI and Labor Market Signaling^††thanks: We are indebted to Adam Kapor, Jakub Kastl, and Alessandro Lizzeri for their invaluable guidance and support. We are also grateful to Elena Aguilar, Nick Buchholz, Jacob Dorn, Michael Droste, Allison Green, Jeff Gortmaker, Kate Ho, Simon Jäger, Quan Le, Thomas Le Barbanchon, Rachel Moore, Stephen Redding, Jordan Richmond, Ben Scuderi, Wilbur Townsend, Carolyn Tsao, So Hye Yoon, and Yuci Zhou for helpful comments and discussions. We thank Matt Barrie and the entire team at Freelancer.com for their openness and willingness to collaborate with academic researchers. We are especially grateful to Andrew Bateman and Will Bosler, without whom this research agenda would not have been possible. We additionally thank Torsha Chakraverty for great research assistance. This research benefited from financial support from the International Economics Section (IES) at Princeton University and the William S. Dietrich II Economic Theory Center, whose support is gratefully acknowledged. We also thank the Research Computing at ITC, Dartmouth College, for their technical support in implementing and scaling up our LLM-based customization measures for freelancers’ job application proposals. All errors are our own.

Freelancer.com is one of the world’s largest DLPs, as measured by both number of users and posted jobs. DLPs such as Freelancer.com and Upwork are global, online marketplaces that match freelance workers with potential employers seeking to outsource digital tasks. These tasks—ranging from software and web development to sales support, marketing, and graphic design—are performed remotely by freelancers hired under short-term contracts. Since its launch in 2009, more than 85 million users from over 247 countries, regions, and territories have registered either to supply or to procure remote work. In fiscal year 2024 alone, employers posted 1.1 million jobs on the platform, generating over $130 million in payments with an average completed job paying $334.⁹⁹9These statistics are based on Freelancer.com’s 2024 Annual Report

We have access to the internal database of Freelancer.com, on which the platform stores data on nearly all market operations such as job posting, job applications, employer hiring decisions, job completion, and worker performance. These data include every application submitted to every job post, along with the precise timestamps and content of any on-platform action (stored as click data) taken by any employer or worker on the platform.

In particular, we observe the complete “lifecycle” of every job post: we observe when the job is posted, the exact text of the job title and job description, whether and when the job is awarded to a worker, whether and when the job is completed, the total payment the worker receives for completing the job, and the star rating ($1-5$) the employer gives the worker upon job completion (if any).

We also observe all on-platform worker-employer interactions (save for the content of private messages). We observe whether and when the job poster contacts each worker via the on-platform messaging system (which is always initiated by the job poster), whether and when the job poster clicks through to each worker’s profile, and whether and when the job poster clicks on each application in any way (e.g., clicking to expand the proposal text, clicking to “favorite” the application, etc.).

Furthermore, we also observe all on-platform actions and information associated with each application. We observe the bid of each application, the text of each application’s proposal, the timestamp that each applying worker first views the full job post (i.e., when workers first click on the job post’s page), the timestamp that each applying worker submits her application, whether the applying worker used the on-platform AI proposal-writing tool, the country of residence of each applying worker, and the complete work history of each applying worker on the platform, including all prior jobs completed, star ratings received, and reviews received. Finally, we observe each application’s reputation score, which, as described above, is the output of the platform’s recommendation algorithm. Thus, we observe the dynamically updated ranking of each application on the “bid list” page at any point in time.

In this subsection, we describe how we construct our sample. The sampling procedure is shaped both by the financial constraints of large-scale text analysis and by the goals of our empirical study.

Our analysis relies on and, we argue, requires (see Section 3.2) the use of LLMs to process the text of millions of job descriptions and worker proposals. Because this computation is costly, it limits the feasible sample size. This creates a trade-off between capturing heterogeneity on the employer side and on the worker side.

Since our primary goal is to study how employers learn about worker ability, and how this learning is affected by the loss of signaling through written proposals, we prioritize worker-side heterogeneity. To do so, we restrict attention to a relatively homogeneous set of job postings.

Specifically, we include all jobs posted on the platform between January 1, 2021—before the mass adoption of LLMs—and July 26, 2024—well after mass adoption and, crucially, after the introduction of the on-platform AI proposal-writing tool on April 18, 2023—that satisfy the following criteria:

• $\bullet$

  Fixed-price jobs;

• $\bullet$

  Transactions conducted in USD (bids and payments);

• $\bullet$

  Budget range of $30–$250 (the modal range on the platform);

• $\bullet$

  Job description written in English;

• $\bullet$

  Posted by employers with verified identities and payment methods;

• $\bullet$

  Posted by employers who, at any point on the platform (including outside our sample period), have hired and paid at least one worker;

• $\bullet$

  Awarded to at most one worker;

• $\bullet$

  Categorized as involving some form of coding;¹⁹¹⁹19Note that we do not include coding-related skill tags that only appear in the post-LLM sample as to make the types of job posts in both the pre-LLM and post-LLM samples comparable. See Table A1 for a list of the top 15 skill tags in our sample.

• $\bullet$

  Job description containing at least 25 words and at least 182 characters.

Applying these restrictions yields a sample of roughly 61,000 job postings, associated with about 2.7 million applications submitted by 212,000 unique workers.

We divide this dataset into two subsamples: one before the release of ChatGPT on November 30, 2022 (“pre-LLM”), and one after (“post-LLM”). The pre-LLM sample contains approximately 33,000 job postings, 960,000 applications, and 92,000 unique workers. The post-LLM sample contains approximately 28,000 postings, 1.7 million applications, and 135,000 workers. Table 1 provides summary statistics for both subsamples.

Our analysis requires a measure of how much cognitive effort each worker expends to produce her written application proposal. We proxy for this effort using the time each worker spends reading the job post and writing her proposal. In particular, we measure the time between when each worker first ever clicks on the job post’s page and when she submits her application. We refer to this raw measurement as “bid time,” and we interchangeably refer to this measure as “signaling effort” or simply “effort.”

We successfully measure valid bid times for 70.45% of all the applications in our sample.²⁰²⁰20The pre-LLM sample has a lower fraction of valid bid times (59.76%) than the post-LLM sample has (76.39%). 6.21% of applications have negative bid times (i.e., the worker’s first click occurs after she submits her application), 17.67% have missing bid times (i.e., we do not observe any clicks by the worker on the job post), 2.54% have implausibly high bid times (i.e., over 12 minutes), and 3.13% have implausibly low bid times (i.e., under 4 seconds).

The applications with missing, negative, or implausibly small bid times likely arise from workers accessing the platform’s job postings via an application programming interface (API) that allows them to scrape job posts without actually registering a click on the job post. Both from inspecting individual applications with missing or negative bid times and from a thorough statistical analysis of the variables we observe, we find that the proposals associated with these applications do not seem to be statistically distinct from the applications for which we observe valid bid times, conditional on an application’s observable characteristics. We treat the incidence of invalid or missing bid times as random, conditional on observables, and we omit these applications from our empirical analyses that directly use bid time, while including them in all other analyses.

Our final sample of bid times thus ranges from 4 seconds to 12 minutes, and we display kernel density estimates of both the pre-LLM and post-LLM distributions of bid time in Figure 2. The average bid time in the pre-LLM sample is 1.47 minutes, which is sensible given the average job post in that period is 93 words long, and the average proposal is 89 words long.

Our analysis requires a measure of the “signal” contained in workers’ written proposals. This paper develops the argument that employers learn about worker ability in equilibrium via worker’s expending costly effort to produce job-post-specific proposals. Thus, to measure signals, we use a novel, LLM-based measurement of how customized and relevant each proposal is to the specific job post to which it is being submitted. In this subsection, we first define our measurement, and then we provide further discussion of its design. Details of this measurement procedure can be found in Appendix C.1, and comparisons to NLP-based alternative measurements can be found in Appendix E.

In this subsection, we describe how we use click, bid ranking, and timestamp data to build our best proxy of which applications job posters actually see and consider.

Many applications submitted to job posts on DLPs are never read by employers, as verified by click data and anecdotal reports. Without measuring which applications are considered by employers, we may introduce significant bias into our demand estimation. For example, if unconsidered applications are more likely than considered ones to have lower bids, then including these unconsidered applications in our demand estimation would introduce attenuation bias into our estimate of employers’ price sensitivity.

Our approach to measuring consideration sets revolves around three key assumptions. First, we assume employers are likely to consider applications that they engage with directly, such as clicking to expand the proposal if its text does not fit within the default bid list display window, clicking to “pin” an application to the top of their bid list, or sending a direct message to the worker who submitted the application. Second, we assume that if an employer engages with an application ranked $n$th at time $t$, then they are likely to consider all applications ranked above it, i.e., with ranks $<n$, at time $t$. Third, we assume that employers are more likely to consider applications the earlier they arrive and the higher they are ranked.

We define consideration sets via an algorithm that combines the three assumptions above. We assign each application a “consideration score” that is a function of (1) whether the employer directly engaged with the application, (2) the reputation score of the application, (3) the time of the application’s submission, and (4) whether the application was scrolled past by the employer before engaging with a lower ranked application. We first define any application as considered if it has a nonzero consideration score, and then adjust these definitions to have stricter consideration score thresholds if a job post has implausibly many or few considered applications. Details of this algorithm can be found in Appendix C.2.

As per our research agreement with Freelancer.com, we do not report the exact fraction of applications that are considered in our sample, but we can report that our measured consideration sets are reasonably sized with about 12 applications per job post on average in the pre-LLM period and about 17 applications per job post on average in the post-LLM period. Reflecting the increase in the number of applications per job post in the post-LLM period, the average fraction of all applications that are considered decreases in the post-LLM period by about 14 percentage points.

See Table A2 for summary statistics on the size and composition of consideration sets in the pre-LLM and post-LLM samples.

Figure 4 displays a binscatter of whether an application was ultimately awarded the job on our signal measure. As per our research agreement with the platform, we do not disclose the level of hiring rates, so we display win rates on the $y$-axis normalized (i.e., divided) by the pre-LLM overall win rate. Before taking a more formal econometric approach that acknowledges the inherent correlation between the outcomes of applications to the same job post, we first present this visual evidence of the demand response to signals. Note that worker competition is intense, with a very low hiring rate per application.

Despite these low hiring rates, we see that pre-LLM applications in the right tail of the signal distribution have nearly four times higher hiring rates than the average application in the pre-LLM sample. These patterns, which are unconditional on consideration sets, hold despite an application’s signal and inclusion in the consideration set having a negative correlation coefficient of -0.078 in the pre-LLM sample.

While suggestive of our signaling hypothesis, this pattern could be driven by other factors than a direct demand response to signals, such as the competition each application faces or other confounding variables correlated with signal.

Table 2 presents the results of a series of estimated multinomial logit models that provide a reduced-form approach to capturing employer demand. These estimated models use a variety of explanatory variables in addition to signal including bid, reputation score, log arrival time (i.e., the time between the job posting and when a worker first clicks on the job post), and an applicant’s country of residence. For each specification, we estimate a version of the model treating all applications as being in the consideration set and a version including only the applications we measured to be in the consideration set.

In both pre-LLM specifications used in Table 2, we find that signals are strong statistically significant predictors of employer demand. Given that the scale of our signal measure is arbitrarily defined (without loss), we interpret the coefficients on signal in Table 2 by computing the decrease in an application’s bid, all else equal, needed to achieve the same predicted increase in demand as a standard deviation increase in signal.

Using the estimates of Column (2) in Table 2, which only include considered applications, we find that in the pre-LLM sample, a one standard deviation increase in signal has the same predicted increase in demand, all else equal, as a $25.67 decrease in bid, which is about 39% of a standard deviation of considered bids in that sample.

Since of course employers do not directly value workers’ proposals themselves, these results suggest that prior to the mass adoption of LLMs, employers were either using workers’ proposals as signals of attributes they do value, or signals were correlated with an attribute employers value that we do not observe. Since we observe virtually everything the employer sees about an application and the worker sending the application, and since over 99.9% of applications in our sample are sent by workers who have never worked with the job poster before, we argue that the most likely explanation for these results is that employers were using workers’ proposals as a signal of something they directly value.

We argue that proposals primarily function as signals of effort expended by workers in reading the job post and crafting a written response. Figure 5 presents binscatters of signal on effort. In the pre-LLM sample, we see a clear increasing and concave relationship between effort and signal. This relationship suggests that signals are produced with effort, i.e., the longer a worker spends reading the job post and/or writing her proposal, the more customized and relevant the proposal is to the job post.

While the visual evidence displayed in Figure 5 suggests a possible causal relationship between effort and signal in the Pre-LLM period, one might question whether the pattern is produced simply by the fact that workers who spend more time reading and writing are also the types of workers to produce better written content. Thus, key to the argument that the relationship is causal is whether the pattern holds true within workers: when a given worker spends more time writing a proposal, does she produce a higher signal?

Table 3 presents Ordinary Least Squares (OLS) estimates that confirm this within-worker relationship between effort and signal, by regressing signal on log effort with worker fixed effects. Of the roughly 92,000 distinct workers we observe sending applications in the pre-LLM sample, around 32,000 of them send multiple applications. The average worker among those 32,000 workers, sends approximately 28 applications in the pre-LLM period. Furthermore, of the workers we see sending multiple applications, the average standard deviation of effort within worker is 1.27 minutes, which is about 65% of the standard deviation of effort in the entire pre-LLM sample. Thus, we argue we have sufficient within-worker variation in effort to credibly test whether the relationship between signal and effort holds within worker.

Column (2) of Table 3 shows that, even within worker, effort is a statistically significant positive predictor of signal. These results suggest that workers’ written proposals function as noisy signals of effort.

Next, we ask why employers might demand signals of effort. Table 4 presents OLS estimates regressing whether a worker completes a job with a 5-star rating conditional on being hired on signal, effort, and various controls.²³²³23Visual evidence of this descriptive relationship between job completion outcomes and signals and effort is presented in Figure A2. Columns (1) and (2) suggest that signals predict job completion outcomes, even after controlling for other variables that employers may use to select workers. Column (3) shows that once we control for effort, the coefficient on signal falls by about 41% and that log effort itself is a strong predictor of job completion outcomes.²⁴²⁴24Table A5 re-estimates column (3) from Table 4 with a corrected measure of effort, accounting for worker fixed effects in signal production, that we implement when estimating the model presented in Section 5. Column (3) of Table A5 shows that once we control for corrected effort, signals are insignificant predictors of job completion outcomes. This result further motivates our claim that proposals are only used as signals in that they are predictors of effort, and that the information contained within signals, conditional on effort, do not predict worker ability. This result implies that while signals do predict job completion outcomes, they do so in large part because they are correlated with effort, which itself is a strong predictor of job completion outcomes.

The regressions presented in Table 4 are susceptible to selection bias since we only observe job completion outcomes for workers who are hired. For example, a worker selected with a low signal may be selected on some other unobservable, to the econometrician, component that positively predicts job completion outcomes. While we maintain that we observe virtually everything the employer sees about an application and the worker sending the application, this selection bias would bias our estimates of the relationship between signal and job completion outcomes towards zero. Thus, we argue that the coefficients on signal and effort in columns (1)-(3) of Table 4 are most likely lower bound estimates of the true relationships.

Taking these results together, we argue that they provide suggestive evidence that prior to the mass adoption of LLMs, workers’ written proposals functioned as Spence-like signals of effort, which employers cannot observe, and that effort itself predicts a worker’s propensity to successfully complete a job. Thus, in other words, we have presented suggestive evidence that written proposals functioned as Spence-like signals of worker ability in the pre-LLM period.

The preceding two subsections built the argument that prior to the mass adoption of LLMs, workers’ written proposals functioned as Spence-like signals of effort, and in equilibrium, functioned as signals of worker ability. In this subsection, we present descriptive evidence that suggests that this signaling story breaks down in the post-LLM period. To do so, we now examine the same empirical patterns presented in the preceding two subsections for the post-LLM period and, in particular, study the subset of applications written with the on-platform AI proposal-writing tool.

Taken together, this section presents suggestive evidence that prior to the mass adoption of LLMs, workers’ written proposals functioned as Spence-like signals of effort, which employers could rely on to predict a worker’s propensity to successfully complete a job. In the post-LLM period, however, this signaling mechanism appears to have collapsed: signals no longer predict effort or job completion outcomes, and thus employers no longer value signals.

Although this descriptive evidence is suggestive, it does not allow us to quantify how LLMs’ lowering of the cost of writing proposals has affected the distribution of ability among hired workers, hiring patterns, or employer and worker welfare. Moreover, descriptive evidence alone cannot isolate LLMs’ effects on signaling from other changes LLMs may have induced by altering the nature of work.

To both quantify and isolate the effect of LLMs’ disruption of labor market signaling, we need to (1) identify the joint distribution of worker abilities and costs of undertaking jobs, (2) identify how employers’ hiring decisions change when LLMs erode the informativeness of signals, and (3) know how workers of differing abilities would change their bidding behavior in an equilibrium without signaling.

To see why these three inferences are necessary, suppose we observe the pattern that workers sending better signals tend to bid higher. First, consider the case in which this pattern results from high-ability workers knowing they can signal their higher-than-average value to employers and thus bidding high to capture more surplus, rather than high-ability workers having higher costs than low-ability workers do. In this case, if LLMs eliminate signaling, high-ability workers would lower their bids to compete with lower-ability workers, and employers may end up hiring a similar distribution of workers as they would with signaling. Now, consider the case in which this pattern results from high-ability workers having higher costs of undertaking the job than low-ability workers do. In this case, if LLMs eliminate signaling, high-ability workers may not be able to lower their bids enough to compete with low-ability workers, and employers may end up hiring far fewer high-ability workers.

We therefore build a model of labor market signaling in the pre-LLM market, which will allow us to identify these three necessary components. Once we have estimated the model, we can simulate a counterfactual market equilibrium without signaling, thereby allowing us to quantify the isolated effect of LLMs disrupting signaling on hiring patterns and welfare.

We model jobs being posted and workers arriving at job posts as being a random exogenous process, and we assume that all workers who arrive at a job post submit an application.²⁶²⁶26See [krasnokutskaya2020role] for a paper studying a similar DLP context in which worker entry to a job post is endogenous. This assumption significantly changes and complicates identification and estimation and is at the core of the empirical challenges overcome by their approach.

There is a large number, $I$, of ex-ante identical employers, each posting one job on a monopolistic platform, hoping to outsource a task. We use $i$ to interchangeably index both the job post and the employer posting the job for notational convenience. Each job post, $i$, has set of $N_{i}$ workers applying to it, indexed by $j\in\{1,\dots,N_{i}\}$.

Each worker $j$ bidding on job post $i$ has a vector of discrete observables $x_{ij}\in X$ that workers do not control.²⁷²⁷27Note that we include the $i$ subscript on $x$ for two distinct reasons. First, since workers are indexed from 1 to $N_{i}$ for each job post $i$, we need a way to distinguish, for example, worker $j=3$ applying to job post $i=1$ from worker $j=3$ applying to job post $i=2$. Second, although the model presented in this section has each worker only appearing once, in our empirical implementation we see workers applying to multiple job posts, and since some dimensions of $x_{ij}$, namely a worker’s arrival time and a worker’s platform reputation at the time of applying to job post $i$, can change over time, it is useful to distinguish between a given worker’s characteristics when applying to one job post and that same worker’s characteristics when applying to another. In our empirical implementation of this model, this vector contains where the worker is from, when the worker arrives at the job post relative to when the job is posted, and the worker’s current reputation on the platform. We discuss the precise construction of this vector in Section 6.

Since these observable characteristics are discrete and $X$ is finite, each worker belongs to one of finitely many observable types, $|X|$. Thus, we can provide an ex-ante characterization of each job post as an $|X|$-dimensional vector, $J_{i}$, of natural numbers that define the number of each $x$-type worker that arrives at job $i$.²⁸²⁸28Recall that we constructed our sample so that job posts are homogenous on most observable dimensions. Thus, we do not include employer or job post observables in our characterization of each job post. We define $\Gamma$ to be a discrete distribution over the set of possible ex-ante worker arrivals, $J_{i}$, that describes how workers jointly arrive at job posts. Worker arrival is completely determined by their observable type $x$, and thus worker arrival is neither endogenous nor selected on unobservables.

Upon arrival, workers draw a two-dimensional type $(c_{ij},a_{ij})|x_{ij}\sim$ i.i.d. from a distribution $F_{c\times a|x}$, which we allow to have any differentiable joint distribution over a compact connected support in $\mathbb{R}^{2}_{+}$. We define $c_{ij}$ to be worker $j$’s opportunity cost of taking the job $i$ and thus represents the amount she would need to be paid to be exactly indifferent between taking the job and not taking the job. We define $a_{ij}$ to be worker $j$’s ability to do the task involved in job post $i$.

After observing their unobservable type $(c_{ij},a_{ij})$, workers choose two actions: a bid $b_{ij}$ and a signaling effort $e_{ij}$. The latter entails a cost $C(e_{ij};a_{ij})$, where the cost function satisfies the following properties:

1. (i)

   $C(e;a)$ is twice continuously differentiable in both arguments for all $e>0$ and all $a$;

2. (ii)

   $\lim_{e\to 0}C(e;a)=0$ for all $a$;

3. (iii)

   $\frac{\partial C(e;a)}{\partial e}>0$ for all $e>0$ and all $a$;

4. (iv)

   $\frac{\partial^{2}C(e;a)}{\partial e^{2}}>0$ for all $e>0$ and all $a$;

5. (v)

   $\frac{\partial C(e;a)}{\partial a}<0$ for all $e>0$ and all $a$.

6. (vi)

   $\frac{\partial^{2}C(e;a)}{\partial e\partial a}<0$ for all $e>0$ and all $a$.

This cost function implies that higher-ability workers can produce any given level of signaling effort at lower cost than lower-ability workers can. Moreover, higher-ability workers face lower marginal costs of increasing effort at any given level of effort.

Workers pay this effort cost upfront, regardless of whether they are hired, but they only receive the payment $b_{ij}$ and incur the opportunity cost of working $c_{ij}$ if they are awarded the job. Thus, a worker’s ex-post utility from bidding $b_{ij}$ and expending effort $e_{ij}$ is given by:

where $w_{ij}$ is an indicator for whether worker $j$ is awarded job $i$.

Workers’ signaling effort $e_{ij}$ is combined with signaling noise $\varepsilon^{s}_{ij}$ to produce a signal $s_{ij}$, conditional on their observable type $x_{ij}$, according to signal production function

where:

1. (i)

   $\varepsilon^{s}_{ij}$ is independent of $(c_{ij},a_{ij},e_{ij})$ conditional on $x_{ij}$;

2. (ii)

   $\varepsilon^{s}_{ij}$ is distributed i.i.d. according to $F_{\varepsilon^{s}|x}$ with $\mathbb{E}[\varepsilon^{s}_{ij}|x_{ij}]=0$ for all $x_{ij}\in X$;

3. (iii)

   for all $e>0$ and all $x\in X$, $f^{s}(e;x)$ is continuously differentiable in $e$;

4. (iv)

   for all $e>0$ and all $x\in X$, $\frac{\partial f^{s}(e;x)}{\partial e}>0$.

By modeling signal production in this way, we are making two key assumptions. First, we are assuming that workers know how much effort they put into a proposal, and they know the distribution of signals that this effort might produce, but they do not know what that realization will be. Second, we are assuming that a worker’s ability $a_{ij}$ only affects the cost of producing signals, not the production of signals themselves.

A key feature of our empirical setting is that employers do not consider all workers who apply to their job posts. Similar to our model of worker arrival, we model consideration sets as being a random exogenous process. In particular, our model incorporates a very specific notion of consideration: consideration is a binary status such that an employer observes an application if and only if it is considered.

We model consideration set formation as being jointly dependent on all the observables of the applications that job post receives, $\{x_{ij}\}_{j\in N_{i}}$.²⁹²⁹29For example, a worker’s reputation score determines her positioning on the bid list only insofar as it compares to her competitors’ reputation scores.

Formally, we model the probability that a worker $j$ is considered by job poster $i$ with the function

where $x_{i,-j}\equiv\Big(x_{ik}\Big)_{k\in\{1,...,N_{i}\}\text{ and }k\neq j}$ is the vector of observable characteristics of all the other workers applying to job post $i$, and $q_{ij}$ is the indicator for whether employer $i$ considers worker $j$. Furthermore, let $Q_{i}\equiv\Big\{j\in\{0,1,...,N_{i}\}\mid q_{ij}=1\Big\}$ be employer $i$’s consideration set, where $j=0$ represents the outside option discussed below.

We model employer labor demand following the tradition of differentiated goods discrete choice demand models. In particular, we model each employer $i$’s indirect utility of hiring worker $j$ as³⁰³⁰30Employers can only hire those applications in their consideration sets, but their utility from hypothetically hiring an unconsidered worker who applies to their job post is still well-defined.:

where $T(x)$ is the utility for hiring a worker with observables $x$, holding fixed the worker’s ability and bid, and $\nu_{ij}$ is a stochastic utility shock distributed i.i.d. according to $F_{\nu}$.

Moreover, employer $i$ receives utility $U_{i0}\equiv U_{0}+\nu_{i0}$ from selecting the outside option—e.g., doing the work themselves, or simply not hiring anyone—where $\nu_{i0}$ is also drawn i.i.d. according to $F_{\nu}$, and we normalize $U_{i0}$ to 0.

We interpret $T(x)$ as potentially including both worker ability, common to all workers from group $x$, that does not enter signaling costs and a taste-based utility from hiring a member of that group.³¹³¹31We are not be able to recover non-signaled worker ability, since we cannot separately identify ability from taste-based utility in $T(x)$, but this poses no challenge to estimating our counterfactuals since we are only changing how workers are able to specifically signal their signaling-relevant ability $a_{ij}$. For ease of exposition, we refer to the signaling-relevant ability $a_{ij}$ simply as “ability” in the remainder of the paper.

In addition, we interpret $\nu_{ij}$ as an employer-worker-specific idiosyncratic taste shock or match effect. Our key assumption is that workers do not observe $\nu_{ij}$ before making their bid and effort choices.³²³²32For example, the taste shock could be derived from looking at a worker’s profile picture or the aesthetics of their writing style. Most likely, given we observe essentially everything that an employer observes about an application, other than a worker’s profile picture and name, this term either captures pure randomness in employer decision-making (e.g., salience bias for certain applications) or it captures elements about the text signal that we do not capture in our measure of signal, i.e., it contains measurement error.

Finally, we assume that employers exogenously abandon their job post, completely independent of the workers who apply to it, with probability $(1-\pi)$, where $\pi\in(0,1)$. This assumption helps rationalize why employers sometimes choose their outside options even when they have very good workers in their consideration sets.

Workers all simultaneously choose their actions after observing their own abilities $a_{ij}$, costs $c_{ij}$, and observable types $x_{ij}$ without observing anything about their competitors, including how many of them there are. Thus, the strategy of each worker $j$ applying to job post $i$ is a function

that maps their type $(c_{ij},a_{ij};x_{ij})$ to their actions $(b_{ij},e_{ij})$. Where useful, we define the functions $\sigma_{ij}^{b}(c,a;x)$ and $\sigma_{ij}^{e}(c,a;x)$ to denote the bid and effort components of worker strategies, respectively, such that $\sigma_{ij}(c,a;x)=\left(\sigma_{ij}^{b}(c,a;x),\sigma_{ij}^{e}(c,a;x)\right)$.

Each employer chooses which worker to hire, or the outside option, after observing only the bids, signals, observable characteristics of the workers in their consideration sets, as well as the utility shocks of those workers and the outside option. Thus, the strategy of each employer $i$ with consideration set $Q$ is a function

that maps the vector of considered bids $\mathbf{b}$, signals $\mathbf{s}$, observable characteristics $\mathbf{x}$, and utility shocks $\boldsymbol{\nu}$ to a discrete choice of which worker in the consideration set to hire, or the outside option.³³³³33The term $3\cdot|Q|-2$ in the domain of $\tau_{i}$ comes from the fact that the outside option has no bid or signal but does have a utility shock, i.e., we have $|Q|-1$ bids, $|Q|-1$ signals, and $|Q|$ utility shocks.

We restrict attention to type-symmetric pure-strategy Perfect Bayesian Equilibria (psPBE), which we define in our setting to be a worker strategy profile $\sigma^{*}$, an employer strategy profile $\tau^{*}$, and a system of beliefs³⁴³⁴34We omit discussion of off-path beliefs, and we further restrict our equilibrium concept to one in which all bids in the allowed range for the job posts in our empirical sample, i.e., $[\mathdollar 30,\mathdollar 250]$ are on-path. Moreover, we note that the signaling noise in each worker’s signal production yields a full support of on-path signals for any on-path effort. This feature of our model in combination with the assumption that employers do not observe worker effort implies beliefs need not be defined for off-path effort. for employers $\mu^{*}$ such that, for all possible realizations of worker types, competitor sets, consideration sets, signaling noise, and utility shocks:

1. 1.

   Employers form correct beliefs³⁵³⁵35Note that because employer utility (Equation 4) is linear in ability, the equilibrium only depends on the mean of employers’ posterior beliefs, rather than the whole posterior belief itself. according to Bayes’ rule:

   $\displaystyle\mu^{*}\left(b,s;x,\sigma^{*}\right)=\mathbbm{E}_{c,a,\varepsilon^{s}}\left[a\>\Big|\>\sigma^{*b}(c,a;x)=b,\>\>s\left(\sigma^{*e}(c,a;x);\varepsilon^{s},x\right)=s;\;x\right]$

2. 2.

   Employers decide whom to hire to maximize their expected utility:

   $\displaystyle\tau^{*}\left(\mathbf{b},\mathbf{s},\boldsymbol{\nu};\mathbf{x},Q\right)=\begin{cases}\displaystyle\operatorname*{arg\,max}_{k\in Q\setminus\{0\}}\left\{U^{E}\left(b_{k},\mu^{*}(b_{k},s_{k};x_{k},\sigma^{*}),\nu_{k};x_{k}\right)\right\}&\hskip-4.2679pt\text{if}\displaystyle\max_{k\in Q\setminus\{0\}}\left\{U^{E}\left(b_{k},\mu^{*}(b_{k},s_{k};x_{k},\sigma^{*}),\nu_{k};x_{k}\right)\right\}>\nu_{0}\\ 0&\hskip-4.2679pt\text{if otherwise}\end{cases}$

3. 3.

   Workers choose bids and efforts to maximize their expected utility:

   $\displaystyle\sigma^{*}\left(c,a;x\right)=\operatorname*{arg\,max}_{(b,e)\in\mathbb{R}^{2}_{+}}\left\{p\left(b,e;x,\sigma^{*},\tau^{*}\right)\cdot(b-c)-C(e;a)\right\},$

   where $p\left(b,e;x,\sigma,\tau\right)$ is the ex-ante probability that worker $j$ is hired:

   $$\mathbb{P}_{c_{-j},a_{-j},x_{-j},\boldsymbol{\varepsilon^{s}},\boldsymbol{\nu},Q}\left(\begin{aligned} &j\in Q\text{ and }\\ &j=\tau\left(b,\;\sigma^{b}_{-j}(c_{-j},a_{-j};x_{-j}),\;s(e;\varepsilon^{s}_{j},x),\;s_{-j}\left(c_{-j},a_{-j};\varepsilon^{s}_{-j},x_{-j},\sigma^{e}\right),\;\boldsymbol{\nu};\;x,\,x_{-j},\;Q\right)\end{aligned}\,\middle|\,b,e;\,x,\sigma,\tau\right)$$

   with

   	$\displaystyle\sigma^{b}_{-j}(c_{-j},a_{-j};x_{-j})\equiv\left(\sigma^{b}(c_{k},a_{k};x_{k})\right)_{k\in Q\setminus\{j,\,0\}},$
   	$\displaystyle s_{-j}\left(c_{-j},a_{-j};\varepsilon^{s}_{-j},x_{-j},\sigma^{e}\right)\equiv\left(s\left(\sigma^{e}(c_{k},a_{k};x_{k});\;\varepsilon^{s}_{k},x_{k}\right)\right)_{k\in Q\setminus\{j,\,0\}},$
   	$\displaystyle c_{-j}\equiv(c_{k})_{k\in Q\setminus\{j,\,0\}},\;a_{-j}\equiv(a_{k})_{k\in Q\setminus\{j,\,0\}},\;x_{-j}\equiv(x_{k})_{k\in Q\setminus\{j,\,0\}},\text{ and }\varepsilon^{s}_{-j}\equiv(\varepsilon^{s}_{k})_{k\in Q\setminus\{j,\,0\}}.$

Identifying the model poses two key challenges. First, to identify workers’ costs and abilities from their observed equilibrium bids and efforts, we must first identify their ex-ante equilibrium beliefs about their chances of being hired as functions of their bids, efforts, and observables.³⁶³⁶36For example, suppose we observe a worker submitting a high bid. Without identifying that worker’s beliefs about her probability of being hired, we cannot disentangle whether she is bidding high because she has a high cost of undertaking the job or because she believes she has a high chance of winning even with a high bid. Second, to identify employers’ preferences from their observed equilibrium choice probabilities, we must first identify their equilibrium beliefs about worker ability as functions of bids, signals, and observables.³⁷³⁷37For example, suppose we observe that employers are willing to hire workers with high bids. Without identifying employers’ beliefs about worker ability, we cannot disentangle whether this pattern is the result of employers being price insensitive or believing that high-bid workers have high ability. Additionally, suppose we see that employer demand is highly responsive to signals. Without identifying how employers form beliefs about ability as a function of those signals, we cannot disentangle whether this responsiveness is due to employers having a high WTP for ability or due to employers’ beliefs about ability being highly responsive to signals.

We overcome these two challenges sequentially by exploiting the information structure of our model. First, we identify worker beliefs from observed conditional hiring probabilities. Because we observe everything that is commonly observable to both workers and firms, a worker’s ex-ante belief about her probability of being hired is independent of her private type (cost and ability), conditional on her bid, effort, and observables. As a result, worker beliefs equal empirical hiring probabilities as functions of bids, efforts, and observables, and thus they are directly identified from data on employer choices. Once we have identified worker beliefs, we can invert workers’ first order conditions (FOCs) to infer their costs and abilities from their observed bids and efforts.

Second, having identified worker costs and abilities, and thus having identified the joint distribution of worker private types, actions, and observables, we can identify employer beliefs about ability as functions of bids, signals, and observables. These identified employer beliefs enable us to identify employer demand through the standard discrete choice arguments.

The remainder of this subsection presents the preceding argument in more detail.

This subsection describes how we estimate labor supply, which includes how we stratify our data by worker observables, how we estimate signal production functions, and how we simulate worker FOCs to recover worker costs and abilities.

To make our estimation of employer beliefs more feasible, we make the assumption that employers form beliefs about workers’ abilities based only on workers’ signals and observables, and not their bids.⁴⁶⁴⁶46This step is where our assumption that bids do not enter employer beliefs simplifies things the most. It is far easier to fit a nonparametric model of a conditional expectation, separately by observable group, in one dimension, rather than two. In principle, our procedure could be modified to allow for bid information to be included in employer beliefs, but this would either require placing more structure on these beliefs or coarsening worker observable groups even further. This assumption is not costless, but we believe it is a reasonable one in our empirical setting.⁴⁷⁴⁷47We argue the reasonability of this assumption in three points. First, the cognitive load employers would face in order to form beliefs based on two-dimensional worker strategies separately for each observable group is quite high, and thus it is plausible that employers would simplify their inference problem by focusing on the dimension of applications more saliently and directly related to ability. Second, given the wide distribution of worker’s countries of residence, employers may not have a good sense of how each worker’s bid relates to her opportunity cost, making it difficult for them to infer ability from bids. Third, anecdotally when reading how workers discuss application strategies on online forums, they rarely mention considering how their bid may be viewed as a signal of ability. Thus, we assume that employers’ equilibrium belief function takes the form $\mu^{*}(s;x)\equiv\mathbb{E}[a|s;x]$.

To estimate employer beliefs, we use the estimated joint distribution of worker types $\widehat{F}_{c\times a|x}$ to fit a flexible nonparametric model of the conditional expectation $\mathbb{E}[a|s;x]$ with the shape restriction that it is weakly increasing in $s$ for each $x$—a condition which our model’s equilibrium implies.

We operationalize this nonparametric estimation by first estimating an isotonic regression of ability on signal, separately by observable group $x$, and then fitting a Piecewise Cubic Hermite Interpolating Polynomial (PCHIP) to the isotonic regression. This approach allows us to capture data-driven non-linearities, while also ensuring that the estimated employer belief function is continuous, differentiable, and monotonically increasing.

Once we have estimated the employer belief function $\widehat{\mu}(s;x)$, we apply it to each application in our data to obtain pseudo-data on employer beliefs.⁴⁸⁴⁸48Note that while we could only estimate labor supply using the subset of workers with valid effort measurements, we can estimate labor demand using all applications in our data since we have observed bids and signals for all applications. We can then estimate labor demand parameters $\alpha$, $\beta$, and $T(x)_{x\in X}$ by maximizing the standard logit likelihood of observed employer choices as functions of bids, beliefs, and observables.

Table 5 presents summary statistics of the estimated marginal distributions of worker costs, abilities, and writing costs. To make our estimates of ability more interpretable, we present them in terms of dollars of employers’ willingness to pay (WTP) computed as $\frac{\widehat{\beta}}{|\widehat{\alpha}|}\cdot\widehat{a}$, where $\widehat{\beta}$ is the estimated coefficient on ability and $\widehat{\alpha}$ is the estimated coefficient on bid, both from estimated employer demand.

We find there to be substantial heterogeneity in both worker costs and abilities, with a range between P20 and P80 of $121.64 and $97.45 respectively. We estimate that over 25% of workers have negative costs, meaning they would be willing to pay to be hired. This finding almost surely arises from the fact that our model ignores dynamics, and thus we are not capturing a worker’s continuation value of being hired for a job, which may be particularly important for workers with low reputations who are trying to build up their on-platform reputation.

The wide range of estimated abilities highlights the importance of signaling in this market. Though Table 5 shows unconditional distributions of costs and abilities, we find that within-group variances of costs and abilities dwarf between-group variances with between group variances only explaining around 2% and 3% of the total variances of costs and abilities respectively. Thus, even conditional on observables, an employer hiring a worker without further information on ability could end up with a wide range of abilities.

Further highlighting the importance of signaling, Figure 10 presents a binscatter of estimated worker ability on signal, with 25th and 75th percentile bands of ability within each signal bin. This figure shows a strong positive correlation (0.547) between signals and abilities, and both the slope of ability on signal and the relatively tight interquartile range show that signals are good predictors of ability in the equilibrium being played in the data.

Finally, Figure 11 presents a binscatter of estimated worker cost on estimated worker ability, with 25th and 75th percentile bands of ability within each cost bin. This figure highlights a key feature of our estimates that impact our counterfactuals: worker costs and abilities are positively correlated. Even though the correlation is not particularly strong (0.193), this correlation generally implies that higher-ability workers also have higher costs. Thus, in a market without signaling, when price competition increases, higher-ability workers may be priced out of the market, when they may otherwise have been hired in a market in which they could signal their ability.

More detailed results on the estimated distributions of worker costs and abilities, equilibrium strategies, and signal production functions can be found in Appendix B.

Table 6 presents our main demand estimates. Our price coefficient generally aligns with our estimated price coefficient from our reduced-form demand estimation, the results of which are presented in Table 2. Our estimated coefficient on ability implies that a one standard deviation increase in ability, all else equal is valued at about $52.16 by employers, as our estimates showed in Table 5. Importantly, this estimate of the WTP for ability should be interpretted as specifically being the WTP for a standard deviation increase in the ability that can be signaled. As we made clear in Section 5, we cannot separate group-level non-signaled ability and taste-based group preferences in our estimates of $T(x)$.

Finally, note that one should only interpret the dollar value estimates of $T(x)$ relative to each other, or relative to the mean outside option value, which is noramlized to zero plus the dollar-valued mean of the T1EV distribution. Our estimates of $T(x)$ broadly fit our expectations from anecdotal evidence from the market. All else equal, employers prefer workers with high ratings over those with middle and low ratings, and they slightly prefer workers with poor ratings that have some experience over those that have no experience on the platform. Employers prefer workers who take their time to arrive at job posts, rather than those that arrive quickly upon the job being posted. Finally, employers prefer workers from non-majority-English-speaking European countries the most, then workers from the large “Other” grouping, then workers from English Speaking countries, and lastly they have a sharp dislike for workers from South Asia. Note that these $T(x)$ estimates are separate from the group level prior beliefs, i.e., before signaling, about ability, i.e., $\mathbb{E}[a|x]$.

In this subsection, we compare the hiring patterns of the NS and SQ equilibria to shed light on how losing access to signaling changes both hiring rates and the composition of hired workers. Broadly, we find that without signaling, as compared to the status quo, employers hire fewer high-ability workers and more low-ability workers, and they hire slightly fewer high-cost workers and slightly more low-cost workers. Accordingly, the market becomes more price-competitive. The average winning bid in the NS equilibrium is $102.68 as compared to $108.02 in the SQ equilibrium, a decrease of 4.94%. Figure 12 presents the distribution of winning bids in both the SQ and NS equilibria, showing that the distribution of winning bids in the NS equilibrium is shifted to the left, with a higher mass of winning bids at lower levels.

These changes to hiring patterns mostly harm workers and leave employers largely unaffected. As we can see in Figure 15, these changes lead to a 4% reduction in worker surplus, and a small, less than 1%, increase in employer surplus. Worker welfare losses are driven by both the modest extensive margin decrease in hiring rates, and the intensive margin decrease in wages. Their losses are mitigated both by the elimination of writing costs, and by the fact that the lower-ability workers tend to have lower costs. Employer surplus is virtually unaffected because the reduction in wages paid to workers roughly balances out the loss of hiring lower-ability workers. Overall, the market becomes less efficient, as total surplus falls by 1%, and significantly less meritocratic, as the new equilibrium favors lower-ability workers over higher ability ones.