Commit Graph

341 Commits (0aada2632e7a02fe5bcda3dd1d91cad9ca0b148d)

Author SHA1 Message Date
Karl-Aksel Puulmann a9d3c03782
Validate the same metric isnt queried multiple times in external stats API (#3871)
* Validate the same metric isnt queried multiple times in external stats API

Issue: https://3.basecamp.com/5308029/buckets/35611491/card_tables/cards/7161347855

* Changelog entry

* Make credo happy
3 years ago
Karl-Aksel Puulmann c6d98397a8
Move `add_percentage` logic into clickhouse (#3854)
* Remove `add_percentage`, calculate percentages in clickhouse queries

This simplifies querying logic and avoids doing extra queries and avoids
race conditions.

* Remove special none handling from breakdowns, handling percentages correctly

* Add (failing) test showing expected add_percentage behavior for user making multiple sessions

* Update add_percentage behavior to use separate subqueries
3 years ago
RobertJoonas 52f584efa9
Group `os_version` by `os` (#3806)
* improve test

* add os to os_version breakdown

...and add operating_system_versions.csv to the CSV export

* fix conversion rate for os_version breakdown

* update changelog

* fix existing CSV tests

* use case instead of cond
3 years ago
RobertJoonas d74b1d5e60
Reapply conversion rate into Stats API + bugfixes (#3805)
* Revert "Revert api conversion rate (#3789)"

This reverts commit 8e8790dd30.

* fix browser_version CR breakdown bug

* changelog bugfix

* inspect data structures before sending to sentry
3 years ago
RobertJoonas 8e8790dd30
Revert api conversion rate (#3789)
* Revert "Unify percentage change for CR and bounce_rate (#3781)"

This reverts commit a6b1a6ebc7.

* Revert "Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)"

This reverts commit 672d682e95.
3 years ago
RobertJoonas a6b1a6ebc7
Unify percentage change for CR and bounce_rate (#3781)
* Fix conversion rate change calculation

The change in conversion rate should be calculated similar to bounce rate.
For example, an increase of 25% -> 50% should not be a 100% change, but
a 25% change instead.

* Use the same comparison function in Stats API and dashboard API

This commit fixes a bug where the percentage change reported by the Stats
API is different from the one returned by the internal dashboard API.

* changelog update
3 years ago
RobertJoonas 672d682e95
Bring Stats API up to speed: Add `conversion_rate` to Aggregate and Breakdown (#3739)
* disable event metric with include_imported in every case

* add missing test for metric validation

* refactor metric validation functions

* implement conversion_rate metric validation

* move calculate_cr function into Stats.Util

* Refactor: Move aggregate CR logic into Stats.aggregate

* define atoms to exist

* Ensure that CR does not depend on visitors being queried

If 'visitors' are already queried, we'll use that value. Otherwise we'll
need to make another query to fetch it.

* confirm Stats API aggregate supports CR (tests only)

* small refactor

This is the only 'event_property' left after pattern matching on all
others in the function clauses defined above.

* Make it possible to optionally query conversion_rate

...in breakdown queries (excluding goal and custom prop breakdown)

* A little refactor asking for revenue metrics

1. The `@revenue_metrics` module attribute is an empty list on full build
   anyway
2. We don't need to query for revenue metrics if there are no revenue goals
   returned in the given query (even if revenue goals exist in site.goals)
3. Revenue metrics are already dropped in prop breakdown without a goal
   filter via (get_revenue_tracking_currency/3)

* Make it possible to optionally query conversion_rate (continuation)

... also from a custom prop and goal breakdown

* Frontend adjustments to the Locations report

* Display conversion rate in Regions and Cities (ListReport view)
* Display total conversions, conversions (visitors), and CR in the
  "Details" modals of Countries, Regions, and Cities
* Move the percentage into a separate column in the Countries details table

* confirm Stats API breakdown supports conversion_rate (tests only)

* small refactor: extract maybe_add_time_on_page function

* Make it possible to query cr alone

... (without the visitors metric). Already supported in aggregate, this
commit only implements it for the breakdown API.

* Reuse Stats.Util helper functions from b02db88 for aggregate API

We can follow the same logic as with breakdown for manually adding
`visitors` into the metrics list and taking it out of the response
later on.

That way we don't have to make another query, e.g. in a case where
only pageviews and conversion rate is queried. Also keeps things
consistent.

* changelog update

* fix test after resolving merge conflict

* Use explicit string->atom mapping instead of casting

* alias Util module instead of importing it

* use Enum.empty instead of Enum.any

* improve readability

* rename special_metrics to computed_metrics and explain with a comment

* rename visitors_without_event_filters to total_visitors

* keep a single function for removing unwanted metrics

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski f8b4d5066a
Add multiple imports per site (#3724)
* Clean up references to no longer active `google_analytics_imports` Oban queue

* Stub CSV importer

* Add SiteImport schema

* Rename `Plausible.Imported` module file to match module name

* Add `import_id` column to `Imported.*` CH schemas

* Implement Importer behavior and manage imports state using new entities

* Implement importer callbacks and maintain site.imported_data for UA

* Keep imports in sync when forgetting all imports

* Scope imported data queries to completed import IDs

* Mark newly imported data with respective import ID

* Clean up Importer implementation a bit

* Test querying legacy and new imported data

* Send Oban notifications on import worker failure too

* Fix checking for forgettable imports and remove redundant function

* Fix UA integration test

* Change site import source to atom enum and add source label

* Add typespecs and reduce repetition in `Plausible.Imported`

* Improve documentation and typespecs

* Add test for purging particular import

* Switch email notification templates depending on import source

* Document running import synchronously

* Fix UA importer args parsing and ensure it's covered by tests

* Clear `site.stats_start_date` on complete import to force recalculation

* Test Oban notifications (h/t @ruslandoga)

* Purge stats on import failure right away to reduce a chance of leaving debris behind

* Fix typos

Co-authored-by: hq1 <hq@mtod.org>

* Fix another typo

* Refactor fetching earliest import and earliest stats start date

* Use `Date.after?` instead of `Timex.after?`

* Cache import data in site virtual fields and limit queried imports to 5

* Ensure always current `stats_start_date` is used

* Work around broken typespec in Timex

* Make `SiteController.forget_imported` action idempotent

* Discard irrecoverably failed import tasks

* Use macros for site import statuses

There's also a fix ensuring only complete imports are considered
where relevant - couldn't isolate it as it was in a common hunk

* Use `import_id` as worker job uniqueness criterion

* Do not load imported stats data in plugins API context

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
hq1 99fe03701e
IP Block List (#3761)
* Add Ecto.Network dependency

* Migration: Add ip block list table

* If Cachex errors out, mark the cache as not ready

* Add IPRule schema

* Seed IPRules

* Add Shields context module

* Implement IPRuleCache

* Start IPRuleCache

* Drop blocklisted IPs on ingestion

* Cosmetic rename

* Add settings sidebar item

* Consider IPRuleCache readiness on health checks

* Fix typo

* Implement IP blocklist live view

* Update moduledocs

* Extend contextual module tests

* Convert IPRules LiveView into LiveComponent

* Keep live flashes on the tabs view

* Update changelog

* Format

* Credo

* Remove garbage

* Update drop reason typespecs

* Update typespecs for cache keys

* Keep track of who added a rule and when

* Test if adding via LV prefills the updated_by tooltip

* Update ecto_network dependency

* s/updated_by/added_by

* s/drop_blocklist_ip/drop_shield_rule_ip

* Add docs link

* s/Updated/Added
3 years ago
hq1 f5129f1b0d
Turn Revenue Goals into Custom Events if the plan doesn't support them (#3768)
* Turn Revenue Goals into Custom Events if the plan runs out

* Tag test with full_build
3 years ago
Adrian Gruntkowski 3738cd9578
Parse referrer filter value when passed separately (#3742)
* Parse referrer filter value when passed separately

* Remove unnecessary test setup
3 years ago
RobertJoonas 04b9067591
Add `event:goal` property to Stats API (#3718)
* add tests for filtering by goal in timeseries and aggregate

* refactor filter parsing

* stop returning custom props in event:goal breakdown

* test breaking down wildcard pageview goals

* extract filter utils

* parse more goal filter options

* add passing tests for new filter types

* do not allow querying session metrics with a goal filter

* remove unused page_match property

* test that non-configured goals are not returned in breakdown

* enforce filtered goals configured

* update changelog

* Allow simple filtering by revenue goals

This does not mean that revenue metrics are supported. If a revenue goal
is filtered by, we treat it like a simple custom event goal in the API.

* use List.wrap
3 years ago
RobertJoonas 403f559b35
Limit Custom Properties access in Stats API (#3670)
* add new function and doc for allowed_props

* limit props access in Stats API queries

* use dot syntax instead
3 years ago
hq1 4569ac09c1
Don't leak internal server errors, use support hash (#3661)
* Add Hahash dependency

* Don't leak internal server error details to the user

* Show the sinking shuttle notice whenever an API error occurs

* Don't render "No data yet" when there's a NetworkError for example

* Use ApiErrorNotice in funnels

* Display either hash or actual error message

The reason "internal-server-error" doesn't work well as a fallback
hash is that e.g. `NetworkError when attempting to fetch resource`
might be completely at client's fault. In such cases it's better
to display the whole thing still.

* Remove unused RocketIcon
3 years ago
RobertJoonas 1342135908
Add `custom_props.csv` back to CSV export for Growth plans (#3656)
* restore special props access for growth plans

* format

* refactor if condition

Co-authored-by: hq1 <hq@mtod.org>

* format

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
RobertJoonas d2270f3c35
Small bugfix + refactor email reports (#3642)
* use more convenient testing functions

* do not display + sign with 0% change in emails

* Rename module/file/function names

before, `weekly_report` was also used for monthly reports and that was a
bit confusing to read in code.

* Refactor send_email_report.ex

This commit improves readability by refactoring the code into smaller
functions and reducing the number of arguments given to functions.

But more importantly, it stops making duplicate stats queries for every
email recipient by moving the queries out of the for loop.

* Refactor: move querying logic out of the worker module

and merge all stats information under a single `stats` assign.
3 years ago
Adrian Gruntkowski 9d97dc1912
Move limit enforcement to accepting site ownership transfer (#3612)
* Move limit enforcement to accepting site ownerhsip transfer

* enforce pageview limit on ownership transfer accept

* Refactor plan limit check logic

* Extract `ensure_can_take_ownership` to `Invitations` context and refactor

* Improve styling of exceeded limits notice in invitation dialog and disable button

* styling improvements to notice

* make transfer_ownership return transfer to self error

* do not allow transferring to user without active subscription WIP

* Add missing typespec and improve existing ones

* Fix formatting

* Explicitly label direct match on function argument for clarity

* Slightly refactor `CreateInvitation.bulk_transfer_ownership_direct`

* Exclude quota enforcement tests from small build test suite

* Remove unused return type from `invite_error()` union type

* Do not block plan upgrade when there's pending ownership transfer

* Don't block and only warn about missing features on transfer

* Remove `x-init` attribute used for debugging

* Add tests for `Quota.monthly_pageview_usage/2`

* Test and improve site admin ownership transfer actions

* Extend tests for `AcceptInvitation.transfer_ownership`

* Test transfer ownership controller level accept action error cases

* Test choosing plan by user without sites but with a pending ownership transfer

* Test invitation x-data in sites LV

* Remove sitelocker trigger in invitation acceptance code and simplify logic

* Add Quota test for `user.allow_next_upgrade_override` being set

* ignore pageview limit only when subscribing to plan

* Use sandbox Paddle instance for staging

* Use sandbox paddle key for staging and dev

---------

Co-authored-by: Robert Joonas <robertjoonas16@gmail.com>
3 years ago
hq1 b96319321c
Add Browser/Version breakdown to CSV (#3599)
* Add csv for browser versions

* add changelog

* Fixup - ensure the dashboard browser version breakdown still works

* Update CHANGELOG

---------

Co-authored-by: Ekaterina Krivich <ekaterinak@heathmont.net>
3 years ago
RobertJoonas ff2c3346d2
Bugfix: Allow breakdown by internally used prop keys for Growth plans (#3562)
* fix bug - allow internally used prop key breakdown for Growth plans

* use case instead of with
3 years ago
hq1 88e1d9dc28
Small build updates (#3546)
* Sites API

* Extract Revenue react api helpers

* !fixup

* Extract JS Money module to /extra

* Extract Revenue full build extras (tests pass for full)

* Update MIX_ENV=small mix test

* Remove dead code

* Add moduledocs

* Add credo config

* Trick dialyzer

* DRY revenue metrics

* Use more concise version of on_full_build macro

* Disable credo check
3 years ago
hq1 b9ec38038c
Add small build option (#3536)
* Update applications

* Clone community config

* Move modules to experimental dir

* Update runtime config

* Apply first set of compile-time conditionals

* Move funnel schemas to experimental

* Make funnel schema-less build compile

* Use experimental/lib for elixir code

* Move JS funnels to experimental

* Clean up conditional rendering

* Tidy up the pipeline

* Make two builds pass tests without warnings

* Reuse existing dotenvs

* Do a bunch of renames

* Clean up naming

* Run secondary CI

* Update router

* Remove RewriteFunnelDupes migration

Tests were disabled already and it was a one-off shot

* Fixup quota mixins

* Add moduledoc

* Change MIX_ENV for seconary test run

* Skip crm on small

* !fixup

* Exclude flags pipeline

* Update lib/plausible_web/controllers/stats_controller.ex

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>

---------

Co-authored-by: Adrian Gruntkowski <adrian.gruntkowski@gmail.com>
3 years ago
Adrian Gruntkowski f464ceae88
Implement pinned sites (#3469)
* Revert "Remove site pins for now"

This reverts commit 5eccf4eaf6.

* Implement basic site pin schema level logic within user specific preferences

* Add vertical ellipsis menu markup

* Implement basic changesets for user preferences

* Implement pin toggling

* Try to fix pin sorting

* Implement pin toggling in LV

* Adjust moduledocs for new schema(s)

* Remove unnecessary `distinct` from query

* Use `button` for pin/unpin action

* Generalize preference setting

* Rename schema and fields for clarity

* Rename `list_type` -> `entry_type`

* Safeguard setting options

* Test `set_option/4` and `toggle_pin/2`

* Add test for listing pinned sites via `Sites.list`

* Disallow pinning sites outside page explicitly

* Test pinning in LV

* Test conditional rendering of site settings in /sites

* Remove unnecessary TODO comment

* Safeguard `Sites.set_option/4` against invalid user/site combo

* Handle pinned sites in dashboard site picker

* Clear flashes upon (un)pinning sites

* Update CHANGELOG

* Prevent blinking of hamburger menu items on first paint

* Highlight hamburger handle on hover in /sites

* Start showing hotkeys in site picker again

* Sort pinned sites in the order they were pinned

* Update sites list order immediately after pin/unpin toggle

* Refactor and split `Sites.list/3`, extracting `Sites.list_with_invitations/3`

* Cap number of pinned sites at 9 per user

* First pass on visual indication of site cards (dis)appearing

* Apply ellipsis gradient+shadow on card hover

* Fix responsive padding of site cards

* Sort by invitations first, pinned sites second and then the rest

* Revert "Apply ellipsis gradient+shadow on card hover"

This reverts commit 0608796612639030ccbb12df639709f78edc1434.

* Apply more subtle hover effect on the ellipsis menu

* Make error and success flash LV boxes use separate component containers

* Promote `pinned_at` in table migration to a column

* Switch logic to using `pinned_at` as a standard schema field

* Refactor `Sites.list*` getting rid of subquery (h/t @ukutaht)

* Remove migration which is already merged upstream

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
3 years ago
Vini Brasil c9bf5827e9
Prepare business tier for release (#3464)
* Change limits for trials

* Keep legacy trial limits for users that registered before the business tier

* Change private preview notice for release

* Run formatter

* Add countdown to private preview notice
3 years ago
Adrian Gruntkowski 07cab27fef
Implement new sites view (#3463)
* Implement complete basics of LV sites

* Reimplement everything in LV except pagination

* Implement basic search capability

* PoC: plot visitors on sites index

* Add rudimentary clipped gradient in minicharts

* Fix clipping gradient, define once

* Format

* Add moduledoc to visitors component

* Move paginator helpers to the top core namespace

* Fix typespec of `Plausible.Sites.list`

* Split sites component into subcomponents

* Add function to uniformly calculate 24h intervals
and visitor totals across multiple sites.

* Integrate batch 24h interval query with plots on sites view

* Don't confuse heex compiler with alpine @ shorthands

* Make linear gradient svg definition truly invisible

* Implement basic pagination

* Extract `site_stats` from site and invitation cards

* Improve pagination

* Tweak css

* Improve filtering on pagination and make WSS fail graceful

* Test `last_24h_visitors_hourly_intervals/2`

* Replace /sites with LV implementation

* Add debounce to search filter

* Fix typespecs

* Fix styling

* Fix mini graph scaling factor calculation

* Fix search consuming itself

* Minimal tweaks to the plots

* Fixup

* Remove magic numbers from the plot

* Create `site_pins` table

* Add `SitePin` schema

* Implement listing invitations, sites and pins in a single query

* Add FIXME note

* Remove site pins for now

* Add tests for `Plausible.Sites.list/3`

* Add a couple more tests to sites dead view

* Remove unnecessary FIXME

* Add LV tests for Sites

* Calculate and display 24h visitors change

* Render the change in bold

* Add clarfying comment on virtual field in `Site` schema

* Remove unnecessary function from Invitations API

* Remove unused list opt from type definition in `Sites`

* Improve joins in list query slightly

* Add comment on manually computing sites list total

* Start searching from a singly character in domain field

* Add typespec to `last_24h_visitors_hourly_intervals`

* Extend moduledoc in visitors component

* Simplify loading sites in LV

* Simplify assigns in LV

* Add missing group for shadow under site card

* Make invitation modal render

* Make HTML in sites LV semantically correct

* Remove autofocus and focus search on `/`

* Remove shadow from search input

* Make search cancel on escape

* Fix tests relying on outdated HTML structure

* Make visitor chart color scheme consistent with dashboard chart

* Update styling of trend labels

* Fix empty state and improve search blur/focus handling

* Use live navigation for pagination

* Implement spinner on load from search

* Remove unused `Plausible.Stats.Clickhouse.last_24h_visitors/1`

* Calculate uniques correctly across hour boundaries

* Swap inlined svg for Heroicons component in invitation modal

* Add order by to base query in 24h hourly intervals

* Revert "Add order by to base query in 24h hourly intervals"

This reverts commit a6be5e3026.

* Query clickhouse 24h visitors only on second mount

* Remove redundant sign from percentage change when negative

* Switch to offset-based pagination

  - offset seems easier to deal with for when actions on
    paginated list will be performed such as site pinning;
    tracking cursor data makes some entries disappear in
    edge cases. The data set is still fairly small and
    static, even for large customers.
  - we're removing Phoenix.Pagination as it doesn't really
    fir any use case, and it was only used to limit the number
    of sites in the site picker
  - site picker is now limited to 9 sites (future: pinned
    sites will be prioritized there)
  - no need to re-query for total count any more
  - BTW, the old /sites template was removed

* Refine the plot queries; Tests pass snapshot

* Add PromEx plugin for LiveView

* Fix tiny plot cut-off at the top

---------

Co-authored-by: Adam Rutkowski <hq@mtod.org>
3 years ago
Vini Brasil 896d78d8fd
Apply feature gates to dashboard queries (#3424)
* Read feature status from Billing.Feature instead of %Site{}

This commit changes data attributes passed to React. Previously the
controller read feature statuses directly from the %Site{} schema. The
Billing.Feature context is aware of the user plan and the features
available.

* Limit funnels internal API based on site owner plan

* Limit props internal API based on site owner plan

* Use site factory in QueryTest

* Limit custom property filter based on site owner plan

* Limit revenue goals queries based on site owner plan
3 years ago
Vini Brasil 303b3509f7
Feature gates (#3401)
This pull request implements limits to funnels, revenue goals and custom props based on the site owner plan. It extends the current "premium feature" notice to account for the new plans, trials and the on-going private preview. Stats API is not in the context of this pull request, but will be implemented likewise.
3 years ago
RobertJoonas 5ba21c04fe
Remove Props FF (#3242)
* stop returning prop_names from /conversions

* remove the old prop-breakdown API route and CSV export

* remove allowed_event_props from CRM

* remove DeprecatedConversions

* remove props FF

* update changelog
3 years ago
hq1 e59cda5d93
Fix filter suggestions (#3265)
* Fix filter suggestions

* Mute credo for now
3 years ago
hq1 bc650c213a
Fix breakdown pagination when filtering by goals (#3251)
* Add a failing test

* Bugfix: breakdown by goal pagination

* Update changelog

* Apply comment about comment
3 years ago
hq1 e467324c5a
Disallow small intervals on large ranges (#3245)
* Update interval dropdown based on custom range selection

* Add `monthsBetweenDates` function

* Reassign interval for custom "period" based on local storage

* Generate intervals by period based on stats_start_date

* Account for "custom" and "all" period intervals dynamically
3 years ago
Ёkaterina Krivich 9e26ea2cdf
Add `referrers.csv` to CSV export (#2624) (#3002)
* Add referrers to csv (#2624)

* Change CHANGELOG

* add missing files

* fix tests

* revert package files changes

* fix typo

* format

* rename
3 years ago
RobertJoonas 232bdd34a1
Conversions listreport (#3215)
* rename conversions.js to deprecated-conversions.js

* add a new Conversions component and switch by props FF

* add a Details view to the new Conversions component

* allow querying conversions with pagination

...and keep the current behaviour for DeprecatedConversions, always returning
page=1 and limit=100

* hide some columns on mobile

* prevent ListReport first column header overflow
3 years ago
RobertJoonas 8ac166b447
Add revenue metrics to Properties report (#3209)
* extract add_exit_rate function

* Change internal API metric names for /entry-pages & /exit-pages

* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,

This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.

This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.

* Change internal API metric names for /conversions

* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,

* return revenue metrics from /custom-prop-values (backend)

* be more explicit about which metric is plotted with Bar

* validate that ListReport input metrics actually exist in the API response

* display revenue metrics in the dashboard Properties section

* limit the number of columns shown on mobile

* add revenue metrics to Properties > Details

* review suggestions

* define hiddenOnMobile per metric instead of keeping the first 3

* rewrite if-else block

---------

Co-authored-by: Vini Brasil <vini@hey.com>
3 years ago
RobertJoonas 17b9e36a19
Refactor: Standardize API metrics used internally (#3208)
* extract add_exit_rate function

* Change internal API metric names for /entry-pages & /exit-pages

* `unique_entrances` -> `visitors`,
* `total_entrances` -> `visits`,
* `unique_exits` -> `visitors`,
* `total_exits` -> `visits`,

This is just a consistency improvement - the `visitors` metric always means
one thing and there's no need to call it different for entry pages internally.

This commit does not change any noticable behavior. The UI labels are kept
the same and the column headers in the CSV export will also remain the same.

* Change internal API metric names for /conversions

* `unique_conversions` -> `visitors`,
* `total_conversions` -> `events`,
3 years ago
RobertJoonas 7b39328d6c
Props details view (#3196)
* make (none) value in custom prop breakdown add +1 to pagination limit

This is needed because the (none) value is always added to the
breakdown_results **after** fetching those from ClickHouse.

* only include (none) values on the first page of results

* fix percentage metric calculation for paginated results

Instead of summing up the number of visitors from the breakdown results
to get the total, we have to make a separate query to `Stats.aggregate`.
Otherwise, the percentages for each results page will wrongly add up to 100%.

Since imported data for aggregated visitors and other properties (such as
browsers, OSs, etc) live in different tables, we have to tweak the tests to
also include the same number of visitors in the `imported_visitors` table.

* add details view for props

* changelog

* exclude imported data from total
3 years ago
RobertJoonas 51cad4c0ec
Props into CSV export (#3167)
* move set_allowed_event_props into context module

* extract parse_csv fn

* add custom_props.csv to CSV export (under FF)

* add test for custom prop export with a prop filter

* add test for goal filtered scenario

* mix format

* fix flaky test

* extract breakdown_custom_prop_values function

* Revert "fix flaky test"

This reverts commit 256b9bb2e7. The flaky
test was already fixed in 86557b1878

* make set_allowed_event_props a bang function

* test that all files are included in the CSV export
3 years ago
ruslandoga f3a1a9d599
derive jason encoder for request (#3182)
* derive jason encoder for request

* test didn't test

---------

Co-authored-by: hq1 <hq@mtod.org>
3 years ago
ruslandoga a9603a44c9
inspect request in sentry context (#3180) 3 years ago
RobertJoonas b9d122c0c7
Refactor: Use ListReport component in Sources (#3153)
* refactor SourceList to use ListReport

* refactor SourceList into a fn comp

* change referrer-drilldown API response format and remove dead code

* use ListReport in referrer-list

* fix CI

* fix flaky test

* remove IO.inspect
3 years ago
RobertJoonas 34fbc3d5bc
New private API to return custom prop values (#3111)
* copy prop_breakdown tests into a separate file

* add a new endpoint for custom props

Duplicate existing goal prop breakdown as a building base for the
new endpoint

* stick to original metric names + CR definitions

We currently use CR with two different definitions, which is inconsistent
and should be changed. This commit just documents the difference for the
time being.

* basic prop breakdown without goal filter

* increase % metric precision to one decimal place

* add some tests without goal filter

* silence credo for TODO comment

* use events metric instead of pageviews

* review feedback

* inline add_cr_a instead
3 years ago
Vini Brasil 35107e2b8f
Add revenue metrics to prop breakdown (#3140)
* Extract <Money> React component

* Unhide prop breakdown for revenue goals

* Query revenue metrics on prop breakdown API
3 years ago
Vini Brasil fd01a67a5f
Plot revenue metrics on main graph (#3112)
* Move money cast function to Stats.Util

* Add revenue metrics to main graph API

* Plot revenue metrics on the main graph

* Return plain numbers instead of a map

* Fix Credo issues

* Fix canMetricBeGraphed function

* Revert canMetricBeGraphed function changes
3 years ago
hq1 fea9bb32ee
Prevent robots from looking too much into public dashboards (#3118)
* Add noindex,nofollow to dashboard pages

* Implement NoRobots plug

* Enable NoRobots plug in the router

* Fixup internal route

* Fix double slash in the router

* Add special bot treatment to plausible.io live demo page

* Revert aggressive protection with agent detection
3 years ago
hq1 bbedeff683
Implement Funnels view on dashboard (#3066)
* Add Funnel react component

assets/js/dashboard/stats/behaviours/funnel.js - restored from:

98a76cbd Remove console.info calls
d94db99d Convert Funnel class component into a functional one
028036ad Review comments
3067a940 Stop doing maths in react
73407cc3 Fix error handling when local storage gets corrupted
e8c6fc52 Format numbers on funnel labels
c815709f Reorganize component responsibility
7a88fe44 Outline basic error handling
94caed7c Chart styling updates
4514608a Add percentages to funnel
d622c32d Add funnel picker

Co-authored-by: Uku Taht <uku.taht@gmail.com>

* Pass funnels list to react via data-funnels

* Implement Funnels react API

lib/plausible_web/controllers/api/stats_controller.ex - restored from:

f36ad234 Adjust to Plausible.Stats interface
9b532273 Test funnel stats controller
028036ad Review comments
bea3725f Remove IO.inspect
7a88fe44 Outline basic error handling
c8ae3eaf Move Funnels to StatsController and use base query
667cf222 Put private functions at the bottom

* Tweak funnel presentation

* Handle errors at the top

* Do not register DataLabels plugin globally

or else all the existing charts are affected

* Calculate drop-off percentage evaluating funnels

* Tweak dark mode + implement nicer tooltips

* Make currently selected funnel bold in the picker

* Count user_ids not session_ids when evaluating funnels

So if a visitor goes:

1. Start session
2. Complete funnel step 1
3. Inactive for 30 minutes
4. Complete funnel step 2

We would not be able to track this funnel completion because of the session timeout.
We like to o measure this as funnel completion even though the session expired in the middle.

cc @ukutaht

* Add extra properties to the funnels API

cc @ukutaht

* Improve tooltips so that step to data is rendered

* Change tooltip number formatting

* Remove debugging remnants

* Quick & dirty mobile view

* Fix mobile view: tweak dark mode & funnel switching

* Ignore DOMException: the operation was aborted

Otherwise this sometimes flashes the space shuttle
screen when navigating quickly via a keyboard.

* Format percentages on the main chart

* Close missing tag 🙈

* Revert "Close missing tag 🙈"

This reverts commit 9c2f970e22fd7e2980503242b414f42ce8bce1d2.

* Use jsx to render funnel tooltip

To get markup validated via lsp mostly...

* Fixup: s/class/className

* Fix className interpolation

* Add a ruler to the tooltip

* Tweak funnel chart style

* Fix font distortion issue on chart/canvas labels

* s/class/className

* Put "Set up funnels" link behind a feature flag

* Refactor internal selection storage

Getting ready for live funnel evaluation

* Don't try to connect LV socket if there's no CRSF token set up

This is perfectly okay for some of the templates/layouts.

* Fix up funnel creation typespecs

Unfortunately we can't define a type with literal string keys,
hence this must suffice.

* Use uniq over count/distinct

* Revert JSX in tooltips

Ref: https://github.com/plausible/analytics/pull/3066#discussion_r1241891155

* Remove the extra query for counting all visitors

cc @ukutaht

* Add premium notice

---------

Co-authored-by: Uku Taht <uku.taht@gmail.com>
3 years ago
Vini Brasil b97290b5cf
Add revenue metrics to top stats (#3059)
This commit adds revenue data to top stats. Average and total are displayed when filtering by a revenue goal (or many if they have the same currency set).
3 years ago
hq1 d8543c81cc
Funnel site settings (#3039)
* Update formatter config

* Install LiveView JS integration & hooks

* Temporarily update endpoint/session config

* Optionally allow preloading funnels for goals

* Site controller

* Implement funnel settings

lib/plausible_web/live/funnel_settings/combo_box.ex - restored from:

054de6e2 Fix the tab/blur bug again
20da4c89 Rename InputPicker to ComboBox

lib/plausible_web/live/funnel_settings/form.ex - restored from:

9bedda3b Remove potential FIXME
20da4c89 Rename InputPicker to ComboBox
028036ad Review comments
aea4ebc4 Access Funnel min/max steps via the __using__/1 macro
0dde27fd Remove inspect call
eed588a7 Start testing the funnel editor
0e95228b Extract funnel settings test module
7b16ace5 Leverage aplinejs to deal with the tyranny
8dc6a3e7 wip
cf228630 wip
30a43fd1 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
7690d50f wip
639c6238 fixup
aa59adeb wip
ff75c00b wip

lib/plausible_web/live/funnel_settings/list.ex - restored from:

4eae122c Fix data-confirm attr interpolation
51f0397d Implement deleting funnels
1f6fe25d Add number of steps to funnels list
298a6a53 wip
ff75c00b wip

test/plausible_web/live/funnel_settings/funnel_settings/combo_box_test.exs - restored from:

20da4c89 Rename InputPicker to ComboBox

test/plausible_web/live/funnel_settings/funnel_settings_test.exs - restored from:

34822ff4 Bootstrap InputPicker tests

lib/plausible_web/live/funnel_settings.ex - restored from:

028036ad Review comments
acd9c4f2 Prepare ephemeral funnel definitions so that users can test funnels
51f0397d Implement deleting funnels
0e95228b Extract funnel settings test module
8dc6a3e7 wip
89f10ecb wip
950a18d9 Dirty funnel save
298a6a53 wip
aa59adeb wip
ff75c00b wip

test/plausible_web/controllers/error_report_controller_test.exs - restored from:

34822ff4 Bootstrap InputPicker tests

test/support/html.ex - restored from:

0a53979d Improve InputPicker tests - include AlpineJS assertions
34822ff4 Bootstrap InputPicker tests

lib/plausible_web/views/layout_view.ex - restored from:

b490403b !ifxup

lib/plausible_web/templates/site/settings_funnels.html.eex - restored from:

51f0397d Implement deleting funnels
ea1315f3 Test funnels list in settings
7b16ace5 Leverage aplinejs to deal with the tyranny
ff75c00b wip
4da25c35 Fixup

lib/plausible_web/templates/layout/app.html.eex - restored from:

ff75c00b wip

* Add funnel settings route

* Warn about funnels deletion when deleting goals

lib/plausible_web/templates/site/settings_goals.html.eex - restored from:

fdd9bcd0 Fixup
f1e6364d Merge remote-tracking branch 'origin/master' into funnels-rebase
9d0b7c6d Fix markup error
4a4ddbdc Optionally preload funnels for goals and stub funnel-goal deletion
ebdc4333 Extend the prompt in case of funnel-goal deletion
639c6238 fixup
aa59adeb wip

* Split new JS LiveView additions

* Put funnels behind a feature flag

* Integrate dashboard feature toggle

* Update signing salt for live view

* Update moduledocs

* Update live reloader config

* Use Phoenix.HTML.Safe for goal names

* Workaround to get flashes working in embedded liveview

* Keep feature toggles idempotent, rename property to setting

We'll still retain the ability to flip bools on a lower level.

* Update moduledocs

* Make live flash disappear after 5s

* Tailwind: purge .heex files too

* Update docs link

* Add live components to tailwind purge config

* Update another flaky test

Ref f0bdf872
cc @vinibrsl

* Fix combobox input length w/ WebKit

* Intoduce generic notice component

* Revert "Fix combobox input length w/ WebKit"

This reverts commit 3c653a6d85d5000167631e10ef45a93c13b41ed1.

* Fix combobox input length on webkit

* Make whole combobox item clickable, not only text

* Fix glitch moving Save button on activation

* Tweak dark mode

* Show funnel form without waiting for funnel name input

* Tweak dark mode

* Include static Phoenix components in tailwind purge

* Tune funnels form into a liveview of its own

This is so that ComboBoxes can publish their selections
and unavailable choices can be propagated to other siblings.

* Push less data over websocket

* Undo Lsp/formatter race condition

* Fixup typespecs

* Bust CI cache
3 years ago
Vini Brasil 2c4cea6dce
Add revenue metrics in StatsController instead of Stats.Breakdown (#3028)
This commit makes total_revenue and average_revenue first-class metrics
instead of adding them magically in `Plausible.Stats.Breakdown`. To
keep the same behavior, revenue metrics are added in StatsController.

Related: https://github.com/plausible/analytics/pull/2957#pullrequestreview-1475017620
3 years ago
hq1 c7b8a8ef27
Bugfix for #3005 (#3025)
* Revert "Revert "UI groundwork: Conversions to Behaviors (#3005)" (#3024)"

This reverts commit dc0853bac7.

* Move ref back to LazyLoader container
3 years ago
hq1 dc0853bac7
Revert "UI groundwork: Conversions to Behaviors (#3005)" (#3024)
This reverts commit ba19f9530e.
3 years ago
RobertJoonas ba19f9530e
UI groundwork: Conversions to Behaviors (#3005)
* copy relevant files from b2ace16540

* make it work and set site.funnels to empty list

* make Behaviours a functional component

* add UI for a setup hint and always display conversions by default

* cherry-pick migration commit

* update site schema with new fields

* backend: implement disable-feature action

* switch between tabs in the behaviors section

Introduces template components to build props and funnels on. Both
only show a setup notice atm, and both are behind feature flags.

* extend API for disabling props and funnels

* render feature setup note directly from the Behaviours component

* fix UI behavior when features are hidden

* update setup notices

* add conversions feature switch to Site Settings > Goals

* mix format

* remove IO.inspect

* change setup notice - use buttons + popup confirmation

* optimize for light mode

* restrict access to setup notices

* some styling improvements

* allow super-admins to enable/disable features

* only show conversions (last 30min) in realtime mode

* use shorter display names for tabs

* optimize for mobile screens

* note about sending custom events

* changelog update + fix CI

* change HTTP verb for the disable-feature action

* change UI label for show/hide goals
3 years ago