E: balasub@cis.ohio-state.edu
D: Wrote SYS V IPC (part of standard kernel since 0.99.10)
+N: Chris Ball
+E: chris@printf.net
+D: Former maintainer of the MMC/SD/SDIO subsystem.
+
N: Dario Ballabio
E: ballabio_dario@emc.com
E: dario.ballabio@tiscalinet.it
--- /dev/null
+Code of Conflict
+----------------
+
+The Linux kernel development effort is a very personal process compared
+to "traditional" ways of developing software. Your code and ideas
+behind it will be carefully reviewed, often resulting in critique and
+criticism. The review will almost always require improvements to the
+code before it can be included in the kernel. Know that this happens
+because everyone involved wants to see the best possible solution for
+the overall success of Linux. This development process has been proven
+to create the most robust operating system kernel ever, and we do not
+want to do anything to cause the quality of submission and eventual
+result to ever decrease.
+
+If however, anyone feels personally abused, threatened, or otherwise
+uncomfortable due to this process, that is not acceptable. If so,
+please contact the Linux Foundation's Technical Advisory Board at
+<tab@lists.linux-foundation.org>, or the individual members, and they
+will work to resolve the issue to the best of their ability. For more
+information on who is on the Technical Advisory Board and what their
+role is, please see:
+ http://www.linuxfoundation.org/programs/advisory-councils/tab
+
+As a reviewer of code, please strive to keep things civil and focused on
+the technical issues involved. We are all humans, and frustrations can
+be high on both sides of the process. Try to keep in mind the immortal
+words of Bill and Ted, "Be excellent to each other."
8. Vendor and device identifications
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
-One is not required to add new device ids to include/linux/pci_ids.h.
-Please add PCI_VENDOR_ID_xxx for vendors and a hex constant for device ids.
+Do not add new device or vendor IDs to include/linux/pci_ids.h unless they
+are shared across multiple drivers. You can add private definitions in
+your driver if they're helpful, or just use plain hex constants.
-PCI_VENDOR_ID_xxx constants are re-used. The device ids are arbitrary
-hex numbers (vendor controlled) and normally used only in a single
-location, the pci_device_id table.
+The device IDs are arbitrary hex numbers (vendor controlled) and normally used
+only in a single location, the pci_device_id table.
-Please DO submit new vendor/device ids to pciids.sourceforge.net project.
+Please DO submit new vendor/device IDs to http://pciids.sourceforge.net/.
APEI Error INJection
~~~~~~~~~~~~~~~~~~~~
-EINJ provides a hardware error injection mechanism
-It is very useful for debugging and testing of other APEI and RAS features.
+EINJ provides a hardware error injection mechanism. It is very useful
+for debugging and testing APEI and RAS features in general.
-To use EINJ, make sure the following are enabled in your kernel
+You need to check whether your BIOS supports EINJ first. For that, look
+for early boot messages similar to this one:
+
+ACPI: EINJ 0x000000007370A000 000150 (v01 INTEL 00000001 INTL 00000001)
+
+which shows that the BIOS is exposing an EINJ table - it is the
+mechanism through which the injection is done.
+
+Alternatively, look in /sys/firmware/acpi/tables for an "EINJ" file,
+which is a different representation of the same thing.
+
+It doesn't necessarily mean that EINJ is not supported if those above
+don't exist: before you give up, go into BIOS setup to see if the BIOS
+has an option to enable error injection. Look for something called WHEA
+or similar. Often, you need to enable an ACPI5 support option prior, in
+order to see the APEI,EINJ,... functionality supported and exposed by
+the BIOS menu.
+
+To use EINJ, make sure the following are options enabled in your kernel
configuration:
CONFIG_DEBUG_FS
CONFIG_ACPI_APEI
CONFIG_ACPI_APEI_EINJ
-The user interface of EINJ is debug file system, under the
-directory apei/einj. The following files are provided.
+The EINJ user interface is in <debugfs mount point>/apei/einj.
+
+The following files belong to it:
- available_error_type
- Reading this file returns the error injection capability of the
- platform, that is, which error types are supported. The error type
- definition is as follow, the left field is the error type value, the
- right field is error description.
-
- 0x00000001 Processor Correctable
- 0x00000002 Processor Uncorrectable non-fatal
- 0x00000004 Processor Uncorrectable fatal
- 0x00000008 Memory Correctable
- 0x00000010 Memory Uncorrectable non-fatal
- 0x00000020 Memory Uncorrectable fatal
- 0x00000040 PCI Express Correctable
- 0x00000080 PCI Express Uncorrectable fatal
- 0x00000100 PCI Express Uncorrectable non-fatal
- 0x00000200 Platform Correctable
- 0x00000400 Platform Uncorrectable non-fatal
- 0x00000800 Platform Uncorrectable fatal
-
- The format of file contents are as above, except there are only the
- available error type lines.
+
+ This file shows which error types are supported:
+
+ Error Type Value Error Description
+ ================ =================
+ 0x00000001 Processor Correctable
+ 0x00000002 Processor Uncorrectable non-fatal
+ 0x00000004 Processor Uncorrectable fatal
+ 0x00000008 Memory Correctable
+ 0x00000010 Memory Uncorrectable non-fatal
+ 0x00000020 Memory Uncorrectable fatal
+ 0x00000040 PCI Express Correctable
+ 0x00000080 PCI Express Uncorrectable fatal
+ 0x00000100 PCI Express Uncorrectable non-fatal
+ 0x00000200 Platform Correctable
+ 0x00000400 Platform Uncorrectable non-fatal
+ 0x00000800 Platform Uncorrectable fatal
+
+ The format of the file contents are as above, except present are only
+ the available error types.
- error_type
- This file is used to set the error type value. The error type value
- is defined in "available_error_type" description.
+
+ Set the value of the error type being injected. Possible error types
+ are defined in the file available_error_type above.
- error_inject
- Write any integer to this file to trigger the error
- injection. Before this, please specify all necessary error
- parameters.
+
+ Write any integer to this file to trigger the error injection. Make
+ sure you have specified all necessary error parameters, i.e. this
+ write should be the last step when injecting errors.
- flags
- Present for kernel version 3.13 and above. Used to specify which
- of param{1..4} are valid and should be used by BIOS during injection.
- Value is a bitmask as specified in ACPI5.0 spec for the
+
+ Present for kernel versions 3.13 and above. Used to specify which
+ of param{1..4} are valid and should be used by the firmware during
+ injection. Value is a bitmask as specified in ACPI5.0 spec for the
SET_ERROR_TYPE_WITH_ADDRESS data structure:
- Bit 0 - Processor APIC field valid (see param3 below)
- Bit 1 - Memory address and mask valid (param1 and param2)
- Bit 2 - PCIe (seg,bus,dev,fn) valid (param4 below)
- If set to zero, legacy behaviour is used where the type of injection
- specifies just one bit set, and param1 is multiplexed.
+
+ Bit 0 - Processor APIC field valid (see param3 below).
+ Bit 1 - Memory address and mask valid (param1 and param2).
+ Bit 2 - PCIe (seg,bus,dev,fn) valid (see param4 below).
+
+ If set to zero, legacy behavior is mimicked where the type of
+ injection specifies just one bit set, and param1 is multiplexed.
- param1
- This file is used to set the first error parameter value. Effect of
- parameter depends on error_type specified. For example, if error
- type is memory related type, the param1 should be a valid physical
- memory address. [Unless "flag" is set - see above]
+
+ This file is used to set the first error parameter value. Its effect
+ depends on the error type specified in error_type. For example, if
+ error type is memory related type, the param1 should be a valid
+ physical memory address. [Unless "flag" is set - see above]
- param2
- This file is used to set the second error parameter value. Effect of
- parameter depends on error_type specified. For example, if error
- type is memory related type, the param2 should be a physical memory
- address mask. Linux requires page or narrower granularity, say,
- 0xfffffffffffff000.
+
+ Same use as param1 above. For example, if error type is of memory
+ related type, then param2 should be a physical memory address mask.
+ Linux requires page or narrower granularity, say, 0xfffffffffffff000.
- param3
- Used when the 0x1 bit is set in "flag" to specify the APIC id
+
+ Used when the 0x1 bit is set in "flags" to specify the APIC id
- param4
- Used when the 0x4 bit is set in "flag" to specify target PCIe device
+ Used when the 0x4 bit is set in "flags" to specify target PCIe device
- notrigger
- The EINJ mechanism is a two step process. First inject the error, then
- perform some actions to trigger it. Setting "notrigger" to 1 skips the
- trigger phase, which *may* allow the user to cause the error in some other
- context by a simple access to the cpu, memory location, or device that is
- the target of the error injection. Whether this actually works depends
- on what operations the BIOS actually includes in the trigger phase.
-
-BIOS versions based in the ACPI 4.0 specification have limited options
-to control where the errors are injected. Your BIOS may support an
-extension (enabled with the param_extension=1 module parameter, or
-boot command line einj.param_extension=1). This allows the address
-and mask for memory injections to be specified by the param1 and
-param2 files in apei/einj.
-
-BIOS versions using the ACPI 5.0 specification have more control over
-the target of the injection. For processor related errors (type 0x1,
-0x2 and 0x4) the APICID of the target should be provided using the
-param1 file in apei/einj. For memory errors (type 0x8, 0x10 and 0x20)
-the address is set using param1 with a mask in param2 (0x0 is equivalent
-to all ones). For PCI express errors (type 0x40, 0x80 and 0x100) the
-segment, bus, device and function are specified using param1:
+
+ The error injection mechanism is a two-step process. First inject the
+ error, then perform some actions to trigger it. Setting "notrigger"
+ to 1 skips the trigger phase, which *may* allow the user to cause the
+ error in some other context by a simple access to the CPU, memory
+ location, or device that is the target of the error injection. Whether
+ this actually works depends on what operations the BIOS actually
+ includes in the trigger phase.
+
+BIOS versions based on the ACPI 4.0 specification have limited options
+in controlling where the errors are injected. Your BIOS may support an
+extension (enabled with the param_extension=1 module parameter, or boot
+command line einj.param_extension=1). This allows the address and mask
+for memory injections to be specified by the param1 and param2 files in
+apei/einj.
+
+BIOS versions based on the ACPI 5.0 specification have more control over
+the target of the injection. For processor-related errors (type 0x1, 0x2
+and 0x4), you can set flags to 0x3 (param3 for bit 0, and param1 and
+param2 for bit 1) so that you have more information added to the error
+signature being injected. The actual data passed is this:
+
+ memory_address = param1;
+ memory_address_range = param2;
+ apicid = param3;
+ pcie_sbdf = param4;
+
+For memory errors (type 0x8, 0x10 and 0x20) the address is set using
+param1 with a mask in param2 (0x0 is equivalent to all ones). For PCI
+express errors (type 0x40, 0x80 and 0x100) the segment, bus, device and
+function are specified using param1:
31 24 23 16 15 11 10 8 7 0
+-------------------------------------------------+
| segment | bus | device | function | reserved |
+-------------------------------------------------+
-An ACPI 5.0 BIOS may also allow vendor specific errors to be injected.
+Anyway, you get the idea, if there's doubt just take a look at the code
+in drivers/acpi/apei/einj.c.
+
+An ACPI 5.0 BIOS may also allow vendor-specific errors to be injected.
In this case a file named vendor will contain identifying information
from the BIOS that hopefully will allow an application wishing to use
-the vendor specific extension to tell that they are running on a BIOS
+the vendor-specific extension to tell that they are running on a BIOS
that supports it. All vendor extensions have the 0x80000000 bit set in
error_type. A file vendor_flags controls the interpretation of param1
and param2 (1 = PROCESSOR, 2 = MEMORY, 4 = PCI). See your BIOS vendor
documentation for details (and expect changes to this API if vendors
creativity in using this feature expands beyond our expectations).
-Example:
+
+An error injection example:
+
# cd /sys/kernel/debug/apei/einj
# cat available_error_type # See which errors can be injected
0x00000002 Processor Uncorrectable non-fatal
0x00000008 Memory Correctable
0x00000010 Memory Uncorrectable non-fatal
# echo 0x12345000 > param1 # Set memory address for injection
-# echo 0xfffffffffffff000 > param2 # Mask - anywhere in this page
+# echo $((-1 << 12)) > param2 # Mask 0xfffffffffffff000 - anywhere in this page
# echo 0x8 > error_type # Choose correctable memory error
# echo 1 > error_inject # Inject now
+You should see something like this in dmesg:
+
+[22715.830801] EDAC sbridge MC3: HANDLING MCE MEMORY ERROR
+[22715.834759] EDAC sbridge MC3: CPU 0: Machine Check Event: 0 Bank 7: 8c00004000010090
+[22715.834759] EDAC sbridge MC3: TSC 0
+[22715.834759] EDAC sbridge MC3: ADDR 12345000 EDAC sbridge MC3: MISC 144780c86
+[22715.834759] EDAC sbridge MC3: PROCESSOR 0:306e7 TIME 1422553404 SOCKET 0 APIC 0
+[22716.616173] EDAC MC3: 1 CE memory read error on CPU_SrcID#0_Channel#0_DIMM#0 (channel:0 slot:0 page:0x12345 offset:0x0 grain:32 syndrome:0x0 - area:DRAM err_code:0001:0090 socket:0 channel_mask:1 rank:0)
For more information about EINJ, please refer to ACPI specification
version 4.0, section 17.5 and ACPI 5.0, section 18.6.
be understood as an underflow into the highest possible value, -2 or
-10M etc. do not work, so it's not consistent.
- memory.low, memory.high, and memory.max will use the string
- "infinity" to indicate and set the highest possible value.
+ memory.low, memory.high, and memory.max will use the string "max" to
+ indicate and set the highest possible value.
5. Planned Changes
- pclkN, clkN: Pairs of parent of input clock and input clock to the
devices in this power domain. Maximum of 4 pairs (N = 0 to 3)
are supported currently.
+- power-domains: phandle pointing to the parent power domain, for more details
+ see Documentation/devicetree/bindings/power/power_domain.txt
Node of a device using power domains must have a power-domains property
defined with a phandle to respective power domain.
--- /dev/null
+Freescale Vybrid Miscellaneous System Control - CPU Configuration
+
+The MSCM IP contains multiple sub modules, this binding describes the first
+block of registers which contains CPU configuration information.
+
+Required properties:
+- compatible: "fsl,vf610-mscm-cpucfg", "syscon"
+- reg: the register range of the MSCM CPU configuration registers
+
+Example:
+ mscm_cpucfg: cpucfg@40001000 {
+ compatible = "fsl,vf610-mscm-cpucfg", "syscon";
+ reg = <0x40001000 0x800>;
+ }
--- /dev/null
+Freescale Vybrid Miscellaneous System Control - Interrupt Router
+
+The MSCM IP contains multiple sub modules, this binding describes the second
+block of registers which control the interrupt router. The interrupt router
+allows to configure the recipient of each peripheral interrupt. Furthermore
+it controls the directed processor interrupts. The module is available in all
+Vybrid SoC's but is only really useful in dual core configurations (VF6xx
+which comes with a Cortex-A5/Cortex-M4 combination).
+
+Required properties:
+- compatible: "fsl,vf610-mscm-ir"
+- reg: the register range of the MSCM Interrupt Router
+- fsl,cpucfg: The handle to the MSCM CPU configuration node, required
+ to get the current CPU ID
+- interrupt-controller: Identifies the node as an interrupt controller
+- #interrupt-cells: Two cells, interrupt number and cells.
+ The hardware interrupt number according to interrupt
+ assignment of the interrupt router is required.
+ Flags get passed only when using GIC as parent. Flags
+ encoding as documented by the GIC bindings.
+- interrupt-parent: Should be the phandle for the interrupt controller of
+ the CPU the device tree is intended to be used on. This
+ is either the node of the GIC or NVIC controller.
+
+Example:
+ mscm_ir: interrupt-controller@40001800 {
+ compatible = "fsl,vf610-mscm-ir";
+ reg = <0x40001800 0x400>;
+ fsl,cpucfg = <&mscm_cpucfg>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
+ interrupt-parent = <&intc>;
+ }
regions, used when the GIC doesn't have banked registers. The offset is
cpu-offset * cpu-nr.
-- arm,routable-irqs : Total number of gic irq inputs which are not directly
- connected from the peripherals, but are routed dynamically
- by a crossbar/multiplexer preceding the GIC. The GIC irq
- input line is assigned dynamically when the corresponding
- peripheral's crossbar line is mapped.
Example:
intc: interrupt-controller@fff11000 {
#interrupt-cells = <3>;
#address-cells = <1>;
interrupt-controller;
- arm,routable-irqs = <160>;
reg = <0xfff11000 0x1000>,
<0xfff10100 0x100>;
};
Required properties:
- compatible : Should be "ti,irq-crossbar"
- reg: Base address and the size of the crossbar registers.
-- ti,max-irqs: Total number of irqs available at the interrupt controller.
+- interrupt-controller: indicates that this block is an interrupt controller.
+- interrupt-parent: the interrupt controller this block is connected to.
+- ti,max-irqs: Total number of irqs available at the parent interrupt controller.
- ti,max-crossbar-sources: Maximum number of crossbar sources that can be routed.
- ti,reg-size: Size of a individual register in bytes. Every individual
register is assumed to be of same size. Valid sizes are 1, 2, 4.
when the interrupt controller irq is unused (when not provided, default is 0)
Examples:
- crossbar_mpu: @4a020000 {
+ crossbar_mpu: crossbar@4a002a48 {
compatible = "ti,irq-crossbar";
reg = <0x4a002a48 0x130>;
ti,max-irqs = <160>;
ti,max-crossbar-sources = <400>;
ti,reg-size = <2>;
- ti,irqs-reserved = <0 1 2 3 5 6 131 132 139 140>;
+ ti,irqs-reserved = <0 1 2 3 5 6 131 132>;
ti,irqs-skip = <10 133 139 140>;
};
An interrupt consumer on an SoC using crossbar will use:
interrupts = <GIC_SPI request_number interrupt_level>
-When the request number is between 0 to that described by
-"ti,max-crossbar-sources", it is assumed to be a crossbar mapping. If the
-request_number is greater than "ti,max-crossbar-sources", then it is mapped as a
-quirky hardware mapping direct to GIC.
Example:
device_x@0x4a023000 {
interrupts = <GIC_SPI 8 IRQ_TYPE_LEVEL_HIGH>;
...
};
-
- device_y@0x4a033000 {
- /* Direct mapped GIC SPI 1 used */
- interrupts = <GIC_SPI DIRECT_IRQ(1) IRQ_TYPE_LEVEL_HIGH>;
- ...
- };
- clocks : list of phandles and specifiers to all input clocks listed in
clock-names property.
+Optional properties:
+
+Some PMUs are capable of behaving as an interrupt controller (mostly
+to wake up a suspended PMU). In which case, they can have the
+following properties:
+
+- interrupt-controller: indicate that said PMU is an interrupt controller
+
+- #interrupt-cells: must be identical to the that of the parent interrupt
+ controller.
+
+- interrupt-parent: a phandle indicating which interrupt controller
+ this PMU signals interrupts to.
+
Example :
pmu_system_controller: system-controller@10040000 {
compatible = "samsung,exynos5250-pmu", "syscon";
reg = <0x10040000 0x5000>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
#clock-cells = <1>;
clock-names = "clkout0", "clkout1", "clkout2", "clkout3",
"clkout4", "clkout8", "clkout9";
Required root node property:
compatible = "st,stih407";
+Boards with the ST STiH410 SoC shall have the following properties:
+Required root node property:
+compatible = "st,stih410";
+
Boards with the ST STiH418 SoC shall have the following properties:
Required root node property:
compatible = "st,stih418";
This binding describes a SATA device.
Required properties:
- - compatible : Must be "st,sti-ahci"
+ - compatible : Must be "st,ahci"
- reg : Physical base addresses and length of register sets
- interrupts : Interrupt associated with the SATA device
- interrupt-names : Associated name must be; "hostc"
- - resets : The power-down and soft-reset lines of SATA IP
- - reset-names : Associated names must be; "pwr-dwn" and "sw-rst"
- clocks : The phandle for the clock
- clock-names : Associated name must be; "ahci_clk"
- - phys : The phandle for the PHY device
+ - phys : The phandle for the PHY port
- phy-names : Associated name must be; "ahci_phy"
+Optional properties:
+ - resets : The power-down, soft-reset and power-reset lines of SATA IP
+ - reset-names : Associated names must be; "pwr-dwn", "sw-rst" and "pwr-rst"
+
Example:
+ /* Example for stih416 */
sata0: sata@fe380000 {
- compatible = "st,sti-ahci";
- reg = <0xfe380000 0x1000>;
- interrupts = <GIC_SPI 157 IRQ_TYPE_NONE>;
- interrupt-names = "hostc";
- phys = <&miphy365x_phy MIPHY_PORT_0 MIPHY_TYPE_SATA>;
- phy-names = "ahci_phy";
- resets = <&powerdown STIH416_SATA0_POWERDOWN>,
+ compatible = "st,ahci";
+ reg = <0xfe380000 0x1000>;
+ interrupts = <GIC_SPI 157 IRQ_TYPE_NONE>;
+ interrupt-names = "hostc";
+ phys = <&phy_port0 PHY_TYPE_SATA>;
+ phy-names = "ahci_phy";
+ resets = <&powerdown STIH416_SATA0_POWERDOWN>,
<&softreset STIH416_SATA0_SOFTRESET>;
- reset-names = "pwr-dwn", "sw-rst";
- clocks = <&clk_s_a0_ls CLK_ICN_REG>;
- clock-names = "ahci_clk";
+ reset-names = "pwr-dwn", "sw-rst";
+ clocks = <&clk_s_a0_ls CLK_ICN_REG>;
+ clock-names = "ahci_clk";
+ };
+
+ /* Example for stih407 family silicon */
+ sata0: sata@9b20000 {
+ compatible = "st,ahci";
+ reg = <0x9b20000 0x1000>;
+ interrupts = <GIC_SPI 159 IRQ_TYPE_NONE>;
+ interrupt-names = "hostc";
+ phys = <&phy_port0 PHY_TYPE_SATA>;
+ phy-names = "ahci_phy";
+ resets = <&powerdown STIH407_SATA0_POWERDOWN>,
+ <&softreset STIH407_SATA0_SOFTRESET>,
+ <&softreset STIH407_SATA0_PWR_SOFTRESET>;
+ reset-names = "pwr-dwn", "sw-rst", "pwr-rst";
+ clocks = <&clk_s_c0_flexgen CLK_ICN_REG>;
+ clock-names = "ahci_clk";
};
Required properties:
- compatible : "gpio-fan"
+
+Optional properties:
- gpios: Specifies the pins that map to bits in the control value,
ordered MSB-->LSB.
- gpio-fan,speed-map: A mapping of possible fan RPM speeds and the
control value that should be set to achieve them. This array
must have the RPM values in ascending order.
-
-Optional properties:
- alarm-gpios: This pin going active indicates something is wrong with
the fan, and a udev event will be fired.
+- cooling-cells: If used as a cooling device, must be <2>
+ Also see: Documentation/devicetree/bindings/thermal/thermal.txt
+ min and max states are derived from the speed-map of the fan.
+
+Note: At least one the "gpios" or "alarm-gpios" properties must be set.
Examples:
6000 2>;
alarm-gpios = <&gpio1 15 1>;
};
+ gpio_fan_cool: gpio_fan {
+ compatible = "gpio-fan";
+ gpios = <&gpio2 14 1
+ &gpio2 13 1>;
+ gpio-fan,speed-map = <0 0>,
+ <3000 1>,
+ <6000 2>;
+ alarm-gpios = <&gpio2 15 1>;
+ #cooling-cells = <2>; /* min followed by max */
+ };
- "fsl,vf610-i2c" for I2C compatible with the one integrated on Vybrid vf610 SoC
- reg : Should contain I2C/HS-I2C registers location and length
- interrupts : Should contain I2C/HS-I2C interrupt
+- clocks : Should contain the I2C/HS-I2C clock specifier
Optional properties:
- clock-frequency : Constains desired I2C/HS-I2C bus clock frequency in Hz.
--- /dev/null
+Dialog Semiconductor DA9150 IIO GPADC bindings
+
+Required properties:
+- compatible: "dlg,da9150-gpadc" for DA9150 IIO GPADC
+- #io-channel-cells: Should be set to <1>
+ (See Documentation/devicetree/bindings/iio/iio-bindings.txt for further info)
+
+For further information on GPADC channels, see device datasheet.
+
+
+Example:
+
+ gpadc: da9150-gpadc {
+ compatible = "dlg,da9150-gpadc";
+ #io-channel-cells = <1>;
+ };
--- /dev/null
+NVIDIA Legacy Interrupt Controller
+
+All Tegra SoCs contain a legacy interrupt controller that routes
+interrupts to the GIC, and also serves as a wakeup source. It is also
+referred to as "ictlr", hence the name of the binding.
+
+The HW block exposes a number of interrupt controllers, each
+implementing a set of 32 interrupts.
+
+Required properties:
+
+- compatible : should be: "nvidia,tegra<chip>-ictlr". The LIC on
+ subsequent SoCs remained backwards-compatible with Tegra30, so on
+ Tegra generations later than Tegra30 the compatible value should
+ include "nvidia,tegra30-ictlr".
+- reg : Specifies base physical address and size of the registers.
+ Each controller must be described separately (Tegra20 has 4 of them,
+ whereas Tegra30 and later have 5"
+- interrupt-controller : Identifies the node as an interrupt controller.
+- #interrupt-cells : Specifies the number of cells needed to encode an
+ interrupt source. The value must be 3.
+- interrupt-parent : a phandle to the GIC these interrupts are routed
+ to.
+
+Notes:
+
+- Because this HW ultimately routes interrupts to the GIC, the
+ interrupt specifier must be that of the GIC.
+- Only SPIs can use the ictlr as an interrupt parent. SGIs and PPIs
+ are explicitly forbidden.
+
+Example:
+
+ ictlr: interrupt-controller@60004000 {
+ compatible = "nvidia,tegra20-ictlr", "nvidia,tegra-ictlr";
+ reg = <0x60004000 64>,
+ <0x60004100 64>,
+ <0x60004200 64>,
+ <0x60004300 64>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&intc>;
+ };
- compatible: has to be "renesas,irqc-<soctype>", "renesas,irqc" as fallback.
Examples with soctypes are:
- - "renesas,irqc-r8a73a4" (R-Mobile AP6)
+ - "renesas,irqc-r8a73a4" (R-Mobile APE6)
- "renesas,irqc-r8a7790" (R-Car H2)
- "renesas,irqc-r8a7791" (R-Car M2-W)
- "renesas,irqc-r8a7792" (R-Car V2H)
- "renesas,irqc-r8a7794" (R-Car E2)
- #interrupt-cells: has to be <2>: an interrupt index and flags, as defined in
interrupts.txt in this directory
+- clocks: Must contain a reference to the functional clock.
Optional properties:
<0 1 IRQ_TYPE_LEVEL_HIGH>,
<0 2 IRQ_TYPE_LEVEL_HIGH>,
<0 3 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&mstp4_clks R8A7790_CLK_IRQC>;
};
--- /dev/null
+STMicroelectronics STi System Configuration Controlled IRQs
+-----------------------------------------------------------
+
+On STi based systems; External, CTI (Core Sight), PMU (Performance Management),
+and PL310 L2 Cache IRQs are controlled using System Configuration registers.
+This driver is used to unmask them prior to use.
+
+Required properties:
+- compatible : Should be set to one of:
+ "st,stih415-irq-syscfg"
+ "st,stih416-irq-syscfg"
+ "st,stih407-irq-syscfg"
+ "st,stid127-irq-syscfg"
+- st,syscfg : Phandle to Cortex-A9 IRQ system config registers
+- st,irq-device : Array of IRQs to enable - should be 2 in length
+- st,fiq-device : Array of FIQs to enable - should be 2 in length
+
+Optional properties:
+- st,invert-ext : External IRQs can be inverted at will. This property inverts
+ these IRQs using bitwise logic. A number of defines have been
+ provided for convenience:
+ ST_IRQ_SYSCFG_EXT_1_INV
+ ST_IRQ_SYSCFG_EXT_2_INV
+ ST_IRQ_SYSCFG_EXT_3_INV
+Example:
+
+irq-syscfg {
+ compatible = "st,stih416-irq-syscfg";
+ st,syscfg = <&syscfg_cpu>;
+ st,irq-device = <ST_IRQ_SYSCFG_PMU_0>,
+ <ST_IRQ_SYSCFG_PMU_1>;
+ st,fiq-device = <ST_IRQ_SYSCFG_DISABLED>,
+ <ST_IRQ_SYSCFG_DISABLED>;
+ st,invert-ext = <(ST_IRQ_SYSCFG_EXT_1_INV | ST_IRQ_SYSCFG_EXT_3_INV)>;
+};
--- /dev/null
+TI OMAP4 Wake-up Generator
+
+All TI OMAP4/5 (and their derivatives) an interrupt controller that
+routes interrupts to the GIC, and also serves as a wakeup source. It
+is also referred to as "WUGEN-MPU", hence the name of the binding.
+
+Reguired properties:
+
+- compatible : should contain at least "ti,omap4-wugen-mpu" or
+ "ti,omap5-wugen-mpu"
+- reg : Specifies base physical address and size of the registers.
+- interrupt-controller : Identifies the node as an interrupt controller.
+- #interrupt-cells : Specifies the number of cells needed to encode an
+ interrupt source. The value must be 3.
+- interrupt-parent : a phandle to the GIC these interrupts are routed
+ to.
+
+Notes:
+
+- Because this HW ultimately routes interrupts to the GIC, the
+ interrupt specifier must be that of the GIC.
+- Only SPIs can use the WUGEN as an interrupt parent. SGIs and PPIs
+ are explicitly forbiden.
+
+Example:
+
+ wakeupgen: interrupt-controller@48281000 {
+ compatible = "ti,omap5-wugen-mpu", "ti,omap4-wugen-mpu";
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ reg = <0x48281000 0x1000>;
+ interrupt-parent = <&gic>;
+ };
--- /dev/null
+Broadcom IPROC SDHCI controller
+
+This file documents differences between the core properties described
+by mmc.txt and the properties that represent the IPROC SDHCI controller.
+
+Required properties:
+- compatible : Should be "brcm,sdhci-iproc-cygnus".
+- clocks : The clock feeding the SDHCI controller.
+
+Optional properties:
+ - sdhci,auto-cmd12: specifies that controller should use auto CMD12.
+
+Example:
+
+sdhci0: sdhci@0x18041000 {
+ compatible = "brcm,sdhci-iproc-cygnus";
+ reg = <0x18041000 0x100>;
+ interrupts = <GIC_SPI 108 IRQ_TYPE_LEVEL_HIGH>;
+ clocks = <&lcpll0_clks BCM_CYGNUS_LCPLL0_SDIO_CLK>;
+ bus-width = <4>;
+ sdhci,auto-cmd12;
+ no-1-8-v;
+};
in transmit mode and CIU clock phase shift value in receive mode for double
data rate mode operation. Refer notes below for the order of the cells and the
valid values.
+* samsung,dw-mshc-hs400-timing: Specifies the value of CIU TX and RX clock phase
+ shift value for hs400 mode operation.
Notes for the sdr-timing and ddr-timing values:
- if CIU clock divider value is 0 (that is divide by 1), both tx and rx
phase shift clocks should be 0.
+* samsung,read-strobe-delay: RCLK (Data strobe) delay to control HS400 mode
+ (Latency value for delay line in Read path)
+
Required properties for a slot (Deprecated - Recommend to use one slot per host):
* gpios: specifies a list of gpios used for command, clock and data bus. The
samsung,dw-mshc-ciu-div = <3>;
samsung,dw-mshc-sdr-timing = <2 3>;
samsung,dw-mshc-ddr-timing = <1 2>;
+ samsung,dw-mshc-hs400-timing = <0 2>;
+ samsung,read-strobe-delay = <90>;
bus-width = <8>;
};
to select a proper data sampling window in case the clock quality is not good
due to signal path is too long on the board. Please refer to eSDHC/uSDHC
chapter, DLL (Delay Line) section in RM for details.
+- voltage-ranges : Specify the voltage range in case there are software
+ transparent level shifters on the outputs of the controller. Two cells are
+ required, first cell specifies minimum slot voltage (mV), second cell
+ specifies maximum slot voltage (mV). Several ranges could be specified.
Examples:
--- /dev/null
+mmc-card / eMMC bindings
+------------------------
+
+This documents describes the devicetree bindings for a mmc-host controller
+child node describing a mmc-card / an eMMC, see "Use of Function subnodes"
+in mmc.txt
+
+Required properties:
+-compatible : Must be "mmc-card"
+-reg : Must be <0>
+
+Optional properties:
+-broken-hpi : Use this to indicate that the mmc-card has a broken hpi
+ implementation, and that hpi should not be used
+
+Example:
+
+&mmc2 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mmc2_pins_a>;
+ vmmc-supply = <®_vcc3v3>;
+ bus-width = <8>;
+ non-removable;
+ status = "okay";
+
+ mmccard: mmccard@0 {
+ reg = <0>;
+ compatible = "mmc-card";
+ broken-hpi;
+ };
+};
used by the sdhci-st driver.
Required properties:
-- compatible : Must be "st,sdhci"
-- clock-names : Should be "mmc"
- See: Documentation/devicetree/bindings/resource-names.txt
-- clocks : Phandle of the clock used by the sdhci controler
- See: Documentation/devicetree/bindings/clock/clock-bindings.txt
+- compatible: Must be "st,sdhci" and it can be compatible to "st,sdhci-stih407"
+ to set the internal glue logic used for configuring the MMC
+ subsystem (mmcss) inside the FlashSS (available in STiH407 SoC
+ family).
+
+- clock-names: Should be "mmc".
+ See: Documentation/devicetree/bindings/resource-names.txt
+- clocks: Phandle to the clock.
+ See: Documentation/devicetree/bindings/clock/clock-bindings.txt
+
+- interrupts: One mmc interrupt should be described here.
+- interrupt-names: Should be "mmcirq".
+
+- pinctrl-names: A pinctrl state names "default" must be defined.
+- pinctrl-0: Phandle referencing pin configuration of the sd/emmc controller.
+ See: Documentation/devicetree/bindings/pinctrl/pinctrl-binding.txt
+
+- reg: This must provide the host controller base address and it can also
+ contain the FlashSS Top register for TX/RX delay used by the driver
+ to configure DLL inside the flashSS, if so reg-names must also be
+ specified.
Optional properties:
-- non-removable: non-removable slot
- See: Documentation/devicetree/bindings/mmc/mmc.txt
-- bus-width: Number of data lines
- See: Documentation/devicetree/bindings/mmc/mmc.txt
+- reg-names: Should be "mmc" and "top-mmc-delay". "top-mmc-delay" is optional
+ for eMMC on stih407 family silicon to configure DLL inside FlashSS.
+
+- non-removable: Non-removable slot. Also used for configuring mmcss in STiH407 SoC
+ family.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
+
+- bus-width: Number of data lines.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
+
+- max-frequency: Can be 200MHz, 100Mz or 50MHz (default) and used for
+ configuring the CCONFIG3 in the mmcss.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
+
+- resets: Phandle and reset specifier pair to softreset line of HC IP.
+ See: Documentation/devicetree/bindings/reset/reset.txt
+
+- vqmmc-supply: Phandle to the regulator dt node, mentioned as the vcc/vdd
+ supply in eMMC/SD specs.
+
+- sd-uhs--sdr50: To enable the SDR50 in the mmcss.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
+
+- sd-uhs-sdr104: To enable the SDR104 in the mmcss.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
+
+- sd-uhs-ddr50: To enable the DDR50 in the mmcss.
+ See: Documentation/devicetree/bindings/mmc/mmc.txt.
Example:
+/* Example stih416e eMMC configuration */
+
mmc0: sdhci@fe81e000 {
compatible = "st,sdhci";
status = "disabled";
pinctrl-0 = <&pinctrl_mmc0>;
clock-names = "mmc";
clocks = <&clk_s_a1_ls 1>;
- bus-width = <8>
+ bus-width = <8>
+
+/* Example SD stih407 family configuration */
+
+mmc1: sdhci@09080000 {
+ compatible = "st,sdhci-stih407", "st,sdhci";
+ status = "disabled";
+ reg = <0x09080000 0x7ff>;
+ reg-names = "mmc";
+ interrupts = <GIC_SPI 90 IRQ_TYPE_NONE>;
+ interrupt-names = "mmcirq";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_sd1>;
+ clock-names = "mmc";
+ clocks = <&clk_s_c0_flexgen CLK_MMC_1>;
+ resets = <&softreset STIH407_MMC1_SOFTRESET>;
+ bus-width = <4>;
+};
+
+/* Example eMMC stih407 family configuration */
+
+mmc0: sdhci@09060000 {
+ compatible = "st,sdhci-stih407", "st,sdhci";
+ status = "disabled";
+ reg = <0x09060000 0x7ff>, <0x9061008 0x20>;
+ reg-names = "mmc", "top-mmc-delay";
+ interrupts = <GIC_SPI 92 IRQ_TYPE_NONE>;
+ interrupt-names = "mmcirq";
+ pinctrl-names = "default";
+ pinctrl-0 = <&pinctrl_mmc0>;
+ clock-names = "mmc";
+ clocks = <&clk_s_c0_flexgen CLK_MMC_0>;
+ vqmmc-supply = <&vmmc_reg>;
+ max-frequency = <200000000>;
+ bus-width = <8>;
+ non-removable;
+ sd-uhs-sdr50;
+ sd-uhs-sdr104;
+ sd-uhs-ddr50;
};
- amd,serdes-cdr-rate: CDR rate speed selection
- amd,serdes-pq-skew: PQ (data sampling) skew
- amd,serdes-tx-amp: TX amplitude boost
+- amd,serdes-dfe-tap-config: DFE taps available to run
+- amd,serdes-dfe-tap-enable: DFE taps to enable
Example:
xgbe_phy@e1240800 {
amd,serdes-cdr-rate = <2>, <2>, <7>;
amd,serdes-pq-skew = <10>, <10>, <30>;
amd,serdes-tx-amp = <15>, <15>, <10>;
+ amd,serdes-dfe-tap-config = <3>, <3>, <1>;
+ amd,serdes-dfe-tap-enable = <0>, <0>, <127>;
};
APM X-Gene SoC.
Required properties for all the ethernet interfaces:
-- compatible: Should be "apm,xgene-enet"
+- compatible: Should state binding information from the following list,
+ - "apm,xgene-enet": RGMII based 1G interface
+ - "apm,xgene1-sgenet": SGMII based 1G interface
+ - "apm,xgene1-xgenet": XFI based 10G interface
- reg: Address and length of the register set for the device. It contains the
information of registers in the same order as described by reg-names
- reg-names: Should contain the register set names
(DSA_MAX_SWITCHES).
Each of these switch child nodes should have the following required properties:
-- reg : Describes the switch address on the MII bus
+- reg : Contains two fields. The first one describes the
+ address on the MII bus. The second is the switch
+ number that must be unique in cascaded configurations
- #address-cells : Must be 1
- #size-cells : Must be 0
--- /dev/null
+* Broadcom iProc PCIe controller with the platform bus interface
+
+Required properties:
+- compatible: Must be "brcm,iproc-pcie"
+- reg: base address and length of the PCIe controller I/O register space
+- #interrupt-cells: set to <1>
+- interrupt-map-mask and interrupt-map, standard PCI properties to define the
+ mapping of the PCIe interface to interrupt numbers
+- linux,pci-domain: PCI domain ID. Should be unique for each host controller
+- bus-range: PCI bus numbers covered
+- #address-cells: set to <3>
+- #size-cells: set to <2>
+- device_type: set to "pci"
+- ranges: ranges for the PCI memory and I/O regions
+
+Optional properties:
+- phys: phandle of the PCIe PHY device
+- phy-names: must be "pcie-phy"
+
+Example:
+ pcie0: pcie@18012000 {
+ compatible = "brcm,iproc-pcie";
+ reg = <0x18012000 0x1000>;
+
+ #interrupt-cells = <1>;
+ interrupt-map-mask = <0 0 0 0>;
+ interrupt-map = <0 0 0 0 &gic GIC_SPI 100 IRQ_TYPE_NONE>;
+
+ linux,pci-domain = <0>;
+
+ bus-range = <0x00 0xff>;
+
+ #address-cells = <3>;
+ #size-cells = <2>;
+ device_type = "pci";
+ ranges = <0x81000000 0 0 0x28000000 0 0x00010000
+ 0x82000000 0 0x20000000 0x20000000 0 0x04000000>;
+
+ phys = <&phy 0 5>;
+ phy-names = "pcie-phy";
+ };
+
+ pcie1: pcie@18013000 {
+ compatible = "brcm,iproc-pcie";
+ reg = <0x18013000 0x1000>;
+
+ #interrupt-cells = <1>;
+ interrupt-map-mask = <0 0 0 0>;
+ interrupt-map = <0 0 0 0 &gic GIC_SPI 106 IRQ_TYPE_NONE>;
+
+ linux,pci-domain = <1>;
+
+ bus-range = <0x00 0xff>;
+
+ #address-cells = <3>;
+ #size-cells = <2>;
+ device_type = "pci";
+ ranges = <0x81000000 0 0 0x48000000 0 0x00010000
+ 0x82000000 0 0x40000000 0x40000000 0 0x04000000>;
+
+ phys = <&phy 1 6>;
+ phy-names = "pcie-phy";
+ };
--- /dev/null
+Dialog Semiconductor DA9150 Charger Power Supply bindings
+
+Required properties:
+- compatible: "dlg,da9150-charger" for DA9150 Charger Power Supply
+
+Optional properties:
+- io-channels: List of phandle and IIO specifier pairs
+- io-channel-names: List of channel names used by charger
+ ["CHAN_IBUS", "CHAN_VBUS", "CHAN_TJUNC", "CHAN_VBAT"]
+ (See Documentation/devicetree/bindings/iio/iio-bindings.txt for further info)
+
+
+Example:
+
+ da9150-charger {
+ compatible = "dlg,da9150-charger";
+
+ io-channels = <&gpadc 0>,
+ <&gpadc 2>,
+ <&gpadc 8>,
+ <&gpadc 5>;
+ io-channel-names = "CHAN_IBUS",
+ "CHAN_VBUS",
+ "CHAN_TJUNC",
+ "CHAN_VBAT";
+ };
providing multiple PM domains (e.g. power controllers), but can be any value
as specified by device tree binding documentation of particular provider.
+Optional properties:
+ - power-domains : A phandle and PM domain specifier as defined by bindings of
+ the power controller specified by phandle.
+ Some power domains might be powered from another power domain (or have
+ other hardware specific dependencies). For representing such dependency
+ a standard PM domain consumer binding is used. When provided, all domains
+ created by the given provider should be subdomains of the domain
+ specified by this binding. More details about power domain specifier are
+ available in the next section.
+
Example:
power: power-controller@12340000 {
The node above defines a power controller that is a PM domain provider and
expects one cell as its phandle argument.
+Example 2:
+
+ parent: power-controller@12340000 {
+ compatible = "foo,power-controller";
+ reg = <0x12340000 0x1000>;
+ #power-domain-cells = <1>;
+ };
+
+ child: power-controller@12340000 {
+ compatible = "foo,power-controller";
+ reg = <0x12341000 0x1000>;
+ power-domains = <&parent 0>;
+ #power-domain-cells = <1>;
+ };
+
+The nodes above define two power controllers: 'parent' and 'child'.
+Domains created by the 'child' power controller are subdomains of '0' power
+domain provided by the 'parent' power controller.
+
==PM domain consumers==
Required properties:
--- /dev/null
+Generic SYSCON mapped register poweroff driver
+
+This is a generic poweroff driver using syscon to map the poweroff register.
+The poweroff is generally performed with a write to the poweroff register
+defined by the register map pointed by syscon reference plus the offset
+with the mask defined in the poweroff node.
+
+Required properties:
+- compatible: should contain "syscon-poweroff"
+- regmap: this is phandle to the register map node
+- offset: offset in the register map for the poweroff register (in bytes)
+- mask: the poweroff value written to the poweroff register (32 bit access)
+
+Default will be little endian mode, 32 bit access only.
+
+Examples:
+
+ poweroff {
+ compatible = "syscon-poweroff";
+ regmap = <®mapnode>;
+ offset = <0x0>;
+ mask = <0x7a>;
+ };
-------------------
Required properties:
-- compatible: "active-semi,act8846" or "active-semi,act8865"
+- compatible: "active-semi,act8846" or "active-semi,act8865" or "active-semi,act8600"
- reg: I2C slave address
Optional properties:
- system-power-controller: Telling whether or not this pmic is controlling
the system power. See Documentation/devicetree/bindings/power/power-controller.txt .
+Optional input supply properties:
+- for act8600:
+ - vp1-supply: The input supply for DCDC_REG1
+ - vp2-supply: The input supply for DCDC_REG2
+ - vp3-supply: The input supply for DCDC_REG3
+ - inl-supply: The input supply for LDO_REG5, LDO_REG6, LDO_REG7 and LDO_REG8
+ SUDCDC_REG4, LDO_REG9 and LDO_REG10 do not have separate supplies.
+- for act8846:
+ - vp1-supply: The input supply for REG1
+ - vp2-supply: The input supply for REG2
+ - vp3-supply: The input supply for REG3
+ - vp4-supply: The input supply for REG4
+ - inl1-supply: The input supply for REG5, REG6 and REG7
+ - inl2-supply: The input supply for REG8 and LDO_REG9
+ - inl3-supply: The input supply for REG10, REG11 and REG12
+- for act8865:
+ - vp1-supply: The input supply for DCDC_REG1
+ - vp2-supply: The input supply for DCDC_REG2
+ - vp3-supply: The input supply for DCDC_REG3
+ - inl45-supply: The input supply for LDO_REG1 and LDO_REG2
+ - inl67-supply: The input supply for LDO_REG3 and LDO_REG4
+
Any standard regulator properties can be used to configure the single regulator.
The valid names for regulators are:
REG1, REG2, REG3, REG4, REG5, REG6, REG7, REG8, REG9, REG10, REG11, REG12
- for act8865:
DCDC_REG1, DCDC_REG2, DCDC_REG3, LDO_REG1, LDO_REG2, LDO_REG3, LDO_REG4.
+ - for act8600:
+ DCDC_REG1, DCDC_REG2, DCDC_REG3, SUDCDC_REG4, LDO_REG5, LDO_REG6, LDO_REG7,
+ LDO_REG8, LDO_REG9, LDO_REG10,
Example:
--------
--- /dev/null
+ETRAX FS UART
+
+Required properties:
+- compatible : "axis,etraxfs-uart"
+- reg: offset and length of the register set for the device.
+- interrupts: device interrupt
+
+Optional properties:
+- {dtr,dsr,ri,cd}-gpios: specify a GPIO for DTR/DSR/RI/CD
+ line respectively.
+
+Example:
+
+serial@b00260000 {
+ compatible = "axis,etraxfs-uart";
+ reg = <0xb0026000 0x1000>;
+ interrupts = <68>;
+ status = "disabled";
+};
- reg-io-width : the size (in bytes) of the IO accesses that should be
performed on the device. If this property is not present then single byte
accesses are used.
+- dcd-override : Override the DCD modem status signal. This signal will always
+ be reported as active instead of being obtained from the modem status
+ register. Define this if your serial port does not use this pin.
+- dsr-override : Override the DTS modem status signal. This signal will always
+ be reported as active instead of being obtained from the modem status
+ register. Define this if your serial port does not use this pin.
+- cts-override : Override the CTS modem status signal. This signal will always
+ be reported as active instead of being obtained from the modem status
+ register. Define this if your serial port does not use this pin.
+- ri-override : Override the RI modem status signal. This signal will always be
+ reported as inactive instead of being obtained from the modem status register.
+ Define this if your serial port does not use this pin.
Example:
interrupts = <10>;
reg-shift = <2>;
reg-io-width = <4>;
+ dcd-override;
+ dsr-override;
+ cts-override;
+ ri-override;
};
Example with one clock:
(CSPI/eCSPI) for i.MX
Required properties:
-- compatible : Should be "fsl,<soc>-cspi" or "fsl,<soc>-ecspi"
+- compatible :
+ - "fsl,imx1-cspi" for SPI compatible with the one integrated on i.MX1
+ - "fsl,imx21-cspi" for SPI compatible with the one integrated on i.MX21
+ - "fsl,imx27-cspi" for SPI compatible with the one integrated on i.MX27
+ - "fsl,imx31-cspi" for SPI compatible with the one integrated on i.MX31
+ - "fsl,imx35-cspi" for SPI compatible with the one integrated on i.MX35
+ - "fsl,imx51-ecspi" for SPI compatible with the one integrated on i.MX51
- reg : Offset and length of the register set for the device
- interrupts : Should contain CSPI/eCSPI interrupt
- fsl,spi-num-chipselects : Contains the number of the chipselect
- cs-gpios : Specifies the gpio pins to be used for chipselects.
+- clocks : Clock specifiers for both ipg and per clocks.
+- clock-names : Clock names should include both "ipg" and "per"
+See the clock consumer binding,
+ Documentation/devicetree/bindings/clock/clock-bindings.txt
- dmas: DMA specifiers for tx and rx dma. See the DMA client binding,
Documentation/devicetree/bindings/dma/dma.txt
- dma-names: DMA request names should include "tx" and "rx" if present.
nodes. If unspecified, a single SPI device without a chip
select can be used.
+- dmas: Two DMA channel specifiers following the convention outlined
+ in bindings/dma/dma.txt
+- dma-names: Names for the dma channels, if present. There must be at
+ least one channel named "tx" for transmit and named "rx" for
+ receive.
SPI slave nodes must be children of the SPI master node and can contain
properties described in Documentation/devicetree/bindings/spi/spi-bus.txt
clocks = <&gcc GCC_BLSP2_QUP2_SPI_APPS_CLK>, <&gcc GCC_BLSP2_AHB_CLK>;
clock-names = "core", "iface";
+ dmas = <&blsp1_bam 13>, <&blsp1_bam 12>;
+ dma-names = "rx", "tx";
+
pinctrl-names = "default";
pinctrl-0 = <&spi8_default>;
in big endian mode, otherwise in native mode(same with CPU), for more
detail please see: Documentation/devicetree/bindings/regmap/regmap.txt.
+Optional SPI slave node properties:
+- fsl,spi-cs-sck-delay: a delay in nanoseconds between activating chip
+ select and the start of clock signal, at the start of a transfer.
+- fsl,spi-sck-cs-delay: a delay in nanoseconds between stopping the clock
+ signal and deactivating chip select, at the end of a transfer.
+
Example:
dspi0@4002c000 {
reg = <0>;
linux,modalias = "m25p80";
modal = "at26df081a";
+ fsl,spi-cs-sck-delay = <100>;
+ fsl,spi-sck-cs-delay = <50>;
};
};
- dma-names: Must include the following entries:
- rx
- tx
+- cs-gpios: Must specify the GPIOs used for chipselect lines.
- #address-cells: Must be 1.
- #size-cells: Must be 0.
- dmas: DMA specifiers for tx and rx dma. See the DMA client binding,
Documentation/devicetree/bindings/dma/dma.txt
- dma-names: DMA request names should include "tx" and "rx" if present.
+- rx-sample-delay-ns: nanoseconds to delay after the SCLK edge before sampling
+ Rx data (may need to be fine tuned for high capacitance lines).
+ No delay (0) by default.
Example:
reg = <0xff110000 0x1000>;
dmas = <&pdma1 11>, <&pdma1 12>;
dma-names = "tx", "rx";
+ rx-sample-delay-ns = <10>;
#address-cells = <1>;
#size-cells = <0>;
interrupts = <GIC_SPI 44 IRQ_TYPE_LEVEL_HIGH>;
devicetree@vger.kernel.org
+ and Cc: the DT maintainers. Use scripts/get_maintainer.pl to identify
+ all of the DT maintainers.
+
3) The Documentation/ portion of the patch should come in the series before
the code implementing the binding.
- compatible : "renesas,thermal-<soctype>", "renesas,rcar-thermal"
as fallback.
Examples with soctypes are:
- - "renesas,thermal-r8a73a4" (R-Mobile AP6)
+ - "renesas,thermal-r8a73a4" (R-Mobile APE6)
- "renesas,thermal-r8a7779" (R-Car H1)
- "renesas,thermal-r8a7790" (R-Car H2)
- "renesas,thermal-r8a7791" (R-Car M2-W)
ams AMS AG
amstaos AMS-Taos Inc.
apm Applied Micro Circuits Corporation (APM)
+arasan Arasan Chip Systems
arm ARM Ltd.
armadeus ARMadeus Systems SARL
asahi-kasei Asahi Kasei Corp.
auo AU Optronics Corporation
avago Avago Technologies
avic Shanghai AVIC Optoelectronics Co., Ltd.
+axis Axis Communications AB
bosch Bosch Sensortec GmbH
brcm Broadcom Corporation
buffalo Buffalo, Inc.
- atmel,disable : Should be present if you want to disable the watchdog.
- atmel,idle-halt : Should be present if you want to stop the watchdog when
entering idle state.
+ CAUTION: This property should be used with care, it actually makes the
+ watchdog not counting when the CPU is in idle state, therefore the
+ watchdog reset time depends on mean CPU usage and will not reset at all
+ if the CPU stop working while it is in idle state, which is probably
+ not what you want.
- atmel,dbg-halt : Should be present if you want to stop the watchdog when
entering debug state.
dlmfs is built with OCFS2 as it requires most of its infrastructure.
-Project web page: http://oss.oracle.com/projects/ocfs2
-Tools web page: http://oss.oracle.com/projects/ocfs2-tools
+Project web page: http://ocfs2.wiki.kernel.org
+Tools web page: https://github.com/markfasheh/ocfs2-tools
OCFS2 mailing lists: http://oss.oracle.com/projects/ocfs2/mailman/
All code copyright 2005 Oracle except when otherwise noted.
You'll want to install the ocfs2-tools package in order to at least
get "mount.ocfs2" and "ocfs2_hb_ctl".
-Project web page: http://oss.oracle.com/projects/ocfs2
-Tools web page: http://oss.oracle.com/projects/ocfs2-tools
+Project web page: http://ocfs2.wiki.kernel.org
+Tools git tree: https://github.com/markfasheh/ocfs2-tools
OCFS2 mailing lists: http://oss.oracle.com/projects/ocfs2/mailman/
All code copyright 2005 Oracle except when otherwise noted.
Prefix: 'it8603'
Addresses scanned: from Super I/O config space (8 I/O ports)
Datasheet: Not publicly available
+ * IT8620E
+ Prefix: 'it8620'
+ Addresses scanned: from Super I/O config space (8 I/O ports)
+ Datasheet: Not publicly available
* IT8705F
Prefix: 'it87'
Addresses scanned: from Super I/O config space (8 I/O ports)
Prefix: 'it8772'
Addresses scanned: from Super I/O config space (8 I/O ports)
Datasheet: Not publicly available
+ * IT8781F
+ Prefix: 'it8781'
+ Addresses scanned: from Super I/O config space (8 I/O ports)
+ Datasheet: Not publicly available
* IT8782F
Prefix: 'it8782'
Addresses scanned: from Super I/O config space (8 I/O ports)
Prefix: 'it8783'
Addresses scanned: from Super I/O config space (8 I/O ports)
Datasheet: Not publicly available
+ * IT8786E
+ Prefix: 'it8786'
+ Addresses scanned: from Super I/O config space (8 I/O ports)
+ Datasheet: Not publicly available
+ * IT8790E
+ Prefix: 'it8790'
+ Addresses scanned: from Super I/O config space (8 I/O ports)
+ Datasheet: Not publicly available
* SiS950 [clone of IT8705F]
Prefix: 'it87'
Addresses scanned: from Super I/O config space (8 I/O ports)
Description
-----------
-This driver implements support for the IT8603E, IT8623E, IT8705F, IT8712F,
-IT8716F, IT8718F, IT8720F, IT8721F, IT8726F, IT8728F, IT8758E, IT8771E,
-IT8772E, IT8782F, IT8783E/F, and SiS950 chips.
+This driver implements support for the IT8603E, IT8620E, IT8623E, IT8705F,
+IT8712F, IT8716F, IT8718F, IT8720F, IT8721F, IT8726F, IT8728F, IT8758E,
+IT8771E, IT8772E, IT8781F, IT8782F, IT8783E/F, IT8786E, IT8790E, and SiS950
+chips.
These chips are 'Super I/O chips', supporting floppy disks, infrared ports,
joysticks and other miscellaneous stuff. For hardware monitoring, they
have support for 2 additional fans. The additional fans are supported by the
driver.
-The IT8716F, IT8718F, IT8720F, IT8721F/IT8758E, IT8782F, IT8783E/F, and late
-IT8712F and IT8705F also have optional 16-bit tachometer counters for fans 1 to
-3. This is better (no more fan clock divider mess) but not compatible with the
-older chips and revisions. The 16-bit tachometer mode is enabled by the driver
-when one of the above chips is detected.
+The IT8716F, IT8718F, IT8720F, IT8721F/IT8758E, IT8781F, IT8782F, IT8783E/F,
+and late IT8712F and IT8705F also have optional 16-bit tachometer counters
+for fans 1 to 3. This is better (no more fan clock divider mess) but not
+compatible with the older chips and revisions. The 16-bit tachometer mode
+is enabled by the driver when one of the above chips is detected.
The IT8726F is just bit enhanced IT8716F with additional hardware
for AMD power sequencing. Therefore the chip will appear as IT8716F
until a datasheet becomes available (hopefully.)
The IT8603E/IT8623E is a custom design, hardware monitoring part is similar to
-IT8728F. It only supports 16-bit fan mode, the full speed mode of the
-fan is not supported (value 0 of pwmX_enable).
+IT8728F. It only supports 3 fans, 16-bit fan mode, and the full speed mode
+of the fan is not supported (value 0 of pwmX_enable).
+
+The IT8620E is another custom design, hardware monitoring part is similar to
+IT8728F. It only supports 16-bit fan mode.
+
+The IT8790E supports up to 3 fans. 16-bit fan mode is always enabled.
Temperatures are measured in degrees Celsius. An alarm is triggered once
when the Overtemperature Shutdown limit is crossed.
0.016 volt (except IT8603E, IT8721F/IT8758E and IT8728F: 0.012 volt.) The
battery voltage in8 does not have limit registers.
-On the IT8603E, IT8721F/IT8758E, IT8782F, and IT8783E/F, some voltage inputs
-are internal and scaled inside the chip:
+On the IT8603E, IT8721F/IT8758E, IT8781F, IT8782F, and IT8783E/F, some
+voltage inputs are internal and scaled inside the chip:
* in3 (optional)
-* in7 (optional for IT8782F and IT8783E/F)
+* in7 (optional for IT8781F, IT8782F, and IT8783E/F)
* in8 (always)
* in9 (relevant for IT8603E only)
The driver handles this transparently so user-space doesn't have to care.
http://www.atmel.com/Images/doc8711.pdf
http://www.atmel.com/Images/Atmel-8852-SEEPROM-AT30TSE002A-Datasheet.pdf
http://www.atmel.com/Images/Atmel-8868-DTS-AT30TSE004A-Datasheet.pdf
- * IDT TSE2002B3, TSE2002GB2, TS3000B3, TS3000GB2
+ * IDT TSE2002B3, TSE2002GB2, TSE2004GB2, TS3000B3, TS3000GB0, TS3000GB2,
+ TS3001GB2
Datasheets:
- http://www.idt.com/sites/default/files/documents/IDT_TSE2002B3C_DST_20100512_120303152056.pdf
- http://www.idt.com/sites/default/files/documents/IDT_TSE2002GB2A1_DST_20111107_120303145914.pdf
- http://www.idt.com/sites/default/files/documents/IDT_TS3000B3A_DST_20101129_120303152013.pdf
- http://www.idt.com/sites/default/files/documents/IDT_TS3000GB2A1_DST_20111104_120303151012.pdf
+ Available from IDT web site
* Maxim MAX6604
Datasheets:
http://datasheets.maxim-ic.com/en/ds/MAX6604.pdf
--- /dev/null
+Kernel driver nct7904
+====================
+
+Supported chip:
+ * Nuvoton NCT7904D
+ Prefix: nct7904
+ Addresses: I2C 0x2d, 0x2e
+ Datasheet: Publicly available at Nuvoton website
+ http://www.nuvoton.com/
+
+Author: Vadim V. Vlasov <vvlasov@dev.rtsoft.ru>
+
+
+Description
+-----------
+
+The NCT7904D is a hardware monitor supporting up to 20 voltage sensors,
+internal temperature sensor, Intel PECI and AMD SB-TSI CPU temperature
+interface, up to 12 fan tachometer inputs, up to 4 fan control channels
+with SmartFan.
+
+
+Sysfs entries
+-------------
+
+Currently, the driver supports only the following features:
+
+in[1-20]_input Input voltage measurements (mV)
+
+fan[1-12]_input Fan tachometer measurements (rpm)
+
+temp1_input Local temperature (1/1000 degree,
+ 0.125 degree resolution)
+
+temp[2-9]_input CPU temperatures (1/1000 degree,
+ 0.125 degree resolution)
+
+fan[1-4]_mode R/W, 0/1 for manual or SmartFan mode
+ Setting SmartFan mode is supported only if it has been
+ previously configured by BIOS (or configuration EEPROM)
+
+fan[1-4]_pwm R/O in SmartFan mode, R/W in manual control mode
+
+The driver checks sensor control registers and does not export the sensors
+that are not enabled. Anyway, a sensor that is enabled may actually be not
+connected and thus provide zero readings.
+
+
+Limitations
+-----------
+
+The following features are not supported in current version:
+
+ - SmartFan control
+ - Watchdog
+ - GPIO
+ - external temperature sensors
+ - SMI
+ - min/max values
+ - many other...
byte 4: 0 y6 y5 y4 y3 y2 y1 y0
byte 5: 0 z6 z5 z4 z3 z2 z1 z0
+Protocol Version 2 DualPoint devices send standard PS/2 mouse packets for
+the DualPoint Stick.
+
Dualpoint device -- interleaved packet format
---------------------------------------------
byte 7: 0 y6 y5 y4 y3 y2 y1 y0
byte 8: 0 z6 z5 z4 z3 z2 z1 z0
+Devices which use the interleaving format normally send standard PS/2 mouse
+packets for the DualPoint Stick + ALPS Absolute Mode packets for the
+touchpad, switching to the interleaved packet format when both the stick and
+the touchpad are used at the same time.
+
ALPS Absolute Mode - Protocol Version 3
---------------------------------------
The kernel does not provide button emulation for such devices but treats
them as any other INPUT_PROP_BUTTONPAD device.
+INPUT_PROP_ACCELEROMETER
+-------------------------
+Directional axes on this device (absolute and/or relative x, y, z) represent
+accelerometer data. All other axes retain their meaning. A device must not mix
+regular directional axes and accelerometer axes on the same event node.
+
Guidelines:
==========
The guidelines below ensure proper single-touch and multi-finger functionality.
The type of approaching tool. A lot of kernel drivers cannot distinguish
between different tool types, such as a finger or a pen. In such cases, the
-event should be omitted. The protocol currently supports MT_TOOL_FINGER and
-MT_TOOL_PEN [2]. For type B devices, this event is handled by input core;
-drivers should instead use input_mt_report_slot_state().
+event should be omitted. The protocol currently supports MT_TOOL_FINGER,
+MT_TOOL_PEN, and MT_TOOL_PALM [2]. For type B devices, this event is handled
+by input core; drivers should instead use input_mt_report_slot_state().
+A contact's ABS_MT_TOOL_TYPE may change over time while still touching the
+device, because the firmware may not be able to determine which tool is being
+used when it first appears.
ABS_MT_BLOB_ID
Format: {"off" | "on" | "skip[mbr]"}
efi= [EFI]
- Format: { "old_map", "nochunk", "noruntime" }
+ Format: { "old_map", "nochunk", "noruntime", "debug" }
old_map [X86-64]: switch to the old ioremap-based EFI
runtime services mapping. 32-bit still uses this one by
default.
boot stub, as chunking can cause problems with some
firmware implementations.
noruntime : disable EFI runtime services support
+ debug: enable misc debug output
efi_no_storage_paranoia [EFI; X86]
Using this parameter you can use more than 50% of
Consumer drivers can request a change in their supply regulator operating mode
by calling :-
-int regulator_set_optimum_mode(struct regulator *regulator, int load_uA);
+int regulator_set_load(struct regulator *regulator, int load_uA);
This will cause the core to recalculate the total load on the regulator (based
on all its consumers) and change operating mode (if necessary and permitted)
The IRQF_NO_SUSPEND flag is used to indicate that to the IRQ subsystem when
requesting a special-purpose interrupt. It causes suspend_device_irqs() to
-leave the corresponding IRQ enabled so as to allow the interrupt to work all
-the time as expected.
+leave the corresponding IRQ enabled so as to allow the interrupt to work as
+expected during the suspend-resume cycle, but does not guarantee that the
+interrupt will wake the system from a suspended state -- for such cases it is
+necessary to use enable_irq_wake().
Note that the IRQF_NO_SUSPEND flag affects the entire IRQ and not just one
user of it. Thus, if the IRQ is shared, all of the interrupt handlers installed
IRQF_NO_SUSPEND and enable_irq_wake()
-------------------------------------
-There are no valid reasons to use both enable_irq_wake() and the IRQF_NO_SUSPEND
-flag on the same IRQ.
+There are very few valid reasons to use both enable_irq_wake() and the
+IRQF_NO_SUSPEND flag on the same IRQ, and it is never valid to use both for the
+same device.
First of all, if the IRQ is not shared, the rules for handling IRQF_NO_SUSPEND
interrupts (interrupt handlers are invoked after suspend_device_irqs()) are
Second, both enable_irq_wake() and IRQF_NO_SUSPEND apply to entire IRQs and not
to individual interrupt handlers, so sharing an IRQ between a system wakeup
-interrupt source and an IRQF_NO_SUSPEND interrupt source does not make sense.
+interrupt source and an IRQF_NO_SUSPEND interrupt source does not generally
+make sense.
+
+In rare cases an IRQ can be shared between a wakeup device driver and an
+IRQF_NO_SUSPEND user. In order for this to be safe, the wakeup device driver
+must be able to discern spurious IRQs from genuine wakeup events (signalling
+the latter to the core with pm_system_wakeup()), must use enable_irq_wake() to
+ensure that the IRQ will function as a wakeup source, and must request the IRQ
+with IRQF_COND_SUSPEND to tell the core that it meets these requirements. If
+these requirements are not met, it is not valid to use IRQF_COND_SUSPEND.
* RTC_PIE_ON, RTC_PIE_OFF: These are also emulated by the generic code.
-If all else fails, check out the rtc-test.c driver!
-
-
--------------------- 8< ---------------- 8< -----------------------------
-
-/*
- * Real Time Clock Driver Test/Example Program
- *
- * Compile with:
- * gcc -s -Wall -Wstrict-prototypes rtctest.c -o rtctest
- *
- * Copyright (C) 1996, Paul Gortmaker.
- *
- * Released under the GNU General Public License, version 2,
- * included herein by reference.
- *
- */
-
-#include <stdio.h>
-#include <linux/rtc.h>
-#include <sys/ioctl.h>
-#include <sys/time.h>
-#include <sys/types.h>
-#include <fcntl.h>
-#include <unistd.h>
-#include <stdlib.h>
-#include <errno.h>
-
-
-/*
- * This expects the new RTC class driver framework, working with
- * clocks that will often not be clones of what the PC-AT had.
- * Use the command line to specify another RTC if you need one.
- */
-static const char default_rtc[] = "/dev/rtc0";
-
-
-int main(int argc, char **argv)
-{
- int i, fd, retval, irqcount = 0;
- unsigned long tmp, data;
- struct rtc_time rtc_tm;
- const char *rtc = default_rtc;
-
- switch (argc) {
- case 2:
- rtc = argv[1];
- /* FALLTHROUGH */
- case 1:
- break;
- default:
- fprintf(stderr, "usage: rtctest [rtcdev]\n");
- return 1;
- }
-
- fd = open(rtc, O_RDONLY);
-
- if (fd == -1) {
- perror(rtc);
- exit(errno);
- }
-
- fprintf(stderr, "\n\t\t\tRTC Driver Test Example.\n\n");
-
- /* Turn on update interrupts (one per second) */
- retval = ioctl(fd, RTC_UIE_ON, 0);
- if (retval == -1) {
- if (errno == ENOTTY) {
- fprintf(stderr,
- "\n...Update IRQs not supported.\n");
- goto test_READ;
- }
- perror("RTC_UIE_ON ioctl");
- exit(errno);
- }
-
- fprintf(stderr, "Counting 5 update (1/sec) interrupts from reading %s:",
- rtc);
- fflush(stderr);
- for (i=1; i<6; i++) {
- /* This read will block */
- retval = read(fd, &data, sizeof(unsigned long));
- if (retval == -1) {
- perror("read");
- exit(errno);
- }
- fprintf(stderr, " %d",i);
- fflush(stderr);
- irqcount++;
- }
-
- fprintf(stderr, "\nAgain, from using select(2) on /dev/rtc:");
- fflush(stderr);
- for (i=1; i<6; i++) {
- struct timeval tv = {5, 0}; /* 5 second timeout on select */
- fd_set readfds;
-
- FD_ZERO(&readfds);
- FD_SET(fd, &readfds);
- /* The select will wait until an RTC interrupt happens. */
- retval = select(fd+1, &readfds, NULL, NULL, &tv);
- if (retval == -1) {
- perror("select");
- exit(errno);
- }
- /* This read won't block unlike the select-less case above. */
- retval = read(fd, &data, sizeof(unsigned long));
- if (retval == -1) {
- perror("read");
- exit(errno);
- }
- fprintf(stderr, " %d",i);
- fflush(stderr);
- irqcount++;
- }
-
- /* Turn off update interrupts */
- retval = ioctl(fd, RTC_UIE_OFF, 0);
- if (retval == -1) {
- perror("RTC_UIE_OFF ioctl");
- exit(errno);
- }
-
-test_READ:
- /* Read the RTC time/date */
- retval = ioctl(fd, RTC_RD_TIME, &rtc_tm);
- if (retval == -1) {
- perror("RTC_RD_TIME ioctl");
- exit(errno);
- }
-
- fprintf(stderr, "\n\nCurrent RTC date/time is %d-%d-%d, %02d:%02d:%02d.\n",
- rtc_tm.tm_mday, rtc_tm.tm_mon + 1, rtc_tm.tm_year + 1900,
- rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec);
-
- /* Set the alarm to 5 sec in the future, and check for rollover */
- rtc_tm.tm_sec += 5;
- if (rtc_tm.tm_sec >= 60) {
- rtc_tm.tm_sec %= 60;
- rtc_tm.tm_min++;
- }
- if (rtc_tm.tm_min == 60) {
- rtc_tm.tm_min = 0;
- rtc_tm.tm_hour++;
- }
- if (rtc_tm.tm_hour == 24)
- rtc_tm.tm_hour = 0;
-
- retval = ioctl(fd, RTC_ALM_SET, &rtc_tm);
- if (retval == -1) {
- if (errno == ENOTTY) {
- fprintf(stderr,
- "\n...Alarm IRQs not supported.\n");
- goto test_PIE;
- }
- perror("RTC_ALM_SET ioctl");
- exit(errno);
- }
-
- /* Read the current alarm settings */
- retval = ioctl(fd, RTC_ALM_READ, &rtc_tm);
- if (retval == -1) {
- perror("RTC_ALM_READ ioctl");
- exit(errno);
- }
-
- fprintf(stderr, "Alarm time now set to %02d:%02d:%02d.\n",
- rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec);
-
- /* Enable alarm interrupts */
- retval = ioctl(fd, RTC_AIE_ON, 0);
- if (retval == -1) {
- perror("RTC_AIE_ON ioctl");
- exit(errno);
- }
-
- fprintf(stderr, "Waiting 5 seconds for alarm...");
- fflush(stderr);
- /* This blocks until the alarm ring causes an interrupt */
- retval = read(fd, &data, sizeof(unsigned long));
- if (retval == -1) {
- perror("read");
- exit(errno);
- }
- irqcount++;
- fprintf(stderr, " okay. Alarm rang.\n");
-
- /* Disable alarm interrupts */
- retval = ioctl(fd, RTC_AIE_OFF, 0);
- if (retval == -1) {
- perror("RTC_AIE_OFF ioctl");
- exit(errno);
- }
-
-test_PIE:
- /* Read periodic IRQ rate */
- retval = ioctl(fd, RTC_IRQP_READ, &tmp);
- if (retval == -1) {
- /* not all RTCs support periodic IRQs */
- if (errno == ENOTTY) {
- fprintf(stderr, "\nNo periodic IRQ support\n");
- goto done;
- }
- perror("RTC_IRQP_READ ioctl");
- exit(errno);
- }
- fprintf(stderr, "\nPeriodic IRQ rate is %ldHz.\n", tmp);
-
- fprintf(stderr, "Counting 20 interrupts at:");
- fflush(stderr);
-
- /* The frequencies 128Hz, 256Hz, ... 8192Hz are only allowed for root. */
- for (tmp=2; tmp<=64; tmp*=2) {
-
- retval = ioctl(fd, RTC_IRQP_SET, tmp);
- if (retval == -1) {
- /* not all RTCs can change their periodic IRQ rate */
- if (errno == ENOTTY) {
- fprintf(stderr,
- "\n...Periodic IRQ rate is fixed\n");
- goto done;
- }
- perror("RTC_IRQP_SET ioctl");
- exit(errno);
- }
-
- fprintf(stderr, "\n%ldHz:\t", tmp);
- fflush(stderr);
-
- /* Enable periodic interrupts */
- retval = ioctl(fd, RTC_PIE_ON, 0);
- if (retval == -1) {
- perror("RTC_PIE_ON ioctl");
- exit(errno);
- }
-
- for (i=1; i<21; i++) {
- /* This blocks */
- retval = read(fd, &data, sizeof(unsigned long));
- if (retval == -1) {
- perror("read");
- exit(errno);
- }
- fprintf(stderr, " %d",i);
- fflush(stderr);
- irqcount++;
- }
-
- /* Disable periodic interrupts */
- retval = ioctl(fd, RTC_PIE_OFF, 0);
- if (retval == -1) {
- perror("RTC_PIE_OFF ioctl");
- exit(errno);
- }
- }
-
-done:
- fprintf(stderr, "\n\n\t\t\t *** Test complete ***\n");
-
- close(fd);
-
- return 0;
-}
+If all else fails, check out the tools/testing/selftests/timers/rtctest.c test!
irqm:1 same as initial settings (assumed BIOS settings)
irqm:2 always totem pole
irqm:0x10 driver will not use IRQF_SHARED flag when requesting irq
- irqm:0x20 driver will not use IRQF_DISABLED flag when requesting irq
(Bits 0x10 and 0x20 can be combined with hardware irq mode option)
may only be needed under Linux when a scatter/gather list is not used and
when the SCSI DATA IN phase is reentered after a phase mismatch.
-14.5 IRQ sharing problems
-
-When an IRQ is shared by devices that are handled by different drivers, it
-may happen that one driver complains about the request of the IRQ having
-failed. Inder Linux-2.0, this may be due to one driver having requested the
-IRQ using the IRQF_DISABLED flag but some other having requested the same IRQ
-without this flag. Under both Linux-2.0 and linux-2.2, this may be caused by
-one driver not having requested the IRQ with the IRQF_SHARED flag.
-
-By default, the ncr53c8xx and sym53c8xx drivers request IRQs with both the
-IRQF_DISABLED and the IRQF_SHARED flag under Linux-2.0 and with only the IRQF_SHARED
-flag under Linux-2.2.
-
-Under Linux-2.0, you can disable use of IRQF_DISABLED flag from the boot
-command line by using the following option:
-
- ncr53c8xx=irqm:0x20 (for the generic ncr53c8xx driver)
- sym53c8xx=irqm:0x20 (for the sym53c8xx driver)
-
-If this does not fix the problem, then you may want to check how all other
-drivers are requesting the IRQ and report the problem. Note that if at least
-a single driver does not request the IRQ with the IRQF_SHARED flag (share IRQ),
-then the request of the IRQ obviously will not succeed for all the drivers.
-
15. SCSI problem troubleshooting
15.1 Problem tracking
MHz PCI bus works for me, though, but I don't recommend using higher clocks
than the 33.33 MHz being in the PCI spec.
-If you want to share the IRQ with another device and the driver refuses to
-do so, you might succeed with changing the DC390_IRQ type in tmscsim.c to
-IRQF_SHARED | IRQF_DISABLED.
-
3.Features
----------
.driver = {
.name = "CHIP",
.owner = THIS_MODULE,
+ .pm = &CHIP_pm_ops,
},
.probe = CHIP_probe,
.remove = CHIP_remove,
- .suspend = CHIP_suspend,
- .resume = CHIP_resume,
};
The driver core will automatically attempt to bind this driver to any SPI
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
+#include <string.h>
#include <getopt.h>
#include <fcntl.h>
#include <sys/ioctl.h>
static uint8_t bits = 8;
static uint32_t speed = 500000;
static uint16_t delay;
+static int verbose;
-static void transfer(int fd)
+uint8_t default_tx[] = {
+ 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+ 0x40, 0x00, 0x00, 0x00, 0x00, 0x95,
+ 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+ 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+ 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
+ 0xF0, 0x0D,
+};
+
+uint8_t default_rx[ARRAY_SIZE(default_tx)] = {0, };
+char *input_tx;
+
+static void hex_dump(const void *src, size_t length, size_t line_size, char *prefix)
+{
+ int i = 0;
+ const unsigned char *address = src;
+ const unsigned char *line = address;
+ unsigned char c;
+
+ printf("%s | ", prefix);
+ while (length-- > 0) {
+ printf("%02X ", *address++);
+ if (!(++i % line_size) || (length == 0 && i % line_size)) {
+ if (length == 0) {
+ while (i++ % line_size)
+ printf("__ ");
+ }
+ printf(" | "); /* right close */
+ while (line < address) {
+ c = *line++;
+ printf("%c", (c < 33 || c == 255) ? 0x2E : c);
+ }
+ printf("\n");
+ if (length > 0)
+ printf("%s | ", prefix);
+ }
+ }
+}
+
+/*
+ * Unescape - process hexadecimal escape character
+ * converts shell input "\x23" -> 0x23
+ */
+int unespcape(char *_dst, char *_src, size_t len)
+{
+ int ret = 0;
+ char *src = _src;
+ char *dst = _dst;
+ unsigned int ch;
+
+ while (*src) {
+ if (*src == '\\' && *(src+1) == 'x') {
+ sscanf(src + 2, "%2x", &ch);
+ src += 4;
+ *dst++ = (unsigned char)ch;
+ } else {
+ *dst++ = *src++;
+ }
+ ret++;
+ }
+ return ret;
+}
+
+static void transfer(int fd, uint8_t const *tx, uint8_t const *rx, size_t len)
{
int ret;
- uint8_t tx[] = {
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
- 0x40, 0x00, 0x00, 0x00, 0x00, 0x95,
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
- 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
- 0xDE, 0xAD, 0xBE, 0xEF, 0xBA, 0xAD,
- 0xF0, 0x0D,
- };
- uint8_t rx[ARRAY_SIZE(tx)] = {0, };
+
struct spi_ioc_transfer tr = {
.tx_buf = (unsigned long)tx,
.rx_buf = (unsigned long)rx,
- .len = ARRAY_SIZE(tx),
+ .len = len,
.delay_usecs = delay,
.speed_hz = speed,
.bits_per_word = bits,
if (ret < 1)
pabort("can't send spi message");
- for (ret = 0; ret < ARRAY_SIZE(tx); ret++) {
- if (!(ret % 6))
- puts("");
- printf("%.2X ", rx[ret]);
- }
- puts("");
+ if (verbose)
+ hex_dump(tx, len, 32, "TX");
+ hex_dump(rx, len, 32, "RX");
}
static void print_usage(const char *prog)
" -L --lsb least significant bit first\n"
" -C --cs-high chip select active high\n"
" -3 --3wire SI/SO signals shared\n"
+ " -v --verbose Verbose (show tx buffer)\n"
+ " -p Send data (e.g. \"1234\\xde\\xad\")\n"
" -N --no-cs no chip select\n"
" -R --ready slave pulls low to pause\n"
" -2 --dual dual transfer\n"
{ "no-cs", 0, 0, 'N' },
{ "ready", 0, 0, 'R' },
{ "dual", 0, 0, '2' },
+ { "verbose", 0, 0, 'v' },
{ "quad", 0, 0, '4' },
{ NULL, 0, 0, 0 },
};
int c;
- c = getopt_long(argc, argv, "D:s:d:b:lHOLC3NR24", lopts, NULL);
+ c = getopt_long(argc, argv, "D:s:d:b:lHOLC3NR24p:v", lopts, NULL);
if (c == -1)
break;
case 'N':
mode |= SPI_NO_CS;
break;
+ case 'v':
+ verbose = 1;
+ break;
case 'R':
mode |= SPI_READY;
break;
+ case 'p':
+ input_tx = optarg;
+ break;
case '2':
mode |= SPI_TX_DUAL;
break;
{
int ret = 0;
int fd;
+ uint8_t *tx;
+ uint8_t *rx;
+ int size;
parse_opts(argc, argv);
printf("bits per word: %d\n", bits);
printf("max speed: %d Hz (%d KHz)\n", speed, speed/1000);
- transfer(fd);
+ if (input_tx) {
+ size = strlen(input_tx+1);
+ tx = malloc(size);
+ rx = malloc(size);
+ size = unespcape((char *)tx, input_tx, size);
+ transfer(fd, tx, rx, size);
+ free(rx);
+ free(tx);
+ } else {
+ transfer(fd, default_tx, default_rx, sizeof(default_tx));
+ }
close(fd);
4.38 KVM_GET_MP_STATE
Capability: KVM_CAP_MP_STATE
-Architectures: x86, s390
+Architectures: x86, s390, arm, arm64
Type: vcpu ioctl
Parameters: struct kvm_mp_state (out)
Returns: 0 on success; -1 on error
Possible values are:
- - KVM_MP_STATE_RUNNABLE: the vcpu is currently running [x86]
+ - KVM_MP_STATE_RUNNABLE: the vcpu is currently running [x86,arm/arm64]
- KVM_MP_STATE_UNINITIALIZED: the vcpu is an application processor (AP)
which has not yet received an INIT signal [x86]
- KVM_MP_STATE_INIT_RECEIVED: the vcpu has received an INIT signal, and is
is waiting for an interrupt [x86]
- KVM_MP_STATE_SIPI_RECEIVED: the vcpu has just received a SIPI (vector
accessible via KVM_GET_VCPU_EVENTS) [x86]
- - KVM_MP_STATE_STOPPED: the vcpu is stopped [s390]
+ - KVM_MP_STATE_STOPPED: the vcpu is stopped [s390,arm/arm64]
- KVM_MP_STATE_CHECK_STOP: the vcpu is in a special error state [s390]
- KVM_MP_STATE_OPERATING: the vcpu is operating (running or halted)
[s390]
in-kernel irqchip, the multiprocessing state must be maintained by userspace on
these architectures.
+For arm/arm64:
+
+The only states that are valid are KVM_MP_STATE_STOPPED and
+KVM_MP_STATE_RUNNABLE which reflect if the vcpu is paused or not.
4.39 KVM_SET_MP_STATE
Capability: KVM_CAP_MP_STATE
-Architectures: x86, s390
+Architectures: x86, s390, arm, arm64
Type: vcpu ioctl
Parameters: struct kvm_mp_state (in)
Returns: 0 on success; -1 on error
in-kernel irqchip, the multiprocessing state must be maintained by userspace on
these architectures.
+For arm/arm64:
+
+The only states that are valid are KVM_MP_STATE_STOPPED and
+KVM_MP_STATE_RUNNABLE which reflect if the vcpu should be paused or not.
4.40 KVM_SET_IDENTITY_MAP_ADDR
MIPS | KVM_REG_MIPS_CP0_STATUS | 32
MIPS | KVM_REG_MIPS_CP0_CAUSE | 32
MIPS | KVM_REG_MIPS_CP0_EPC | 64
+ MIPS | KVM_REG_MIPS_CP0_PRID | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG1 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG2 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG3 | 32
+ MIPS | KVM_REG_MIPS_CP0_CONFIG4 | 32
+ MIPS | KVM_REG_MIPS_CP0_CONFIG5 | 32
MIPS | KVM_REG_MIPS_CP0_CONFIG7 | 32
MIPS | KVM_REG_MIPS_CP0_ERROREPC | 64
MIPS | KVM_REG_MIPS_COUNT_CTL | 64
MIPS | KVM_REG_MIPS_COUNT_RESUME | 64
MIPS | KVM_REG_MIPS_COUNT_HZ | 64
+ MIPS | KVM_REG_MIPS_FPR_32(0..31) | 32
+ MIPS | KVM_REG_MIPS_FPR_64(0..31) | 64
+ MIPS | KVM_REG_MIPS_VEC_128(0..31) | 128
+ MIPS | KVM_REG_MIPS_FCR_IR | 32
+ MIPS | KVM_REG_MIPS_FCR_CSR | 32
+ MIPS | KVM_REG_MIPS_MSA_IR | 32
+ MIPS | KVM_REG_MIPS_MSA_CSR | 32
ARM registers are mapped using the lower 32 bits. The upper 16 of that
is the register group type, or coprocessor number:
MIPS KVM control registers (see above) have the following id bit patterns:
0x7030 0000 0002 <reg:16>
+MIPS FPU registers (see KVM_REG_MIPS_FPR_{32,64}() above) have the following
+id bit patterns depending on the size of the register being accessed. They are
+always accessed according to the current guest FPU mode (Status.FR and
+Config5.FRE), i.e. as the guest would see them, and they become unpredictable
+if the guest FPU mode is changed. MIPS SIMD Architecture (MSA) vector
+registers (see KVM_REG_MIPS_VEC_128() above) have similar patterns as they
+overlap the FPU registers:
+ 0x7020 0000 0003 00 <0:3> <reg:5> (32-bit FPU registers)
+ 0x7030 0000 0003 00 <0:3> <reg:5> (64-bit FPU registers)
+ 0x7040 0000 0003 00 <0:3> <reg:5> (128-bit MSA vector registers)
+
+MIPS FPU control registers (see KVM_REG_MIPS_FCR_{IR,CSR} above) have the
+following id bit patterns:
+ 0x7020 0000 0003 01 <0:3> <reg:5>
+
+MIPS MSA control registers (see KVM_REG_MIPS_MSA_{IR,CSR} above) have the
+following id bit patterns:
+ 0x7020 0000 0003 02 <0:3> <reg:5>
+
4.69 KVM_GET_ONE_REG
4.75 KVM_IRQFD
Capability: KVM_CAP_IRQFD
-Architectures: x86 s390
+Architectures: x86 s390 arm arm64
Type: vm ioctl
Parameters: struct kvm_irqfd (in)
Returns: 0 on success, -1 on error
irqfd. The KVM_IRQFD_FLAG_RESAMPLE is only necessary on assignment
and need not be specified with KVM_IRQFD_FLAG_DEASSIGN.
+On ARM/ARM64, the gsi field in the kvm_irqfd struct specifies the Shared
+Peripheral Interrupt (SPI) index, such that the GIC interrupt ID is
+given by gsi + 32.
+
4.76 KVM_PPC_ALLOCATE_HTAB
Capability: KVM_CAP_PPC_ALLOC_HTAB
eax, ebx, ecx, edx: the values returned by the cpuid instruction for
this function/index combination
+4.89 KVM_S390_MEM_OP
+
+Capability: KVM_CAP_S390_MEM_OP
+Architectures: s390
+Type: vcpu ioctl
+Parameters: struct kvm_s390_mem_op (in)
+Returns: = 0 on success,
+ < 0 on generic error (e.g. -EFAULT or -ENOMEM),
+ > 0 if an exception occurred while walking the page tables
+
+Read or write data from/to the logical (virtual) memory of a VPCU.
+
+Parameters are specified via the following structure:
+
+struct kvm_s390_mem_op {
+ __u64 gaddr; /* the guest address */
+ __u64 flags; /* flags */
+ __u32 size; /* amount of bytes */
+ __u32 op; /* type of operation */
+ __u64 buf; /* buffer in userspace */
+ __u8 ar; /* the access register number */
+ __u8 reserved[31]; /* should be set to 0 */
+};
+
+The type of operation is specified in the "op" field. It is either
+KVM_S390_MEMOP_LOGICAL_READ for reading from logical memory space or
+KVM_S390_MEMOP_LOGICAL_WRITE for writing to logical memory space. The
+KVM_S390_MEMOP_F_CHECK_ONLY flag can be set in the "flags" field to check
+whether the corresponding memory access would create an access exception
+(without touching the data in the memory at the destination). In case an
+access exception occurred while walking the MMU tables of the guest, the
+ioctl returns a positive error number to indicate the type of exception.
+This exception is also raised directly at the corresponding VCPU if the
+flag KVM_S390_MEMOP_F_INJECT_EXCEPTION is set in the "flags" field.
+
+The start address of the memory region has to be specified in the "gaddr"
+field, and the length of the region in the "size" field. "buf" is the buffer
+supplied by the userspace application where the read data should be written
+to for KVM_S390_MEMOP_LOGICAL_READ, or where the data that should be written
+is stored for a KVM_S390_MEMOP_LOGICAL_WRITE. "buf" is unused and can be NULL
+when KVM_S390_MEMOP_F_CHECK_ONLY is specified. "ar" designates the access
+register number to be used.
+
+The "reserved" field is meant for future extensions. It is not used by
+KVM with the currently defined set of flags.
+
+4.90 KVM_S390_GET_SKEYS
+
+Capability: KVM_CAP_S390_SKEYS
+Architectures: s390
+Type: vm ioctl
+Parameters: struct kvm_s390_skeys
+Returns: 0 on success, KVM_S390_GET_KEYS_NONE if guest is not using storage
+ keys, negative value on error
+
+This ioctl is used to get guest storage key values on the s390
+architecture. The ioctl takes parameters via the kvm_s390_skeys struct.
+
+struct kvm_s390_skeys {
+ __u64 start_gfn;
+ __u64 count;
+ __u64 skeydata_addr;
+ __u32 flags;
+ __u32 reserved[9];
+};
+
+The start_gfn field is the number of the first guest frame whose storage keys
+you want to get.
+
+The count field is the number of consecutive frames (starting from start_gfn)
+whose storage keys to get. The count field must be at least 1 and the maximum
+allowed value is defined as KVM_S390_SKEYS_ALLOC_MAX. Values outside this range
+will cause the ioctl to return -EINVAL.
+
+The skeydata_addr field is the address to a buffer large enough to hold count
+bytes. This buffer will be filled with storage key data by the ioctl.
+
+4.91 KVM_S390_SET_SKEYS
+
+Capability: KVM_CAP_S390_SKEYS
+Architectures: s390
+Type: vm ioctl
+Parameters: struct kvm_s390_skeys
+Returns: 0 on success, negative value on error
+
+This ioctl is used to set guest storage key values on the s390
+architecture. The ioctl takes parameters via the kvm_s390_skeys struct.
+See section on KVM_S390_GET_SKEYS for struct definition.
+
+The start_gfn field is the number of the first guest frame whose storage keys
+you want to set.
+
+The count field is the number of consecutive frames (starting from start_gfn)
+whose storage keys to get. The count field must be at least 1 and the maximum
+allowed value is defined as KVM_S390_SKEYS_ALLOC_MAX. Values outside this range
+will cause the ioctl to return -EINVAL.
+
+The skeydata_addr field is the address to a buffer containing count bytes of
+storage keys. Each byte in the buffer will be set as the storage key for a
+single frame starting at start_gfn for count frames.
+
+Note: If any architecturally invalid key value is found in the given data then
+the ioctl will return -EINVAL.
+
+4.92 KVM_S390_IRQ
+
+Capability: KVM_CAP_S390_INJECT_IRQ
+Architectures: s390
+Type: vcpu ioctl
+Parameters: struct kvm_s390_irq (in)
+Returns: 0 on success, -1 on error
+Errors:
+ EINVAL: interrupt type is invalid
+ type is KVM_S390_SIGP_STOP and flag parameter is invalid value
+ type is KVM_S390_INT_EXTERNAL_CALL and code is bigger
+ than the maximum of VCPUs
+ EBUSY: type is KVM_S390_SIGP_SET_PREFIX and vcpu is not stopped
+ type is KVM_S390_SIGP_STOP and a stop irq is already pending
+ type is KVM_S390_INT_EXTERNAL_CALL and an external call interrupt
+ is already pending
+
+Allows to inject an interrupt to the guest.
+
+Using struct kvm_s390_irq as a parameter allows
+to inject additional payload which is not
+possible via KVM_S390_INTERRUPT.
+
+Interrupt parameters are passed via kvm_s390_irq:
+
+struct kvm_s390_irq {
+ __u64 type;
+ union {
+ struct kvm_s390_io_info io;
+ struct kvm_s390_ext_info ext;
+ struct kvm_s390_pgm_info pgm;
+ struct kvm_s390_emerg_info emerg;
+ struct kvm_s390_extcall_info extcall;
+ struct kvm_s390_prefix_info prefix;
+ struct kvm_s390_stop_info stop;
+ struct kvm_s390_mchk_info mchk;
+ char reserved[64];
+ } u;
+};
+
+type can be one of the following:
+
+KVM_S390_SIGP_STOP - sigp stop; parameter in .stop
+KVM_S390_PROGRAM_INT - program check; parameters in .pgm
+KVM_S390_SIGP_SET_PREFIX - sigp set prefix; parameters in .prefix
+KVM_S390_RESTART - restart; no parameters
+KVM_S390_INT_CLOCK_COMP - clock comparator interrupt; no parameters
+KVM_S390_INT_CPU_TIMER - CPU timer interrupt; no parameters
+KVM_S390_INT_EMERGENCY - sigp emergency; parameters in .emerg
+KVM_S390_INT_EXTERNAL_CALL - sigp external call; parameters in .extcall
+KVM_S390_MCHK - machine check interrupt; parameters in .mchk
+
+
+Note that the vcpu ioctl is asynchronous to vcpu execution.
+
+4.94 KVM_S390_GET_IRQ_STATE
+
+Capability: KVM_CAP_S390_IRQ_STATE
+Architectures: s390
+Type: vcpu ioctl
+Parameters: struct kvm_s390_irq_state (out)
+Returns: >= number of bytes copied into buffer,
+ -EINVAL if buffer size is 0,
+ -ENOBUFS if buffer size is too small to fit all pending interrupts,
+ -EFAULT if the buffer address was invalid
+
+This ioctl allows userspace to retrieve the complete state of all currently
+pending interrupts in a single buffer. Use cases include migration
+and introspection. The parameter structure contains the address of a
+userspace buffer and its length:
+
+struct kvm_s390_irq_state {
+ __u64 buf;
+ __u32 flags;
+ __u32 len;
+ __u32 reserved[4];
+};
+
+Userspace passes in the above struct and for each pending interrupt a
+struct kvm_s390_irq is copied to the provided buffer.
+
+If -ENOBUFS is returned the buffer provided was too small and userspace
+may retry with a bigger buffer.
+
+4.95 KVM_S390_SET_IRQ_STATE
+
+Capability: KVM_CAP_S390_IRQ_STATE
+Architectures: s390
+Type: vcpu ioctl
+Parameters: struct kvm_s390_irq_state (in)
+Returns: 0 on success,
+ -EFAULT if the buffer address was invalid,
+ -EINVAL for an invalid buffer length (see below),
+ -EBUSY if there were already interrupts pending,
+ errors occurring when actually injecting the
+ interrupt. See KVM_S390_IRQ.
+
+This ioctl allows userspace to set the complete state of all cpu-local
+interrupts currently pending for the vcpu. It is intended for restoring
+interrupt state after a migration. The input parameter is a userspace buffer
+containing a struct kvm_s390_irq_state:
+
+struct kvm_s390_irq_state {
+ __u64 buf;
+ __u32 len;
+ __u32 pad;
+};
+
+The userspace memory referenced by buf contains a struct kvm_s390_irq
+for each interrupt to be injected into the guest.
+If one of the interrupts could not be injected for some reason the
+ioctl aborts.
+
+len must be a multiple of sizeof(struct kvm_s390_irq). It must be > 0
+and it must not exceed (max_vcpus + 32) * sizeof(struct kvm_s390_irq),
+which is the maximum number of possibly pending cpu-local interrupts.
+
5. The kvm_run structure
------------------------
This capability enables the in-kernel irqchip for s390. Please refer to
"4.24 KVM_CREATE_IRQCHIP" for details.
+6.9 KVM_CAP_MIPS_FPU
+
+Architectures: mips
+Target: vcpu
+Parameters: args[0] is reserved for future use (should be 0).
+
+This capability allows the use of the host Floating Point Unit by the guest. It
+allows the Config1.FP bit to be set to enable the FPU in the guest. Once this is
+done the KVM_REG_MIPS_FPR_* and KVM_REG_MIPS_FCR_* registers can be accessed
+(depending on the current guest FPU register mode), and the Status.FR,
+Config5.FRE bits are accessible via the KVM API and also from the guest,
+depending on them being supported by the FPU.
+
+6.10 KVM_CAP_MIPS_MSA
+
+Architectures: mips
+Target: vcpu
+Parameters: args[0] is reserved for future use (should be 0).
+
+This capability allows the use of the MIPS SIMD Architecture (MSA) by the guest.
+It allows the Config3.MSAP bit to be set to enable the use of MSA by the guest.
+Once this is done the KVM_REG_MIPS_VEC_* and KVM_REG_MIPS_MSA_* registers can be
+accessed, and the Config5.MSAEn bit is accessible via the KVM API and also from
+the guest.
+
7. Capabilities that can be enabled on VMs
------------------------------------------
Only privileged operation exceptions will be checked for in the kernel (or even
in the hardware prior to interception). If this capability is not enabled, the
old way of handling SIGP orders is used (partially in kernel and user space).
+
+7.3 KVM_CAP_S390_VECTOR_REGISTERS
+
+Architectures: s390
+Parameters: none
+Returns: 0 on success, negative value on error
+
+Allows use of the vector registers introduced with z13 processor, and
+provides for the synchronization between host and user space. Will
+return -EINVAL if the machine does not support vectors.
+
+7.4 KVM_CAP_S390_USER_STSI
+
+Architectures: s390
+Parameters: none
+
+This capability allows post-handlers for the STSI instruction. After
+initial handling in the kernel, KVM exits to user space with
+KVM_EXIT_S390_STSI to allow user space to insert further data.
+
+Before exiting to userspace, kvm handlers should fill in s390_stsi field of
+vcpu->run:
+struct {
+ __u64 addr;
+ __u8 ar;
+ __u8 reserved;
+ __u8 fc;
+ __u8 sel1;
+ __u16 sel2;
+} s390_stsi;
+
+@addr - guest address of STSI SYSIB
+@fc - function code
+@sel1 - selector 1
+@sel2 - selector 2
+@ar - access register number
+
+KVM handlers should exit to userspace with rc = -EREMOTE.
Copies all floating interrupts into a buffer provided by userspace.
When the buffer is too small it returns -ENOMEM, which is the indication
for userspace to try again with a bigger buffer.
+ -ENOBUFS is returned when the allocation of a kernelspace buffer has
+ failed.
+ -EFAULT is returned when copying data to userspace failed.
All interrupts remain pending, i.e. are not deleted from the list of
currently pending interrupts.
attr->addr contains the userspace address of the buffer into which all
- If 0, the protected-mode code is loaded at 0x10000.
- If 1, the protected-mode code is loaded at 0x100000.
+ Bit 1 (kernel internal): ALSR_FLAG
+ - Used internally by the compressed kernel to communicate
+ KASLR status to kernel proper.
+ If 1, KASLR enabled.
+ If 0, KASLR disabled.
+
Bit 5 (write): QUIET_FLAG
- If 0, print early messages.
- If 1, suppress early messages.
F: include/uapi/linux/kfd_ioctl.h
AMD MICROCODE UPDATE SUPPORT
-M: Andreas Herrmann <herrmann.der.user@googlemail.com>
-L: amd64-microcode@amd64.org
+M: Borislav Petkov <bp@alien8.de>
S: Maintained
F: arch/x86/kernel/cpu/microcode/amd*
F: arch/arm/boot/dts/imx*
F: arch/arm/configs/imx*_defconfig
+ARM/FREESCALE VYBRID ARM ARCHITECTURE
+M: Shawn Guo <shawn.guo@linaro.org>
+M: Sascha Hauer <kernel@pengutronix.de>
+R: Stefan Agner <stefan@agner.ch>
+L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
+S: Maintained
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/shawnguo/linux.git
+F: arch/arm/mach-imx/*vf610*
+F: arch/arm/boot/dts/vf*
+
ARM/GLOMATION GESBC9312SX MACHINE SUPPORT
M: Lennert Buytenhek <kernel@wantstofly.org>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: arch/arm/mach-mvebu/
-F: drivers/rtc/armada38x-rtc
+F: drivers/rtc/rtc-armada38x.c
ARM/Marvell Berlin SoC support
M: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
M: Jason Cooper <jason@lakedaemon.net>
M: Andrew Lunn <andrew@lunn.ch>
M: Sebastian Hesselbarth <sebastian.hesselbarth@gmail.com>
+M: Gregory Clement <gregory.clement@free-electrons.com>
L: linux-arm-kernel@lists.infradead.org (moderated for non-subscribers)
S: Maintained
F: arch/arm/mach-dove/
F: drivers/*/*rockchip*
F: drivers/*/*/*rockchip*
F: sound/soc/rockchip/
+N: rockchip
ARM/SAMSUNG EXYNOS ARM ARCHITECTURES
M: Kukjin Kim <kgene@kernel.org>
F: include/linux/platform_data/at24.h
ATA OVER ETHERNET (AOE) DRIVER
-M: "Ed L. Cashin" <ecashin@coraid.com>
-W: http://support.coraid.com/support/linux
+M: "Ed L. Cashin" <ed.cashin@acm.org>
+W: http://www.openaoe.org/
S: Supported
F: Documentation/aoe/
F: drivers/block/aoe/
F: drivers/net/ethernet/atheros/
ATM
-M: Chas Williams <chas@cmf.nrl.navy.mil>
+M: Chas Williams <3chas3@gmail.com>
L: linux-atm-general@lists.sourceforge.net (moderated for non-subscribers)
L: netdev@vger.kernel.org
W: http://linux-atm.sourceforge.net
BONDING DRIVER
M: Jay Vosburgh <j.vosburgh@gmail.com>
M: Veaceslav Falico <vfalico@gmail.com>
-M: Andy Gospodarek <andy@greyhouse.net>
+M: Andy Gospodarek <gospo@cumulusnetworks.com>
L: netdev@vger.kernel.org
W: http://sourceforge.net/projects/bonding/
S: Supported
BROADCOM BCM281XX/BCM11XXX/BCM216XX ARM ARCHITECTURE
M: Christian Daudt <bcm@fixthebug.org>
-M: Matt Porter <mporter@linaro.org>
M: Florian Fainelli <f.fainelli@gmail.com>
L: bcm-kernel-feedback-list@broadcom.com
T: git git://github.com/broadcom/mach-bcm
CAN NETWORK LAYER
M: Oliver Hartkopp <socketcan@hartkopp.net>
+M: Marc Kleine-Budde <mkl@pengutronix.de>
L: linux-can@vger.kernel.org
-W: http://gitorious.org/linux-can
+W: https://github.com/linux-can
T: git git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can-next.git
S: Maintained
M: Wolfgang Grandegger <wg@grandegger.com>
M: Marc Kleine-Budde <mkl@pengutronix.de>
L: linux-can@vger.kernel.org
-W: http://gitorious.org/linux-can
+W: https://github.com/linux-can
T: git git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can.git
T: git git://git.kernel.org/pub/scm/linux/kernel/git/mkl/linux-can-next.git
S: Maintained
F: Documentation/hwmon/da90??
F: drivers/gpio/gpio-da90??.c
F: drivers/hwmon/da90??-hwmon.c
+F: drivers/iio/adc/da91??-*.c
F: drivers/input/misc/da90??_onkey.c
F: drivers/input/touchscreen/da9052_tsi.c
F: drivers/leds/leds-da90??.c
F: drivers/mfd/da903x.c
F: drivers/mfd/da90??-*.c
+F: drivers/mfd/da91??-*.c
F: drivers/power/da9052-battery.c
+F: drivers/power/da91??-*.c
F: drivers/regulator/da903x.c
F: drivers/regulator/da9???-regulator.[ch]
F: drivers/rtc/rtc-da90??.c
F: include/linux/mfd/da9052/
F: include/linux/mfd/da9055/
F: include/linux/mfd/da9063/
+F: include/linux/mfd/da9150/
F: include/sound/da[79]*.h
F: sound/soc/codecs/da[79]*.[ch]
F: Documentation/hwmon/dme1737
F: drivers/hwmon/dme1737.c
+DMI/SMBIOS SUPPORT
+M: Jean Delvare <jdelvare@suse.de>
+S: Maintained
+F: drivers/firmware/dmi-id.c
+F: drivers/firmware/dmi_scan.c
+F: include/linux/dmi.h
+
DOCKING STATION DRIVER
M: Shaohua Li <shaohua.li@intel.com>
L: linux-acpi@vger.kernel.org
F: drivers/platform/x86/intel_menlow.c
INTEL IA32 MICROCODE UPDATE SUPPORT
-M: Tigran Aivazian <tigran@aivazian.fsnet.co.uk>
+M: Borislav Petkov <bp@alien8.de>
S: Maintained
F: arch/x86/kernel/cpu/microcode/core*
F: arch/x86/kernel/cpu/microcode/intel*
S: Maintained
F: drivers/char/hw_random/ixp4xx-rng.c
-INTEL ETHERNET DRIVERS (e100/e1000/e1000e/fm10k/igb/igbvf/ixgb/ixgbe/ixgbevf/i40e/i40evf)
+INTEL ETHERNET DRIVERS
M: Jeff Kirsher <jeffrey.t.kirsher@intel.com>
-M: Jesse Brandeburg <jesse.brandeburg@intel.com>
-M: Bruce Allan <bruce.w.allan@intel.com>
-M: Carolyn Wyborny <carolyn.wyborny@intel.com>
-M: Don Skidmore <donald.c.skidmore@intel.com>
-M: Greg Rose <gregory.v.rose@intel.com>
-M: Matthew Vick <matthew.vick@intel.com>
-M: John Ronciak <john.ronciak@intel.com>
-M: Mitch Williams <mitch.a.williams@intel.com>
-M: Linux NICS <linux.nics@intel.com>
-L: e1000-devel@lists.sourceforge.net
+R: Jesse Brandeburg <jesse.brandeburg@intel.com>
+R: Shannon Nelson <shannon.nelson@intel.com>
+R: Carolyn Wyborny <carolyn.wyborny@intel.com>
+R: Don Skidmore <donald.c.skidmore@intel.com>
+R: Matthew Vick <matthew.vick@intel.com>
+R: John Ronciak <john.ronciak@intel.com>
+R: Mitch Williams <mitch.a.williams@intel.com>
+L: intel-wired-lan@lists.osuosl.org
W: http://www.intel.com/support/feedback.htm
W: http://e1000.sourceforge.net/
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net.git
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-next.git
+Q: http://patchwork.ozlabs.org/project/intel-wired-lan/list/
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/net-queue.git
+T: git git://git.kernel.org/pub/scm/linux/kernel/git/jkirsher/next-queue.git
S: Supported
F: Documentation/networking/e100.txt
F: Documentation/networking/e1000.txt
F: Documentation/*/kvm*.txt
F: Documentation/virtual/kvm/
F: arch/*/kvm/
+F: arch/x86/kernel/kvm.c
+F: arch/x86/kernel/kvmclock.c
F: arch/*/include/asm/kvm*
F: include/linux/kvm*
F: include/uapi/linux/kvm*
F: include/linux/mfd/
MULTIMEDIA CARD (MMC), SECURE DIGITAL (SD) AND SDIO SUBSYSTEM
-M: Chris Ball <chris@printf.net>
M: Ulf Hansson <ulf.hansson@linaro.org>
L: linux-mmc@vger.kernel.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/cjb/mmc.git
T: git git://git.linaro.org/people/ulf.hansson/mmc.git
S: Maintained
F: drivers/mmc/
M: Mark Fasheh <mfasheh@suse.com>
M: Joel Becker <jlbec@evilplan.org>
L: ocfs2-devel@oss.oracle.com (moderated for non-subscribers)
-W: http://oss.oracle.com/projects/ocfs2/
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/jlbec/ocfs2.git
+W: http://ocfs2.wiki.kernel.org
S: Supported
F: Documentation/filesystems/ocfs2.txt
F: Documentation/filesystems/dlmfs.txt
L: netdev@vger.kernel.org
F: drivers/net/ethernet/samsung/sxgbe/
+SAMSUNG THERMAL DRIVER
+M: Lukasz Majewski <l.majewski@samsung.com>
+L: linux-pm@vger.kernel.org
+L: linux-samsung-soc@vger.kernel.org
+S: Supported
+T: https://github.com/lmajewski/linux-samsung-thermal.git
+F: drivers/thermal/samsung/
+
SAMSUNG USB2 PHY DRIVER
M: Kamil Debski <k.debski@samsung.com>
L: linux-kernel@vger.kernel.org
F: kernel/time/clocksource.c
F: kernel/time/time*.c
F: kernel/time/ntp.c
+F: tools/testing/selftests/timers/
SC1200 WDT DRIVER
M: Zwane Mwaikambo <zwanem@gmail.com>
F: drivers/mmc/host/sdricoh_cs.c
SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) DRIVER
-M: Chris Ball <chris@printf.net>
L: linux-mmc@vger.kernel.org
-T: git git://git.kernel.org/pub/scm/linux/kernel/git/cjb/mmc.git
-S: Maintained
+S: Orphan
F: drivers/mmc/host/sdhci.*
F: drivers/mmc/host/sdhci-pltfm.[ch]
K: \bsecure_computing
K: \bTIF_SECCOMP\b
-SECURE DIGITAL HOST CONTROLLER INTERFACE, OPEN FIRMWARE BINDINGS (SDHCI-OF)
-M: Anton Vorontsov <anton@enomsg.org>
-L: linuxppc-dev@lists.ozlabs.org
-L: linux-mmc@vger.kernel.org
-S: Maintained
-F: drivers/mmc/host/sdhci-pltfm.[ch]
-
SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) SAMSUNG DRIVER
M: Ben Dooks <ben-linux@fluff.org>
+M: Jaehoon Chung <jh80.chung@samsung.com>
L: linux-mmc@vger.kernel.org
S: Maintained
-F: drivers/mmc/host/sdhci-s3c.c
+F: drivers/mmc/host/sdhci-s3c*
SECURE DIGITAL HOST CONTROLLER INTERFACE (SDHCI) ST SPEAR DRIVER
M: Viresh Kumar <viresh.linux@gmail.com>
F: Documentation/usb/ohci.txt
F: drivers/usb/host/ohci*
+USB OTG FSM (Finite State Machine)
+M: Peter Chen <Peter.Chen@freescale.com>
+T: git git://github.com/hzpeterchen/linux-usb.git
+L: linux-usb@vger.kernel.org
+S: Maintained
+F: drivers/usb/common/usb-otg-fsm.c
+
USB OVER IP DRIVER
M: Valentina Manea <valentina.manea.m@gmail.com>
M: Shuah Khan <shuah.kh@samsung.com>
VERSION = 4
PATCHLEVEL = 0
SUBLEVEL = 0
-EXTRAVERSION = -rc1
+EXTRAVERSION =
NAME = Hurr durr I'ma sheep
# *DOCUMENTATION*
# check for 'asm goto'
ifeq ($(shell $(CONFIG_SHELL) $(srctree)/scripts/gcc-goto.sh $(CC)), y)
KBUILD_CFLAGS += -DCC_HAVE_ASM_GOTO
+ KBUILD_AFLAGS += -DCC_HAVE_ASM_GOTO
endif
include $(srctree)/scripts/Makefile.kasan
bus = pci_scan_root_bus(NULL, next_busno, alpha_mv.pci_ops,
hose, &resources);
+ if (!bus)
+ continue;
hose->bus = bus;
hose->need_domain_info = need_domain_info;
next_busno = bus->busn_res.end + 1;
pci_assign_unassigned_resources();
pci_fixup_irqs(alpha_mv.pci_swizzle, alpha_mv.pci_map_irq);
+ for (hose = hose_head; hose; hose = hose->next) {
+ bus = hose->bus;
+ if (bus)
+ pci_bus_add_devices(bus);
+ }
}
}
static int
-alpha_rtc_set_mmss(struct device *dev, unsigned long nowtime)
+alpha_rtc_set_mmss(struct device *dev, time64_t nowtime)
{
int retval = 0;
int real_seconds, real_minutes, cmos_minutes;
static const struct rtc_class_ops alpha_rtc_ops = {
.read_time = alpha_rtc_read_time,
.set_time = alpha_rtc_set_time,
- .set_mmss = alpha_rtc_set_mmss,
+ .set_mmss64 = alpha_rtc_set_mmss,
.ioctl = alpha_rtc_ioctl,
};
}
static int
-remote_set_mmss(struct device *dev, unsigned long now)
+remote_set_mmss(struct device *dev, time64_t now)
{
union remote_data x;
if (smp_processor_id() != boot_cpuid) {
static const struct rtc_class_ops remote_rtc_ops = {
.read_time = remote_read_time,
.set_time = remote_set_time,
- .set_mmss = remote_set_mmss,
+ .set_mmss64 = remote_set_mmss,
.ioctl = alpha_rtc_ioctl,
};
#endif
/* Scan our single hose. */
bus = pci_scan_bus(0, alpha_mv.pci_ops, hose);
+ if (!bus)
+ return;
+
hose->bus = bus;
pcibios_claim_one_bus(bus);
for the root bus, so just clear it. */
bus->self = NULL;
pci_fixup_irqs(alpha_mv.pci_swizzle, alpha_mv.pci_map_irq);
+ pci_bus_add_devices(bus);
}
/*
/* Forward declaration, a strange C thing */
struct task_struct;
-/* Return saved PC of a blocked thread */
-unsigned long thread_saved_pc(struct task_struct *t);
-
#define task_pt_regs(p) \
((struct pt_regs *)(THREAD_SIZE + (void *)task_stack_page(p)) - 1)
#define release_segments(mm) do { } while (0)
#define KSTK_EIP(tsk) (task_pt_regs(tsk)->ret)
+#define KSTK_ESP(tsk) (task_pt_regs(tsk)->sp)
/*
* Where abouts of Task's sp, fp, blink when it was last seen in kernel mode.
* Look in process.c for details of kernel stack layout
*/
-#define KSTK_ESP(tsk) (tsk->thread.ksp)
+#define TSK_K_ESP(tsk) (tsk->thread.ksp)
-#define KSTK_REG(tsk, off) (*((unsigned int *)(KSTK_ESP(tsk) + \
+#define TSK_K_REG(tsk, off) (*((unsigned int *)(TSK_K_ESP(tsk) + \
sizeof(struct callee_regs) + off)))
-#define KSTK_BLINK(tsk) KSTK_REG(tsk, 4)
-#define KSTK_FP(tsk) KSTK_REG(tsk, 0)
+#define TSK_K_BLINK(tsk) TSK_K_REG(tsk, 4)
+#define TSK_K_FP(tsk) TSK_K_REG(tsk, 0)
+
+#define thread_saved_pc(tsk) TSK_K_BLINK(tsk)
extern void start_thread(struct pt_regs * regs, unsigned long pc,
unsigned long usp);
--- /dev/null
+/*
+ * Copyright (C) 2014-15 Synopsys, Inc. (www.synopsys.com)
+ * Copyright (C) 2007-2010, 2011-2012 Synopsys, Inc. (www.synopsys.com)
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef __ASM_STACKTRACE_H
+#define __ASM_STACKTRACE_H
+
+#include <linux/sched.h>
+
+/**
+ * arc_unwind_core - Unwind the kernel mode stack for an execution context
+ * @tsk: NULL for current task, specific task otherwise
+ * @regs: pt_regs used to seed the unwinder {SP, FP, BLINK, PC}
+ * If NULL, use pt_regs of @tsk (if !NULL) otherwise
+ * use the current values of {SP, FP, BLINK, PC}
+ * @consumer_fn: Callback invoked for each frame unwound
+ * Returns 0 to continue unwinding, -1 to stop
+ * @arg: Arg to callback
+ *
+ * Returns the address of first function in stack
+ *
+ * Semantics:
+ * - synchronous unwinding (e.g. dump_stack): @tsk NULL, @regs NULL
+ * - Asynchronous unwinding of sleeping task: @tsk !NULL, @regs NULL
+ * - Asynchronous unwinding of intr/excp etc: @tsk !NULL, @regs !NULL
+ */
+notrace noinline unsigned int arc_unwind_core(
+ struct task_struct *tsk, struct pt_regs *regs,
+ int (*consumer_fn) (unsigned int, void *),
+ void *arg);
+
+#endif /* __ASM_STACKTRACE_H */
return 0;
}
-/*
- * API: expected by schedular Code: If thread is sleeping where is that.
- * What is this good for? it will be always the scheduler or ret_from_fork.
- * So we hard code that anyways.
- */
-unsigned long thread_saved_pc(struct task_struct *t)
-{
- struct pt_regs *regs = task_pt_regs(t);
- unsigned long blink = 0;
-
- /*
- * If the thread being queried for in not itself calling this, then it
- * implies it is not executing, which in turn implies it is sleeping,
- * which in turn implies it got switched OUT by the schedular.
- * In that case, it's kernel mode blink can reliably retrieved as per
- * the picture above (right above pt_regs).
- */
- if (t != current && t->state != TASK_RUNNING)
- blink = *((unsigned int *)regs - 1);
-
- return blink;
-}
-
int elf_check_arch(const struct elf32_hdr *x)
{
unsigned int eflags;
sigset_t *set)
{
int err;
- err = __copy_to_user(&(sf->uc.uc_mcontext.regs), regs,
+ err = __copy_to_user(&(sf->uc.uc_mcontext.regs.scratch), regs,
sizeof(sf->uc.uc_mcontext.regs.scratch));
err |= __copy_to_user(&sf->uc.uc_sigmask, set, sizeof(sigset_t));
if (!err)
set_current_blocked(&set);
- err |= __copy_from_user(regs, &(sf->uc.uc_mcontext.regs),
+ err |= __copy_from_user(regs, &(sf->uc.uc_mcontext.regs.scratch),
sizeof(sf->uc.uc_mcontext.regs.scratch));
return err;
/* Don't restart from sigreturn */
syscall_wont_restart(regs);
+ /*
+ * Ensure that sigreturn always returns to user mode (in case the
+ * regs saved on user stack got fudged between save and sigreturn)
+ * Otherwise it is easy to panic the kernel with a custom
+ * signal handler and/or restorer which clobberes the status32/ret
+ * to return to a bogus location in kernel mode.
+ */
+ regs->status32 |= STATUS_U_MASK;
+
return regs->r0;
badframe:
/*
* handler returns using sigreturn stub provided already by userpsace
+ * If not, nuke the process right away
*/
- BUG_ON(!(ksig->ka.sa.sa_flags & SA_RESTORER));
+ if(!(ksig->ka.sa.sa_flags & SA_RESTORER))
+ return 1;
+
regs->blink = (unsigned long)ksig->ka.sa.sa_restorer;
/* User Stack for signal handler will be above the frame just carved */
handle_signal(struct ksignal *ksig, struct pt_regs *regs)
{
sigset_t *oldset = sigmask_to_save();
- int ret;
+ int failed;
/* Set up the stack frame */
- ret = setup_rt_frame(ksig, oldset, regs);
+ failed = setup_rt_frame(ksig, oldset, regs);
- signal_setup_done(ret, ksig, 0);
+ signal_setup_done(failed, ksig, 0);
}
void do_signal(struct pt_regs *regs)
struct pt_regs *regs,
struct unwind_frame_info *frame_info)
{
+ /*
+ * synchronous unwinding (e.g. dump_stack)
+ * - uses current values of SP and friends
+ */
if (tsk == NULL && regs == NULL) {
unsigned long fp, sp, blink, ret;
frame_info->task = current;
frame_info->regs.r63 = ret;
frame_info->call_frame = 0;
} else if (regs == NULL) {
+ /*
+ * Asynchronous unwinding of sleeping task
+ * - Gets SP etc from task's pt_regs (saved bottom of kernel
+ * mode stack of task)
+ */
frame_info->task = tsk;
- frame_info->regs.r27 = KSTK_FP(tsk);
- frame_info->regs.r28 = KSTK_ESP(tsk);
- frame_info->regs.r31 = KSTK_BLINK(tsk);
+ frame_info->regs.r27 = TSK_K_FP(tsk);
+ frame_info->regs.r28 = TSK_K_ESP(tsk);
+ frame_info->regs.r31 = TSK_K_BLINK(tsk);
frame_info->regs.r63 = (unsigned int)__switch_to;
/* In the prologue of __switch_to, first FP is saved on stack
frame_info->call_frame = 0;
} else {
+ /*
+ * Asynchronous unwinding of intr/exception
+ * - Just uses the pt_regs passed
+ */
frame_info->task = tsk;
frame_info->regs.r27 = regs->fp;
#endif
-static noinline unsigned int
+notrace noinline unsigned int
arc_unwind_core(struct task_struct *tsk, struct pt_regs *regs,
int (*consumer_fn) (unsigned int, void *), void *arg)
{
*/
#include <linux/types.h>
+#include <linux/perf_event.h>
#include <linux/ptrace.h>
#include <linux/uaccess.h>
#include <asm/disasm.h>
}
}
+ perf_sw_event(PERF_COUNT_SW_ALIGNMENT_FAULTS, 1, regs, address);
return 0;
fault:
#include <linux/ptrace.h>
#include <linux/uaccess.h>
#include <linux/kdebug.h>
+#include <linux/perf_event.h>
#include <asm/pgalloc.h>
#include <asm/mmu.h>
return;
}
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS, 1, regs, address);
+
if (likely(!(fault & VM_FAULT_ERROR))) {
if (flags & FAULT_FLAG_ALLOW_RETRY) {
/* To avoid updating stats twice for retry case */
- if (fault & VM_FAULT_MAJOR)
+ if (fault & VM_FAULT_MAJOR) {
tsk->maj_flt++;
- else
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MAJ, 1,
+ regs, address);
+ } else {
tsk->min_flt++;
+ perf_sw_event(PERF_COUNT_SW_PAGE_FAULTS_MIN, 1,
+ regs, address);
+ }
if (fault & VM_FAULT_RETRY) {
flags &= ~FAULT_FLAG_ALLOW_RETRY;
select GENERIC_CLOCKEVENTS
select GPIO_PXA
select HAVE_IDE
+ select IRQ_DOMAIN
select MULTI_IRQ_HANDLER
select PLAT_PXA
select SPARSE_IRQ
machine-$(CONFIG_ARCH_CLPS711X) += clps711x
machine-$(CONFIG_ARCH_CNS3XXX) += cns3xxx
machine-$(CONFIG_ARCH_DAVINCI) += davinci
+machine-$(CONFIG_ARCH_DIGICOLOR) += digicolor
machine-$(CONFIG_ARCH_DOVE) += dove
machine-$(CONFIG_ARCH_EBSA110) += ebsa110
machine-$(CONFIG_ARCH_EFM32) += efm32
&usb0 {
status = "okay";
+ dr_mode = "peripheral";
};
&usb1 {
cd-gpios = <&gpio0 6 GPIO_ACTIVE_HIGH>;
cd-inverted;
};
+
+&aes {
+ status = "okay";
+};
+
+&sham {
+ status = "okay";
+};
&mmc1 {
vmmc-supply = <&ldo3_reg>;
};
-
-&sham {
- status = "okay";
-};
-
-&aes {
- status = "okay";
-};
dual_emac_res_vlan = <3>;
};
+&phy_sel {
+ rmii-clock-ext;
+};
+
&mac {
pinctrl-names = "default", "sleep";
pinctrl-0 = <&cpsw_default>;
ehrpwm0_tbclk: ehrpwm0_tbclk@44e10664 {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <0>;
reg = <0x0664>;
};
ehrpwm1_tbclk: ehrpwm1_tbclk@44e10664 {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <1>;
reg = <0x0664>;
};
ehrpwm2_tbclk: ehrpwm2_tbclk@44e10664 {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <2>;
reg = <0x0664>;
};
/ {
compatible = "ti,am4372", "ti,am43";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&wakeupgen>;
aliases {
#interrupt-cells = <3>;
reg = <0x48241000 0x1000>,
<0x48240100 0x0100>;
+ interrupt-parent = <&gic>;
+ };
+
+ wakeupgen: interrupt-controller@48281000 {
+ compatible = "ti,omap4-wugen-mpu";
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ reg = <0x48281000 0x1000>;
+ interrupt-parent = <&gic>;
};
l2-cache-controller@48242000 {
reg = <0x24>;
compatible = "ti,tps65218";
interrupts = <GIC_SPI 7 IRQ_TYPE_NONE>; /* NMIn */
- interrupt-parent = <&gic>;
interrupt-controller;
#interrupt-cells = <2>;
>;
};
- i2c1_pins_default: i2c1_pins_default {
- pinctrl-single,pins = <
- 0x15c (PIN_INPUT | SLEWCTRL_FAST | MUX_MODE2) /* spi0_cs0.i2c1_scl */
- 0x158 (PIN_INPUT | SLEWCTRL_FAST | MUX_MODE2) /* spi0_d1.i2c1_sda */
- >;
- };
-
- i2c1_pins_sleep: i2c1_pins_sleep {
- pinctrl-single,pins = <
- 0x15c (PIN_INPUT_PULLDOWN | MUX_MODE7) /* spi0_cs0.i2c1_scl */
- 0x158 (PIN_INPUT_PULLDOWN | MUX_MODE7) /* spi0_d1.i2c1_sda */
- >;
- };
-
mmc1_pins_default: pinmux_mmc1_pins_default {
pinctrl-single,pins = <
0x100 (PIN_INPUT | MUX_MODE0) /* mmc0_clk.mmc0_clk */
status = "okay";
pinctrl-names = "default", "sleep";
pinctrl-0 = <&i2c0_pins_default>;
- pinctrl-1 = <&i2c0_pins_default>;
+ pinctrl-1 = <&i2c0_pins_sleep>;
clock-frequency = <400000>;
at24@50 {
pagesize = <64>;
reg = <0x50>;
};
-};
-
-&i2c1 {
- status = "okay";
- pinctrl-names = "default", "sleep";
- pinctrl-0 = <&i2c1_pins_default>;
- pinctrl-1 = <&i2c1_pins_default>;
- clock-frequency = <400000>;
tps: tps62362@60 {
compatible = "ti,tps62362";
+ reg = <0x60>;
regulator-name = "VDD_MPU";
regulator-min-microvolt = <950000>;
regulator-max-microvolt = <1330000>;
tps@24 {
compatible = "ti,tps65218";
reg = <0x24>;
- interrupt-parent = <&gic>;
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>;
interrupt-controller;
#interrupt-cells = <2>;
reg = <0x24>;
compatible = "ti,tps65218";
interrupts = <GIC_SPI 7 IRQ_TYPE_NONE>; /* NMIn */
- interrupt-parent = <&gic>;
interrupt-controller;
#interrupt-cells = <2>;
ehrpwm0_tbclk: ehrpwm0_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <0>;
reg = <0x0664>;
};
ehrpwm1_tbclk: ehrpwm1_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <1>;
reg = <0x0664>;
};
ehrpwm2_tbclk: ehrpwm2_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <2>;
reg = <0x0664>;
};
ehrpwm3_tbclk: ehrpwm3_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <4>;
reg = <0x0664>;
};
ehrpwm4_tbclk: ehrpwm4_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <5>;
reg = <0x0664>;
};
ehrpwm5_tbclk: ehrpwm5_tbclk {
#clock-cells = <0>;
compatible = "ti,gate-clock";
- clocks = <&dpll_per_m2_ck>;
+ clocks = <&l4ls_gclk>;
ti,bit-shift = <6>;
reg = <0x0664>;
};
mcp_rtc: rtc@6f {
compatible = "microchip,mcp7941x";
reg = <0x6f>;
- interrupt-parent = <&gic>;
interrupts = <GIC_SPI 2 IRQ_TYPE_LEVEL_LOW>; /* IRQ_SYS_1N */
pinctrl-names = "default";
&uart3 {
status = "okay";
- interrupts-extended = <&gic GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&crossbar_mpu GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>,
<&dra7_pmx_core 0x248>;
pinctrl-names = "default";
pinctrl-0 = <&usb1_pins>;
};
-&omap_dwc3_1 {
- extcon = <&extcon_usb1>;
-};
-
-&omap_dwc3_2 {
- extcon = <&extcon_usb2>;
-};
-
&usb2 {
dr_mode = "peripheral";
};
pinctrl_usart3_rts: usart3_rts-0 {
atmel,pins =
- <AT91_PIOB 8 AT91_PERIPH_B AT91_PINCTRL_NONE>; /* PC8 periph B */
+ <AT91_PIOC 8 AT91_PERIPH_B AT91_PINCTRL_NONE>;
};
pinctrl_usart3_cts: usart3_cts-0 {
atmel,pins =
- <AT91_PIOB 10 AT91_PERIPH_B AT91_PINCTRL_NONE>; /* PC10 periph B */
+ <AT91_PIOC 10 AT91_PERIPH_B AT91_PINCTRL_NONE>;
};
};
};
usb1: gadget@fffa4000 {
- compatible = "atmel,at91rm9200-udc";
+ compatible = "atmel,at91sam9260-udc";
reg = <0xfffa4000 0x4000>;
interrupts = <10 IRQ_TYPE_LEVEL_HIGH 2>;
clocks = <&udc_clk>, <&udpck>;
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
};
usb1: gadget@fffa4000 {
- compatible = "atmel,at91rm9200-udc";
+ compatible = "atmel,at91sam9261-udc";
reg = <0xfffa4000 0x4000>;
interrupts = <10 IRQ_TYPE_LEVEL_HIGH 2>;
- clocks = <&usb>, <&udc_clk>, <&udpck>;
- clock-names = "usb_clk", "udc_clk", "udpck";
+ clocks = <&udc_clk>, <&udpck>;
+ clock-names = "pclk", "hclk";
+ atmel,matrix = <&matrix>;
status = "disabled";
};
};
matrix: matrix@ffffee00 {
- compatible = "atmel,at91sam9260-bus-matrix";
+ compatible = "atmel,at91sam9260-bus-matrix", "syscon";
reg = <0xffffee00 0x200>;
};
sram1: sram@00500000 {
compatible = "mmio-sram";
- reg = <0x00300000 0x4000>;
+ reg = <0x00500000 0x4000>;
};
ahb {
};
usb1: gadget@fff78000 {
- compatible = "atmel,at91rm9200-udc";
+ compatible = "atmel,at91sam9263-udc";
reg = <0xfff78000 0x4000>;
interrupts = <24 IRQ_TYPE_LEVEL_HIGH 2>;
clocks = <&udc_clk>, <&udpck>;
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
compatible = "atmel,at91sam9g45-ehci", "usb-ehci";
reg = <0x00800000 0x100000>;
interrupts = <22 IRQ_TYPE_LEVEL_HIGH 2>;
- clocks = <&usb>, <&uhphs_clk>, <&uhphs_clk>, <&uhpck>;
+ clocks = <&utmi>, <&uhphs_clk>, <&uhphs_clk>, <&uhpck>;
clock-names = "usb_clk", "ehci_clk", "hclk", "uhpck";
status = "disabled";
};
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
reg = <0x00500000 0x80000
0xf803c000 0x400>;
interrupts = <23 IRQ_TYPE_LEVEL_HIGH 0>;
- clocks = <&usb>, <&udphs_clk>;
+ clocks = <&utmi>, <&udphs_clk>;
clock-names = "hclk", "pclk";
status = "disabled";
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
compatible = "atmel,at91sam9g45-ehci", "usb-ehci";
reg = <0x00700000 0x100000>;
interrupts = <22 IRQ_TYPE_LEVEL_HIGH 2>;
- clocks = <&usb>, <&uhphs_clk>, <&uhpck>;
+ clocks = <&utmi>, <&uhphs_clk>, <&uhpck>;
clock-names = "usb_clk", "ehci_clk", "uhpck";
status = "disabled";
};
DM816X_IOPAD(0x0aac, PIN_INPUT | MUX_MODE0) /* SPI_D1 */
>;
};
+
+ mmc_pins: pinmux_mmc_pins {
+ pinctrl-single,pins = <
+ DM816X_IOPAD(0x0a70, MUX_MODE0) /* SD_POW */
+ DM816X_IOPAD(0x0a74, MUX_MODE0) /* SD_CLK */
+ DM816X_IOPAD(0x0a78, MUX_MODE0) /* SD_CMD */
+ DM816X_IOPAD(0x0a7C, MUX_MODE0) /* SD_DAT0 */
+ DM816X_IOPAD(0x0a80, MUX_MODE0) /* SD_DAT1 */
+ DM816X_IOPAD(0x0a84, MUX_MODE0) /* SD_DAT2 */
+ DM816X_IOPAD(0x0a88, MUX_MODE0) /* SD_DAT2 */
+ DM816X_IOPAD(0x0a8c, MUX_MODE2) /* GP1[7] */
+ DM816X_IOPAD(0x0a90, MUX_MODE2) /* GP1[8] */
+ >;
+ };
+
+ usb0_pins: pinmux_usb0_pins {
+ pinctrl-single,pins = <
+ DM816X_IOPAD(0x0d00, MUX_MODE0) /* USB0_DRVVBUS */
+ >;
+ };
+
+ usb1_pins: pinmux_usb0_pins {
+ pinctrl-single,pins = <
+ DM816X_IOPAD(0x0d04, MUX_MODE0) /* USB1_DRVVBUS */
+ >;
+ };
};
&i2c1 {
};
&mmc1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&mmc_pins>;
vmmc-supply = <&vmmcsd_fixed>;
+ bus-width = <4>;
+ cd-gpios = <&gpio2 7 GPIO_ACTIVE_LOW>;
+ wp-gpios = <&gpio2 8 GPIO_ACTIVE_LOW>;
+};
+
+/* At least dm8168-evm rev c won't support multipoint, later may */
+&usb0 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&usb0_pins>;
+ mentor,multipoint = <0>;
+};
+
+&usb1 {
+ pinctrl-names = "default";
+ pinctrl-0 = <&usb1_pins>;
+ mentor,multipoint = <0>;
};
/* Device Configuration Registers */
scm_conf: syscon@600 {
- compatible = "syscon";
+ compatible = "syscon", "simple-bus";
reg = <0x600 0x110>;
#address-cells = <1>;
#size-cells = <1>;
+ ranges = <0 0x600 0x110>;
+
+ usb_phy0: usb-phy@20 {
+ compatible = "ti,dm8168-usb-phy";
+ reg = <0x20 0x8>;
+ reg-names = "phy";
+ clocks = <&main_fapll 6>;
+ clock-names = "refclk";
+ #phy-cells = <0>;
+ syscon = <&scm_conf>;
+ };
+
+ usb_phy1: usb-phy@28 {
+ compatible = "ti,dm8168-usb-phy";
+ reg = <0x28 0x8>;
+ reg-names = "phy";
+ clocks = <&main_fapll 6>;
+ clock-names = "refclk";
+ #phy-cells = <0>;
+ syscon = <&scm_conf>;
+ };
};
scrm_clocks: clocks {
};
gpio1: gpio@48032000 {
- compatible = "ti,omap3-gpio";
+ compatible = "ti,omap4-gpio";
ti,hwmods = "gpio1";
+ ti,gpio-always-on;
reg = <0x48032000 0x1000>;
- interrupts = <97>;
+ interrupts = <96>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
};
gpio2: gpio@4804c000 {
- compatible = "ti,omap3-gpio";
+ compatible = "ti,omap4-gpio";
ti,hwmods = "gpio2";
+ ti,gpio-always-on;
reg = <0x4804c000 0x1000>;
- interrupts = <99>;
+ interrupts = <98>;
+ gpio-controller;
+ #gpio-cells = <2>;
+ interrupt-controller;
+ #interrupt-cells = <2>;
};
gpmc: gpmc@50000000 {
reg-names = "mc", "control";
interrupts = <18>;
interrupt-names = "mc";
- dr_mode = "otg";
+ dr_mode = "host";
+ interface-type = <0>;
+ phys = <&usb_phy0>;
+ phy-names = "usb2-phy";
mentor,multipoint = <1>;
mentor,num-eps = <16>;
mentor,ram-bits = <12>;
usb1: usb@47401800 {
compatible = "ti,musb-am33xx";
- status = "disabled";
reg = <0x47401c00 0x400
0x47401800 0x200>;
reg-names = "mc", "control";
interrupts = <19>;
interrupt-names = "mc";
- dr_mode = "otg";
+ dr_mode = "host";
+ interface-type = <0>;
+ phys = <&usb_phy1>;
+ phy-names = "usb2-phy";
mentor,multipoint = <1>;
mentor,num-eps = <16>;
mentor,ram-bits = <12>;
dcan1_pins_default: dcan1_pins_default {
pinctrl-single,pins = <
- 0x3d0 (PIN_OUTPUT | MUX_MODE0) /* dcan1_tx */
- 0x3d4 (MUX_MODE15) /* dcan1_rx.off */
- 0x418 (PULL_DIS | MUX_MODE1) /* wakeup0.dcan1_rx */
+ 0x3d0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* dcan1_tx */
+ 0x418 (PULL_UP | MUX_MODE1) /* wakeup0.dcan1_rx */
>;
};
dcan1_pins_sleep: dcan1_pins_sleep {
pinctrl-single,pins = <
- 0x3d0 (MUX_MODE15) /* dcan1_tx.off */
- 0x3d4 (MUX_MODE15) /* dcan1_rx.off */
- 0x418 (MUX_MODE15) /* wakeup0.off */
+ 0x3d0 (MUX_MODE15 | PULL_UP) /* dcan1_tx.off */
+ 0x418 (MUX_MODE15 | PULL_UP) /* wakeup0.off */
>;
};
};
status = "okay";
pinctrl-names = "default";
pinctrl-0 = <&uart1_pins>;
- interrupts-extended = <&gic GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
+ interrupts-extended = <&crossbar_mpu GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>,
<&dra7_pmx_core 0x3e0>;
};
};
};
-&omap_dwc3_1 {
- extcon = <&extcon_usb1>;
-};
-
-&omap_dwc3_2 {
- extcon = <&extcon_usb2>;
-};
-
&usb1 {
dr_mode = "peripheral";
pinctrl-names = "default";
#include "skeleton.dtsi"
#define MAX_SOURCES 400
-#define DIRECT_IRQ(irq) (MAX_SOURCES + irq)
/ {
#address-cells = <1>;
#size-cells = <1>;
compatible = "ti,dra7xx";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&crossbar_mpu>;
aliases {
i2c0 = &i2c1;
<GIC_PPI 14 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 11 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 10 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_LOW)>;
+ interrupt-parent = <&gic>;
};
gic: interrupt-controller@48211000 {
compatible = "arm,cortex-a15-gic";
interrupt-controller;
#interrupt-cells = <3>;
- arm,routable-irqs = <192>;
reg = <0x48211000 0x1000>,
<0x48212000 0x1000>,
<0x48214000 0x2000>,
<0x48216000 0x2000>;
interrupts = <GIC_PPI 9 (GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_HIGH)>;
+ interrupt-parent = <&gic>;
+ };
+
+ wakeupgen: interrupt-controller@48281000 {
+ compatible = "ti,omap5-wugen-mpu", "ti,omap4-wugen-mpu";
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ reg = <0x48281000 0x1000>;
+ interrupt-parent = <&gic>;
};
/*
ti,hwmods = "l3_main_1", "l3_main_2";
reg = <0x44000000 0x1000000>,
<0x45000000 0x1000>;
- interrupts = <GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI DIRECT_IRQ(10) IRQ_TYPE_LEVEL_HIGH>;
+ interrupts-extended = <&crossbar_mpu GIC_SPI 4 IRQ_TYPE_LEVEL_HIGH>,
+ <&wakeupgen GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
prm: prm@4ae06000 {
compatible = "ti,dra7-prm";
<GIC_SPI 9 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 10 IRQ_TYPE_LEVEL_HIGH>;
#dma-cells = <1>;
- #dma-channels = <32>;
- #dma-requests = <127>;
+ dma-channels = <32>;
+ dma-requests = <127>;
};
gpio1: gpio@4ae10000 {
uart1: serial@4806a000 {
compatible = "ti,omap4-uart";
reg = <0x4806a000 0x100>;
- interrupts-extended = <&gic GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts-extended = <&crossbar_mpu GIC_SPI 67 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart1";
clock-frequency = <48000000>;
status = "disabled";
uart2: serial@4806c000 {
compatible = "ti,omap4-uart";
reg = <0x4806c000 0x100>;
- interrupts-extended = <&gic GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 68 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart2";
clock-frequency = <48000000>;
status = "disabled";
uart3: serial@48020000 {
compatible = "ti,omap4-uart";
reg = <0x48020000 0x100>;
- interrupts-extended = <&gic GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 69 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart3";
clock-frequency = <48000000>;
status = "disabled";
uart4: serial@4806e000 {
compatible = "ti,omap4-uart";
reg = <0x4806e000 0x100>;
- interrupts-extended = <&gic GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 65 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart4";
clock-frequency = <48000000>;
status = "disabled";
uart5: serial@48066000 {
compatible = "ti,omap4-uart";
reg = <0x48066000 0x100>;
- interrupts-extended = <&gic GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 100 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart5";
clock-frequency = <48000000>;
status = "disabled";
uart6: serial@48068000 {
compatible = "ti,omap4-uart";
reg = <0x48068000 0x100>;
- interrupts-extended = <&gic GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 101 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart6";
clock-frequency = <48000000>;
status = "disabled";
uart7: serial@48420000 {
compatible = "ti,omap4-uart";
reg = <0x48420000 0x100>;
- interrupts-extended = <&gic GIC_SPI 218 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 218 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart7";
clock-frequency = <48000000>;
status = "disabled";
uart8: serial@48422000 {
compatible = "ti,omap4-uart";
reg = <0x48422000 0x100>;
- interrupts-extended = <&gic GIC_SPI 219 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 219 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart8";
clock-frequency = <48000000>;
status = "disabled";
uart9: serial@48424000 {
compatible = "ti,omap4-uart";
reg = <0x48424000 0x100>;
- interrupts-extended = <&gic GIC_SPI 220 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 220 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart9";
clock-frequency = <48000000>;
status = "disabled";
uart10: serial@4ae2b000 {
compatible = "ti,omap4-uart";
reg = <0x4ae2b000 0x100>;
- interrupts-extended = <&gic GIC_SPI 221 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 221 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart10";
clock-frequency = <48000000>;
status = "disabled";
<0x4A096800 0x40>; /* pll_ctrl */
reg-names = "phy_rx", "phy_tx", "pll_ctrl";
ctrl-module = <&omap_control_sata>;
- clocks = <&sys_clkin1>;
- clock-names = "sysclk";
+ clocks = <&sys_clkin1>, <&sata_ref_clk>;
+ clock-names = "sysclk", "refclk";
#phy-cells = <0>;
};
"wkupclk", "refclk",
"div-clk", "phy-div";
#phy-cells = <0>;
- ti,hwmods = "pcie1-phy";
};
pcie2_phy: pciephy@4a095000 {
"wkupclk", "refclk",
"div-clk", "phy-div";
#phy-cells = <0>;
- ti,hwmods = "pcie2-phy";
status = "disabled";
};
};
status = "disabled";
};
- crossbar_mpu: crossbar@4a020000 {
+ crossbar_mpu: crossbar@4a002a48 {
compatible = "ti,irq-crossbar";
reg = <0x4a002a48 0x130>;
+ interrupt-controller;
+ interrupt-parent = <&wakeupgen>;
+ #interrupt-cells = <3>;
ti,max-irqs = <160>;
ti,max-crossbar-sources = <MAX_SOURCES>;
ti,reg-size = <2>;
dcan1_pins_default: dcan1_pins_default {
pinctrl-single,pins = <
- 0x3d0 (PIN_OUTPUT | MUX_MODE0) /* dcan1_tx */
- 0x3d4 (MUX_MODE15) /* dcan1_rx.off */
- 0x418 (PULL_DIS | MUX_MODE1) /* wakeup0.dcan1_rx */
+ 0x3d0 (PIN_OUTPUT_PULLUP | MUX_MODE0) /* dcan1_tx */
+ 0x418 (PULL_UP | MUX_MODE1) /* wakeup0.dcan1_rx */
>;
};
dcan1_pins_sleep: dcan1_pins_sleep {
pinctrl-single,pins = <
- 0x3d0 (MUX_MODE15) /* dcan1_tx.off */
- 0x3d4 (MUX_MODE15) /* dcan1_rx.off */
- 0x418 (MUX_MODE15) /* wakeup0.off */
+ 0x3d0 (MUX_MODE15 | PULL_UP) /* dcan1_tx.off */
+ 0x418 (MUX_MODE15 | PULL_UP) /* wakeup0.off */
>;
};
pinctrl-0 = <&tps65917_pins_default>;
interrupts = <GIC_SPI 2 IRQ_TYPE_NONE>; /* IRQ_SYS_1N */
- interrupt-parent = <&gic>;
interrupt-controller;
#interrupt-cells = <2>;
phy-supply = <&ldo4_reg>;
};
-&omap_dwc3_1 {
- extcon = <&extcon_usb1>;
-};
-
-&omap_dwc3_2 {
- extcon = <&extcon_usb2>;
-};
-
&usb1 {
dr_mode = "peripheral";
pinctrl-names = "default";
pmu {
compatible = "arm,cortex-a15-pmu";
- interrupts = <GIC_SPI DIRECT_IRQ(131) IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-parent = <&wakeupgen>;
+ interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>;
};
};
pmu {
compatible = "arm,cortex-a15-pmu";
- interrupts = <GIC_SPI DIRECT_IRQ(131) IRQ_TYPE_LEVEL_HIGH>,
- <GIC_SPI DIRECT_IRQ(132) IRQ_TYPE_LEVEL_HIGH>;
+ interrupt-parent = <&wakeupgen>;
+ interrupts = <GIC_SPI 131 IRQ_TYPE_LEVEL_HIGH>,
+ <GIC_SPI 132 IRQ_TYPE_LEVEL_HIGH>;
};
ocp {
ti,invert-autoidle-bit;
};
+ dpll_core_byp_mux: dpll_core_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ ti,bit-shift = <23>;
+ reg = <0x012c>;
+ };
+
dpll_core_ck: dpll_core_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-core-clock";
- clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ clocks = <&sys_clkin1>, <&dpll_core_byp_mux>;
reg = <0x0120>, <0x0124>, <0x012c>, <0x0128>;
};
clock-div = <1>;
};
+ dpll_dsp_byp_mux: dpll_dsp_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&dsp_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x0240>;
+ };
+
dpll_dsp_ck: dpll_dsp_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&dsp_dpll_hs_clk_div>;
+ clocks = <&sys_clkin1>, <&dpll_dsp_byp_mux>;
reg = <0x0234>, <0x0238>, <0x0240>, <0x023c>;
};
clock-div = <1>;
};
+ dpll_iva_byp_mux: dpll_iva_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&iva_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x01ac>;
+ };
+
dpll_iva_ck: dpll_iva_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&iva_dpll_hs_clk_div>;
+ clocks = <&sys_clkin1>, <&dpll_iva_byp_mux>;
reg = <0x01a0>, <0x01a4>, <0x01ac>, <0x01a8>;
};
clock-div = <1>;
};
+ dpll_gpu_byp_mux: dpll_gpu_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ ti,bit-shift = <23>;
+ reg = <0x02e4>;
+ };
+
dpll_gpu_ck: dpll_gpu_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ clocks = <&sys_clkin1>, <&dpll_gpu_byp_mux>;
reg = <0x02d8>, <0x02dc>, <0x02e4>, <0x02e0>;
};
clock-div = <1>;
};
+ dpll_ddr_byp_mux: dpll_ddr_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ ti,bit-shift = <23>;
+ reg = <0x021c>;
+ };
+
dpll_ddr_ck: dpll_ddr_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ clocks = <&sys_clkin1>, <&dpll_ddr_byp_mux>;
reg = <0x0210>, <0x0214>, <0x021c>, <0x0218>;
};
ti,invert-autoidle-bit;
};
+ dpll_gmac_byp_mux: dpll_gmac_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ ti,bit-shift = <23>;
+ reg = <0x02b4>;
+ };
+
dpll_gmac_ck: dpll_gmac_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&dpll_abe_m3x2_ck>;
+ clocks = <&sys_clkin1>, <&dpll_gmac_byp_mux>;
reg = <0x02a8>, <0x02ac>, <0x02b4>, <0x02b0>;
};
clock-div = <1>;
};
+ dpll_eve_byp_mux: dpll_eve_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&eve_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x0290>;
+ };
+
dpll_eve_ck: dpll_eve_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&eve_dpll_hs_clk_div>;
+ clocks = <&sys_clkin1>, <&dpll_eve_byp_mux>;
reg = <0x0284>, <0x0288>, <0x0290>, <0x028c>;
};
clock-div = <1>;
};
+ dpll_per_byp_mux: dpll_per_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&per_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x014c>;
+ };
+
dpll_per_ck: dpll_per_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin1>, <&per_dpll_hs_clk_div>;
+ clocks = <&sys_clkin1>, <&dpll_per_byp_mux>;
reg = <0x0140>, <0x0144>, <0x014c>, <0x0148>;
};
clock-div = <1>;
};
+ dpll_usb_byp_mux: dpll_usb_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin1>, <&usb_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x018c>;
+ };
+
dpll_usb_ck: dpll_usb_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-j-type-clock";
- clocks = <&sys_clkin1>, <&usb_dpll_hs_clk_div>;
+ clocks = <&sys_clkin1>, <&dpll_usb_byp_mux>;
reg = <0x0180>, <0x0184>, <0x018c>, <0x0188>;
};
*/
#include "skeleton.dtsi"
+#include "exynos4-cpu-thermal.dtsi"
#include <dt-bindings/clock/exynos3250.h>
/ {
pmu_system_controller: system-controller@10020000 {
compatible = "samsung,exynos3250-pmu", "syscon";
reg = <0x10020000 0x4000>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
};
mipi_phy: video-phy@10020710 {
compatible = "samsung,exynos3250-rtc";
reg = <0x10070000 0x100>;
interrupts = <0 73 0>, <0 74 0>;
+ interrupt-parent = <&pmu_system_controller>;
status = "disabled";
};
interrupts = <0 216 0>;
clocks = <&cmu CLK_TMU_APBIF>;
clock-names = "tmu_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
status = "disabled";
};
--- /dev/null
+/*
+ * Device tree sources for Exynos4 thermal zone
+ *
+ * Copyright (c) 2014 Lukasz Majewski <l.majewski@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <dt-bindings/thermal/thermal.h>
+
+/ {
+thermal-zones {
+ cpu_thermal: cpu-thermal {
+ thermal-sensors = <&tmu 0>;
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+ trips {
+ cpu_alert0: cpu-alert-0 {
+ temperature = <70000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu_alert1: cpu-alert-1 {
+ temperature = <95000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu_alert2: cpu-alert-2 {
+ temperature = <110000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu_crit0: cpu-crit-0 {
+ temperature = <120000>; /* millicelsius */
+ hysteresis = <0>; /* millicelsius */
+ type = "critical";
+ };
+ };
+ cooling-maps {
+ map0 {
+ trip = <&cpu_alert0>;
+ };
+ map1 {
+ trip = <&cpu_alert1>;
+ };
+ };
+ };
+};
+};
i2c5 = &i2c_5;
i2c6 = &i2c_6;
i2c7 = &i2c_7;
+ i2c8 = &i2c_8;
csis0 = &csis_0;
csis1 = &csis_1;
fimc0 = &fimc_0;
compatible = "samsung,exynos4210-pd";
reg = <0x10023C20 0x20>;
#power-domain-cells = <0>;
+ power-domains = <&pd_lcd0>;
};
pd_cam: cam-power-domain@10023C00 {
pmu_system_controller: system-controller@10020000 {
compatible = "samsung,exynos4210-pmu", "syscon";
reg = <0x10020000 0x4000>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
};
dsi_0: dsi@11C80000 {
rtc@10070000 {
compatible = "samsung,s3c6410-rtc";
reg = <0x10070000 0x100>;
+ interrupt-parent = <&pmu_system_controller>;
interrupts = <0 44 0>, <0 45 0>;
clocks = <&clock CLK_RTC>;
clock-names = "rtc";
status = "disabled";
};
+ i2c_8: i2c@138E0000 {
+ #address-cells = <1>;
+ #size-cells = <0>;
+ compatible = "samsung,s3c2440-hdmiphy-i2c";
+ reg = <0x138E0000 0x100>;
+ interrupts = <0 93 0>;
+ clocks = <&clock CLK_I2C_HDMI>;
+ clock-names = "i2c";
+ status = "disabled";
+
+ hdmi_i2c_phy: hdmiphy@38 {
+ compatible = "exynos4210-hdmiphy";
+ reg = <0x38>;
+ };
+ };
+
spi_0: spi@13920000 {
compatible = "samsung,exynos4210-spi";
reg = <0x13920000 0x100>;
status = "disabled";
};
+ tmu: tmu@100C0000 {
+ #include "exynos4412-tmu-sensor-conf.dtsi"
+ };
+
+ hdmi: hdmi@12D00000 {
+ compatible = "samsung,exynos4210-hdmi";
+ reg = <0x12D00000 0x70000>;
+ interrupts = <0 92 0>;
+ clock-names = "hdmi", "sclk_hdmi", "sclk_pixel", "sclk_hdmiphy",
+ "mout_hdmi";
+ clocks = <&clock CLK_HDMI>, <&clock CLK_SCLK_HDMI>,
+ <&clock CLK_SCLK_PIXEL>, <&clock CLK_SCLK_HDMIPHY>,
+ <&clock CLK_MOUT_HDMI>;
+ phy = <&hdmi_i2c_phy>;
+ power-domains = <&pd_tv>;
+ samsung,syscon-phandle = <&pmu_system_controller>;
+ status = "disabled";
+ };
+
+ mixer: mixer@12C10000 {
+ compatible = "samsung,exynos4210-mixer";
+ interrupts = <0 91 0>;
+ reg = <0x12C10000 0x2100>, <0x12c00000 0x300>;
+ power-domains = <&pd_tv>;
+ status = "disabled";
+ };
+
ppmu_dmc0: ppmu_dmc0@106a0000 {
compatible = "samsung,exynos-ppmu";
reg = <0x106a0000 0x2000>;
status = "okay";
};
+ tmu@100C0000 {
+ status = "okay";
+ };
+
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ cooling-maps {
+ map0 {
+ /* Corresponds to 800MHz at freq_table */
+ cooling-device = <&cpu0 2 2>;
+ };
+ map1 {
+ /* Corresponds to 200MHz at freq_table */
+ cooling-device = <&cpu0 4 4>;
+ };
+ };
+ };
+ };
+
camera {
pinctrl-names = "default";
pinctrl-0 = <>;
assigned-clock-rates = <0>, <160000000>;
};
};
+
+ hdmi_en: voltage-regulator-hdmi-5v {
+ compatible = "regulator-fixed";
+ regulator-name = "HDMI_5V";
+ regulator-min-microvolt = <5000000>;
+ regulator-max-microvolt = <5000000>;
+ gpio = <&gpe0 1 0>;
+ enable-active-high;
+ };
+
+ hdmi_ddc: i2c-ddc {
+ compatible = "i2c-gpio";
+ gpios = <&gpe4 2 0 &gpe4 3 0>;
+ i2c-gpio,delay-us = <100>;
+ #address-cells = <1>;
+ #size-cells = <0>;
+
+ pinctrl-0 = <&i2c_ddc_bus>;
+ pinctrl-names = "default";
+ status = "okay";
+ };
+
+ mixer@12C10000 {
+ status = "okay";
+ };
+
+ hdmi@12D00000 {
+ hpd-gpio = <&gpx3 7 0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&hdmi_hpd>;
+ hdmi-en-supply = <&hdmi_en>;
+ vdd-supply = <&ldo3_reg>;
+ vdd_osc-supply = <&ldo4_reg>;
+ vdd_pll-supply = <&ldo3_reg>;
+ ddc = <&hdmi_ddc>;
+ status = "okay";
+ };
+
+ i2c@138E0000 {
+ status = "okay";
+ };
+};
+
+&pinctrl_1 {
+ hdmi_hpd: hdmi-hpd {
+ samsung,pins = "gpx3-7";
+ samsung,pin-pud = <0>;
+ };
+};
+
+&pinctrl_0 {
+ i2c_ddc_bus: i2c-ddc-bus {
+ samsung,pins = "gpe4-2", "gpe4-3";
+ samsung,pin-function = <2>;
+ samsung,pin-pud = <3>;
+ samsung,pin-drv = <0>;
+ };
};
&mdma1 {
#include "exynos4.dtsi"
#include "exynos4210-pinctrl.dtsi"
+#include "exynos4-cpu-thermal.dtsi"
/ {
compatible = "samsung,exynos4210", "samsung,exynos4";
#address-cells = <1>;
#size-cells = <0>;
- cpu@900 {
+ cpu0: cpu@900 {
device_type = "cpu";
compatible = "arm,cortex-a9";
reg = <0x900>;
+ cooling-min-level = <4>;
+ cooling-max-level = <2>;
+ #cooling-cells = <2>; /* min followed by max */
};
cpu@901 {
reg = <0x03860000 0x1000>;
};
- tmu@100C0000 {
+ tmu: tmu@100C0000 {
compatible = "samsung,exynos4210-tmu";
interrupt-parent = <&combiner>;
reg = <0x100C0000 0x100>;
interrupts = <2 4>;
clocks = <&clock CLK_TMU_APBIF>;
clock-names = "tmu_apbif";
+ samsung,tmu_gain = <15>;
+ samsung,tmu_reference_voltage = <7>;
status = "disabled";
};
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+ thermal-sensors = <&tmu 0>;
+
+ trips {
+ cpu_alert0: cpu-alert-0 {
+ temperature = <85000>; /* millicelsius */
+ };
+ cpu_alert1: cpu-alert-1 {
+ temperature = <100000>; /* millicelsius */
+ };
+ cpu_alert2: cpu-alert-2 {
+ temperature = <110000>; /* millicelsius */
+ };
+ };
+ };
+ };
+
g2d@12800000 {
compatible = "samsung,s5pv210-g2d";
reg = <0x12800000 0x1000>;
};
};
+ mixer: mixer@12C10000 {
+ clock-names = "mixer", "hdmi", "sclk_hdmi", "vp", "mout_mixer",
+ "sclk_mixer";
+ clocks = <&clock CLK_MIXER>, <&clock CLK_HDMI>,
+ <&clock CLK_SCLK_HDMI>, <&clock CLK_VP>,
+ <&clock CLK_MOUT_MIXER>, <&clock CLK_SCLK_MIXER>;
+ };
+
ppmu_lcd1: ppmu_lcd1@12240000 {
compatible = "samsung,exynos-ppmu";
reg = <0x12240000 0x2000>;
#address-cells = <1>;
#size-cells = <0>;
- cpu@A00 {
+ cpu0: cpu@A00 {
device_type = "cpu";
compatible = "arm,cortex-a9";
reg = <0xA00>;
+ cooling-min-level = <13>;
+ cooling-max-level = <7>;
+ #cooling-cells = <2>; /* min followed by max */
};
cpu@A01 {
regulator-always-on;
};
+ ldo8_reg: ldo@8 {
+ regulator-compatible = "LDO8";
+ regulator-name = "VDD10_HDMI_1.0V";
+ regulator-min-microvolt = <1000000>;
+ regulator-max-microvolt = <1000000>;
+ };
+
+ ldo10_reg: ldo@10 {
+ regulator-compatible = "LDO10";
+ regulator-name = "VDDQ_MIPIHSI_1.8V";
+ regulator-min-microvolt = <1800000>;
+ regulator-max-microvolt = <1800000>;
+ };
+
ldo11_reg: LDO11 {
regulator-name = "VDD18_ABB1_1.8V";
regulator-min-microvolt = <1800000>;
ehci: ehci@12580000 {
status = "okay";
};
+
+ tmu@100C0000 {
+ vtmu-supply = <&ldo10_reg>;
+ status = "okay";
+ };
+
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ cooling-maps {
+ map0 {
+ /* Corresponds to 800MHz at freq_table */
+ cooling-device = <&cpu0 7 7>;
+ };
+ map1 {
+ /* Corresponds to 200MHz at freq_table */
+ cooling-device = <&cpu0 13 13>;
+ };
+ };
+ };
+ };
+
+ mixer: mixer@12C10000 {
+ status = "okay";
+ };
+
+ hdmi@12D00000 {
+ hpd-gpio = <&gpx3 7 0>;
+ pinctrl-names = "default";
+ pinctrl-0 = <&hdmi_hpd>;
+ vdd-supply = <&ldo8_reg>;
+ vdd_osc-supply = <&ldo10_reg>;
+ vdd_pll-supply = <&ldo8_reg>;
+ ddc = <&hdmi_ddc>;
+ status = "okay";
+ };
+
+ hdmi_ddc: i2c@13880000 {
+ status = "okay";
+ pinctrl-names = "default";
+ pinctrl-0 = <&i2c2_bus>;
+ };
+
+ i2c@138E0000 {
+ status = "okay";
+ };
};
&pinctrl_1 {
samsung,pin-pud = <0>;
samsung,pin-drv = <0>;
};
+
+ hdmi_hpd: hdmi-hpd {
+ samsung,pins = "gpx3-7";
+ samsung,pin-pud = <1>;
+ };
};
--- /dev/null
+/*
+ * Device tree sources for Exynos4412 TMU sensor configuration
+ *
+ * Copyright (c) 2014 Lukasz Majewski <l.majewski@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <dt-bindings/thermal/thermal_exynos.h>
+
+#thermal-sensor-cells = <0>;
+samsung,tmu_gain = <8>;
+samsung,tmu_reference_voltage = <16>;
+samsung,tmu_noise_cancel_mode = <4>;
+samsung,tmu_efuse_value = <55>;
+samsung,tmu_min_efuse_value = <40>;
+samsung,tmu_max_efuse_value = <100>;
+samsung,tmu_first_point_trim = <25>;
+samsung,tmu_second_point_trim = <85>;
+samsung,tmu_default_temp_offset = <50>;
+samsung,tmu_cal_type = <TYPE_ONE_POINT_TRIMMING>;
pulldown-ohm = <100000>; /* 100K */
io-channels = <&adc 2>; /* Battery temperature */
};
+
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ cooling-maps {
+ map0 {
+ /* Corresponds to 800MHz at freq_table */
+ cooling-device = <&cpu0 7 7>;
+ };
+ map1 {
+ /* Corresponds to 200MHz at freq_table */
+ cooling-device = <&cpu0 13 13>;
+ };
+ };
+ };
+ };
};
&pmu_system_controller {
#address-cells = <1>;
#size-cells = <0>;
- cpu@A00 {
+ cpu0: cpu@A00 {
device_type = "cpu";
compatible = "arm,cortex-a9";
reg = <0xA00>;
+ cooling-min-level = <13>;
+ cooling-max-level = <7>;
+ #cooling-cells = <2>; /* min followed by max */
};
cpu@A01 {
#include "exynos4.dtsi"
#include "exynos4x12-pinctrl.dtsi"
+#include "exynos4-cpu-thermal.dtsi"
/ {
aliases {
clock-names = "tmu_apbif";
status = "disabled";
};
+
+ hdmi: hdmi@12D00000 {
+ compatible = "samsung,exynos4212-hdmi";
+ };
+
+ mixer: mixer@12C10000 {
+ compatible = "samsung,exynos4212-mixer";
+ clock-names = "mixer", "hdmi", "sclk_hdmi", "vp";
+ clocks = <&clock CLK_MIXER>, <&clock CLK_HDMI>,
+ <&clock CLK_SCLK_HDMI>, <&clock CLK_VP>;
+ };
};
#include <dt-bindings/clock/exynos5250.h>
#include "exynos5.dtsi"
#include "exynos5250-pinctrl.dtsi"
-
+#include "exynos4-cpu-thermal.dtsi"
#include <dt-bindings/clock/exynos-audss-clk.h>
/ {
#address-cells = <1>;
#size-cells = <0>;
- cpu@0 {
+ cpu0: cpu@0 {
device_type = "cpu";
compatible = "arm,cortex-a15";
reg = <0>;
clock-frequency = <1700000000>;
+ cooling-min-level = <15>;
+ cooling-max-level = <9>;
+ #cooling-cells = <2>; /* min followed by max */
};
cpu@1 {
device_type = "cpu";
#power-domain-cells = <0>;
};
+ pd_disp1: disp1-power-domain@100440A0 {
+ compatible = "samsung,exynos4210-pd";
+ reg = <0x100440A0 0x20>;
+ #power-domain-cells = <0>;
+ };
+
clock: clock-controller@10010000 {
compatible = "samsung,exynos5250-clock";
reg = <0x10010000 0x30000>;
clock-names = "clkout16";
clocks = <&clock CLK_FIN_PLL>;
#clock-cells = <1>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
};
sysreg_system_controller: syscon@10050000 {
rtc: rtc@101E0000 {
clocks = <&clock CLK_RTC>;
clock-names = "rtc";
+ interrupt-parent = <&pmu_system_controller>;
status = "disabled";
};
- tmu@10060000 {
+ tmu: tmu@10060000 {
compatible = "samsung,exynos5250-tmu";
reg = <0x10060000 0x100>;
interrupts = <0 65 0>;
clocks = <&clock CLK_TMU>;
clock-names = "tmu_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
+ };
+
+ thermal-zones {
+ cpu_thermal: cpu-thermal {
+ polling-delay-passive = <0>;
+ polling-delay = <0>;
+ thermal-sensors = <&tmu 0>;
+
+ cooling-maps {
+ map0 {
+ /* Corresponds to 800MHz at freq_table */
+ cooling-device = <&cpu0 9 9>;
+ };
+ map1 {
+ /* Corresponds to 200MHz at freq_table */
+ cooling-device = <&cpu0 15 15>;
+ };
+ };
+ };
};
serial@12C00000 {
hdmi: hdmi {
compatible = "samsung,exynos4212-hdmi";
reg = <0x14530000 0x70000>;
+ power-domains = <&pd_disp1>;
interrupts = <0 95 0>;
clocks = <&clock CLK_HDMI>, <&clock CLK_SCLK_HDMI>,
<&clock CLK_SCLK_PIXEL>, <&clock CLK_SCLK_HDMIPHY>,
mixer {
compatible = "samsung,exynos5250-mixer";
reg = <0x14450000 0x10000>;
+ power-domains = <&pd_disp1>;
interrupts = <0 94 0>;
- clocks = <&clock CLK_MIXER>, <&clock CLK_SCLK_HDMI>;
- clock-names = "mixer", "sclk_hdmi";
+ clocks = <&clock CLK_MIXER>, <&clock CLK_HDMI>,
+ <&clock CLK_SCLK_HDMI>;
+ clock-names = "mixer", "hdmi", "sclk_hdmi";
};
dp_phy: video-phy@10040720 {
};
dp: dp-controller@145B0000 {
+ power-domains = <&pd_disp1>;
clocks = <&clock CLK_DP>;
clock-names = "dp";
phys = <&dp_phy>;
};
fimd: fimd@14400000 {
+ power-domains = <&pd_disp1>;
clocks = <&clock CLK_SCLK_FIMD1>, <&clock CLK_FIMD1>;
clock-names = "sclk_fimd", "fimd";
};
--- /dev/null
+/*
+ * Device tree sources for default Exynos5420 thermal zone definition
+ *
+ * Copyright (c) 2014 Lukasz Majewski <l.majewski@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+polling-delay-passive = <0>;
+polling-delay = <0>;
+trips {
+ cpu-alert-0 {
+ temperature = <85000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu-alert-1 {
+ temperature = <103000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu-alert-2 {
+ temperature = <110000>; /* millicelsius */
+ hysteresis = <10000>; /* millicelsius */
+ type = "active";
+ };
+ cpu-crit-0 {
+ temperature = <1200000>; /* millicelsius */
+ hysteresis = <0>; /* millicelsius */
+ type = "critical";
+ };
+};
rtc: rtc@101E0000 {
clocks = <&clock CLK_RTC>;
clock-names = "rtc";
+ interrupt-parent = <&pmu_system_controller>;
status = "disabled";
};
compatible = "samsung,exynos5420-mixer";
reg = <0x14450000 0x10000>;
interrupts = <0 94 0>;
- clocks = <&clock CLK_MIXER>, <&clock CLK_SCLK_HDMI>;
- clock-names = "mixer", "sclk_hdmi";
+ clocks = <&clock CLK_MIXER>, <&clock CLK_HDMI>,
+ <&clock CLK_SCLK_HDMI>;
+ clock-names = "mixer", "hdmi", "sclk_hdmi";
power-domains = <&disp_pd>;
};
clock-names = "clkout16";
clocks = <&clock CLK_FIN_PLL>;
#clock-cells = <1>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
};
sysreg_system_controller: syscon@10050000 {
interrupts = <0 65 0>;
clocks = <&clock CLK_TMU>;
clock-names = "tmu_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
};
tmu_cpu1: tmu@10064000 {
interrupts = <0 183 0>;
clocks = <&clock CLK_TMU>;
clock-names = "tmu_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
};
tmu_cpu2: tmu@10068000 {
interrupts = <0 184 0>;
clocks = <&clock CLK_TMU>, <&clock CLK_TMU>;
clock-names = "tmu_apbif", "tmu_triminfo_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
};
tmu_cpu3: tmu@1006c000 {
interrupts = <0 185 0>;
clocks = <&clock CLK_TMU>, <&clock CLK_TMU_GPU>;
clock-names = "tmu_apbif", "tmu_triminfo_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
};
tmu_gpu: tmu@100a0000 {
interrupts = <0 215 0>;
clocks = <&clock CLK_TMU_GPU>, <&clock CLK_TMU>;
clock-names = "tmu_apbif", "tmu_triminfo_apbif";
+ #include "exynos4412-tmu-sensor-conf.dtsi"
+ };
+
+ thermal-zones {
+ cpu0_thermal: cpu0-thermal {
+ thermal-sensors = <&tmu_cpu0>;
+ #include "exynos5420-trip-points.dtsi"
+ };
+ cpu1_thermal: cpu1-thermal {
+ thermal-sensors = <&tmu_cpu1>;
+ #include "exynos5420-trip-points.dtsi"
+ };
+ cpu2_thermal: cpu2-thermal {
+ thermal-sensors = <&tmu_cpu2>;
+ #include "exynos5420-trip-points.dtsi"
+ };
+ cpu3_thermal: cpu3-thermal {
+ thermal-sensors = <&tmu_cpu3>;
+ #include "exynos5420-trip-points.dtsi"
+ };
+ gpu_thermal: gpu-thermal {
+ thermal-sensors = <&tmu_gpu>;
+ #include "exynos5420-trip-points.dtsi"
+ };
};
watchdog: watchdog@101D0000 {
--- /dev/null
+/*
+ * Device tree sources for Exynos5440 TMU sensor configuration
+ *
+ * Copyright (c) 2014 Lukasz Majewski <l.majewski@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+#include <dt-bindings/thermal/thermal_exynos.h>
+
+#thermal-sensor-cells = <0>;
+samsung,tmu_gain = <5>;
+samsung,tmu_reference_voltage = <16>;
+samsung,tmu_noise_cancel_mode = <4>;
+samsung,tmu_efuse_value = <0x5d2d>;
+samsung,tmu_min_efuse_value = <16>;
+samsung,tmu_max_efuse_value = <76>;
+samsung,tmu_first_point_trim = <25>;
+samsung,tmu_second_point_trim = <70>;
+samsung,tmu_default_temp_offset = <25>;
+samsung,tmu_cal_type = <TYPE_ONE_POINT_TRIMMING>;
--- /dev/null
+/*
+ * Device tree sources for default Exynos5440 thermal zone definition
+ *
+ * Copyright (c) 2014 Lukasz Majewski <l.majewski@samsung.com>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ */
+
+polling-delay-passive = <0>;
+polling-delay = <0>;
+trips {
+ cpu-alert-0 {
+ temperature = <100000>; /* millicelsius */
+ hysteresis = <0>; /* millicelsius */
+ type = "active";
+ };
+ cpu-crit-0 {
+ temperature = <1050000>; /* millicelsius */
+ hysteresis = <0>; /* millicelsius */
+ type = "critical";
+ };
+};
interrupts = <0 58 0>;
clocks = <&clock CLK_B_125>;
clock-names = "tmu_apbif";
+ #include "exynos5440-tmu-sensor-conf.dtsi"
};
tmuctrl_1: tmuctrl@16011C {
interrupts = <0 58 0>;
clocks = <&clock CLK_B_125>;
clock-names = "tmu_apbif";
+ #include "exynos5440-tmu-sensor-conf.dtsi"
};
tmuctrl_2: tmuctrl@160120 {
interrupts = <0 58 0>;
clocks = <&clock CLK_B_125>;
clock-names = "tmu_apbif";
+ #include "exynos5440-tmu-sensor-conf.dtsi"
+ };
+
+ thermal-zones {
+ cpu0_thermal: cpu0-thermal {
+ thermal-sensors = <&tmuctrl_0>;
+ #include "exynos5440-trip-points.dtsi"
+ };
+ cpu1_thermal: cpu1-thermal {
+ thermal-sensors = <&tmuctrl_1>;
+ #include "exynos5440-trip-points.dtsi"
+ };
+ cpu2_thermal: cpu2-thermal {
+ thermal-sensors = <&tmuctrl_2>;
+ #include "exynos5440-trip-points.dtsi"
+ };
};
sata@210000 {
regulator-max-microvolt = <5000000>;
gpio = <&gpio3 22 0>;
enable-active-high;
+ vin-supply = <&swbst_reg>;
};
reg_usb_h1_vbus: regulator@1 {
regulator-max-microvolt = <5000000>;
gpio = <&gpio1 29 0>;
enable-active-high;
+ vin-supply = <&swbst_reg>;
};
reg_audio: regulator@2 {
regulator-max-microvolt = <5000000>;
gpio = <&gpio4 0 0>;
enable-active-high;
+ vin-supply = <&swbst_reg>;
};
reg_usb_otg2_vbus: regulator@1 {
regulator-max-microvolt = <5000000>;
gpio = <&gpio4 2 0>;
enable-active-high;
+ vin-supply = <&swbst_reg>;
};
reg_aud3v: regulator@2 {
<14>,
<15>;
#dma-cells = <1>;
- #dma-channels = <32>;
- #dma-requests = <64>;
+ dma-channels = <32>;
+ dma-requests = <64>;
};
i2c1: i2c@48070000 {
model = "Nokia N900";
compatible = "nokia,omap3-n900", "ti,omap3430", "ti,omap3";
+ aliases {
+ i2c0;
+ i2c1 = &i2c1;
+ i2c2 = &i2c2;
+ i2c3 = &i2c3;
+ };
+
cpus {
cpu@0 {
cpu0-supply = <&vcc>;
compatible = "smsc,lan91c94";
interrupt-parent = <&gpio2>;
interrupts = <22 IRQ_TYPE_LEVEL_HIGH>; /* gpio54 */
- reg = <1 0x300 0xf>; /* 16 byte IO range at offset 0x300 */
+ reg = <1 0 0xf>; /* 16 byte IO range */
bank-width = <2>;
pinctrl-names = "default";
pinctrl-0 = <ðernet_pins>;
ti,hwmods = "aes";
reg = <0x480c5000 0x50>;
interrupts = <0>;
+ dmas = <&sdma 65 &sdma 66>;
+ dma-names = "tx", "rx";
};
prm: prm@48306000 {
<14>,
<15>;
#dma-cells = <1>;
- #dma-channels = <32>;
- #dma-requests = <96>;
+ dma-channels = <32>;
+ dma-requests = <96>;
};
omap3_pmx_core: pinmux@48002030 {
ti,hwmods = "sham";
reg = <0x480c3000 0x64>;
interrupts = <49>;
+ dmas = <&sdma 69>;
+ dma-names = "rx";
};
smartreflex_core: smartreflex@480cb000 {
twl: twl@48 {
reg = <0x48>;
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_1N cascaded to gic */
- interrupt-parent = <&gic>;
};
twl6040: twl@4b {
compatible = "ti,twl6040";
reg = <0x4b>;
interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_2N cascaded to gic */
- interrupt-parent = <&gic>;
ti,audpwron-gpio = <&gpio6 0 GPIO_ACTIVE_HIGH>; /* gpio_160 */
vio-supply = <&v1v8>;
reg = <0x48>;
/* IRQ# = 7 */
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_1N cascaded to gic */
- interrupt-parent = <&gic>;
};
twl6040: twl@4b {
/* IRQ# = 119 */
interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_2N cascaded to gic */
- interrupt-parent = <&gic>;
ti,audpwron-gpio = <&gpio4 31 GPIO_ACTIVE_HIGH>; /* gpio line 127 */
vio-supply = <&v1v8>;
};
&uart2 {
- interrupts-extended = <&gic GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART2_RX>;
};
&uart3 {
- interrupts-extended = <&gic GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART3_RX>;
};
&uart4 {
- interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART4_RX>;
};
reg = <0x48>;
/* SPI = 0, IRQ# = 7, 4 = active high level-sensitive */
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_1N cascaded to gic */
- interrupt-parent = <&gic>;
};
twl6040: twl@4b {
/* SPI = 0, IRQ# = 119, 4 = active high level-sensitive */
interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_2N cascaded to gic */
- interrupt-parent = <&gic>;
ti,audpwron-gpio = <&gpio4 31 0>; /* gpio line 127 */
vio-supply = <&v1v8>;
};
&uart2 {
- interrupts-extended = <&gic GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART2_RX>;
pinctrl-names = "default";
pinctrl-0 = <&uart2_pins>;
};
&uart3 {
- interrupts-extended = <&gic GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART3_RX>;
pinctrl-names = "default";
pinctrl-0 = <&uart3_pins>;
};
&uart4 {
- interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH
+ interrupts-extended = <&wakeupgen GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH
&omap4_pmx_core OMAP4_UART4_RX>;
pinctrl-names = "default";
pinctrl-0 = <&uart4_pins>;
reg = <0x48>;
/* SPI = 0, IRQ# = 7, 4 = active high level-sensitive */
interrupts = <GIC_SPI 7 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_1N cascaded to gic */
- interrupt-parent = <&gic>;
};
twl6040: twl@4b {
/* SPI = 0, IRQ# = 119, 4 = active high level-sensitive */
interrupts = <GIC_SPI 119 IRQ_TYPE_LEVEL_HIGH>; /* IRQ_SYS_2N cascaded to gic */
- interrupt-parent = <&gic>;
ti,audpwron-gpio = <&gpio6 22 0>; /* gpio 182 */
vio-supply = <&v1v8>;
/ {
compatible = "ti,omap4430", "ti,omap4";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&wakeupgen>;
aliases {
i2c0 = &i2c1;
#interrupt-cells = <3>;
reg = <0x48241000 0x1000>,
<0x48240100 0x0100>;
+ interrupt-parent = <&gic>;
};
L2: l2-cache-controller@48242000 {
clocks = <&mpu_periphclk>;
reg = <0x48240600 0x20>;
interrupts = <GIC_PPI 13 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_LEVEL_HIGH)>;
+ interrupt-parent = <&gic>;
+ };
+
+ wakeupgen: interrupt-controller@48281000 {
+ compatible = "ti,omap4-wugen-mpu";
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ reg = <0x48281000 0x1000>;
+ interrupt-parent = <&gic>;
};
/*
<GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>;
#dma-cells = <1>;
- #dma-channels = <32>;
- #dma-requests = <127>;
+ dma-channels = <32>;
+ dma-requests = <127>;
};
gpio1: gpio@4a310000 {
uart2: serial@4806c000 {
compatible = "ti,omap4-uart";
reg = <0x4806c000 0x100>;
- interrupts-extended = <&gic GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart2";
clock-frequency = <48000000>;
};
uart3: serial@48020000 {
compatible = "ti,omap4-uart";
reg = <0x48020000 0x100>;
- interrupts-extended = <&gic GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart3";
clock-frequency = <48000000>;
};
uart4: serial@4806e000 {
compatible = "ti,omap4-uart";
reg = <0x4806e000 0x100>;
- interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart4";
clock-frequency = <48000000>;
};
palmas: palmas@48 {
compatible = "ti,palmas";
interrupts = <GIC_SPI 7 IRQ_TYPE_NONE>; /* IRQ_SYS_1N */
- interrupt-parent = <&gic>;
reg = <0x48>;
interrupt-controller;
#interrupt-cells = <2>;
core_thermal: core_thermal {
polling-delay-passive = <250>; /* milliseconds */
- polling-delay = <1000>; /* milliseconds */
+ polling-delay = <500>; /* milliseconds */
/* sensor ID */
thermal-sensors = <&bandgap 2>;
gpu_thermal: gpu_thermal {
polling-delay-passive = <250>; /* milliseconds */
- polling-delay = <1000>; /* milliseconds */
+ polling-delay = <500>; /* milliseconds */
/* sensor ID */
thermal-sensors = <&bandgap 1>;
palmas: palmas@48 {
compatible = "ti,palmas";
interrupts = <GIC_SPI 7 IRQ_TYPE_NONE>; /* IRQ_SYS_1N */
- interrupt-parent = <&gic>;
reg = <0x48>;
interrupt-controller;
#interrupt-cells = <2>;
pinctrl-0 = <&twl6040_pins>;
interrupts = <GIC_SPI 119 IRQ_TYPE_NONE>; /* IRQ_SYS_2N cascaded to gic */
- interrupt-parent = <&gic>;
ti,audpwron-gpio = <&gpio5 13 0>; /* gpio line 141 */
vio-supply = <&smps7_reg>;
#size-cells = <1>;
compatible = "ti,omap5";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&wakeupgen>;
aliases {
i2c0 = &i2c1;
<GIC_PPI 14 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 11 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 10 (GIC_CPU_MASK_RAW(3) | IRQ_TYPE_LEVEL_LOW)>;
+ interrupt-parent = <&gic>;
};
pmu {
<0x48212000 0x1000>,
<0x48214000 0x2000>,
<0x48216000 0x2000>;
+ interrupt-parent = <&gic>;
+ };
+
+ wakeupgen: interrupt-controller@48281000 {
+ compatible = "ti,omap5-wugen-mpu", "ti,omap4-wugen-mpu";
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ reg = <0x48281000 0x1000>;
+ interrupt-parent = <&gic>;
};
/*
<GIC_SPI 14 IRQ_TYPE_LEVEL_HIGH>,
<GIC_SPI 15 IRQ_TYPE_LEVEL_HIGH>;
#dma-cells = <1>;
- #dma-channels = <32>;
- #dma-requests = <127>;
+ dma-channels = <32>;
+ dma-requests = <127>;
};
gpio1: gpio@4ae10000 {
uart1: serial@4806a000 {
compatible = "ti,omap4-uart";
reg = <0x4806a000 0x100>;
- interrupts-extended = <&gic GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 72 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart1";
clock-frequency = <48000000>;
};
uart2: serial@4806c000 {
compatible = "ti,omap4-uart";
reg = <0x4806c000 0x100>;
- interrupts-extended = <&gic GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 73 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart2";
clock-frequency = <48000000>;
};
uart3: serial@48020000 {
compatible = "ti,omap4-uart";
reg = <0x48020000 0x100>;
- interrupts-extended = <&gic GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 74 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart3";
clock-frequency = <48000000>;
};
uart4: serial@4806e000 {
compatible = "ti,omap4-uart";
reg = <0x4806e000 0x100>;
- interrupts-extended = <&gic GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 70 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart4";
clock-frequency = <48000000>;
};
uart5: serial@48066000 {
compatible = "ti,omap4-uart";
reg = <0x48066000 0x100>;
- interrupts-extended = <&gic GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 105 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart5";
clock-frequency = <48000000>;
};
uart6: serial@48068000 {
compatible = "ti,omap4-uart";
reg = <0x48068000 0x100>;
- interrupts-extended = <&gic GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
+ interrupts = <GIC_SPI 106 IRQ_TYPE_LEVEL_HIGH>;
ti,hwmods = "uart6";
clock-frequency = <48000000>;
};
usbhsohci: ohci@4a064800 {
compatible = "ti,ohci-omap3";
reg = <0x4a064800 0x400>;
- interrupt-parent = <&gic>;
interrupts = <GIC_SPI 76 IRQ_TYPE_LEVEL_HIGH>;
};
usbhsehci: ehci@4a064c00 {
compatible = "ti,ehci-omap";
reg = <0x4a064c00 0x400>;
- interrupt-parent = <&gic>;
interrupts = <GIC_SPI 77 IRQ_TYPE_LEVEL_HIGH>;
};
};
<0x4A096800 0x40>; /* pll_ctrl */
reg-names = "phy_rx", "phy_tx", "pll_ctrl";
ctrl-module = <&omap_control_sata>;
- clocks = <&sys_clkin>;
- clock-names = "sysclk";
+ clocks = <&sys_clkin>, <&sata_ref_clk>;
+ clock-names = "sysclk", "refclk";
#phy-cells = <0>;
};
};
};
};
+&cpu_thermal {
+ polling-delay = <500>; /* milliseconds */
+};
+
/include/ "omap54xx-clocks.dtsi"
ti,index-starts-at-one;
};
+ dpll_core_byp_mux: dpll_core_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin>, <&dpll_abe_m3x2_ck>;
+ ti,bit-shift = <23>;
+ reg = <0x012c>;
+ };
+
dpll_core_ck: dpll_core_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-core-clock";
- clocks = <&sys_clkin>, <&dpll_abe_m3x2_ck>;
+ clocks = <&sys_clkin>, <&dpll_core_byp_mux>;
reg = <0x0120>, <0x0124>, <0x012c>, <0x0128>;
};
clock-div = <1>;
};
+ dpll_iva_byp_mux: dpll_iva_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin>, <&iva_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x01ac>;
+ };
+
dpll_iva_ck: dpll_iva_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin>, <&iva_dpll_hs_clk_div>;
+ clocks = <&sys_clkin>, <&dpll_iva_byp_mux>;
reg = <0x01a0>, <0x01a4>, <0x01ac>, <0x01a8>;
};
};
};
&cm_core_clocks {
+
+ dpll_per_byp_mux: dpll_per_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin>, <&per_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x014c>;
+ };
+
dpll_per_ck: dpll_per_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-clock";
- clocks = <&sys_clkin>, <&per_dpll_hs_clk_div>;
+ clocks = <&sys_clkin>, <&dpll_per_byp_mux>;
reg = <0x0140>, <0x0144>, <0x014c>, <0x0148>;
};
ti,index-starts-at-one;
};
+ dpll_usb_byp_mux: dpll_usb_byp_mux {
+ #clock-cells = <0>;
+ compatible = "ti,mux-clock";
+ clocks = <&sys_clkin>, <&usb_dpll_hs_clk_div>;
+ ti,bit-shift = <23>;
+ reg = <0x018c>;
+ };
+
dpll_usb_ck: dpll_usb_ck {
#clock-cells = <0>;
compatible = "ti,omap4-dpll-j-type-clock";
- clocks = <&sys_clkin>, <&usb_dpll_hs_clk_div>;
+ clocks = <&sys_clkin>, <&dpll_usb_byp_mux>;
reg = <0x0180>, <0x0184>, <0x018c>, <0x0188>;
};
"mac_clk_rx", "mac_clk_tx",
"clk_mac_ref", "clk_mac_refout",
"aclk_mac", "pclk_mac";
+ status = "disabled";
};
usb_host0_ehci: usb@ff500000 {
atmel,watchdog-type = "hardware";
atmel,reset-type = "all";
atmel,dbg-halt;
- atmel,idle-halt;
status = "disabled";
};
compatible = "atmel,at91sam9g45-ehci", "usb-ehci";
reg = <0x00700000 0x100000>;
interrupts = <32 IRQ_TYPE_LEVEL_HIGH 2>;
- clocks = <&usb>, <&uhphs_clk>, <&uhpck>;
+ clocks = <&utmi>, <&uhphs_clk>, <&uhpck>;
clock-names = "usb_clk", "ehci_clk", "uhpck";
status = "disabled";
};
gpio4 = &pioE;
tcb0 = &tcb0;
tcb1 = &tcb1;
+ i2c0 = &i2c0;
i2c2 = &i2c2;
};
cpus {
compatible = "atmel,at91sam9g45-ehci", "usb-ehci";
reg = <0x00600000 0x100000>;
interrupts = <46 IRQ_TYPE_LEVEL_HIGH 2>;
- clocks = <&usb>, <&uhphs_clk>, <&uhpck>;
+ clocks = <&utmi>, <&uhphs_clk>, <&uhpck>;
clock-names = "usb_clk", "ehci_clk", "uhpck";
status = "disabled";
};
lcdck: lcdck {
#clock-cells = <0>;
- reg = <4>;
- clocks = <&smd>;
+ reg = <3>;
+ clocks = <&mck>;
};
smdck: smdck {
reg = <50>;
};
- lcd_clk: lcd_clk {
+ lcdc_clk: lcdc_clk {
#clock-cells = <0>;
reg = <51>;
};
#address-cells = <1>;
#size-cells = <0>;
reg = <0xfff01000 0x1000>;
- interrupts = <0 156 4>;
+ interrupts = <0 155 4>;
num-cs = <4>;
clocks = <&spi_m_clk>;
status = "disabled";
reg-shift = <2>;
reg-io-width = <4>;
clocks = <&l4_sp_clk>;
+ dmas = <&pdma 28>,
+ <&pdma 29>;
+ dma-names = "tx", "rx";
};
uart1: serial1@ffc03000 {
reg-shift = <2>;
reg-io-width = <4>;
clocks = <&l4_sp_clk>;
+ dmas = <&pdma 30>,
+ <&pdma 31>;
+ dma-names = "tx", "rx";
};
rst: rstmgr@ffd05000 {
model = "Olimex A10-OLinuXino-LIME";
compatible = "olimex,a10-olinuxino-lime", "allwinner,sun4i-a10";
+ cpus {
+ cpu0: cpu@0 {
+ /*
+ * The A10-Lime is known to be unstable
+ * when running at 1008 MHz
+ */
+ operating-points = <
+ /* kHz uV */
+ 912000 1350000
+ 864000 1300000
+ 624000 1250000
+ >;
+ cooling-max-level = <2>;
+ };
+ };
+
soc@01c00000 {
emac: ethernet@01c0b000 {
pinctrl-names = "default";
clock-latency = <244144>; /* 8 32k periods */
operating-points = <
/* kHz uV */
- 1056000 1500000
1008000 1400000
912000 1350000
864000 1300000
>;
#cooling-cells = <2>;
cooling-min-level = <0>;
- cooling-max-level = <4>;
+ cooling-max-level = <3>;
};
};
clock-latency = <244144>; /* 8 32k periods */
operating-points = <
/* kHz uV */
- 1104000 1500000
1008000 1400000
912000 1350000
864000 1300000
>;
#cooling-cells = <2>;
cooling-min-level = <0>;
- cooling-max-level = <6>;
+ cooling-max-level = <5>;
};
};
clock-latency = <244144>; /* 8 32k periods */
operating-points = <
/* kHz uV */
- 1008000 1450000
960000 1400000
912000 1400000
864000 1300000
>;
#cooling-cells = <2>;
cooling-min-level = <0>;
- cooling-max-level = <7>;
+ cooling-max-level = <6>;
};
cpu@1 {
/ {
compatible = "nvidia,tegra114";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&lic>;
host1x@50000000 {
compatible = "nvidia,tegra114-host1x", "simple-bus";
<0x50046000 0x2000>;
interrupts = <GIC_PPI 9
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
+ interrupt-parent = <&gic>;
+ };
+
+ lic: interrupt-controller@60004000 {
+ compatible = "nvidia,tegra114-ictlr", "nvidia,tegra30-ictlr";
+ reg = <0x60004000 0x100>,
+ <0x60004100 0x50>,
+ <0x60004200 0x50>,
+ <0x60004300 0x50>,
+ <0x60004400 0x50>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
};
timer@60005000 {
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 10
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>;
+ interrupt-parent = <&gic>;
};
};
/ {
compatible = "nvidia,tegra124";
- interrupt-parent = <&gic>;
+ interrupt-parent = <&lic>;
#address-cells = <2>;
#size-cells = <2>;
<0x0 0x50046000 0x0 0x2000>;
interrupts = <GIC_PPI 9
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
+ interrupt-parent = <&gic>;
};
gpu@0,57000000 {
status = "disabled";
};
+ lic: interrupt-controller@60004000 {
+ compatible = "nvidia,tegra124-ictlr", "nvidia,tegra30-ictlr";
+ reg = <0x0 0x60004000 0x0 0x100>,
+ <0x0 0x60004100 0x0 0x100>,
+ <0x0 0x60004200 0x0 0x100>,
+ <0x0 0x60004300 0x0 0x100>,
+ <0x0 0x60004400 0x0 0x100>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&gic>;
+ };
+
timer@0,60005000 {
compatible = "nvidia,tegra124-timer", "nvidia,tegra20-timer";
reg = <0x0 0x60005000 0x0 0x400>;
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>,
<GIC_PPI 10
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_LOW)>;
+ interrupt-parent = <&gic>;
};
};
/ {
compatible = "nvidia,tegra20";
- interrupt-parent = <&intc>;
+ interrupt-parent = <&lic>;
host1x@50000000 {
compatible = "nvidia,tegra20-host1x", "simple-bus";
timer@50040600 {
compatible = "arm,cortex-a9-twd-timer";
+ interrupt-parent = <&intc>;
reg = <0x50040600 0x20>;
interrupts = <GIC_PPI 13
(GIC_CPU_MASK_SIMPLE(2) | IRQ_TYPE_LEVEL_HIGH)>;
0x50040100 0x0100>;
interrupt-controller;
#interrupt-cells = <3>;
+ interrupt-parent = <&intc>;
};
cache-controller@50043000 {
cache-level = <2>;
};
+ lic: interrupt-controller@60004000 {
+ compatible = "nvidia,tegra20-ictlr";
+ reg = <0x60004000 0x100>,
+ <0x60004100 0x50>,
+ <0x60004200 0x50>,
+ <0x60004300 0x50>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&intc>;
+ };
+
timer@60005000 {
compatible = "nvidia,tegra20-timer";
reg = <0x60005000 0x60>;
/ {
compatible = "nvidia,tegra30";
- interrupt-parent = <&intc>;
+ interrupt-parent = <&lic>;
pcie-controller@00003000 {
compatible = "nvidia,tegra30-pcie";
timer@50040600 {
compatible = "arm,cortex-a9-twd-timer";
reg = <0x50040600 0x20>;
+ interrupt-parent = <&intc>;
interrupts = <GIC_PPI 13
(GIC_CPU_MASK_SIMPLE(4) | IRQ_TYPE_LEVEL_HIGH)>;
clocks = <&tegra_car TEGRA30_CLK_TWD>;
0x50040100 0x0100>;
interrupt-controller;
#interrupt-cells = <3>;
+ interrupt-parent = <&intc>;
};
cache-controller@50043000 {
cache-level = <2>;
};
+ lic: interrupt-controller@60004000 {
+ compatible = "nvidia,tegra30-ictlr";
+ reg = <0x60004000 0x100>,
+ <0x60004100 0x50>,
+ <0x60004200 0x50>,
+ <0x60004300 0x50>,
+ <0x60004400 0x50>;
+ interrupt-controller;
+ #interrupt-cells = <3>;
+ interrupt-parent = <&intc>;
+ };
+
timer@60005000 {
compatible = "nvidia,tegra30-timer", "nvidia,tegra20-timer";
reg = <0x60005000 0x400>;
unsigned int mpidr, this_cpu, that_cpu;
unsigned int ob_mpidr, ob_cpu, ob_cluster, ib_mpidr, ib_cpu, ib_cluster;
struct completion inbound_alive;
- struct tick_device *tdev;
- enum clock_event_mode tdev_mode;
long volatile *handshake_ptr;
int ipi_nr, ret;
/* redirect GIC's SGIs to our counterpart */
gic_migrate_target(bL_gic_id[ib_cpu][ib_cluster]);
- tdev = tick_get_device(this_cpu);
- if (tdev && !cpumask_equal(tdev->evtdev->cpumask, cpumask_of(this_cpu)))
- tdev = NULL;
- if (tdev) {
- tdev_mode = tdev->evtdev->mode;
- clockevents_set_mode(tdev->evtdev, CLOCK_EVT_MODE_SHUTDOWN);
- }
+ tick_suspend_local();
ret = cpu_pm_enter();
ret = cpu_pm_exit();
- if (tdev) {
- clockevents_set_mode(tdev->evtdev, tdev_mode);
- clockevents_program_event(tdev->evtdev,
- tdev->evtdev->next_event, 1);
- }
+ tick_resume_local();
trace_cpu_migrate_finish(ktime_get_real_ns(), ib_mpidr);
local_fiq_enable();
CONFIG_BLK_DEV_SD=y
# CONFIG_SCSI_LOWLEVEL is not set
CONFIG_NETDEVICES=y
+CONFIG_ARM_AT91_ETHER=y
CONFIG_MACB=y
# CONFIG_NET_VENDOR_BROADCOM is not set
CONFIG_DM9000=y
CONFIG_ARCH_STI=y
CONFIG_ARCH_EXYNOS=y
CONFIG_EXYNOS5420_MCPM=y
+CONFIG_ARCH_SHMOBILE_MULTI=y
+CONFIG_ARCH_EMEV2=y
+CONFIG_ARCH_R7S72100=y
+CONFIG_ARCH_R8A73A4=y
+CONFIG_ARCH_R8A7740=y
+CONFIG_ARCH_R8A7779=y
+CONFIG_ARCH_R8A7790=y
+CONFIG_ARCH_R8A7791=y
+CONFIG_ARCH_R8A7794=y
+CONFIG_ARCH_SH73A0=y
+CONFIG_MACH_MARZEN=y
CONFIG_ARCH_SUNXI=y
CONFIG_ARCH_SIRF=y
CONFIG_ARCH_TEGRA=y
CONFIG_PCI_MSI=y
CONFIG_PCI_MVEBU=y
CONFIG_PCI_TEGRA=y
+CONFIG_PCI_RCAR_GEN2=y
+CONFIG_PCI_RCAR_GEN2_PCIE=y
CONFIG_PCIEPORTBUS=y
CONFIG_SMP=y
-CONFIG_NR_CPUS=8
+CONFIG_NR_CPUS=16
CONFIG_HIGHPTE=y
CONFIG_CMA=y
CONFIG_ARM_APPENDED_DTB=y
CONFIG_DMA_CMA=y
CONFIG_CMA_SIZE_MBYTES=64
CONFIG_OMAP_OCP2SCP=y
+CONFIG_SIMPLE_PM_BUS=y
CONFIG_MTD=y
CONFIG_MTD_CMDLINE_PARTS=y
CONFIG_MTD_BLOCK=y
CONFIG_AHCI_TEGRA=y
CONFIG_SATA_HIGHBANK=y
CONFIG_SATA_MV=y
+CONFIG_SATA_RCAR=y
CONFIG_NETDEVICES=y
CONFIG_HIX5HD2_GMAC=y
CONFIG_SUN4I_EMAC=y
CONFIG_MVNETA=y
CONFIG_KS8851=y
CONFIG_R8169=y
+CONFIG_SH_ETH=y
CONFIG_SMSC911X=y
CONFIG_STMMAC_ETH=y
CONFIG_TI_CPSW=y
CONFIG_XILINX_EMACLITE=y
CONFIG_AT803X_PHY=y
CONFIG_MARVELL_PHY=y
+CONFIG_SMSC_PHY=y
CONFIG_BROADCOM_PHY=y
CONFIG_ICPLUS_PHY=y
+CONFIG_MICREL_PHY=y
CONFIG_USB_PEGASUS=y
CONFIG_USB_USBNET=y
CONFIG_USB_NET_SMSC75XX=y
CONFIG_MOUSE_PS2_ELANTECH=y
CONFIG_INPUT_TOUCHSCREEN=y
CONFIG_TOUCHSCREEN_ATMEL_MXT=y
+CONFIG_TOUCHSCREEN_ST1232=m
CONFIG_TOUCHSCREEN_STMPE=y
CONFIG_TOUCHSCREEN_SUN4I=y
CONFIG_INPUT_MISC=y
CONFIG_INPUT_MPU3050=y
CONFIG_INPUT_AXP20X_PEK=y
+CONFIG_INPUT_ADXL34X=m
CONFIG_SERIO_AMBAKMI=y
CONFIG_SERIAL_8250=y
CONFIG_SERIAL_8250_CONSOLE=y
CONFIG_SERIAL_8250_DW=y
+CONFIG_SERIAL_8250_EM=y
CONFIG_SERIAL_8250_MT6577=y
CONFIG_SERIAL_AMBA_PL011=y
CONFIG_SERIAL_AMBA_PL011_CONSOLE=y
CONFIG_SERIAL_TEGRA=y
CONFIG_SERIAL_IMX=y
CONFIG_SERIAL_IMX_CONSOLE=y
+CONFIG_SERIAL_SH_SCI=y
+CONFIG_SERIAL_SH_SCI_NR_UARTS=20
+CONFIG_SERIAL_SH_SCI_CONSOLE=y
CONFIG_SERIAL_MSM=y
CONFIG_SERIAL_MSM_CONSOLE=y
CONFIG_SERIAL_VT8500=y
CONFIG_I2C_MUX_PINCTRL=y
CONFIG_I2C_CADENCE=y
CONFIG_I2C_DESIGNWARE_PLATFORM=y
+CONFIG_I2C_GPIO=m
CONFIG_I2C_EXYNOS5=y
CONFIG_I2C_MV64XXX=y
+CONFIG_I2C_RIIC=y
CONFIG_I2C_S3C2410=y
+CONFIG_I2C_SH_MOBILE=y
CONFIG_I2C_SIRF=y
-CONFIG_I2C_TEGRA=y
CONFIG_I2C_ST=y
-CONFIG_SPI=y
+CONFIG_I2C_TEGRA=y
CONFIG_I2C_XILINX=y
-CONFIG_SPI_DAVINCI=y
+CONFIG_I2C_RCAR=y
+CONFIG_SPI=y
CONFIG_SPI_CADENCE=y
+CONFIG_SPI_DAVINCI=y
CONFIG_SPI_OMAP24XX=y
CONFIG_SPI_ORION=y
CONFIG_SPI_PL022=y
+CONFIG_SPI_RSPI=y
+CONFIG_SPI_SH_MSIOF=m
+CONFIG_SPI_SH_HSPI=y
CONFIG_SPI_SIRF=y
CONFIG_SPI_SUN4I=y
CONFIG_SPI_SUN6I=y
CONFIG_PINCTRL_APQ8084=y
CONFIG_GPIO_SYSFS=y
CONFIG_GPIO_GENERIC_PLATFORM=y
-CONFIG_GPIO_DWAPB=y
CONFIG_GPIO_DAVINCI=y
+CONFIG_GPIO_DWAPB=y
+CONFIG_GPIO_EM=y
+CONFIG_GPIO_RCAR=y
CONFIG_GPIO_XILINX=y
CONFIG_GPIO_ZYNQ=y
CONFIG_GPIO_PCA953X=y
CONFIG_GPIO_PCA953X_IRQ=y
+CONFIG_GPIO_PCF857X=y
CONFIG_GPIO_TWL4030=y
CONFIG_GPIO_PALMAS=y
CONFIG_GPIO_SYSCON=y
CONFIG_POWER_RESET_GPIO=y
CONFIG_POWER_RESET_KEYSTONE=y
CONFIG_POWER_RESET_SUN6I=y
+CONFIG_POWER_RESET_RMOBILE=y
CONFIG_SENSORS_LM90=y
CONFIG_SENSORS_LM95245=y
CONFIG_THERMAL=y
CONFIG_CPU_THERMAL=y
+CONFIG_RCAR_THERMAL=y
CONFIG_ARMADA_THERMAL=y
CONFIG_DAVINCI_WATCHDOG
CONFIG_ST_THERMAL_SYSCFG=y
CONFIG_ORION_WATCHDOG=y
CONFIG_SUNXI_WATCHDOG=y
CONFIG_MESON_WATCHDOG=y
+CONFIG_MFD_AS3711=y
CONFIG_MFD_AS3722=y
CONFIG_MFD_BCM590XX=y
CONFIG_MFD_AXP20X=y
CONFIG_MFD_TPS6586X=y
CONFIG_MFD_TPS65910=y
CONFIG_REGULATOR_AB8500=y
+CONFIG_REGULATOR_AS3711=y
CONFIG_REGULATOR_AS3722=y
CONFIG_REGULATOR_AXP20X=y
CONFIG_REGULATOR_BCM590XX=y
+CONFIG_REGULATOR_DA9210=y
CONFIG_REGULATOR_GPIO=y
CONFIG_MFD_SYSCON=y
CONFIG_POWER_RESET_SYSCON=y
CONFIG_REGULATOR_MAX8907=y
+CONFIG_REGULATOR_MAX8973=y
CONFIG_REGULATOR_MAX77686=y
CONFIG_REGULATOR_PALMAS=y
CONFIG_REGULATOR_S2MPS11=y
CONFIG_REGULATOR_VEXPRESS=y
CONFIG_MEDIA_SUPPORT=y
CONFIG_MEDIA_CAMERA_SUPPORT=y
+CONFIG_MEDIA_CONTROLLER=y
+CONFIG_VIDEO_V4L2_SUBDEV_API=y
CONFIG_MEDIA_USB_SUPPORT=y
CONFIG_USB_VIDEO_CLASS=y
CONFIG_USB_GSPCA=y
+CONFIG_V4L_PLATFORM_DRIVERS=y
+CONFIG_SOC_CAMERA=m
+CONFIG_SOC_CAMERA_PLATFORM=m
+CONFIG_VIDEO_RCAR_VIN=m
+CONFIG_V4L_MEM2MEM_DRIVERS=y
+CONFIG_VIDEO_RENESAS_VSP1=m
+# CONFIG_MEDIA_SUBDRV_AUTOSELECT is not set
+CONFIG_VIDEO_ADV7180=m
CONFIG_DRM=y
+CONFIG_DRM_RCAR_DU=m
CONFIG_DRM_TEGRA=y
CONFIG_DRM_PANEL_SIMPLE=y
CONFIG_FB_ARMCLCD=y
CONFIG_FB_WM8505=y
+CONFIG_FB_SH_MOBILE_LCDC=y
CONFIG_FB_SIMPLE=y
+CONFIG_FB_SH_MOBILE_MERAM=y
CONFIG_BACKLIGHT_LCD_SUPPORT=y
CONFIG_BACKLIGHT_CLASS_DEVICE=y
CONFIG_BACKLIGHT_PWM=y
+CONFIG_BACKLIGHT_AS3711=y
CONFIG_FRAMEBUFFER_CONSOLE=y
CONFIG_FRAMEBUFFER_CONSOLE_ROTATION=y
CONFIG_SOUND=y
CONFIG_SND_DYNAMIC_MINORS=y
CONFIG_SND_USB_AUDIO=y
CONFIG_SND_SOC=y
+CONFIG_SND_SOC_SH4_FSI=m
+CONFIG_SND_SOC_RCAR=m
CONFIG_SND_SOC_TEGRA=y
CONFIG_SND_SOC_TEGRA_RT5640=y
CONFIG_SND_SOC_TEGRA_WM8753=y
CONFIG_SND_SOC_TEGRA_TRIMSLICE=y
CONFIG_SND_SOC_TEGRA_ALC5632=y
CONFIG_SND_SOC_TEGRA_MAX98090=y
+CONFIG_SND_SOC_AK4642=m
+CONFIG_SND_SOC_WM8978=m
CONFIG_USB=y
CONFIG_USB_XHCI_HCD=y
CONFIG_USB_XHCI_MVEBU=y
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_OHCI_HCD_STI=y
CONFIG_USB_OHCI_HCD_PLATFORM=y
+CONFIG_USB_R8A66597_HCD=m
+CONFIG_USB_RENESAS_USBHS=m
CONFIG_USB_STORAGE=y
CONFIG_USB_DWC3=y
CONFIG_USB_CHIPIDEA=y
CONFIG_USB_GPIO_VBUS=y
CONFIG_USB_ISP1301=y
CONFIG_USB_MXS_PHY=y
+CONFIG_USB_RCAR_PHY=m
+CONFIG_USB_RCAR_GEN2_PHY=m
+CONFIG_USB_GADGET=y
+CONFIG_USB_RENESAS_USBHS_UDC=m
CONFIG_MMC=y
CONFIG_MMC_BLOCK_MINORS=16
CONFIG_MMC_ARMMMCI=y
CONFIG_MMC_OMAP=y
CONFIG_MMC_OMAP_HS=y
CONFIG_MMC_MVSDIO=y
-CONFIG_MMC_SUNXI=y
+CONFIG_MMC_SDHI=y
CONFIG_MMC_DW=y
CONFIG_MMC_DW_IDMAC=y
CONFIG_MMC_DW_PLTFM=y
CONFIG_MMC_DW_EXYNOS=y
CONFIG_MMC_DW_ROCKCHIP=y
+CONFIG_MMC_SH_MMCIF=y
+CONFIG_MMC_SUNXI=y
CONFIG_NEW_LEDS=y
CONFIG_LEDS_CLASS=y
CONFIG_LEDS_GPIO=y
CONFIG_RTC_DRV_DS1307=y
CONFIG_RTC_DRV_MAX8907=y
CONFIG_RTC_DRV_MAX77686=y
+CONFIG_RTC_DRV_RS5C372=m
CONFIG_RTC_DRV_PALMAS=y
CONFIG_RTC_DRV_TWL4030=y
CONFIG_RTC_DRV_TPS6586X=y
CONFIG_RTC_DRV_TPS65910=y
+CONFIG_RTC_DRV_S35390A=m
CONFIG_RTC_DRV_EM3027=y
CONFIG_RTC_DRV_PL031=y
CONFIG_RTC_DRV_VT8500=y
CONFIG_DW_DMAC=y
CONFIG_MV_XOR=y
CONFIG_TEGRA20_APB_DMA=y
+CONFIG_SH_DMAE=y
+CONFIG_RCAR_AUDMAC_PP=m
+CONFIG_RCAR_DMAC=y
CONFIG_STE_DMA40=y
CONFIG_SIRF_DMA=y
CONFIG_TI_EDMA=y
CONFIG_XILINX_XADC=y
CONFIG_AK8975=y
CONFIG_PWM=y
+CONFIG_PWM_RENESAS_TPU=y
CONFIG_PWM_TEGRA=y
CONFIG_PWM_VT8500=y
CONFIG_PHY_HIX5HD2_SATA=y
CONFIG_MTD_NAND=y
CONFIG_MTD_NAND_ECC_BCH=y
CONFIG_MTD_NAND_OMAP2=y
+CONFIG_MTD_NAND_OMAP_BCH=y
CONFIG_MTD_ONENAND=y
CONFIG_MTD_ONENAND_VERIFY_WRITE=y
CONFIG_MTD_ONENAND_OMAP2=y
CONFIG_REGULATOR_PALMAS=y
CONFIG_REGULATOR_PBIAS=y
CONFIG_REGULATOR_TI_ABB=y
+CONFIG_REGULATOR_TPS62360=m
CONFIG_REGULATOR_TPS65023=y
CONFIG_REGULATOR_TPS6507X=y
CONFIG_REGULATOR_TPS65217=y
CONFIG_PWM_TWL=m
CONFIG_PWM_TWL_LED=m
CONFIG_OMAP_USB2=m
-CONFIG_TI_PIPE3=m
+CONFIG_TI_PIPE3=y
+CONFIG_TWL4030_USB=m
CONFIG_EXT2_FS=y
CONFIG_EXT3_FS=y
# CONFIG_EXT3_FS_XATTR is not set
CONFIG_SYSVIPC=y
CONFIG_IRQ_DOMAIN_DEBUG=y
CONFIG_LOG_BUF_SHIFT=14
-CONFIG_SYSFS_DEPRECATED=y
-CONFIG_SYSFS_DEPRECATED_V2=y
CONFIG_BLK_DEV_INITRD=y
CONFIG_EMBEDDED=y
CONFIG_SLAB=y
CONFIG_PERF_EVENTS=y
CONFIG_ARCH_SUNXI=y
CONFIG_SMP=y
+CONFIG_NR_CPUS=8
CONFIG_AEABI=y
CONFIG_HIGHMEM=y
CONFIG_HIGHPTE=y
CONFIG_USB=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_MON=y
-CONFIG_USB_ISP1760_HCD=y
CONFIG_USB_STORAGE=y
+CONFIG_USB_ISP1760=y
CONFIG_MMC=y
CONFIG_MMC_ARMMMCI=y
CONFIG_NEW_LEDS=y
# define VFP_ABI_FRAME 0
# define BSAES_ASM_EXTENDED_KEY
# define XTS_CHAIN_TWEAK
-# define __ARM_ARCH__ 7
+# define __ARM_ARCH__ __LINUX_ARM_ARCH__
+# define __ARM_MAX_ARCH__ 7
#endif
#ifdef __thumb__
# define adrl adr
#endif
-#if __ARM_ARCH__>=7
+#if __ARM_MAX_ARCH__>=7
+.arch armv7-a
+.fpu neon
+
.text
.syntax unified @ ARMv7-capable assembler is expected to handle this
#ifdef __thumb2__
.code 32
#endif
-.fpu neon
-
.type _bsaes_decrypt8,%function
.align 4
_bsaes_decrypt8:
vld1.8 {q8}, [r0] @ initial tweak
adr r2, .Lxts_magic
+#ifndef XTS_CHAIN_TWEAK
tst r9, #0xf @ if not multiple of 16
it ne @ Thumb2 thing, sanity check in ARM
subne r9, #0x10 @ subtract another 16 bytes
+#endif
subs r9, #0x80
blo .Lxts_dec_short
# define VFP_ABI_FRAME 0
# define BSAES_ASM_EXTENDED_KEY
# define XTS_CHAIN_TWEAK
-# define __ARM_ARCH__ 7
+# define __ARM_ARCH__ __LINUX_ARM_ARCH__
+# define __ARM_MAX_ARCH__ 7
#endif
#ifdef __thumb__
# define adrl adr
#endif
-#if __ARM_ARCH__>=7
+#if __ARM_MAX_ARCH__>=7
+.arch armv7-a
+.fpu neon
+
.text
.syntax unified @ ARMv7-capable assembler is expected to handle this
#ifdef __thumb2__
.code 32
#endif
-.fpu neon
-
.type _bsaes_decrypt8,%function
.align 4
_bsaes_decrypt8:
vld1.8 {@XMM[8]}, [r0] @ initial tweak
adr $magic, .Lxts_magic
+#ifndef XTS_CHAIN_TWEAK
tst $len, #0xf @ if not multiple of 16
it ne @ Thumb2 thing, sanity check in ARM
subne $len, #0x10 @ subtract another 16 bytes
+#endif
subs $len, #0x80
blo .Lxts_dec_short
#ifndef _ASM_ARM_JUMP_LABEL_H
#define _ASM_ARM_JUMP_LABEL_H
-#ifdef __KERNEL__
+#ifndef __ASSEMBLY__
#include <linux/types.h>
return true;
}
-#endif /* __KERNEL__ */
-
typedef u32 jump_label_t;
struct jump_entry {
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif
#define HSR_COND (0xfU << HSR_COND_SHIFT)
#define FSC_FAULT (0x04)
+#define FSC_ACCESS (0x08)
#define FSC_PERM (0x0c)
/* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
#include <asm/fpstate.h>
#include <kvm/arm_arch_timer.h>
+#define __KVM_HAVE_ARCH_INTC_INITIALIZED
+
#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
#else
unsigned long kvm_arm_num_regs(struct kvm_vcpu *vcpu);
int kvm_arm_copy_reg_indices(struct kvm_vcpu *vcpu, u64 __user *indices);
+int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end);
+int kvm_test_age_hva(struct kvm *kvm, unsigned long hva);
/* We do not have shadow page tables, hence the empty hooks */
-static inline int kvm_age_hva(struct kvm *kvm, unsigned long start,
- unsigned long end)
-{
- return 0;
-}
-
-static inline int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
-{
- return 0;
-}
-
static inline void kvm_arch_mmu_notifier_invalidate_page(struct kvm *kvm,
unsigned long address)
{
bool sign_extend;
};
-/*
- * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
- * which is an anonymous type. Use our own type instead.
- */
-struct kvm_exit_mmio {
- phys_addr_t phys_addr;
- u8 data[8];
- u32 len;
- bool is_write;
- void *private;
-};
-
-static inline void kvm_prepare_mmio(struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
-{
- run->mmio.phys_addr = mmio->phys_addr;
- run->mmio.len = mmio->len;
- run->mmio.is_write = mmio->is_write;
- memcpy(run->mmio.data, mmio->data, mmio->len);
- run->exit_reason = KVM_EXIT_MMIO;
-}
-
int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run);
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa);
(__boundary - 1 < (end) - 1)? __boundary: (end); \
})
+#define kvm_pgd_index(addr) pgd_index(addr)
+
static inline bool kvm_page_empty(void *ptr)
{
struct page *ptr_page = virt_to_page(ptr);
return page_count(ptr_page) == 1;
}
-
#define kvm_pte_table_empty(kvm, ptep) kvm_page_empty(ptep)
#define kvm_pmd_table_empty(kvm, pmdp) kvm_page_empty(pmdp)
#define kvm_pud_table_empty(kvm, pudp) (0)
#define KVM_PREALLOC_LEVEL 0
-static inline int kvm_prealloc_hwpgd(struct kvm *kvm, pgd_t *pgd)
+static inline void *kvm_get_hwpgd(struct kvm *kvm)
{
- return 0;
+ return kvm->arch.pgd;
}
-static inline void kvm_free_hwpgd(struct kvm *kvm) { }
-
-static inline void *kvm_get_hwpgd(struct kvm *kvm)
+static inline unsigned int kvm_get_hwpgd_size(void)
{
- return kvm->arch.pgd;
+ return PTRS_PER_S2_PGD * sizeof(pgd_t);
}
struct kvm;
bool need_flush = !vcpu_has_cache_enabled(vcpu) || ipa_uncached;
- VM_BUG_ON(size & PAGE_MASK);
+ VM_BUG_ON(size & ~PAGE_MASK);
if (!need_flush && !icache_is_pipt())
goto vipt_cache;
extern void timer_tick(void);
-struct timespec;
-typedef void (*clock_access_fn)(struct timespec *);
+typedef void (*clock_access_fn)(struct timespec64 *);
extern int register_persistent_clock(clock_access_fn read_boot,
clock_access_fn read_persistent);
#define AT91_DBGU 0xfc00c000 /* SAMA5D4_BASE_USART3 */
#endif
-/* Keep in sync with mach-at91/include/mach/hardware.h */
+#ifdef CONFIG_MMU
#define AT91_IO_P2V(x) ((x) - 0x01000000)
+#else
+#define AT91_IO_P2V(x) (x)
+#endif
#define AT91_DBGU_SR (0x14) /* Status Register */
#define AT91_DBGU_THR (0x1c) /* Transmitter Holding Register */
/* Highest supported SPI, from VGIC_NR_IRQS */
#define KVM_ARM_IRQ_GIC_MAX 127
+/* One single KVM irqchip, ie. the VGIC */
+#define KVM_NR_IRQCHIPS 1
+
/* PSCI interface */
#define KVM_PSCI_FN_BASE 0x95c1ba5e
#define KVM_PSCI_FN(n) (KVM_PSCI_FN_BASE + (n))
DEFINE(VCPU_HxFAR, offsetof(struct kvm_vcpu, arch.fault.hxfar));
DEFINE(VCPU_HPFAR, offsetof(struct kvm_vcpu, arch.fault.hpfar));
DEFINE(VCPU_HYP_PC, offsetof(struct kvm_vcpu, arch.fault.hyp_pc));
-#ifdef CONFIG_KVM_ARM_VGIC
DEFINE(VCPU_VGIC_CPU, offsetof(struct kvm_vcpu, arch.vgic_cpu));
DEFINE(VGIC_V2_CPU_HCR, offsetof(struct vgic_cpu, vgic_v2.vgic_hcr));
DEFINE(VGIC_V2_CPU_VMCR, offsetof(struct vgic_cpu, vgic_v2.vgic_vmcr));
DEFINE(VGIC_V2_CPU_APR, offsetof(struct vgic_cpu, vgic_v2.vgic_apr));
DEFINE(VGIC_V2_CPU_LR, offsetof(struct vgic_cpu, vgic_v2.vgic_lr));
DEFINE(VGIC_CPU_NR_LR, offsetof(struct vgic_cpu, nr_lr));
-#ifdef CONFIG_KVM_ARM_TIMER
DEFINE(VCPU_TIMER_CNTV_CTL, offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_ctl));
DEFINE(VCPU_TIMER_CNTV_CVAL, offsetof(struct kvm_vcpu, arch.timer_cpu.cntv_cval));
DEFINE(KVM_TIMER_CNTVOFF, offsetof(struct kvm, arch.timer.cntvoff));
DEFINE(KVM_TIMER_ENABLED, offsetof(struct kvm, arch.timer.enabled));
-#endif
DEFINE(KVM_VGIC_VCTRL, offsetof(struct kvm, arch.vgic.vctrl_base));
-#endif
DEFINE(KVM_VTTBR, offsetof(struct kvm, arch.vttbr));
#endif
return 0;
if (cpu_arch)
cpu_arch += CPU_ARCH_ARMv3;
} else if ((read_cpuid_id() & 0x000f0000) == 0x000f0000) {
- unsigned int mmfr0;
-
/* Revised CPUID format. Read the Memory Model Feature
* Register 0 and check for VMSAv7 or PMSAv7 */
- asm("mrc p15, 0, %0, c0, c1, 4"
- : "=r" (mmfr0));
+ unsigned int mmfr0 = read_cpuid_ext(CPUID_EXT_MMFR0);
if ((mmfr0 & 0x0000000f) >= 0x00000003 ||
(mmfr0 & 0x000000f0) >= 0x00000030)
cpu_arch = CPU_ARCH_ARMv7;
}
#endif
-static void dummy_clock_access(struct timespec *ts)
+static void dummy_clock_access(struct timespec64 *ts)
{
ts->tv_sec = 0;
ts->tv_nsec = 0;
static clock_access_fn __read_persistent_clock = dummy_clock_access;
static clock_access_fn __read_boot_clock = dummy_clock_access;;
-void read_persistent_clock(struct timespec *ts)
+void read_persistent_clock64(struct timespec64 *ts)
{
__read_persistent_clock(ts);
}
-void read_boot_clock(struct timespec *ts)
+void read_boot_clock64(struct timespec64 *ts)
{
__read_boot_clock(ts);
}
config KVM
bool "Kernel-based Virtual Machine (KVM) support"
+ depends on MMU && OF
select PREEMPT_NOTIFIERS
select ANON_INODES
select HAVE_KVM_CPU_RELAX_INTERCEPT
select KVM_ARM_HOST
select KVM_GENERIC_DIRTYLOG_READ_PROTECT
select SRCU
- depends on ARM_VIRT_EXT && ARM_LPAE
+ select MMU_NOTIFIER
+ select HAVE_KVM_EVENTFD
+ select HAVE_KVM_IRQFD
+ depends on ARM_VIRT_EXT && ARM_LPAE && ARM_ARCH_TIMER
---help---
- Support hosting virtualized guest machines. You will also
- need to select one or more of the processor modules below.
+ Support hosting virtualized guest machines.
This module provides access to the hardware capabilities through
a character device node named /dev/kvm.
If unsure, say N.
config KVM_ARM_HOST
- bool "KVM host support for ARM cpus."
- depends on KVM
- depends on MMU
- select MMU_NOTIFIER
+ bool
---help---
Provides host support for ARM processors.
large, so only choose a reasonable number that you expect to
actually use.
-config KVM_ARM_VGIC
- bool "KVM support for Virtual GIC"
- depends on KVM_ARM_HOST && OF
- select HAVE_KVM_IRQCHIP
- default y
- ---help---
- Adds support for a hardware assisted, in-kernel GIC emulation.
-
-config KVM_ARM_TIMER
- bool "KVM support for Architected Timers"
- depends on KVM_ARM_VGIC && ARM_ARCH_TIMER
- select HAVE_KVM_IRQCHIP
- default y
- ---help---
- Adds support for the Architected Timers in virtual machines
-
endif # VIRTUALIZATION
plus_virt_def := -DREQUIRES_VIRT=1
endif
-ccflags-y += -Ivirt/kvm -Iarch/arm/kvm
+ccflags-y += -Iarch/arm/kvm
CFLAGS_arm.o := -I. $(plus_virt_def)
CFLAGS_mmu.o := -I.
AFLAGS_interrupts.o := -Wa,-march=armv7-a$(plus_virt)
KVM := ../../../virt/kvm
-kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o
+kvm-arm-y = $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o
obj-y += kvm-arm.o init.o interrupts.o
obj-y += arm.o handle_exit.o guest.o mmu.o emulate.o reset.o
obj-y += coproc.o coproc_a15.o coproc_a7.o mmio.o psci.o perf.o
-obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
-obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2.o
-obj-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2-emul.o
-obj-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
+obj-y += $(KVM)/arm/vgic.o
+obj-y += $(KVM)/arm/vgic-v2.o
+obj-y += $(KVM)/arm/vgic-v2-emul.o
+obj-y += $(KVM)/arm/arch_timer.o
static u8 kvm_next_vmid;
static DEFINE_SPINLOCK(kvm_vmid_lock);
-static bool vgic_present;
-
static void kvm_arm_set_running_vcpu(struct kvm_vcpu *vcpu)
{
BUG_ON(preemptible());
int r;
switch (ext) {
case KVM_CAP_IRQCHIP:
- r = vgic_present;
- break;
+ case KVM_CAP_IRQFD:
+ case KVM_CAP_IOEVENTFD:
case KVM_CAP_DEVICE_CTRL:
case KVM_CAP_USER_MEMORY:
case KVM_CAP_SYNC_MMU:
case KVM_CAP_ARM_PSCI:
case KVM_CAP_ARM_PSCI_0_2:
case KVM_CAP_READONLY_MEM:
+ case KVM_CAP_MP_STATE:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
{
- return 0;
+ return kvm_timer_should_fire(vcpu);
}
int kvm_arch_vcpu_init(struct kvm_vcpu *vcpu)
int kvm_arch_vcpu_ioctl_get_mpstate(struct kvm_vcpu *vcpu,
struct kvm_mp_state *mp_state)
{
- return -EINVAL;
+ if (vcpu->arch.pause)
+ mp_state->mp_state = KVM_MP_STATE_STOPPED;
+ else
+ mp_state->mp_state = KVM_MP_STATE_RUNNABLE;
+
+ return 0;
}
int kvm_arch_vcpu_ioctl_set_mpstate(struct kvm_vcpu *vcpu,
struct kvm_mp_state *mp_state)
{
- return -EINVAL;
+ switch (mp_state->mp_state) {
+ case KVM_MP_STATE_RUNNABLE:
+ vcpu->arch.pause = false;
+ break;
+ case KVM_MP_STATE_STOPPED:
+ vcpu->arch.pause = true;
+ break;
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
}
/**
return 0;
}
+bool kvm_arch_intc_initialized(struct kvm *kvm)
+{
+ return vgic_initialized(kvm);
+}
+
static void vcpu_pause(struct kvm_vcpu *vcpu)
{
wait_queue_head_t *wq = kvm_arch_vcpu_wq(vcpu);
vcpu->mode = OUTSIDE_GUEST_MODE;
kvm_guest_exit();
- trace_kvm_exit(*vcpu_pc(vcpu));
+ trace_kvm_exit(kvm_vcpu_trap_get_class(vcpu), *vcpu_pc(vcpu));
/*
* We may have taken a host interrupt in HYP mode (ie
* while executing the guest). This interrupt is still
switch (dev_id) {
case KVM_ARM_DEVICE_VGIC_V2:
- if (!vgic_present)
- return -ENXIO;
return kvm_vgic_addr(kvm, type, &dev_addr->addr, true);
default:
return -ENODEV;
switch (ioctl) {
case KVM_CREATE_IRQCHIP: {
- if (vgic_present)
- return kvm_vgic_create(kvm, KVM_DEV_TYPE_ARM_VGIC_V2);
- else
- return -ENXIO;
+ return kvm_vgic_create(kvm, KVM_DEV_TYPE_ARM_VGIC_V2);
}
case KVM_ARM_SET_DEVICE_ADDR: {
struct kvm_arm_device_addr dev_addr;
if (err)
goto out_free_context;
-#ifdef CONFIG_KVM_ARM_VGIC
- vgic_present = true;
-#endif
-
/*
* Init HYP architected timer support
*/
return -EINVAL;
}
-#ifndef CONFIG_KVM_ARM_TIMER
-
-#define NUM_TIMER_REGS 0
-
-static int copy_timer_indices(struct kvm_vcpu *vcpu, u64 __user *uindices)
-{
- return 0;
-}
-
-static bool is_timer_reg(u64 index)
-{
- return false;
-}
-
-#else
-
#define NUM_TIMER_REGS 3
static bool is_timer_reg(u64 index)
return 0;
}
-#endif
-
static int set_timer_reg(struct kvm_vcpu *vcpu, const struct kvm_one_reg *reg)
{
void __user *uaddr = (void __user *)(long)reg->addr;
* Assumes vcpu pointer in vcpu reg
*/
.macro save_vgic_state
-#ifdef CONFIG_KVM_ARM_VGIC
/* Get VGIC VCTRL base into r2 */
ldr r2, [vcpu, #VCPU_KVM]
ldr r2, [r2, #KVM_VGIC_VCTRL]
subs r4, r4, #1
bne 1b
2:
-#endif
.endm
/*
* Assumes vcpu pointer in vcpu reg
*/
.macro restore_vgic_state
-#ifdef CONFIG_KVM_ARM_VGIC
/* Get VGIC VCTRL base into r2 */
ldr r2, [vcpu, #VCPU_KVM]
ldr r2, [r2, #KVM_VGIC_VCTRL]
subs r4, r4, #1
bne 1b
2:
-#endif
.endm
#define CNTHCTL_PL1PCTEN (1 << 0)
* Clobbers r2-r5
*/
.macro save_timer_state
-#ifdef CONFIG_KVM_ARM_TIMER
ldr r4, [vcpu, #VCPU_KVM]
ldr r2, [r4, #KVM_TIMER_ENABLED]
cmp r2, #0
mcrr p15, 4, r2, r2, c14 @ CNTVOFF
1:
-#endif
@ Allow physical timer/counter access for the host
mrc p15, 4, r2, c14, c1, 0 @ CNTHCTL
orr r2, r2, #(CNTHCTL_PL1PCEN | CNTHCTL_PL1PCTEN)
bic r2, r2, #CNTHCTL_PL1PCEN
mcr p15, 4, r2, c14, c1, 0 @ CNTHCTL
-#ifdef CONFIG_KVM_ARM_TIMER
ldr r4, [vcpu, #VCPU_KVM]
ldr r2, [r4, #KVM_TIMER_ENABLED]
cmp r2, #0
and r2, r2, #3
mcr p15, 0, r2, c14, c3, 1 @ CNTV_CTL
1:
-#endif
.endm
.equ vmentry, 0
return 0;
}
-static int decode_hsr(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa,
- struct kvm_exit_mmio *mmio)
+static int decode_hsr(struct kvm_vcpu *vcpu, bool *is_write, int *len)
{
unsigned long rt;
- int len;
- bool is_write, sign_extend;
+ int access_size;
+ bool sign_extend;
if (kvm_vcpu_dabt_isextabt(vcpu)) {
/* cache operation on I/O addr, tell guest unsupported */
return 1;
}
- len = kvm_vcpu_dabt_get_as(vcpu);
- if (unlikely(len < 0))
- return len;
+ access_size = kvm_vcpu_dabt_get_as(vcpu);
+ if (unlikely(access_size < 0))
+ return access_size;
- is_write = kvm_vcpu_dabt_iswrite(vcpu);
+ *is_write = kvm_vcpu_dabt_iswrite(vcpu);
sign_extend = kvm_vcpu_dabt_issext(vcpu);
rt = kvm_vcpu_dabt_get_rd(vcpu);
- mmio->is_write = is_write;
- mmio->phys_addr = fault_ipa;
- mmio->len = len;
+ *len = access_size;
vcpu->arch.mmio_decode.sign_extend = sign_extend;
vcpu->arch.mmio_decode.rt = rt;
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa)
{
- struct kvm_exit_mmio mmio;
unsigned long data;
unsigned long rt;
int ret;
+ bool is_write;
+ int len;
+ u8 data_buf[8];
/*
- * Prepare MMIO operation. First stash it in a private
- * structure that we can use for in-kernel emulation. If the
- * kernel can't handle it, copy it into run->mmio and let user
- * space do its magic.
+ * Prepare MMIO operation. First decode the syndrome data we get
+ * from the CPU. Then try if some in-kernel emulation feels
+ * responsible, otherwise let user space do its magic.
*/
-
if (kvm_vcpu_dabt_isvalid(vcpu)) {
- ret = decode_hsr(vcpu, fault_ipa, &mmio);
+ ret = decode_hsr(vcpu, &is_write, &len);
if (ret)
return ret;
} else {
rt = vcpu->arch.mmio_decode.rt;
- if (mmio.is_write) {
- data = vcpu_data_guest_to_host(vcpu, *vcpu_reg(vcpu, rt),
- mmio.len);
+ if (is_write) {
+ data = vcpu_data_guest_to_host(vcpu, *vcpu_reg(vcpu, rt), len);
+
+ trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, len, fault_ipa, data);
+ mmio_write_buf(data_buf, len, data);
- trace_kvm_mmio(KVM_TRACE_MMIO_WRITE, mmio.len,
- fault_ipa, data);
- mmio_write_buf(mmio.data, mmio.len, data);
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, fault_ipa, len,
+ data_buf);
} else {
- trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, mmio.len,
+ trace_kvm_mmio(KVM_TRACE_MMIO_READ_UNSATISFIED, len,
fault_ipa, 0);
+
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, fault_ipa, len,
+ data_buf);
}
- if (vgic_handle_mmio(vcpu, run, &mmio))
+ /* Now prepare kvm_run for the potential return to userland. */
+ run->mmio.is_write = is_write;
+ run->mmio.phys_addr = fault_ipa;
+ run->mmio.len = len;
+ memcpy(run->mmio.data, data_buf, len);
+
+ if (!ret) {
+ /* We handled the access successfully in the kernel. */
+ kvm_handle_mmio_return(vcpu, run);
return 1;
+ }
- kvm_prepare_mmio(run, &mmio);
+ run->exit_reason = KVM_EXIT_MMIO;
return 0;
}
phys_addr_t addr = start, end = start + size;
phys_addr_t next;
- pgd = pgdp + pgd_index(addr);
+ pgd = pgdp + kvm_pgd_index(addr);
do {
next = kvm_pgd_addr_end(addr, end);
if (!pgd_none(*pgd))
phys_addr_t next;
pgd_t *pgd;
- pgd = kvm->arch.pgd + pgd_index(addr);
+ pgd = kvm->arch.pgd + kvm_pgd_index(addr);
do {
next = kvm_pgd_addr_end(addr, end);
stage2_flush_puds(kvm, pgd, addr, next);
__phys_to_pfn(phys_addr), PAGE_HYP_DEVICE);
}
+/* Free the HW pgd, one page at a time */
+static void kvm_free_hwpgd(void *hwpgd)
+{
+ free_pages_exact(hwpgd, kvm_get_hwpgd_size());
+}
+
+/* Allocate the HW PGD, making sure that each page gets its own refcount */
+static void *kvm_alloc_hwpgd(void)
+{
+ unsigned int size = kvm_get_hwpgd_size();
+
+ return alloc_pages_exact(size, GFP_KERNEL | __GFP_ZERO);
+}
+
/**
* kvm_alloc_stage2_pgd - allocate level-1 table for stage-2 translation.
* @kvm: The KVM struct pointer for the VM.
*/
int kvm_alloc_stage2_pgd(struct kvm *kvm)
{
- int ret;
pgd_t *pgd;
+ void *hwpgd;
if (kvm->arch.pgd != NULL) {
kvm_err("kvm_arch already initialized?\n");
return -EINVAL;
}
+ hwpgd = kvm_alloc_hwpgd();
+ if (!hwpgd)
+ return -ENOMEM;
+
+ /* When the kernel uses more levels of page tables than the
+ * guest, we allocate a fake PGD and pre-populate it to point
+ * to the next-level page table, which will be the real
+ * initial page table pointed to by the VTTBR.
+ *
+ * When KVM_PREALLOC_LEVEL==2, we allocate a single page for
+ * the PMD and the kernel will use folded pud.
+ * When KVM_PREALLOC_LEVEL==1, we allocate 2 consecutive PUD
+ * pages.
+ */
if (KVM_PREALLOC_LEVEL > 0) {
+ int i;
+
/*
* Allocate fake pgd for the page table manipulation macros to
* work. This is not used by the hardware and we have no
*/
pgd = (pgd_t *)kmalloc(PTRS_PER_S2_PGD * sizeof(pgd_t),
GFP_KERNEL | __GFP_ZERO);
+
+ if (!pgd) {
+ kvm_free_hwpgd(hwpgd);
+ return -ENOMEM;
+ }
+
+ /* Plug the HW PGD into the fake one. */
+ for (i = 0; i < PTRS_PER_S2_PGD; i++) {
+ if (KVM_PREALLOC_LEVEL == 1)
+ pgd_populate(NULL, pgd + i,
+ (pud_t *)hwpgd + i * PTRS_PER_PUD);
+ else if (KVM_PREALLOC_LEVEL == 2)
+ pud_populate(NULL, pud_offset(pgd, 0) + i,
+ (pmd_t *)hwpgd + i * PTRS_PER_PMD);
+ }
} else {
/*
* Allocate actual first-level Stage-2 page table used by the
* hardware for Stage-2 page table walks.
*/
- pgd = (pgd_t *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, S2_PGD_ORDER);
+ pgd = (pgd_t *)hwpgd;
}
- if (!pgd)
- return -ENOMEM;
-
- ret = kvm_prealloc_hwpgd(kvm, pgd);
- if (ret)
- goto out_err;
-
kvm_clean_pgd(pgd);
kvm->arch.pgd = pgd;
return 0;
-out_err:
- if (KVM_PREALLOC_LEVEL > 0)
- kfree(pgd);
- else
- free_pages((unsigned long)pgd, S2_PGD_ORDER);
- return ret;
}
/**
return;
unmap_stage2_range(kvm, 0, KVM_PHYS_SIZE);
- kvm_free_hwpgd(kvm);
+ kvm_free_hwpgd(kvm_get_hwpgd(kvm));
if (KVM_PREALLOC_LEVEL > 0)
kfree(kvm->arch.pgd);
- else
- free_pages((unsigned long)kvm->arch.pgd, S2_PGD_ORDER);
+
kvm->arch.pgd = NULL;
}
pgd_t *pgd;
pud_t *pud;
- pgd = kvm->arch.pgd + pgd_index(addr);
+ pgd = kvm->arch.pgd + kvm_pgd_index(addr);
if (WARN_ON(pgd_none(*pgd))) {
if (!cache)
return NULL;
pgd_t *pgd;
phys_addr_t next;
- pgd = kvm->arch.pgd + pgd_index(addr);
+ pgd = kvm->arch.pgd + kvm_pgd_index(addr);
do {
/*
* Release kvm_mmu_lock periodically if the memory region is
out_unlock:
spin_unlock(&kvm->mmu_lock);
+ kvm_set_pfn_accessed(pfn);
kvm_release_pfn_clean(pfn);
return ret;
}
+/*
+ * Resolve the access fault by making the page young again.
+ * Note that because the faulting entry is guaranteed not to be
+ * cached in the TLB, we don't need to invalidate anything.
+ */
+static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)
+{
+ pmd_t *pmd;
+ pte_t *pte;
+ pfn_t pfn;
+ bool pfn_valid = false;
+
+ trace_kvm_access_fault(fault_ipa);
+
+ spin_lock(&vcpu->kvm->mmu_lock);
+
+ pmd = stage2_get_pmd(vcpu->kvm, NULL, fault_ipa);
+ if (!pmd || pmd_none(*pmd)) /* Nothing there */
+ goto out;
+
+ if (kvm_pmd_huge(*pmd)) { /* THP, HugeTLB */
+ *pmd = pmd_mkyoung(*pmd);
+ pfn = pmd_pfn(*pmd);
+ pfn_valid = true;
+ goto out;
+ }
+
+ pte = pte_offset_kernel(pmd, fault_ipa);
+ if (pte_none(*pte)) /* Nothing there either */
+ goto out;
+
+ *pte = pte_mkyoung(*pte); /* Just a page... */
+ pfn = pte_pfn(*pte);
+ pfn_valid = true;
+out:
+ spin_unlock(&vcpu->kvm->mmu_lock);
+ if (pfn_valid)
+ kvm_set_pfn_accessed(pfn);
+}
+
/**
* kvm_handle_guest_abort - handles all 2nd stage aborts
* @vcpu: the VCPU pointer
/* Check the stage-2 fault is trans. fault or write fault */
fault_status = kvm_vcpu_trap_get_fault_type(vcpu);
- if (fault_status != FSC_FAULT && fault_status != FSC_PERM) {
+ if (fault_status != FSC_FAULT && fault_status != FSC_PERM &&
+ fault_status != FSC_ACCESS) {
kvm_err("Unsupported FSC: EC=%#x xFSC=%#lx ESR_EL2=%#lx\n",
kvm_vcpu_trap_get_class(vcpu),
(unsigned long)kvm_vcpu_trap_get_fault(vcpu),
/* Userspace should not be able to register out-of-bounds IPAs */
VM_BUG_ON(fault_ipa >= KVM_PHYS_SIZE);
+ if (fault_status == FSC_ACCESS) {
+ handle_access_fault(vcpu, fault_ipa);
+ ret = 1;
+ goto out_unlock;
+ }
+
ret = user_mem_abort(vcpu, fault_ipa, memslot, hva, fault_status);
if (ret == 0)
ret = 1;
return ret;
}
-static void handle_hva_to_gpa(struct kvm *kvm,
- unsigned long start,
- unsigned long end,
- void (*handler)(struct kvm *kvm,
- gpa_t gpa, void *data),
- void *data)
+static int handle_hva_to_gpa(struct kvm *kvm,
+ unsigned long start,
+ unsigned long end,
+ int (*handler)(struct kvm *kvm,
+ gpa_t gpa, void *data),
+ void *data)
{
struct kvm_memslots *slots;
struct kvm_memory_slot *memslot;
+ int ret = 0;
slots = kvm_memslots(kvm);
for (; gfn < gfn_end; ++gfn) {
gpa_t gpa = gfn << PAGE_SHIFT;
- handler(kvm, gpa, data);
+ ret |= handler(kvm, gpa, data);
}
}
+
+ return ret;
}
-static void kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, void *data)
+static int kvm_unmap_hva_handler(struct kvm *kvm, gpa_t gpa, void *data)
{
unmap_stage2_range(kvm, gpa, PAGE_SIZE);
+ return 0;
}
int kvm_unmap_hva(struct kvm *kvm, unsigned long hva)
return 0;
}
-static void kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, void *data)
+static int kvm_set_spte_handler(struct kvm *kvm, gpa_t gpa, void *data)
{
pte_t *pte = (pte_t *)data;
* through this calling path.
*/
stage2_set_pte(kvm, NULL, gpa, pte, 0);
+ return 0;
}
handle_hva_to_gpa(kvm, hva, end, &kvm_set_spte_handler, &stage2_pte);
}
+static int kvm_age_hva_handler(struct kvm *kvm, gpa_t gpa, void *data)
+{
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pmd = stage2_get_pmd(kvm, NULL, gpa);
+ if (!pmd || pmd_none(*pmd)) /* Nothing there */
+ return 0;
+
+ if (kvm_pmd_huge(*pmd)) { /* THP, HugeTLB */
+ if (pmd_young(*pmd)) {
+ *pmd = pmd_mkold(*pmd);
+ return 1;
+ }
+
+ return 0;
+ }
+
+ pte = pte_offset_kernel(pmd, gpa);
+ if (pte_none(*pte))
+ return 0;
+
+ if (pte_young(*pte)) {
+ *pte = pte_mkold(*pte); /* Just a page... */
+ return 1;
+ }
+
+ return 0;
+}
+
+static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, void *data)
+{
+ pmd_t *pmd;
+ pte_t *pte;
+
+ pmd = stage2_get_pmd(kvm, NULL, gpa);
+ if (!pmd || pmd_none(*pmd)) /* Nothing there */
+ return 0;
+
+ if (kvm_pmd_huge(*pmd)) /* THP, HugeTLB */
+ return pmd_young(*pmd);
+
+ pte = pte_offset_kernel(pmd, gpa);
+ if (!pte_none(*pte)) /* Just a page... */
+ return pte_young(*pte);
+
+ return 0;
+}
+
+int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end)
+{
+ trace_kvm_age_hva(start, end);
+ return handle_hva_to_gpa(kvm, start, end, kvm_age_hva_handler, NULL);
+}
+
+int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
+{
+ trace_kvm_test_age_hva(hva);
+ return handle_hva_to_gpa(kvm, hva, hva, kvm_test_age_hva_handler, NULL);
+}
+
void kvm_mmu_free_memory_caches(struct kvm_vcpu *vcpu)
{
mmu_free_memory_cache(&vcpu->arch.mmu_page_cache);
);
TRACE_EVENT(kvm_exit,
- TP_PROTO(unsigned long vcpu_pc),
- TP_ARGS(vcpu_pc),
+ TP_PROTO(unsigned int exit_reason, unsigned long vcpu_pc),
+ TP_ARGS(exit_reason, vcpu_pc),
TP_STRUCT__entry(
+ __field( unsigned int, exit_reason )
__field( unsigned long, vcpu_pc )
),
TP_fast_assign(
+ __entry->exit_reason = exit_reason;
__entry->vcpu_pc = vcpu_pc;
),
- TP_printk("PC: 0x%08lx", __entry->vcpu_pc)
+ TP_printk("HSR_EC: 0x%04x, PC: 0x%08lx",
+ __entry->exit_reason,
+ __entry->vcpu_pc)
);
TRACE_EVENT(kvm_guest_fault,
__entry->hxfar, __entry->vcpu_pc)
);
+TRACE_EVENT(kvm_access_fault,
+ TP_PROTO(unsigned long ipa),
+ TP_ARGS(ipa),
+
+ TP_STRUCT__entry(
+ __field( unsigned long, ipa )
+ ),
+
+ TP_fast_assign(
+ __entry->ipa = ipa;
+ ),
+
+ TP_printk("IPA: %lx", __entry->ipa)
+);
+
TRACE_EVENT(kvm_irq_line,
TP_PROTO(unsigned int type, int vcpu_idx, int irq_num, int level),
TP_ARGS(type, vcpu_idx, irq_num, level),
TP_printk("mmu notifier set pte hva: %#08lx", __entry->hva)
);
+TRACE_EVENT(kvm_age_hva,
+ TP_PROTO(unsigned long start, unsigned long end),
+ TP_ARGS(start, end),
+
+ TP_STRUCT__entry(
+ __field( unsigned long, start )
+ __field( unsigned long, end )
+ ),
+
+ TP_fast_assign(
+ __entry->start = start;
+ __entry->end = end;
+ ),
+
+ TP_printk("mmu notifier age hva: %#08lx -- %#08lx",
+ __entry->start, __entry->end)
+);
+
+TRACE_EVENT(kvm_test_age_hva,
+ TP_PROTO(unsigned long hva),
+ TP_ARGS(hva),
+
+ TP_STRUCT__entry(
+ __field( unsigned long, hva )
+ ),
+
+ TP_fast_assign(
+ __entry->hva = hva;
+ ),
+
+ TP_printk("mmu notifier test age hva: %#08lx", __entry->hva)
+);
+
TRACE_EVENT(kvm_hvc,
TP_PROTO(unsigned long vcpu_pc, unsigned long r0, unsigned long imm),
TP_ARGS(vcpu_pc, r0, imm),
bool "Alphascale ASM9260"
depends on ARCH_MULTI_V5
select CPU_ARM926T
+ select ASM9260_TIMER
+ select GENERIC_CLOCKEVENTS
help
Support for Alphascale ASM9260 based platform.
phys_addr_t sram_pbase;
unsigned long sram_base;
struct device_node *node;
- struct platform_device *pdev;
+ struct platform_device *pdev = NULL;
- node = of_find_compatible_node(NULL, NULL, "mmio-sram");
- if (!node) {
- pr_warn("%s: failed to find sram node!\n", __func__);
- return;
+ for_each_compatible_node(node, NULL, "mmio-sram") {
+ pdev = of_find_device_by_node(node);
+ if (pdev) {
+ of_node_put(node);
+ break;
+ }
}
- pdev = of_find_device_by_node(node);
if (!pdev) {
pr_warn("%s: failed to find sram device!\n", __func__);
- goto put_node;
+ return;
}
sram_pool = dev_get_gen_pool(&pdev->dev);
if (!sram_pool) {
pr_warn("%s: sram pool unavailable!\n", __func__);
- goto put_node;
+ return;
}
sram_base = gen_pool_alloc(sram_pool, at91_slow_clock_sz);
if (!sram_base) {
pr_warn("%s: unable to alloc ocram!\n", __func__);
- goto put_node;
+ return;
}
sram_pbase = gen_pool_virt_to_phys(sram_pool, sram_base);
slow_clock = __arm_ioremap_exec(sram_pbase, at91_slow_clock_sz, false);
-
-put_node:
- of_node_put(node);
}
#endif
" mcr p15, 0, %0, c7, c0, 4\n\t"
" str %5, [%1, %2]"
:
- : "r" (0), "r" (AT91_BASE_SYS), "r" (AT91RM9200_SDRAMC_LPR),
+ : "r" (0), "r" (at91_ramc_base[0]), "r" (AT91RM9200_SDRAMC_LPR),
"r" (1), "r" (AT91RM9200_SDRAMC_SRR),
"r" (lpr));
}
*/
#undef SLOWDOWN_MASTER_CLOCK
-#define MCKRDY_TIMEOUT 1000
-#define MOSCRDY_TIMEOUT 1000
-#define PLLALOCK_TIMEOUT 1000
-#define PLLBLOCK_TIMEOUT 1000
-
pmc .req r0
sdramc .req r1
ramc1 .req r2
* Wait until master clock is ready (after switching master clock source)
*/
.macro wait_mckrdy
- mov tmp2, #MCKRDY_TIMEOUT
-1: sub tmp2, tmp2, #1
- cmp tmp2, #0
- beq 2f
- ldr tmp1, [pmc, #AT91_PMC_SR]
+1: ldr tmp1, [pmc, #AT91_PMC_SR]
tst tmp1, #AT91_PMC_MCKRDY
beq 1b
-2:
.endm
/*
* Wait until master oscillator has stabilized.
*/
.macro wait_moscrdy
- mov tmp2, #MOSCRDY_TIMEOUT
-1: sub tmp2, tmp2, #1
- cmp tmp2, #0
- beq 2f
- ldr tmp1, [pmc, #AT91_PMC_SR]
+1: ldr tmp1, [pmc, #AT91_PMC_SR]
tst tmp1, #AT91_PMC_MOSCS
beq 1b
-2:
.endm
/*
* Wait until PLLA has locked.
*/
.macro wait_pllalock
- mov tmp2, #PLLALOCK_TIMEOUT
-1: sub tmp2, tmp2, #1
- cmp tmp2, #0
- beq 2f
- ldr tmp1, [pmc, #AT91_PMC_SR]
+1: ldr tmp1, [pmc, #AT91_PMC_SR]
tst tmp1, #AT91_PMC_LOCKA
beq 1b
-2:
.endm
/*
* Wait until PLLB has locked.
*/
.macro wait_pllblock
- mov tmp2, #PLLBLOCK_TIMEOUT
-1: sub tmp2, tmp2, #1
- cmp tmp2, #0
- beq 2f
- ldr tmp1, [pmc, #AT91_PMC_SR]
+1: ldr tmp1, [pmc, #AT91_PMC_SR]
tst tmp1, #AT91_PMC_LOCKB
beq 1b
-2:
.endm
.text
+ .arm
+
/* void at91_slow_clock(void __iomem *pmc, void __iomem *sdramc,
* void __iomem *ramc1, int memctrl)
*/
cmp memctrl, #AT91_MEMCTRL_DDRSDR
bne sdr_sr_enable
+ /* LPDDR1 --> force DDR2 mode during self-refresh */
+ ldr tmp1, [sdramc, #AT91_DDRSDRC_MDR]
+ str tmp1, .saved_sam9_mdr
+ bic tmp1, tmp1, #~AT91_DDRSDRC_MD
+ cmp tmp1, #AT91_DDRSDRC_MD_LOW_POWER_DDR
+ ldreq tmp1, [sdramc, #AT91_DDRSDRC_MDR]
+ biceq tmp1, tmp1, #AT91_DDRSDRC_MD
+ orreq tmp1, tmp1, #AT91_DDRSDRC_MD_DDR2
+ streq tmp1, [sdramc, #AT91_DDRSDRC_MDR]
+
/* prepare for DDRAM self-refresh mode */
ldr tmp1, [sdramc, #AT91_DDRSDRC_LPR]
str tmp1, .saved_sam9_lpr
/* figure out if we use the second ram controller */
cmp ramc1, #0
- ldrne tmp2, [ramc1, #AT91_DDRSDRC_LPR]
- strne tmp2, .saved_sam9_lpr1
- bicne tmp2, #AT91_DDRSDRC_LPCB
- orrne tmp2, #AT91_DDRSDRC_LPCB_SELF_REFRESH
+ beq ddr_no_2nd_ctrl
+
+ ldr tmp2, [ramc1, #AT91_DDRSDRC_MDR]
+ str tmp2, .saved_sam9_mdr1
+ bic tmp2, tmp2, #~AT91_DDRSDRC_MD
+ cmp tmp2, #AT91_DDRSDRC_MD_LOW_POWER_DDR
+ ldreq tmp2, [ramc1, #AT91_DDRSDRC_MDR]
+ biceq tmp2, tmp2, #AT91_DDRSDRC_MD
+ orreq tmp2, tmp2, #AT91_DDRSDRC_MD_DDR2
+ streq tmp2, [ramc1, #AT91_DDRSDRC_MDR]
+
+ ldr tmp2, [ramc1, #AT91_DDRSDRC_LPR]
+ str tmp2, .saved_sam9_lpr1
+ bic tmp2, #AT91_DDRSDRC_LPCB
+ orr tmp2, #AT91_DDRSDRC_LPCB_SELF_REFRESH
/* Enable DDRAM self-refresh mode */
+ str tmp2, [ramc1, #AT91_DDRSDRC_LPR]
+ddr_no_2nd_ctrl:
str tmp1, [sdramc, #AT91_DDRSDRC_LPR]
- strne tmp2, [ramc1, #AT91_DDRSDRC_LPR]
b sdr_sr_done
/* Turn off the main oscillator */
ldr tmp1, [pmc, #AT91_CKGR_MOR]
bic tmp1, tmp1, #AT91_PMC_MOSCEN
+ orr tmp1, tmp1, #AT91_PMC_KEY
str tmp1, [pmc, #AT91_CKGR_MOR]
/* Wait for interrupt */
/* Turn on the main oscillator */
ldr tmp1, [pmc, #AT91_CKGR_MOR]
orr tmp1, tmp1, #AT91_PMC_MOSCEN
+ orr tmp1, tmp1, #AT91_PMC_KEY
str tmp1, [pmc, #AT91_CKGR_MOR]
wait_moscrdy
*/
cmp memctrl, #AT91_MEMCTRL_DDRSDR
bne sdr_en_restore
+ /* Restore MDR in case of LPDDR1 */
+ ldr tmp1, .saved_sam9_mdr
+ str tmp1, [sdramc, #AT91_DDRSDRC_MDR]
/* Restore LPR on AT91 with DDRAM */
ldr tmp1, .saved_sam9_lpr
str tmp1, [sdramc, #AT91_DDRSDRC_LPR]
/* if we use the second ram controller */
cmp ramc1, #0
+ ldrne tmp2, .saved_sam9_mdr1
+ strne tmp2, [ramc1, #AT91_DDRSDRC_MDR]
ldrne tmp2, .saved_sam9_lpr1
strne tmp2, [ramc1, #AT91_DDRSDRC_LPR]
.saved_sam9_lpr1:
.word 0
+.saved_sam9_mdr:
+ .word 0
+
+.saved_sam9_mdr1:
+ .word 0
+
ENTRY(at91_slow_clock_sz)
.word .-at91_slow_clock
static struct pci_bus __init *
dove_pcie_scan_bus(int nr, struct pci_sys_data *sys)
{
- struct pci_bus *bus;
-
- if (nr < num_pcie_ports) {
- bus = pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
- &sys->resources);
- } else {
- bus = NULL;
+ if (nr >= num_pcie_ports) {
BUG();
+ return NULL;
}
- return bus;
+ return pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
+ &sys->resources);
}
static int __init dove_pcie_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
exynos_map_io();
}
+/*
+ * Apparently, these SoCs are not able to wake-up from suspend using
+ * the PMU. Too bad. Should they suddenly become capable of such a
+ * feat, the matches below should be moved to suspend.c.
+ */
static const struct of_device_id exynos_dt_pmu_match[] = {
- { .compatible = "samsung,exynos3250-pmu" },
- { .compatible = "samsung,exynos4210-pmu" },
- { .compatible = "samsung,exynos4212-pmu" },
- { .compatible = "samsung,exynos4412-pmu" },
- { .compatible = "samsung,exynos4415-pmu" },
- { .compatible = "samsung,exynos5250-pmu" },
{ .compatible = "samsung,exynos5260-pmu" },
{ .compatible = "samsung,exynos5410-pmu" },
- { .compatible = "samsung,exynos5420-pmu" },
{ /*sentinel*/ },
};
np = of_find_matching_node(NULL, exynos_dt_pmu_match);
if (np)
pmu_base_addr = of_iomap(np, 0);
-
- if (!pmu_base_addr)
- panic("failed to find exynos pmu register\n");
}
static void __init exynos_init_irq(void)
*/
void exynos_cpu_power_down(int cpu)
{
- if (cpu == 0 && (of_machine_is_compatible("samsung,exynos5420") ||
- of_machine_is_compatible("samsung,exynos5800"))) {
+ if (cpu == 0 && (soc_is_exynos5420() || soc_is_exynos5800())) {
/*
* Bypass power down for CPU0 during suspend. Check for
* the SYS_PWR_REG value to decide if we are suspending
of_genpd_add_provider_simple(np, &pd->pd);
}
+ /* Assign the child power domains to their parents */
+ for_each_compatible_node(np, NULL, "samsung,exynos4210-pd") {
+ struct generic_pm_domain *child_domain, *parent_domain;
+ struct of_phandle_args args;
+
+ args.np = np;
+ args.args_count = 0;
+ child_domain = of_genpd_get_from_provider(&args);
+ if (!child_domain)
+ continue;
+
+ if (of_parse_phandle_with_args(np, "power-domains",
+ "#power-domain-cells", 0, &args) != 0)
+ continue;
+
+ parent_domain = of_genpd_get_from_provider(&args);
+ if (!parent_domain)
+ continue;
+
+ if (pm_genpd_add_subdomain(parent_domain, child_domain))
+ pr_warn("%s failed to add subdomain: %s\n",
+ parent_domain->name, child_domain->name);
+ else
+ pr_info("%s has as child subdomain: %s.\n",
+ parent_domain->name, child_domain->name);
+ of_node_put(np);
+ }
+
return 0;
}
arch_initcall(exynos4_pm_init_power_domain);
#include <linux/syscore_ops.h>
#include <linux/cpu_pm.h>
#include <linux/io.h>
-#include <linux/irqchip/arm-gic.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/of_address.h>
#include <linux/err.h>
#include <linux/regulator/machine.h>
#define EXYNOS5420_CPU_STATE 0x28
/**
- * struct exynos_wkup_irq - Exynos GIC to PMU IRQ mapping
- * @hwirq: Hardware IRQ signal of the GIC
+ * struct exynos_wkup_irq - PMU IRQ to mask mapping
+ * @hwirq: Hardware IRQ signal of the PMU
* @mask: Mask in PMU wake-up mask register
*/
struct exynos_wkup_irq {
static u32 exynos_irqwake_intmask = 0xffffffff;
static const struct exynos_wkup_irq exynos3250_wkup_irq[] = {
- { 73, BIT(1) }, /* RTC alarm */
- { 74, BIT(2) }, /* RTC tick */
+ { 105, BIT(1) }, /* RTC alarm */
+ { 106, BIT(2) }, /* RTC tick */
{ /* sentinel */ },
};
static const struct exynos_wkup_irq exynos4_wkup_irq[] = {
- { 76, BIT(1) }, /* RTC alarm */
- { 77, BIT(2) }, /* RTC tick */
+ { 44, BIT(1) }, /* RTC alarm */
+ { 45, BIT(2) }, /* RTC tick */
{ /* sentinel */ },
};
static const struct exynos_wkup_irq exynos5250_wkup_irq[] = {
- { 75, BIT(1) }, /* RTC alarm */
- { 76, BIT(2) }, /* RTC tick */
+ { 43, BIT(1) }, /* RTC alarm */
+ { 44, BIT(2) }, /* RTC tick */
{ /* sentinel */ },
};
return -ENOENT;
}
+static struct irq_chip exynos_pmu_chip = {
+ .name = "PMU",
+ .irq_eoi = irq_chip_eoi_parent,
+ .irq_mask = irq_chip_mask_parent,
+ .irq_unmask = irq_chip_unmask_parent,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .irq_set_wake = exynos_irq_set_wake,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+#endif
+};
+
+static int exynos_pmu_domain_xlate(struct irq_domain *domain,
+ struct device_node *controller,
+ const u32 *intspec,
+ unsigned int intsize,
+ unsigned long *out_hwirq,
+ unsigned int *out_type)
+{
+ if (domain->of_node != controller)
+ return -EINVAL; /* Shouldn't happen, really... */
+ if (intsize != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (intspec[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ *out_hwirq = intspec[1];
+ *out_type = intspec[2];
+ return 0;
+}
+
+static int exynos_pmu_domain_alloc(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs, void *data)
+{
+ struct of_phandle_args *args = data;
+ struct of_phandle_args parent_args;
+ irq_hw_number_t hwirq;
+ int i;
+
+ if (args->args_count != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (args->args[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ hwirq = args->args[1];
+
+ for (i = 0; i < nr_irqs; i++)
+ irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+ &exynos_pmu_chip, NULL);
+
+ parent_args = *args;
+ parent_args.np = domain->parent->of_node;
+ return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, &parent_args);
+}
+
+static struct irq_domain_ops exynos_pmu_domain_ops = {
+ .xlate = exynos_pmu_domain_xlate,
+ .alloc = exynos_pmu_domain_alloc,
+ .free = irq_domain_free_irqs_common,
+};
+
+static int __init exynos_pmu_irq_init(struct device_node *node,
+ struct device_node *parent)
+{
+ struct irq_domain *parent_domain, *domain;
+
+ if (!parent) {
+ pr_err("%s: no parent, giving up\n", node->full_name);
+ return -ENODEV;
+ }
+
+ parent_domain = irq_find_host(parent);
+ if (!parent_domain) {
+ pr_err("%s: unable to obtain parent domain\n", node->full_name);
+ return -ENXIO;
+ }
+
+ pmu_base_addr = of_iomap(node, 0);
+
+ if (!pmu_base_addr) {
+ pr_err("%s: failed to find exynos pmu register\n",
+ node->full_name);
+ return -ENOMEM;
+ }
+
+ domain = irq_domain_add_hierarchy(parent_domain, 0, 0,
+ node, &exynos_pmu_domain_ops,
+ NULL);
+ if (!domain) {
+ iounmap(pmu_base_addr);
+ return -ENOMEM;
+ }
+
+ return 0;
+}
+
+#define EXYNOS_PMU_IRQ(symbol, name) OF_DECLARE_2(irqchip, symbol, name, exynos_pmu_irq_init)
+
+EXYNOS_PMU_IRQ(exynos3250_pmu_irq, "samsung,exynos3250-pmu");
+EXYNOS_PMU_IRQ(exynos4210_pmu_irq, "samsung,exynos4210-pmu");
+EXYNOS_PMU_IRQ(exynos4212_pmu_irq, "samsung,exynos4212-pmu");
+EXYNOS_PMU_IRQ(exynos4412_pmu_irq, "samsung,exynos4412-pmu");
+EXYNOS_PMU_IRQ(exynos4415_pmu_irq, "samsung,exynos4415-pmu");
+EXYNOS_PMU_IRQ(exynos5250_pmu_irq, "samsung,exynos5250-pmu");
+EXYNOS_PMU_IRQ(exynos5420_pmu_irq, "samsung,exynos5420-pmu");
+
static int exynos_cpu_do_idle(void)
{
/* issue the standby signal into the pm unit. */
void __init exynos_pm_init(void)
{
const struct of_device_id *match;
+ struct device_node *np;
u32 tmp;
- of_find_matching_node_and_match(NULL, exynos_pmu_of_device_ids, &match);
- if (!match) {
+ np = of_find_matching_node_and_match(NULL, exynos_pmu_of_device_ids, &match);
+ if (!np) {
pr_err("Failed to find PMU node\n");
return;
}
- pm_data = (struct exynos_pm_data *) match->data;
- /* Platform-specific GIC callback */
- gic_arch_extn.irq_set_wake = exynos_irq_set_wake;
+ if (WARN_ON(!of_find_property(np, "interrupt-controller", NULL)))
+ pr_warn("Outdated DT detected, suspend/resume will NOT work\n");
+
+ pm_data = (struct exynos_pm_data *) match->data;
/* All wakeup disable */
tmp = pmu_raw_readl(S5P_WAKEUP_MASK);
config SOC_VF610
bool "Vybrid Family VF610 support"
+ select IRQ_DOMAIN_HIERARCHY
select ARM_GIC
select PINCTRL_VF610
select PL310_ERRATA_769419 if CACHE_L2X0
* set bit IOMUXC_GPR1[21]. Or the PTP clock must be from pad
* (external OSC), and we need to clear the bit.
*/
- clksel = ptp_clk == enet_ref ? IMX6Q_GPR1_ENET_CLK_SEL_ANATOP :
- IMX6Q_GPR1_ENET_CLK_SEL_PAD;
+ clksel = clk_is_match(ptp_clk, enet_ref) ?
+ IMX6Q_GPR1_ENET_CLK_SEL_ANATOP :
+ IMX6Q_GPR1_ENET_CLK_SEL_PAD;
gpr = syscon_regmap_lookup_by_compatible("fsl,imx6q-iomuxc-gpr");
if (!IS_ERR(gpr))
regmap_update_bits(gpr, IOMUXC_GPR1,
#include <linux/input.h>
#include <linux/io.h>
#include <linux/delay.h>
+#include <linux/smc91x.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
[1] = {
.start = MSM_GPIO_TO_INT(49),
.end = MSM_GPIO_TO_INT(49),
- .flags = IORESOURCE_IRQ,
+ .flags = IORESOURCE_IRQ | IORESOURCE_IRQ_HIGHLEVEL,
},
};
+static struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_16BIT | SMC91X_NOWAIT,
+};
+
static struct platform_device smc91x_device = {
.name = "smc91x",
.id = 0,
.num_resources = ARRAY_SIZE(smc91x_resources),
.resource = smc91x_resources,
+ .dev.platform_data = &smc91x_platdata,
};
static struct platform_device *devices[] __initdata = {
#include <linux/usb/msm_hsusb.h>
#include <linux/err.h>
#include <linux/clkdev.h>
+#include <linux/smc91x.h>
#include <asm/mach-types.h>
#include <asm/mach/arch.h>
.flags = IORESOURCE_MEM,
},
[1] = {
- .flags = IORESOURCE_IRQ,
+ .flags = IORESOURCE_IRQ | IORESOURCE_IRQ_HIGHLEVEL,
},
};
+static struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_16BIT | SMC91X_NOWAIT,
+};
+
static struct platform_device smc91x_device = {
.name = "smc91x",
.id = 0,
.num_resources = ARRAY_SIZE(smc91x_resources),
.resource = smc91x_resources,
+ .dev.platform_data = &smc91x_platdata,
};
static int __init msm_init_smc91x(void)
static struct pci_bus __init *
mv78xx0_pcie_scan_bus(int nr, struct pci_sys_data *sys)
{
- struct pci_bus *bus;
-
- if (nr < num_pcie_ports) {
- bus = pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
- &sys->resources);
- } else {
- bus = NULL;
+ if (nr >= num_pcie_ports) {
BUG();
+ return NULL;
}
- return bus;
+ return pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
+ &sys->resources);
}
static int __init mv78xx0_pcie_map_irq(const struct pci_dev *dev, u8 slot,
#include <linux/cpuidle.h>
#include <linux/cpu_pm.h>
#include <linux/export.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <asm/cpuidle.h>
#include <asm/proc-fns.h>
{
struct idle_statedata *cx = state_ptr + index;
u32 mpuss_can_lose_context = 0;
- int cpu_id = smp_processor_id();
/*
* CPU0 has to wait and stay ON until CPU1 is OFF state.
mpuss_can_lose_context = (cx->mpu_state == PWRDM_POWER_RET) &&
(cx->mpu_logic_state == PWRDM_POWER_OFF);
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu_id);
+ tick_broadcast_enter();
/*
* Call idle CPU PM enter notifier chain so that
if (dev->cpu == 0 && mpuss_can_lose_context)
cpu_cluster_pm_exit();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu_id);
+ tick_broadcast_exit();
fail:
cpuidle_coupled_parallel_barrier(dev, &abort_barrier);
*/
static void omap_setup_broadcast_timer(void *arg)
{
- int cpu = smp_processor_id();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ON, &cpu);
+ tick_broadcast_enable();
}
static struct cpuidle_driver omap4_idle_driver = {
static inline void omap_hsmmc_mux(struct omap_hsmmc_platform_data
*mmc_controller, int controller_nr)
{
- if (gpio_is_valid(mmc_controller->switch_pin) &&
- (mmc_controller->switch_pin < OMAP_MAX_GPIO_LINES))
- omap_mux_init_gpio(mmc_controller->switch_pin,
+ if (gpio_is_valid(mmc_controller->gpio_cd) &&
+ (mmc_controller->gpio_cd < OMAP_MAX_GPIO_LINES))
+ omap_mux_init_gpio(mmc_controller->gpio_cd,
+ OMAP_PIN_INPUT_PULLUP);
+ if (gpio_is_valid(mmc_controller->gpio_cod) &&
+ (mmc_controller->gpio_cod < OMAP_MAX_GPIO_LINES))
+ omap_mux_init_gpio(mmc_controller->gpio_cod,
OMAP_PIN_INPUT_PULLUP);
if (gpio_is_valid(mmc_controller->gpio_wp) &&
(mmc_controller->gpio_wp < OMAP_MAX_GPIO_LINES))
mmc->internal_clock = !c->ext_clock;
mmc->reg_offset = 0;
- mmc->switch_pin = c->gpio_cd;
+ if (c->cover_only) {
+ /* detect if mobile phone cover removed */
+ mmc->gpio_cd = -EINVAL;
+ mmc->gpio_cod = c->gpio_cd;
+ } else {
+ /* card detect pin on the mmc socket itself */
+ mmc->gpio_cd = c->gpio_cd;
+ mmc->gpio_cod = -EINVAL;
+ }
mmc->gpio_wp = c->gpio_wp;
mmc->remux = c->remux;
mmc->init_card = c->init_card;
- if (c->cover_only)
- mmc->cover = 1;
-
if (c->nonremovable)
mmc->nonremovable = 1;
if (!mmc_pdata)
continue;
- mmc_pdata->switch_pin = c->gpio_cd;
+ if (c->cover_only) {
+ /* detect if mobile phone cover removed */
+ mmc_pdata->gpio_cd = -EINVAL;
+ mmc_pdata->gpio_cod = c->gpio_cd;
+ } else {
+ /* card detect pin on the mmc socket itself */
+ mmc_pdata->gpio_cd = c->gpio_cd;
+ mmc_pdata->gpio_cod = -EINVAL;
+ }
mmc_pdata->gpio_wp = c->gpio_wp;
res = omap_device_register(pdev);
return kasprintf(GFP_KERNEL, "OMAP4");
else if (soc_is_omap54xx())
return kasprintf(GFP_KERNEL, "OMAP5");
+ else if (soc_is_am33xx() || soc_is_am335x())
+ return kasprintf(GFP_KERNEL, "AM33xx");
else if (soc_is_am43xx())
return kasprintf(GFP_KERNEL, "AM43xx");
else if (soc_is_dra7xx())
#include <linux/init.h>
#include <linux/io.h>
#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/of_address.h>
#include <linux/platform_device.h>
#include <linux/cpu.h>
#include <linux/notifier.h>
#include <linux/cpu_pm.h>
-#include <linux/irqchip/arm-gic.h>
#include "omap-wakeupgen.h"
#include "omap-secure.h"
static inline int _wakeupgen_get_irq_info(u32 irq, u32 *bit_posn, u8 *reg_index)
{
- unsigned int spi_irq;
-
- /*
- * PPIs and SGIs are not supported.
- */
- if (irq < OMAP44XX_IRQ_GIC_START)
- return -EINVAL;
-
- /*
- * Subtract the GIC offset.
- */
- spi_irq = irq - OMAP44XX_IRQ_GIC_START;
- if (spi_irq > MAX_IRQS) {
- pr_err("omap wakeupGen: Invalid IRQ%d\n", irq);
- return -EINVAL;
- }
-
/*
* Each WakeupGen register controls 32 interrupt.
* i.e. 1 bit per SPI IRQ
*/
- *reg_index = spi_irq >> 5;
- *bit_posn = spi_irq %= 32;
+ *reg_index = irq >> 5;
+ *bit_posn = irq %= 32;
return 0;
}
raw_spin_lock_irqsave(&wakeupgen_lock, flags);
_wakeupgen_clear(d->hwirq, irq_target_cpu[d->hwirq]);
raw_spin_unlock_irqrestore(&wakeupgen_lock, flags);
+ irq_chip_mask_parent(d);
}
/*
raw_spin_lock_irqsave(&wakeupgen_lock, flags);
_wakeupgen_set(d->hwirq, irq_target_cpu[d->hwirq]);
raw_spin_unlock_irqrestore(&wakeupgen_lock, flags);
+ irq_chip_unmask_parent(d);
}
#ifdef CONFIG_HOTPLUG_CPU
return omap_secure_apis;
}
+static struct irq_chip wakeupgen_chip = {
+ .name = "WUGEN",
+ .irq_eoi = irq_chip_eoi_parent,
+ .irq_mask = wakeupgen_mask,
+ .irq_unmask = wakeupgen_unmask,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+#endif
+};
+
+static int wakeupgen_domain_xlate(struct irq_domain *domain,
+ struct device_node *controller,
+ const u32 *intspec,
+ unsigned int intsize,
+ unsigned long *out_hwirq,
+ unsigned int *out_type)
+{
+ if (domain->of_node != controller)
+ return -EINVAL; /* Shouldn't happen, really... */
+ if (intsize != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (intspec[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ *out_hwirq = intspec[1];
+ *out_type = intspec[2];
+ return 0;
+}
+
+static int wakeupgen_domain_alloc(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs, void *data)
+{
+ struct of_phandle_args *args = data;
+ struct of_phandle_args parent_args;
+ irq_hw_number_t hwirq;
+ int i;
+
+ if (args->args_count != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (args->args[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ hwirq = args->args[1];
+ if (hwirq >= MAX_IRQS)
+ return -EINVAL; /* Can't deal with this */
+
+ for (i = 0; i < nr_irqs; i++)
+ irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+ &wakeupgen_chip, NULL);
+
+ parent_args = *args;
+ parent_args.np = domain->parent->of_node;
+ return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, &parent_args);
+}
+
+static struct irq_domain_ops wakeupgen_domain_ops = {
+ .xlate = wakeupgen_domain_xlate,
+ .alloc = wakeupgen_domain_alloc,
+ .free = irq_domain_free_irqs_common,
+};
+
/*
* Initialise the wakeupgen module.
*/
-int __init omap_wakeupgen_init(void)
+static int __init wakeupgen_init(struct device_node *node,
+ struct device_node *parent)
{
+ struct irq_domain *parent_domain, *domain;
int i;
unsigned int boot_cpu = smp_processor_id();
u32 val;
+ if (!parent) {
+ pr_err("%s: no parent, giving up\n", node->full_name);
+ return -ENODEV;
+ }
+
+ parent_domain = irq_find_host(parent);
+ if (!parent_domain) {
+ pr_err("%s: unable to obtain parent domain\n", node->full_name);
+ return -ENXIO;
+ }
/* Not supported on OMAP4 ES1.0 silicon */
if (omap_rev() == OMAP4430_REV_ES1_0) {
WARN(1, "WakeupGen: Not supported on OMAP4430 ES1.0\n");
}
/* Static mapping, never released */
- wakeupgen_base = ioremap(OMAP_WKUPGEN_BASE, SZ_4K);
+ wakeupgen_base = of_iomap(node, 0);
if (WARN_ON(!wakeupgen_base))
return -ENOMEM;
max_irqs = AM43XX_IRQS;
}
+ domain = irq_domain_add_hierarchy(parent_domain, 0, max_irqs,
+ node, &wakeupgen_domain_ops,
+ NULL);
+ if (!domain) {
+ iounmap(wakeupgen_base);
+ return -ENOMEM;
+ }
+
/* Clear all IRQ bitmasks at wakeupGen level */
for (i = 0; i < irq_banks; i++) {
wakeupgen_writel(0, i, CPU0_ID);
wakeupgen_writel(0, i, CPU1_ID);
}
- /*
- * Override GIC architecture specific functions to add
- * OMAP WakeupGen interrupt controller along with GIC
- */
- gic_arch_extn.irq_mask = wakeupgen_mask;
- gic_arch_extn.irq_unmask = wakeupgen_unmask;
- gic_arch_extn.flags = IRQCHIP_MASK_ON_SUSPEND | IRQCHIP_SKIP_SET_WAKE;
-
/*
* FIXME: Add support to set_smp_affinity() once the core
* GIC code has necessary hooks in place.
return 0;
}
+
+/*
+ * We cannot use the IRQCHIP_DECLARE macro that lives in
+ * drivers/irqchip, so we're forced to roll our own. Not very nice.
+ */
+OF_DECLARE_2(irqchip, ti_wakeupgen, "ti,omap4-wugen-mpu", wakeupgen_init);
#define OMAP_TIMESTAMPCYCLELO 0xc08
#define OMAP_TIMESTAMPCYCLEHI 0xc0c
-extern int __init omap_wakeupgen_init(void);
extern void __iomem *omap_get_wakeupgen_base(void);
extern int omap_secure_apis_support(void);
#endif
#include <linux/of_platform.h>
#include <linux/export.h>
#include <linux/irqchip/arm-gic.h>
-#include <linux/irqchip/irq-crossbar.h>
#include <linux/of_address.h>
#include <linux/reboot.h>
#include <linux/genalloc.h>
}
omap_early_initcall(omap4_sar_ram_init);
-static const struct of_device_id gic_match[] = {
- { .compatible = "arm,cortex-a9-gic", },
- { .compatible = "arm,cortex-a15-gic", },
+static const struct of_device_id intc_match[] = {
+ { .compatible = "ti,omap4-wugen-mpu", },
+ { .compatible = "ti,omap5-wugen-mpu", },
{ },
};
-static struct device_node *gic_node;
+static struct device_node *intc_node;
unsigned int omap4_xlate_irq(unsigned int hwirq)
{
struct of_phandle_args irq_data;
unsigned int irq;
- if (!gic_node)
- gic_node = of_find_matching_node(NULL, gic_match);
+ if (!intc_node)
+ intc_node = of_find_matching_node(NULL, intc_match);
- if (WARN_ON(!gic_node))
+ if (WARN_ON(!intc_node))
return hwirq;
- irq_data.np = gic_node;
+ irq_data.np = intc_node;
irq_data.args_count = 3;
irq_data.args[0] = 0;
irq_data.args[1] = hwirq - OMAP44XX_IRQ_GIC_START;
{
struct device_node *np;
+ intc_node = of_find_matching_node(NULL, intc_match);
+ if (WARN_ON(!intc_node)) {
+ pr_err("No WUGEN found in DT, system will misbehave.\n");
+ pr_err("UPDATE YOUR DEVICE TREE!\n");
+ }
+
/* Extract GIC distributor and TWD bases for OMAP4460 ROM Errata WA */
if (!cpu_is_omap446x())
goto skip_errata_init;
WARN_ON(!twd_base);
skip_errata_init:
- omap_wakeupgen_init();
-#ifdef CONFIG_IRQ_CROSSBAR
- irqcrossbar_init();
-#endif
irqchip_init();
}
if (ret == -EBUSY)
pr_warn("omap_hwmod: %s: failed to hardreset\n", oh->name);
- if (!ret) {
+ if (oh->clkdm) {
/*
* Set the clockdomain to HW_AUTO, assuming that the
* previous state was HW_AUTO.
*/
- if (oh->clkdm && hwsup)
+ if (hwsup)
clkdm_allow_idle(oh->clkdm);
- } else {
- if (oh->clkdm)
- clkdm_hwmod_disable(oh->clkdm, oh);
+
+ clkdm_hwmod_disable(oh->clkdm, oh);
}
return ret;
INIT_LIST_HEAD(&oh->master_ports);
INIT_LIST_HEAD(&oh->slave_ports);
spin_lock_init(&oh->_lock);
+ lockdep_set_class(&oh->_lock, &oh->hwmod_key);
oh->_state = _HWMOD_STATE_REGISTERED;
u32 _sysc_cache;
void __iomem *_mpu_rt_va;
spinlock_t _lock;
+ struct lock_class_key hwmod_key; /* unique lock class */
struct list_head node;
struct omap_hwmod_ocp_if *_mpu_port;
unsigned int (*xlate_irq)(unsigned int);
*
*/
-static struct omap_hwmod_class dra7xx_pcie_hwmod_class = {
+static struct omap_hwmod_class dra7xx_pciess_hwmod_class = {
.name = "pcie",
};
/* pcie1 */
-static struct omap_hwmod dra7xx_pcie1_hwmod = {
+static struct omap_hwmod dra7xx_pciess1_hwmod = {
.name = "pcie1",
- .class = &dra7xx_pcie_hwmod_class,
+ .class = &dra7xx_pciess_hwmod_class,
.clkdm_name = "pcie_clkdm",
.main_clk = "l4_root_clk_div",
- .prcm = {
- .omap4 = {
- .clkctrl_offs = DRA7XX_CM_PCIE_CLKSTCTRL_OFFSET,
- .modulemode = MODULEMODE_SWCTRL,
- },
- },
-};
-
-/* pcie2 */
-static struct omap_hwmod dra7xx_pcie2_hwmod = {
- .name = "pcie2",
- .class = &dra7xx_pcie_hwmod_class,
- .clkdm_name = "pcie_clkdm",
- .main_clk = "l4_root_clk_div",
- .prcm = {
- .omap4 = {
- .clkctrl_offs = DRA7XX_CM_PCIE_CLKSTCTRL_OFFSET,
- .modulemode = MODULEMODE_SWCTRL,
- },
- },
-};
-
-/*
- * 'PCIE PHY' class
- *
- */
-
-static struct omap_hwmod_class dra7xx_pcie_phy_hwmod_class = {
- .name = "pcie-phy",
-};
-
-/* pcie1 phy */
-static struct omap_hwmod dra7xx_pcie1_phy_hwmod = {
- .name = "pcie1-phy",
- .class = &dra7xx_pcie_phy_hwmod_class,
- .clkdm_name = "l3init_clkdm",
- .main_clk = "l4_root_clk_div",
.prcm = {
.omap4 = {
.clkctrl_offs = DRA7XX_CM_L3INIT_PCIESS1_CLKCTRL_OFFSET,
},
};
-/* pcie2 phy */
-static struct omap_hwmod dra7xx_pcie2_phy_hwmod = {
- .name = "pcie2-phy",
- .class = &dra7xx_pcie_phy_hwmod_class,
- .clkdm_name = "l3init_clkdm",
+/* pcie2 */
+static struct omap_hwmod dra7xx_pciess2_hwmod = {
+ .name = "pcie2",
+ .class = &dra7xx_pciess_hwmod_class,
+ .clkdm_name = "pcie_clkdm",
.main_clk = "l4_root_clk_div",
.prcm = {
.omap4 = {
.user = OCP_USER_MPU | OCP_USER_SDMA,
};
-/* l3_main_1 -> pcie1 */
-static struct omap_hwmod_ocp_if dra7xx_l3_main_1__pcie1 = {
+/* l3_main_1 -> pciess1 */
+static struct omap_hwmod_ocp_if dra7xx_l3_main_1__pciess1 = {
.master = &dra7xx_l3_main_1_hwmod,
- .slave = &dra7xx_pcie1_hwmod,
+ .slave = &dra7xx_pciess1_hwmod,
.clk = "l3_iclk_div",
.user = OCP_USER_MPU | OCP_USER_SDMA,
};
-/* l4_cfg -> pcie1 */
-static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pcie1 = {
+/* l4_cfg -> pciess1 */
+static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pciess1 = {
.master = &dra7xx_l4_cfg_hwmod,
- .slave = &dra7xx_pcie1_hwmod,
+ .slave = &dra7xx_pciess1_hwmod,
.clk = "l4_root_clk_div",
.user = OCP_USER_MPU | OCP_USER_SDMA,
};
-/* l3_main_1 -> pcie2 */
-static struct omap_hwmod_ocp_if dra7xx_l3_main_1__pcie2 = {
+/* l3_main_1 -> pciess2 */
+static struct omap_hwmod_ocp_if dra7xx_l3_main_1__pciess2 = {
.master = &dra7xx_l3_main_1_hwmod,
- .slave = &dra7xx_pcie2_hwmod,
+ .slave = &dra7xx_pciess2_hwmod,
.clk = "l3_iclk_div",
.user = OCP_USER_MPU | OCP_USER_SDMA,
};
-/* l4_cfg -> pcie2 */
-static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pcie2 = {
- .master = &dra7xx_l4_cfg_hwmod,
- .slave = &dra7xx_pcie2_hwmod,
- .clk = "l4_root_clk_div",
- .user = OCP_USER_MPU | OCP_USER_SDMA,
-};
-
-/* l4_cfg -> pcie1 phy */
-static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pcie1_phy = {
- .master = &dra7xx_l4_cfg_hwmod,
- .slave = &dra7xx_pcie1_phy_hwmod,
- .clk = "l4_root_clk_div",
- .user = OCP_USER_MPU | OCP_USER_SDMA,
-};
-
-/* l4_cfg -> pcie2 phy */
-static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pcie2_phy = {
+/* l4_cfg -> pciess2 */
+static struct omap_hwmod_ocp_if dra7xx_l4_cfg__pciess2 = {
.master = &dra7xx_l4_cfg_hwmod,
- .slave = &dra7xx_pcie2_phy_hwmod,
+ .slave = &dra7xx_pciess2_hwmod,
.clk = "l4_root_clk_div",
.user = OCP_USER_MPU | OCP_USER_SDMA,
};
&dra7xx_l4_cfg__mpu,
&dra7xx_l4_cfg__ocp2scp1,
&dra7xx_l4_cfg__ocp2scp3,
- &dra7xx_l3_main_1__pcie1,
- &dra7xx_l4_cfg__pcie1,
- &dra7xx_l3_main_1__pcie2,
- &dra7xx_l4_cfg__pcie2,
- &dra7xx_l4_cfg__pcie1_phy,
- &dra7xx_l4_cfg__pcie2_phy,
+ &dra7xx_l3_main_1__pciess1,
+ &dra7xx_l4_cfg__pciess1,
+ &dra7xx_l3_main_1__pciess2,
+ &dra7xx_l4_cfg__pciess2,
&dra7xx_l3_main_1__qspi,
&dra7xx_l4_per3__rtcss,
&dra7xx_l4_cfg__sata,
static void __init omap3_evm_legacy_init(void)
{
+ hsmmc2_internal_input_clk();
legacy_init_wl12xx(WL12XX_REFCLOCK_38, 0, 149);
}
{
saved_mask[0] =
omap4_prm_read_inst_reg(OMAP4430_PRM_OCP_SOCKET_INST,
- OMAP4_PRM_IRQSTATUS_MPU_OFFSET);
+ OMAP4_PRM_IRQENABLE_MPU_OFFSET);
saved_mask[1] =
omap4_prm_read_inst_reg(OMAP4430_PRM_OCP_SOCKET_INST,
- OMAP4_PRM_IRQSTATUS_MPU_2_OFFSET);
+ OMAP4_PRM_IRQENABLE_MPU_2_OFFSET);
omap4_prm_write_inst_reg(0, OMAP4430_PRM_OCP_SOCKET_INST,
OMAP4_PRM_IRQENABLE_MPU_OFFSET);
int __init orion5x_pci_sys_setup(int nr, struct pci_sys_data *sys)
{
- int ret = 0;
-
vga_base = ORION5X_PCIE_MEM_PHYS_BASE;
if (nr == 0) {
orion_pcie_set_local_bus_nr(PCIE_BASE, sys->busnr);
- ret = pcie_setup(sys);
- } else if (nr == 1 && !orion5x_pci_disabled) {
+ return pcie_setup(sys);
+ }
+
+ if (nr == 1 && !orion5x_pci_disabled) {
orion5x_pci_set_bus_nr(sys->busnr);
- ret = pci_setup(sys);
+ return pci_setup(sys);
}
- return ret;
+ return 0;
}
struct pci_bus __init *orion5x_pci_sys_scan_bus(int nr, struct pci_sys_data *sys)
{
- struct pci_bus *bus;
+ if (nr == 0)
+ return pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
+ &sys->resources);
- if (nr == 0) {
- bus = pci_scan_root_bus(NULL, sys->busnr, &pcie_ops, sys,
- &sys->resources);
- } else if (nr == 1 && !orion5x_pci_disabled) {
- bus = pci_scan_root_bus(NULL, sys->busnr, &pci_ops, sys,
- &sys->resources);
- } else {
- bus = NULL;
- BUG();
- }
+ if (nr == 1 && !orion5x_pci_disabled)
+ return pci_scan_root_bus(NULL, sys->busnr, &pci_ops, sys,
+ &sys->resources);
- return bus;
+ BUG();
+ return NULL;
}
int __init orion5x_pci_map_irq(const struct pci_dev *dev, u8 slot, u8 pin)
#include <linux/platform_data/video-pxafb.h>
#include <mach/bitfield.h>
#include <linux/platform_data/mmc-pxamci.h>
+#include <linux/smc91x.h>
#include "generic.h"
#include "devices.h"
}
};
+static struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_32BIT | SMC91X_USE_DMA | SMC91X_NOWAIT,
+};
+
static struct platform_device smc91x_device = {
.name = "smc91x",
.id = 0,
.num_resources = ARRAY_SIZE(smc91x_resources),
.resource = smc91x_resources,
+ .dev.platform_data = &smc91x_platdata,
};
static void idp_backlight_power(int on)
* it under the terms of the GNU General Public License version 2 as
* published by the Free Software Foundation.
*/
+#include <linux/bitops.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/interrupt.h>
#define ICHP_VAL_IRQ (1 << 31)
#define ICHP_IRQ(i) (((i) >> 16) & 0x7fff)
#define IPR_VALID (1 << 31)
-#define IRQ_BIT(n) (((n) - PXA_IRQ(0)) & 0x1f)
#define MAX_INTERNAL_IRQS 128
static void __iomem *pxa_irq_base;
static int pxa_internal_irq_nr;
static bool cpu_has_ipr;
+static struct irq_domain *pxa_irq_domain;
static inline void __iomem *irq_base(int i)
{
void pxa_mask_irq(struct irq_data *d)
{
void __iomem *base = irq_data_get_irq_chip_data(d);
+ irq_hw_number_t irq = irqd_to_hwirq(d);
uint32_t icmr = __raw_readl(base + ICMR);
- icmr &= ~(1 << IRQ_BIT(d->irq));
+ icmr &= ~BIT(irq & 0x1f);
__raw_writel(icmr, base + ICMR);
}
void pxa_unmask_irq(struct irq_data *d)
{
void __iomem *base = irq_data_get_irq_chip_data(d);
+ irq_hw_number_t irq = irqd_to_hwirq(d);
uint32_t icmr = __raw_readl(base + ICMR);
- icmr |= 1 << IRQ_BIT(d->irq);
+ icmr |= BIT(irq & 0x1f);
__raw_writel(icmr, base + ICMR);
}
} while (1);
}
-void __init pxa_init_irq(int irq_nr, int (*fn)(struct irq_data *, unsigned int))
+static int pxa_irq_map(struct irq_domain *h, unsigned int virq,
+ irq_hw_number_t hw)
{
- int irq, i, n;
+ void __iomem *base = irq_base(hw / 32);
- BUG_ON(irq_nr > MAX_INTERNAL_IRQS);
+ /* initialize interrupt priority */
+ if (cpu_has_ipr)
+ __raw_writel(hw | IPR_VALID, pxa_irq_base + IPR(hw));
+
+ irq_set_chip_and_handler(virq, &pxa_internal_irq_chip,
+ handle_level_irq);
+ irq_set_chip_data(virq, base);
+ set_irq_flags(virq, IRQF_VALID);
+
+ return 0;
+}
+
+static struct irq_domain_ops pxa_irq_ops = {
+ .map = pxa_irq_map,
+ .xlate = irq_domain_xlate_onecell,
+};
+
+static __init void
+pxa_init_irq_common(struct device_node *node, int irq_nr,
+ int (*fn)(struct irq_data *, unsigned int))
+{
+ int n;
pxa_internal_irq_nr = irq_nr;
- cpu_has_ipr = !cpu_is_pxa25x();
- pxa_irq_base = io_p2v(0x40d00000);
+ pxa_irq_domain = irq_domain_add_legacy(node, irq_nr,
+ PXA_IRQ(0), 0,
+ &pxa_irq_ops, NULL);
+ if (!pxa_irq_domain)
+ panic("Unable to add PXA IRQ domain\n");
+ irq_set_default_host(pxa_irq_domain);
for (n = 0; n < irq_nr; n += 32) {
void __iomem *base = irq_base(n >> 5);
__raw_writel(0, base + ICMR); /* disable all IRQs */
__raw_writel(0, base + ICLR); /* all IRQs are IRQ, not FIQ */
- for (i = n; (i < (n + 32)) && (i < irq_nr); i++) {
- /* initialize interrupt priority */
- if (cpu_has_ipr)
- __raw_writel(i | IPR_VALID, pxa_irq_base + IPR(i));
-
- irq = PXA_IRQ(i);
- irq_set_chip_and_handler(irq, &pxa_internal_irq_chip,
- handle_level_irq);
- irq_set_chip_data(irq, base);
- set_irq_flags(irq, IRQF_VALID);
- }
}
-
/* only unmasked interrupts kick us out of idle */
__raw_writel(1, irq_base(0) + ICCR);
pxa_internal_irq_chip.irq_set_wake = fn;
}
+void __init pxa_init_irq(int irq_nr, int (*fn)(struct irq_data *, unsigned int))
+{
+ BUG_ON(irq_nr > MAX_INTERNAL_IRQS);
+
+ pxa_irq_base = io_p2v(0x40d00000);
+ cpu_has_ipr = !cpu_is_pxa25x();
+ pxa_init_irq_common(NULL, irq_nr, fn);
+}
+
#ifdef CONFIG_PM
static unsigned long saved_icmr[MAX_INTERNAL_IRQS/32];
static unsigned long saved_ipr[MAX_INTERNAL_IRQS];
};
#ifdef CONFIG_OF
-static struct irq_domain *pxa_irq_domain;
-
-static int pxa_irq_map(struct irq_domain *h, unsigned int virq,
- irq_hw_number_t hw)
-{
- void __iomem *base = irq_base(hw / 32);
-
- /* initialize interrupt priority */
- if (cpu_has_ipr)
- __raw_writel(hw | IPR_VALID, pxa_irq_base + IPR(hw));
-
- irq_set_chip_and_handler(hw, &pxa_internal_irq_chip,
- handle_level_irq);
- irq_set_chip_data(hw, base);
- set_irq_flags(hw, IRQF_VALID);
-
- return 0;
-}
-
-static struct irq_domain_ops pxa_irq_ops = {
- .map = pxa_irq_map,
- .xlate = irq_domain_xlate_onecell,
-};
-
static const struct of_device_id intc_ids[] __initconst = {
{ .compatible = "marvell,pxa-intc", },
{}
{
struct device_node *node;
struct resource res;
- int n, ret;
+ int ret;
node = of_find_matching_node(NULL, intc_ids);
if (!node) {
return;
}
- pxa_irq_domain = irq_domain_add_legacy(node, pxa_internal_irq_nr, 0, 0,
- &pxa_irq_ops, NULL);
- if (!pxa_irq_domain)
- panic("Unable to add PXA IRQ domain\n");
-
- irq_set_default_host(pxa_irq_domain);
-
- for (n = 0; n < pxa_internal_irq_nr; n += 32) {
- void __iomem *base = irq_base(n >> 5);
-
- __raw_writel(0, base + ICMR); /* disable all IRQs */
- __raw_writel(0, base + ICLR); /* all IRQs are IRQ, not FIQ */
- }
-
- /* only unmasked interrupts kick us out of idle */
- __raw_writel(1, irq_base(0) + ICCR);
-
- pxa_internal_irq_chip.irq_set_wake = fn;
+ pxa_init_irq_common(node, pxa_internal_irq_nr, fn);
}
#endif /* CONFIG_OF */
#include <linux/mtd/mtd.h>
#include <linux/mtd/partitions.h>
#include <linux/pwm_backlight.h>
+#include <linux/smc91x.h>
#include <asm/types.h>
#include <asm/setup.h>
[1] = {
.start = LPD270_ETHERNET_IRQ,
.end = LPD270_ETHERNET_IRQ,
- .flags = IORESOURCE_IRQ,
+ .flags = IORESOURCE_IRQ | IORESOURCE_IRQ_HIGHEDGE,
},
};
+struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_16BIT | SMC91X_NOWAIT,
+};
+
static struct platform_device smc91x_device = {
.name = "smc91x",
.id = 0,
.num_resources = ARRAY_SIZE(smc91x_resources),
.resource = smc91x_resources,
+ .dev.platform_data = &smc91x_platdata,
};
static struct resource lpd270_flash_resources[] = {
struct power_supply *psy =
power_supply_get_by_name(raumfeld_power_supplicants[0]);
- if (psy)
+ if (psy) {
power_supply_set_battery_charged(psy);
+ power_supply_put(psy);
+ }
}
static int raumfeld_power_resume(void)
};
static struct platform_device can_regulator_device = {
- .name = "reg-fixed-volage",
+ .name = "reg-fixed-voltage",
.id = 0,
.dev = {
.platform_data = &can_regulator_pdata,
#include <linux/platform_data/video-clcd-versatile.h>
#include <linux/io.h>
#include <linux/smsc911x.h>
+#include <linux/smc91x.h>
#include <linux/ata_platform.h>
#include <linux/amba/mmci.h>
#include <linux/gfp.h>
.phy_interface = PHY_INTERFACE_MODE_MII,
};
+static struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_32BIT | SMC91X_NOWAIT,
+};
+
static struct platform_device realview_eth_device = {
.name = "smsc911x",
.id = 0,
realview_eth_device.resource = res;
if (strcmp(realview_eth_device.name, "smsc911x") == 0)
realview_eth_device.dev.platform_data = &smsc911x_config;
+ else
+ realview_eth_device.dev.platform_data = &smc91x_platdata;
return platform_device_register(&realview_eth_device);
}
[1] = {
.start = IRQ_EB_ETH,
.end = IRQ_EB_ETH,
- .flags = IORESOURCE_IRQ,
+ .flags = IORESOURCE_IRQ | IORESOURCE_IRQ_HIGHEDGE,
},
};
#include <linux/pm.h>
#include <linux/serial_core.h>
#include <linux/slab.h>
+#include <linux/smc91x.h>
#include <asm/mach-types.h>
#include <asm/mach/map.h>
0x02000000, "smc91x-attrib"),
{ .flags = IORESOURCE_IRQ },
};
+ struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_8BIT | SMC91X_IO_SHIFT_2 | SMC91X_NOWAIT,
+ };
struct platform_device_info smc91x_devinfo = {
.parent = &dev->dev,
.name = "smc91x",
.id = 0,
.res = smc91x_resources,
.num_res = ARRAY_SIZE(smc91x_resources),
+ .data = &smc91x_platdata,
+ .size_data = sizeof(smc91x_platdata),
};
int ret, irq;
#include <linux/irq.h>
#include <linux/io.h>
#include <linux/mtd/partitions.h>
+#include <linux/smc91x.h>
#include <mach/hardware.h>
#include <asm/setup.h>
#endif
};
+static struct smc91x_platdata smc91x_platdata = {
+ .flags = SMC91X_USE_16BIT | SMC91X_NOWAIT,
+};
static struct platform_device smc91x_device = {
.name = "smc91x",
.id = 0,
.num_resources = ARRAY_SIZE(smc91x_resources),
.resource = smc91x_resources,
+ .dev = {
+ .platform_data = &smc91x_platdata,
+ },
};
static struct platform_device *devices[] __initdata = {
return IRQ_HANDLED;
}
-static int sh73a0_set_wake(struct irq_data *data, unsigned int on)
-{
- return 0; /* always allow wakeup */
-}
-
#define PINTER0_PHYS 0xe69000a0
#define PINTER1_PHYS 0xe69000a4
#define PINTER0_VIRT IOMEM(0xe69000a0)
void __iomem *gic_cpu_base = IOMEM(0xf0000100);
void __iomem *intevtsa = ioremap_nocache(0xffd20100, PAGE_SIZE);
+ gic_set_irqchip_flags(IRQCHIP_SKIP_SET_WAKE);
gic_init(0, 29, gic_dist_base, gic_cpu_base);
- gic_arch_extn.irq_set_wake = sh73a0_set_wake;
register_intc_controller(&intcs_desc);
register_intc_controller(&intc_pint0_desc);
}
#ifdef CONFIG_USE_OF
-static int r8a7779_set_wake(struct irq_data *data, unsigned int on)
-{
- return 0; /* always allow wakeup */
-}
-
void __init r8a7779_init_irq_dt(void)
{
#ifdef CONFIG_ARCH_SHMOBILE_LEGACY
void __iomem *gic_dist_base = ioremap_nocache(0xf0001000, 0x1000);
void __iomem *gic_cpu_base = ioremap_nocache(0xf0000100, 0x1000);
#endif
- gic_arch_extn.irq_set_wake = r8a7779_set_wake;
+ gic_set_irqchip_flags(IRQCHIP_SKIP_SET_WAKE);
#ifdef CONFIG_ARCH_SHMOBILE_LEGACY
gic_init(0, 29, gic_dist_base, gic_cpu_base);
extern unsigned long socfpga_cpu1start_addr;
-#define SOCFPGA_SCU_VIRT_BASE 0xfffec000
+#define SOCFPGA_SCU_VIRT_BASE 0xfee00000
#endif
#include <asm/hardware/cache-l2x0.h>
#include <asm/mach/arch.h>
#include <asm/mach/map.h>
+#include <asm/cacheflush.h>
#include "core.h"
(u32 *) &socfpga_cpu1start_addr))
pr_err("SMP: Need cpu1-start-addr in device tree.\n");
+ /* Ensure that socfpga_cpu1start_addr is visible to other CPUs */
+ smp_wmb();
+ sync_cache_w(&socfpga_cpu1start_addr);
+
sys_manager_base_addr = of_iomap(np, 0);
np = of_find_compatible_node(NULL, NULL, "altr,rst-mgr");
"st,stih415",
"st,stih416",
"st,stih407",
+ "st,stih410",
"st,stih418",
NULL
};
menuconfig ARCH_SUNXI
bool "Allwinner SoCs" if ARCH_MULTI_V7
select ARCH_REQUIRE_GPIOLIB
+ select ARCH_HAS_RESET_CONTROLLER
select CLKSRC_MMIO
select GENERIC_IRQ_CHIP
select PINCTRL
select SUN4I_TIMER
+ select RESET_CONTROLLER
if ARCH_SUNXI
config MACH_SUN6I
bool "Allwinner A31 (sun6i) SoCs support"
default ARCH_SUNXI
- select ARCH_HAS_RESET_CONTROLLER
select ARM_GIC
select MFD_SUN6I_PRCM
- select RESET_CONTROLLER
select SUN5I_HSTIMER
config MACH_SUN7I
config MACH_SUN8I
bool "Allwinner A23 (sun8i) SoCs support"
default ARCH_SUNXI
- select ARCH_HAS_RESET_CONTROLLER
select ARM_GIC
select MFD_SUN6I_PRCM
- select RESET_CONTROLLER
config MACH_SUN9I
bool "Allwinner (sun9i) SoCs support"
default ARCH_SUNXI
- select ARCH_HAS_RESET_CONTROLLER
select ARM_GIC
- select RESET_CONTROLLER
endif
*/
#include <asm/firmware.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/cpuidle.h>
#include <linux/cpu_pm.h>
#include <linux/kernel.h>
tegra_set_cpu_in_lp2();
cpu_pm_enter();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+ tick_broadcast_enter();
call_firmware_op(prepare_idle);
if (call_firmware_op(do_idle, 0) == -ENOSYS)
cpu_suspend(0, tegra30_sleep_cpu_secondary_finish);
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
cpu_pm_exit();
tegra_clear_cpu_in_lp2();
*/
#include <linux/clk/tegra.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/cpuidle.h>
#include <linux/cpu_pm.h>
#include <linux/kernel.h>
if (tegra20_reset_cpu_1() || !tegra_cpu_rail_off_ready())
return false;
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+ tick_broadcast_enter();
tegra_idle_lp2_last();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
if (cpu_online(1))
tegra20_wake_cpu1_from_reset();
struct cpuidle_driver *drv,
int index)
{
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+ tick_broadcast_enter();
cpu_suspend(0, tegra20_sleep_cpu_secondary_finish);
tegra20_cpu_clear_resettable();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
return true;
}
*/
#include <linux/clk/tegra.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/cpuidle.h>
#include <linux/cpu_pm.h>
#include <linux/kernel.h>
return false;
}
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+ tick_broadcast_enter();
tegra_idle_lp2_last();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
return true;
}
struct cpuidle_driver *drv,
int index)
{
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu);
+ tick_broadcast_enter();
smp_wmb();
cpu_suspend(0, tegra30_sleep_cpu_secondary_finish);
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
return true;
}
#define TEGRA_ARM_INT_DIST_BASE 0x50041000
#define TEGRA_ARM_INT_DIST_SIZE SZ_4K
-#define TEGRA_PRIMARY_ICTLR_BASE 0x60004000
-#define TEGRA_PRIMARY_ICTLR_SIZE SZ_64
-
-#define TEGRA_SECONDARY_ICTLR_BASE 0x60004100
-#define TEGRA_SECONDARY_ICTLR_SIZE SZ_64
-
-#define TEGRA_TERTIARY_ICTLR_BASE 0x60004200
-#define TEGRA_TERTIARY_ICTLR_SIZE SZ_64
-
-#define TEGRA_QUATERNARY_ICTLR_BASE 0x60004300
-#define TEGRA_QUATERNARY_ICTLR_SIZE SZ_64
-
-#define TEGRA_QUINARY_ICTLR_BASE 0x60004400
-#define TEGRA_QUINARY_ICTLR_SIZE SZ_64
-
#define TEGRA_TMR1_BASE 0x60005000
#define TEGRA_TMR1_SIZE SZ_8
#include "board.h"
#include "iomap.h"
-#define ICTLR_CPU_IEP_VFIQ 0x08
-#define ICTLR_CPU_IEP_FIR 0x14
-#define ICTLR_CPU_IEP_FIR_SET 0x18
-#define ICTLR_CPU_IEP_FIR_CLR 0x1c
-
-#define ICTLR_CPU_IER 0x20
-#define ICTLR_CPU_IER_SET 0x24
-#define ICTLR_CPU_IER_CLR 0x28
-#define ICTLR_CPU_IEP_CLASS 0x2C
-
-#define ICTLR_COP_IER 0x30
-#define ICTLR_COP_IER_SET 0x34
-#define ICTLR_COP_IER_CLR 0x38
-#define ICTLR_COP_IEP_CLASS 0x3c
-
-#define FIRST_LEGACY_IRQ 32
-#define TEGRA_MAX_NUM_ICTLRS 5
-
#define SGI_MASK 0xFFFF
-static int num_ictlrs;
-
-static void __iomem *ictlr_reg_base[] = {
- IO_ADDRESS(TEGRA_PRIMARY_ICTLR_BASE),
- IO_ADDRESS(TEGRA_SECONDARY_ICTLR_BASE),
- IO_ADDRESS(TEGRA_TERTIARY_ICTLR_BASE),
- IO_ADDRESS(TEGRA_QUATERNARY_ICTLR_BASE),
- IO_ADDRESS(TEGRA_QUINARY_ICTLR_BASE),
-};
-
#ifdef CONFIG_PM_SLEEP
-static u32 cop_ier[TEGRA_MAX_NUM_ICTLRS];
-static u32 cop_iep[TEGRA_MAX_NUM_ICTLRS];
-static u32 cpu_ier[TEGRA_MAX_NUM_ICTLRS];
-static u32 cpu_iep[TEGRA_MAX_NUM_ICTLRS];
-
-static u32 ictlr_wake_mask[TEGRA_MAX_NUM_ICTLRS];
static void __iomem *tegra_gic_cpu_base;
#endif
return false;
}
-static inline void tegra_irq_write_mask(unsigned int irq, unsigned long reg)
-{
- void __iomem *base;
- u32 mask;
-
- BUG_ON(irq < FIRST_LEGACY_IRQ ||
- irq >= FIRST_LEGACY_IRQ + num_ictlrs * 32);
-
- base = ictlr_reg_base[(irq - FIRST_LEGACY_IRQ) / 32];
- mask = BIT((irq - FIRST_LEGACY_IRQ) % 32);
-
- __raw_writel(mask, base + reg);
-}
-
-static void tegra_mask(struct irq_data *d)
-{
- if (d->hwirq < FIRST_LEGACY_IRQ)
- return;
-
- tegra_irq_write_mask(d->hwirq, ICTLR_CPU_IER_CLR);
-}
-
-static void tegra_unmask(struct irq_data *d)
-{
- if (d->hwirq < FIRST_LEGACY_IRQ)
- return;
-
- tegra_irq_write_mask(d->hwirq, ICTLR_CPU_IER_SET);
-}
-
-static void tegra_ack(struct irq_data *d)
-{
- if (d->hwirq < FIRST_LEGACY_IRQ)
- return;
-
- tegra_irq_write_mask(d->hwirq, ICTLR_CPU_IEP_FIR_CLR);
-}
-
-static void tegra_eoi(struct irq_data *d)
-{
- if (d->hwirq < FIRST_LEGACY_IRQ)
- return;
-
- tegra_irq_write_mask(d->hwirq, ICTLR_CPU_IEP_FIR_CLR);
-}
-
-static int tegra_retrigger(struct irq_data *d)
-{
- if (d->hwirq < FIRST_LEGACY_IRQ)
- return 0;
-
- tegra_irq_write_mask(d->hwirq, ICTLR_CPU_IEP_FIR_SET);
-
- return 1;
-}
-
#ifdef CONFIG_PM_SLEEP
-static int tegra_set_wake(struct irq_data *d, unsigned int enable)
-{
- u32 irq = d->hwirq;
- u32 index, mask;
-
- if (irq < FIRST_LEGACY_IRQ ||
- irq >= FIRST_LEGACY_IRQ + num_ictlrs * 32)
- return -EINVAL;
-
- index = ((irq - FIRST_LEGACY_IRQ) / 32);
- mask = BIT((irq - FIRST_LEGACY_IRQ) % 32);
- if (enable)
- ictlr_wake_mask[index] |= mask;
- else
- ictlr_wake_mask[index] &= ~mask;
-
- return 0;
-}
-
-static int tegra_legacy_irq_suspend(void)
-{
- unsigned long flags;
- int i;
-
- local_irq_save(flags);
- for (i = 0; i < num_ictlrs; i++) {
- void __iomem *ictlr = ictlr_reg_base[i];
- /* Save interrupt state */
- cpu_ier[i] = readl_relaxed(ictlr + ICTLR_CPU_IER);
- cpu_iep[i] = readl_relaxed(ictlr + ICTLR_CPU_IEP_CLASS);
- cop_ier[i] = readl_relaxed(ictlr + ICTLR_COP_IER);
- cop_iep[i] = readl_relaxed(ictlr + ICTLR_COP_IEP_CLASS);
-
- /* Disable COP interrupts */
- writel_relaxed(~0ul, ictlr + ICTLR_COP_IER_CLR);
-
- /* Disable CPU interrupts */
- writel_relaxed(~0ul, ictlr + ICTLR_CPU_IER_CLR);
-
- /* Enable the wakeup sources of ictlr */
- writel_relaxed(ictlr_wake_mask[i], ictlr + ICTLR_CPU_IER_SET);
- }
- local_irq_restore(flags);
-
- return 0;
-}
-
-static void tegra_legacy_irq_resume(void)
-{
- unsigned long flags;
- int i;
-
- local_irq_save(flags);
- for (i = 0; i < num_ictlrs; i++) {
- void __iomem *ictlr = ictlr_reg_base[i];
- writel_relaxed(cpu_iep[i], ictlr + ICTLR_CPU_IEP_CLASS);
- writel_relaxed(~0ul, ictlr + ICTLR_CPU_IER_CLR);
- writel_relaxed(cpu_ier[i], ictlr + ICTLR_CPU_IER_SET);
- writel_relaxed(cop_iep[i], ictlr + ICTLR_COP_IEP_CLASS);
- writel_relaxed(~0ul, ictlr + ICTLR_COP_IER_CLR);
- writel_relaxed(cop_ier[i], ictlr + ICTLR_COP_IER_SET);
- }
- local_irq_restore(flags);
-}
-
-static struct syscore_ops tegra_legacy_irq_syscore_ops = {
- .suspend = tegra_legacy_irq_suspend,
- .resume = tegra_legacy_irq_resume,
-};
-
-int tegra_legacy_irq_syscore_init(void)
-{
- register_syscore_ops(&tegra_legacy_irq_syscore_ops);
-
- return 0;
-}
-
static int tegra_gic_notifier(struct notifier_block *self,
unsigned long cmd, void *v)
{
cpu_pm_register_notifier(&tegra_gic_notifier_block);
}
#else
-#define tegra_set_wake NULL
static void tegra114_gic_cpu_pm_registration(void) { }
#endif
+static const struct of_device_id tegra_ictlr_match[] __initconst = {
+ { .compatible = "nvidia,tegra20-ictlr" },
+ { .compatible = "nvidia,tegra30-ictlr" },
+ { }
+};
+
void __init tegra_init_irq(void)
{
- int i;
- void __iomem *distbase;
-
- distbase = IO_ADDRESS(TEGRA_ARM_INT_DIST_BASE);
- num_ictlrs = readl_relaxed(distbase + GIC_DIST_CTR) & 0x1f;
-
- if (num_ictlrs > ARRAY_SIZE(ictlr_reg_base)) {
- WARN(1, "Too many (%d) interrupt controllers found. Maximum is %d.",
- num_ictlrs, ARRAY_SIZE(ictlr_reg_base));
- num_ictlrs = ARRAY_SIZE(ictlr_reg_base);
- }
-
- for (i = 0; i < num_ictlrs; i++) {
- void __iomem *ictlr = ictlr_reg_base[i];
- writel(~0, ictlr + ICTLR_CPU_IER_CLR);
- writel(0, ictlr + ICTLR_CPU_IEP_CLASS);
- }
-
- gic_arch_extn.irq_ack = tegra_ack;
- gic_arch_extn.irq_eoi = tegra_eoi;
- gic_arch_extn.irq_mask = tegra_mask;
- gic_arch_extn.irq_unmask = tegra_unmask;
- gic_arch_extn.irq_retrigger = tegra_retrigger;
- gic_arch_extn.irq_set_wake = tegra_set_wake;
- gic_arch_extn.flags = IRQCHIP_MASK_ON_SUSPEND;
-
- /*
- * Check if there is a devicetree present, since the GIC will be
- * initialized elsewhere under DT.
- */
- if (!of_have_populated_dt())
- gic_init(0, 29, distbase,
- IO_ADDRESS(TEGRA_ARM_PERIF_BASE + 0x100));
+ if (WARN_ON(!of_find_matching_node(NULL, tegra_ictlr_match)))
+ pr_warn("Outdated DT detected, suspend/resume will NOT work\n");
tegra114_gic_cpu_pm_registration();
}
bool tegra_pending_sgi(void);
-#ifdef CONFIG_PM_SLEEP
-int tegra_legacy_irq_syscore_init(void);
-#else
-static inline int tegra_legacy_irq_syscore_init(void) { return 0; }
-#endif
-
#endif
{
tegra_init_irq();
irqchip_init();
- tegra_legacy_irq_syscore_init();
}
static void __init tegra_dt_init(void)
*/
void __init ux500_init_irq(void)
{
- gic_arch_extn.flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND;
+ gic_set_irqchip_flags(IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND);
irqchip_init();
/*
static void __init zynq_irq_init(void)
{
- gic_arch_extn.flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND;
+ gic_set_irqchip_flags(IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND);
irqchip_init();
}
}
ret = l2x0_cache_size_of_parse(np, aux_val, aux_mask, &assoc, SZ_512K);
- if (ret)
- return;
-
- switch (assoc) {
- case 16:
- *aux_val &= ~L2X0_AUX_CTRL_ASSOC_MASK;
- *aux_val |= L310_AUX_CTRL_ASSOCIATIVITY_16;
- *aux_mask &= ~L2X0_AUX_CTRL_ASSOC_MASK;
- break;
- case 8:
- *aux_val &= ~L2X0_AUX_CTRL_ASSOC_MASK;
- *aux_mask &= ~L2X0_AUX_CTRL_ASSOC_MASK;
- break;
- default:
- pr_err("L2C-310 OF cache associativity %d invalid, only 8 or 16 permitted\n",
- assoc);
- break;
+ if (!ret) {
+ switch (assoc) {
+ case 16:
+ *aux_val &= ~L2X0_AUX_CTRL_ASSOC_MASK;
+ *aux_val |= L310_AUX_CTRL_ASSOCIATIVITY_16;
+ *aux_mask &= ~L2X0_AUX_CTRL_ASSOC_MASK;
+ break;
+ case 8:
+ *aux_val &= ~L2X0_AUX_CTRL_ASSOC_MASK;
+ *aux_mask &= ~L2X0_AUX_CTRL_ASSOC_MASK;
+ break;
+ default:
+ pr_err("L2C-310 OF cache associativity %d invalid, only 8 or 16 permitted\n",
+ assoc);
+ break;
+ }
}
prefetch = l2x0_saved_regs.prefetch_ctrl;
*/
if (sizeof(mask) != sizeof(dma_addr_t) &&
mask > (dma_addr_t)~0 &&
- dma_to_pfn(dev, ~0) < max_pfn) {
+ dma_to_pfn(dev, ~0) < max_pfn - 1) {
if (warn) {
dev_warn(dev, "Coherent DMA mask %#llx is larger than dma_addr_t allows\n",
mask);
if (!iommu)
return false;
+ /*
+ * currently arm_iommu_create_mapping() takes a max of size_t
+ * for size param. So check this limit for now.
+ */
+ if (size > SIZE_MAX)
+ return false;
+
mapping = arm_iommu_create_mapping(dev->bus, dma_base, size);
if (IS_ERR(mapping)) {
pr_warn("Failed to create %llu-byte IOMMU mapping for device %s\n",
pr_alert("Unhandled fault: %s (0x%03x) at 0x%08lx\n",
inf->name, fsr, addr);
+ show_pte(current->mm, addr);
info.si_signo = inf->sig;
info.si_errno = 0;
WARN_ON_ONCE(1);
}
- if (!is_module_address(start) || !is_module_address(end - 1))
+ if (start < MODULES_VADDR || start >= MODULES_END)
+ return -EINVAL;
+
+ if (end < MODULES_VADDR || start >= MODULES_END)
return -EINVAL;
data.set_mask = set_mask;
}
/**
- * omap_read_persistent_clock - Return time from a persistent clock.
+ * omap_read_persistent_clock64 - Return time from a persistent clock.
*
* Reads the time from a source which isn't disabled during PM, the
* 32k sync timer. Convert the cycles elapsed since last read into
- * nsecs and adds to a monotonically increasing timespec.
+ * nsecs and adds to a monotonically increasing timespec64.
*/
-static struct timespec persistent_ts;
+static struct timespec64 persistent_ts;
static cycles_t cycles;
static unsigned int persistent_mult, persistent_shift;
-static DEFINE_SPINLOCK(read_persistent_clock_lock);
-static void omap_read_persistent_clock(struct timespec *ts)
+static void omap_read_persistent_clock64(struct timespec64 *ts)
{
unsigned long long nsecs;
cycles_t last_cycles;
- unsigned long flags;
-
- spin_lock_irqsave(&read_persistent_clock_lock, flags);
last_cycles = cycles;
cycles = sync32k_cnt_reg ? readl_relaxed(sync32k_cnt_reg) : 0;
nsecs = clocksource_cyc2ns(cycles - last_cycles,
persistent_mult, persistent_shift);
- timespec_add_ns(&persistent_ts, nsecs);
+ timespec64_add_ns(&persistent_ts, nsecs);
*ts = persistent_ts;
-
- spin_unlock_irqrestore(&read_persistent_clock_lock, flags);
}
/**
/*
* 120000 rough estimate from the calculations in
- * __clocksource_updatefreq_scale.
+ * __clocksource_update_freq_scale.
*/
clocks_calc_mult_shift(&persistent_mult, &persistent_shift,
32768, NSEC_PER_SEC, 120000);
}
sched_clock_register(omap_32k_read_sched_clock, 32, 32768);
- register_persistent_clock(NULL, omap_read_persistent_clock);
+ register_persistent_clock(NULL, omap_read_persistent_clock64);
pr_info("OMAP clocksource: 32k_counter at 32768 Hz\n");
return 0;
struct device *dev = &pdev->dev;
const struct of_device_id *match;
const struct dmtimer_platform_data *pdata;
+ int ret;
match = of_match_device(of_match_ptr(omap_timer_match), dev);
pdata = match ? match->data : dev->platform_data;
}
if (!timer->reserved) {
- pm_runtime_get_sync(dev);
+ ret = pm_runtime_get_sync(dev);
+ if (ret < 0) {
+ dev_err(dev, "%s: pm_runtime_get_sync failed!\n",
+ __func__);
+ goto err_get_sync;
+ }
__omap_dm_timer_init_regs(timer);
pm_runtime_put(dev);
}
dev_dbg(dev, "Device Probed.\n");
return 0;
+
+err_get_sync:
+ pm_runtime_put_noidle(dev);
+ pm_runtime_disable(dev);
+ return ret;
}
/**
}
spin_unlock_irqrestore(&dm_timer_lock, flags);
+ pm_runtime_disable(&pdev->dev);
+
return ret;
}
};
sgenet0: ethernet@1f210000 {
- compatible = "apm,xgene-enet";
+ compatible = "apm,xgene1-sgenet";
status = "disabled";
reg = <0x0 0x1f210000 0x0 0xd100>,
<0x0 0x1f200000 0x0 0Xc300>,
};
xgenet: ethernet@1f610000 {
- compatible = "apm,xgene-enet";
+ compatible = "apm,xgene1-xgenet";
status = "disabled";
reg = <0x0 0x1f610000 0x0 0xd100>,
<0x0 0x1f600000 0x0 0Xc300>,
reg = <0x0 0x0>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@1 {
device_type = "cpu";
reg = <0x0 0x1>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@2 {
device_type = "cpu";
reg = <0x0 0x2>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@3 {
device_type = "cpu";
reg = <0x0 0x3>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
+ };
+
+ L2_0: l2-cache0 {
+ compatible = "cache";
};
};
*/
/* SoC fixed clocks */
- soc_uartclk: refclk72738khz {
+ soc_uartclk: refclk7273800hz {
compatible = "fixed-clock";
#clock-cells = <0>;
clock-frequency = <7273800>;
reg = <0x0 0x0>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A57_L2>;
};
A57_1: cpu@1 {
reg = <0x0 0x1>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A57_L2>;
};
A53_0: cpu@100 {
reg = <0x0 0x100>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A53_L2>;
};
A53_1: cpu@101 {
reg = <0x0 0x101>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A53_L2>;
};
A53_2: cpu@102 {
reg = <0x0 0x102>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A53_L2>;
};
A53_3: cpu@103 {
reg = <0x0 0x103>;
device_type = "cpu";
enable-method = "psci";
+ next-level-cache = <&A53_L2>;
+ };
+
+ A57_L2: l2-cache0 {
+ compatible = "cache";
+ };
+
+ A53_L2: l2-cache1 {
+ compatible = "cache";
};
};
reg = <0x0 0x0>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@1 {
device_type = "cpu";
reg = <0x0 0x1>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@2 {
device_type = "cpu";
reg = <0x0 0x2>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
};
cpu@3 {
device_type = "cpu";
reg = <0x0 0x3>;
enable-method = "spin-table";
cpu-release-addr = <0x0 0x8000fff8>;
+ next-level-cache = <&L2_0>;
+ };
+
+ L2_0: l2-cache0 {
+ compatible = "cache";
};
};
obj-$(CONFIG_CRYPTO_AES_ARM64_NEON_BLK) += aes-neon-blk.o
aes-neon-blk-y := aes-glue-neon.o aes-neon.o
-AFLAGS_aes-ce.o := -DINTERLEAVE=2 -DINTERLEAVE_INLINE
+AFLAGS_aes-ce.o := -DINTERLEAVE=4
AFLAGS_aes-neon.o := -DINTERLEAVE=4
CFLAGS_aes-glue-ce.o := -DUSE_V8_CRYPTO_EXTENSIONS
#error "Only include this from assembly code"
#endif
+#ifndef __ASM_ASSEMBLER_H
+#define __ASM_ASSEMBLER_H
+
#include <asm/ptrace.h>
#include <asm/thread_info.h>
#endif
orr \rd, \lbits, \hbits, lsl #32
.endm
+
+#endif /* __ASM_ASSEMBLER_H */
__ret; \
})
-#define this_cpu_cmpxchg_1(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n)
-#define this_cpu_cmpxchg_2(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n)
-#define this_cpu_cmpxchg_4(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n)
-#define this_cpu_cmpxchg_8(ptr, o, n) cmpxchg_local(raw_cpu_ptr(&(ptr)), o, n)
-
-#define this_cpu_cmpxchg_double_8(ptr1, ptr2, o1, o2, n1, n2) \
- cmpxchg_double_local(raw_cpu_ptr(&(ptr1)), raw_cpu_ptr(&(ptr2)), \
- o1, o2, n1, n2)
+#define _protect_cmpxchg_local(pcp, o, n) \
+({ \
+ typeof(*raw_cpu_ptr(&(pcp))) __ret; \
+ preempt_disable(); \
+ __ret = cmpxchg_local(raw_cpu_ptr(&(pcp)), o, n); \
+ preempt_enable(); \
+ __ret; \
+})
+
+#define this_cpu_cmpxchg_1(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+#define this_cpu_cmpxchg_2(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+#define this_cpu_cmpxchg_4(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+#define this_cpu_cmpxchg_8(ptr, o, n) _protect_cmpxchg_local(ptr, o, n)
+
+#define this_cpu_cmpxchg_double_8(ptr1, ptr2, o1, o2, n1, n2) \
+({ \
+ int __ret; \
+ preempt_disable(); \
+ __ret = cmpxchg_double_local( raw_cpu_ptr(&(ptr1)), \
+ raw_cpu_ptr(&(ptr2)), \
+ o1, o2, n1, n2); \
+ preempt_enable(); \
+ __ret; \
+})
#define cmpxchg64(ptr,o,n) cmpxchg((ptr),(o),(n))
#define cmpxchg64_local(ptr,o,n) cmpxchg_local((ptr),(o),(n))
#ifndef __ASM_CPUIDLE_H
#define __ASM_CPUIDLE_H
+#include <asm/proc-fns.h>
+
#ifdef CONFIG_CPU_IDLE
extern int cpu_init_idle(unsigned int cpu);
extern int cpu_suspend(unsigned long arg);
#define ESR_ELx_FSC (0x3F)
#define ESR_ELx_FSC_TYPE (0x3C)
#define ESR_ELx_FSC_EXTABT (0x10)
+#define ESR_ELx_FSC_ACCESS (0x08)
#define ESR_ELx_FSC_FAULT (0x04)
#define ESR_ELx_FSC_PERM (0x0C)
#define ESR_ELx_CV (UL(1) << 24)
__AARCH64_INSN_FUNCS(bics, 0x7F200000, 0x6A200000)
__AARCH64_INSN_FUNCS(b, 0xFC000000, 0x14000000)
__AARCH64_INSN_FUNCS(bl, 0xFC000000, 0x94000000)
-__AARCH64_INSN_FUNCS(cbz, 0xFE000000, 0x34000000)
-__AARCH64_INSN_FUNCS(cbnz, 0xFE000000, 0x35000000)
+__AARCH64_INSN_FUNCS(cbz, 0x7F000000, 0x34000000)
+__AARCH64_INSN_FUNCS(cbnz, 0x7F000000, 0x35000000)
+__AARCH64_INSN_FUNCS(tbz, 0x7F000000, 0x36000000)
+__AARCH64_INSN_FUNCS(tbnz, 0x7F000000, 0x37000000)
__AARCH64_INSN_FUNCS(bcond, 0xFF000010, 0x54000000)
__AARCH64_INSN_FUNCS(svc, 0xFFE0001F, 0xD4000001)
__AARCH64_INSN_FUNCS(hvc, 0xFFE0001F, 0xD4000002)
*/
#ifndef __ASM_JUMP_LABEL_H
#define __ASM_JUMP_LABEL_H
+
+#ifndef __ASSEMBLY__
+
#include <linux/types.h>
#include <asm/insn.h>
-#ifdef __KERNEL__
-
#define JUMP_LABEL_NOP_SIZE AARCH64_INSN_SIZE
static __always_inline bool arch_static_branch(struct static_key *key)
return true;
}
-#endif /* __KERNEL__ */
-
typedef u64 jump_label_t;
struct jump_entry {
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif /* __ASM_JUMP_LABEL_H */
* 40 bits wide (T0SZ = 24). Systems with a PARange smaller than 40 bits are
* not known to exist and will break with this configuration.
*
+ * VTCR_EL2.PS is extracted from ID_AA64MMFR0_EL1.PARange at boot time
+ * (see hyp-init.S).
+ *
* Note that when using 4K pages, we concatenate two first level page tables
* together.
*
#ifdef CONFIG_ARM64_64K_PAGES
/*
* Stage2 translation configuration:
- * 40bits output (PS = 2)
* 40bits input (T0SZ = 24)
* 64kB pages (TG0 = 1)
* 2 level page tables (SL = 1)
#else
/*
* Stage2 translation configuration:
- * 40bits output (PS = 2)
* 40bits input (T0SZ = 24)
* 4kB pages (TG0 = 0)
* 3 level page tables (SL = 1)
/* For compatibility with fault code shared with 32-bit */
#define FSC_FAULT ESR_ELx_FSC_FAULT
+#define FSC_ACCESS ESR_ELx_FSC_ACCESS
#define FSC_PERM ESR_ELx_FSC_PERM
/* Hyp Prefetch Fault Address Register (HPFAR/HDFAR) */
#include <asm/kvm_asm.h>
#include <asm/kvm_mmio.h>
+#define __KVM_HAVE_ARCH_INTC_INITIALIZED
+
#if defined(CONFIG_KVM_ARM_MAX_VCPUS)
#define KVM_MAX_VCPUS CONFIG_KVM_ARM_MAX_VCPUS
#else
int kvm_unmap_hva_range(struct kvm *kvm,
unsigned long start, unsigned long end);
void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);
+int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end);
+int kvm_test_age_hva(struct kvm *kvm, unsigned long hva);
/* We do not have shadow page tables, hence the empty hooks */
-static inline int kvm_age_hva(struct kvm *kvm, unsigned long start,
- unsigned long end)
-{
- return 0;
-}
-
-static inline int kvm_test_age_hva(struct kvm *kvm, unsigned long hva)
-{
- return 0;
-}
-
static inline void kvm_arch_mmu_notifier_invalidate_page(struct kvm *kvm,
unsigned long address)
{
bool sign_extend;
};
-/*
- * The in-kernel MMIO emulation code wants to use a copy of run->mmio,
- * which is an anonymous type. Use our own type instead.
- */
-struct kvm_exit_mmio {
- phys_addr_t phys_addr;
- u8 data[8];
- u32 len;
- bool is_write;
- void *private;
-};
-
-static inline void kvm_prepare_mmio(struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
-{
- run->mmio.phys_addr = mmio->phys_addr;
- run->mmio.len = mmio->len;
- run->mmio.is_write = mmio->is_write;
- memcpy(run->mmio.data, mmio->data, mmio->len);
- run->exit_reason = KVM_EXIT_MMIO;
-}
-
int kvm_handle_mmio_return(struct kvm_vcpu *vcpu, struct kvm_run *run);
int io_mem_abort(struct kvm_vcpu *vcpu, struct kvm_run *run,
phys_addr_t fault_ipa);
#define PTRS_PER_S2_PGD (1 << PTRS_PER_S2_PGD_SHIFT)
#define S2_PGD_ORDER get_order(PTRS_PER_S2_PGD * sizeof(pgd_t))
+#define kvm_pgd_index(addr) (((addr) >> PGDIR_SHIFT) & (PTRS_PER_S2_PGD - 1))
+
/*
* If we are concatenating first level stage-2 page tables, we would have less
* than or equal to 16 pointers in the fake PGD, because that's what the
#define KVM_PREALLOC_LEVEL (0)
#endif
-/**
- * kvm_prealloc_hwpgd - allocate inital table for VTTBR
- * @kvm: The KVM struct pointer for the VM.
- * @pgd: The kernel pseudo pgd
- *
- * When the kernel uses more levels of page tables than the guest, we allocate
- * a fake PGD and pre-populate it to point to the next-level page table, which
- * will be the real initial page table pointed to by the VTTBR.
- *
- * When KVM_PREALLOC_LEVEL==2, we allocate a single page for the PMD and
- * the kernel will use folded pud. When KVM_PREALLOC_LEVEL==1, we
- * allocate 2 consecutive PUD pages.
- */
-static inline int kvm_prealloc_hwpgd(struct kvm *kvm, pgd_t *pgd)
-{
- unsigned int i;
- unsigned long hwpgd;
-
- if (KVM_PREALLOC_LEVEL == 0)
- return 0;
-
- hwpgd = __get_free_pages(GFP_KERNEL | __GFP_ZERO, PTRS_PER_S2_PGD_SHIFT);
- if (!hwpgd)
- return -ENOMEM;
-
- for (i = 0; i < PTRS_PER_S2_PGD; i++) {
- if (KVM_PREALLOC_LEVEL == 1)
- pgd_populate(NULL, pgd + i,
- (pud_t *)hwpgd + i * PTRS_PER_PUD);
- else if (KVM_PREALLOC_LEVEL == 2)
- pud_populate(NULL, pud_offset(pgd, 0) + i,
- (pmd_t *)hwpgd + i * PTRS_PER_PMD);
- }
-
- return 0;
-}
-
static inline void *kvm_get_hwpgd(struct kvm *kvm)
{
pgd_t *pgd = kvm->arch.pgd;
return pmd_offset(pud, 0);
}
-static inline void kvm_free_hwpgd(struct kvm *kvm)
+static inline unsigned int kvm_get_hwpgd_size(void)
{
- if (KVM_PREALLOC_LEVEL > 0) {
- unsigned long hwpgd = (unsigned long)kvm_get_hwpgd(kvm);
- free_pages(hwpgd, PTRS_PER_S2_PGD_SHIFT);
- }
+ if (KVM_PREALLOC_LEVEL > 0)
+ return PTRS_PER_S2_PGD * PAGE_SIZE;
+ return PTRS_PER_S2_PGD * sizeof(pgd_t);
}
static inline bool kvm_page_empty(void *ptr)
{
unsigned int cpu = smp_processor_id();
+ /*
+ * init_mm.pgd does not contain any user mappings and it is always
+ * active for kernel addresses in TTBR1. Just set the reserved TTBR0.
+ */
+ if (next == &init_mm) {
+ cpu_set_reserved_ttbr0();
+ return;
+ }
+
if (!cpumask_test_and_set_cpu(cpu, mm_cpumask(next)) || prev != next)
check_and_switch_context(next, tsk);
}
return ret;
}
+#define _percpu_read(pcp) \
+({ \
+ typeof(pcp) __retval; \
+ preempt_disable(); \
+ __retval = (typeof(pcp))__percpu_read(raw_cpu_ptr(&(pcp)), \
+ sizeof(pcp)); \
+ preempt_enable(); \
+ __retval; \
+})
+
+#define _percpu_write(pcp, val) \
+do { \
+ preempt_disable(); \
+ __percpu_write(raw_cpu_ptr(&(pcp)), (unsigned long)(val), \
+ sizeof(pcp)); \
+ preempt_enable(); \
+} while(0) \
+
+#define _pcp_protect(operation, pcp, val) \
+({ \
+ typeof(pcp) __retval; \
+ preempt_disable(); \
+ __retval = (typeof(pcp))operation(raw_cpu_ptr(&(pcp)), \
+ (val), sizeof(pcp)); \
+ preempt_enable(); \
+ __retval; \
+})
+
#define _percpu_add(pcp, val) \
- __percpu_add(raw_cpu_ptr(&(pcp)), val, sizeof(pcp))
+ _pcp_protect(__percpu_add, pcp, val)
-#define _percpu_add_return(pcp, val) (typeof(pcp)) (_percpu_add(pcp, val))
+#define _percpu_add_return(pcp, val) _percpu_add(pcp, val)
#define _percpu_and(pcp, val) \
- __percpu_and(raw_cpu_ptr(&(pcp)), val, sizeof(pcp))
+ _pcp_protect(__percpu_and, pcp, val)
#define _percpu_or(pcp, val) \
- __percpu_or(raw_cpu_ptr(&(pcp)), val, sizeof(pcp))
-
-#define _percpu_read(pcp) (typeof(pcp)) \
- (__percpu_read(raw_cpu_ptr(&(pcp)), sizeof(pcp)))
-
-#define _percpu_write(pcp, val) \
- __percpu_write(raw_cpu_ptr(&(pcp)), (unsigned long)(val), sizeof(pcp))
+ _pcp_protect(__percpu_or, pcp, val)
#define _percpu_xchg(pcp, val) (typeof(pcp)) \
- (__percpu_xchg(raw_cpu_ptr(&(pcp)), (unsigned long)(val), sizeof(pcp)))
+ _pcp_protect(__percpu_xchg, pcp, (unsigned long)(val))
#define this_cpu_add_1(pcp, val) _percpu_add(pcp, val)
#define this_cpu_add_2(pcp, val) _percpu_add(pcp, val)
static inline pte_t pte_modify(pte_t pte, pgprot_t newprot)
{
const pteval_t mask = PTE_USER | PTE_PXN | PTE_UXN | PTE_RDONLY |
- PTE_PROT_NONE | PTE_VALID | PTE_WRITE;
+ PTE_PROT_NONE | PTE_WRITE | PTE_TYPE_MASK;
pte_val(pte) = (pte_val(pte) & ~mask) | (pgprot_val(newprot) & mask);
return pte;
}
#include <asm/memory.h>
-#define cpu_switch_mm(pgd,mm) cpu_do_switch_mm(virt_to_phys(pgd),mm)
+#define cpu_switch_mm(pgd,mm) \
+do { \
+ BUG_ON(pgd == swapper_pg_dir); \
+ cpu_do_switch_mm(virt_to_phys(pgd),mm); \
+} while (0)
#define cpu_get_pgd() \
({ \
#define STACK_TOP STACK_TOP_MAX
#endif /* CONFIG_COMPAT */
-#define ARCH_LOW_ADDRESS_LIMIT PHYS_MASK
+extern phys_addr_t arm64_dma_phys_limit;
+#define ARCH_LOW_ADDRESS_LIMIT (arm64_dma_phys_limit - 1)
#endif /* __KERNEL__ */
struct debug_info {
static inline void __pte_free_tlb(struct mmu_gather *tlb, pgtable_t pte,
unsigned long addr)
{
+ __flush_tlb_pgtable(tlb->mm, addr);
pgtable_page_dtor(pte);
tlb_remove_entry(tlb, pte);
}
static inline void __pmd_free_tlb(struct mmu_gather *tlb, pmd_t *pmdp,
unsigned long addr)
{
+ __flush_tlb_pgtable(tlb->mm, addr);
tlb_remove_entry(tlb, virt_to_page(pmdp));
}
#endif
static inline void __pud_free_tlb(struct mmu_gather *tlb, pud_t *pudp,
unsigned long addr)
{
+ __flush_tlb_pgtable(tlb->mm, addr);
tlb_remove_entry(tlb, virt_to_page(pudp));
}
#endif
#include <linux/sched.h>
#include <asm/cputype.h>
-extern void __cpu_flush_user_tlb_range(unsigned long, unsigned long, struct vm_area_struct *);
-extern void __cpu_flush_kern_tlb_range(unsigned long, unsigned long);
-
-extern struct cpu_tlb_fns cpu_tlb;
-
/*
* TLB Management
* ==============
flush_tlb_all();
}
+/*
+ * Used to invalidate the TLB (walk caches) corresponding to intermediate page
+ * table levels (pgd/pud/pmd).
+ */
+static inline void __flush_tlb_pgtable(struct mm_struct *mm,
+ unsigned long uaddr)
+{
+ unsigned long addr = uaddr >> 12 | ((unsigned long)ASID(mm) << 48);
+
+ dsb(ishst);
+ asm("tlbi vae1is, %0" : : "r" (addr));
+ dsb(ish);
+}
/*
* On AArch64, the cache coherency is handled via the set_pte_at() function.
*/
/* Highest supported SPI, from VGIC_NR_IRQS */
#define KVM_ARM_IRQ_GIC_MAX 127
+/* One single KVM irqchip, ie. the VGIC */
+#define KVM_NR_IRQCHIPS 1
+
/* PSCI interface */
#define KVM_PSCI_FN_BASE 0x95c1ba5e
#define KVM_PSCI_FN(n) (KVM_PSCI_FN_BASE + (n))
arm64-obj-y := cputable.o debug-monitors.o entry.o irq.o fpsimd.o \
entry-fpsimd.o process.o ptrace.o setup.o signal.o \
sys.o stacktrace.o time.o traps.o io.o vdso.o \
- hyp-stub.o psci.o cpu_ops.o insn.o return_address.o \
- cpuinfo.o cpu_errata.o alternative.o cacheinfo.o
+ hyp-stub.o psci.o psci-call.o cpu_ops.o insn.o \
+ return_address.o cpuinfo.o cpu_errata.o \
+ alternative.o cacheinfo.o
arm64-obj-$(CONFIG_COMPAT) += sys32.o kuser32.o signal32.o \
sys_compat.o entry32.o \
static void efi_set_pgd(struct mm_struct *mm)
{
- cpu_switch_mm(mm->pgd, mm);
+ if (mm == &init_mm)
+ cpu_set_reserved_ttbr0();
+ else
+ cpu_switch_mm(mm->pgd, mm);
+
flush_tlb_all();
if (icache_is_aivivt())
__flush_icache_all();
efi_set_pgd(current->active_mm);
preempt_enable();
}
+
+/*
+ * UpdateCapsule() depends on the system being shutdown via
+ * ResetSystem().
+ */
+bool efi_poweroff_required(void)
+{
+ return efi_enabled(EFI_RUNTIME_SERVICES);
+}
branch = aarch64_insn_gen_branch_imm(pc,
(unsigned long)ftrace_graph_caller,
- AARCH64_INSN_BRANCH_LINK);
+ AARCH64_INSN_BRANCH_NOLINK);
nop = aarch64_insn_gen_nop();
if (enable)
* zeroing of .bss would clobber it.
*/
.pushsection .data..cacheline_aligned
-ENTRY(__boot_cpu_mode)
.align L1_CACHE_SHIFT
+ENTRY(__boot_cpu_mode)
.long BOOT_CPU_MODE_EL2
.long 0
.popsection
if (module && IS_ENABLED(CONFIG_DEBUG_SET_MODULE_RONX))
page = vmalloc_to_page(addr);
- else
+ else if (!module && IS_ENABLED(CONFIG_DEBUG_RODATA))
page = virt_to_page(addr);
+ else
+ return addr;
BUG_ON(!page);
set_fixmap(fixmap, page_to_phys(page));
#include <stdarg.h>
#include <linux/compat.h>
+#include <linux/efi.h>
#include <linux/export.h>
#include <linux/sched.h>
#include <linux/kernel.h>
local_irq_disable();
smp_send_stop();
+ /*
+ * UpdateCapsule() depends on the system being reset via
+ * ResetSystem().
+ */
+ if (efi_enabled(EFI_RUNTIME_SERVICES))
+ efi_reboot(reboot_mode, NULL);
+
/* Now call the architecture specific reboot code. */
if (arm_pm_restart)
arm_pm_restart(reboot_mode, cmd);
--- /dev/null
+/*
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * Copyright (C) 2015 ARM Limited
+ *
+ * Author: Will Deacon <will.deacon@arm.com>
+ */
+
+#include <linux/linkage.h>
+
+/* int __invoke_psci_fn_hvc(u64 function_id, u64 arg0, u64 arg1, u64 arg2) */
+ENTRY(__invoke_psci_fn_hvc)
+ hvc #0
+ ret
+ENDPROC(__invoke_psci_fn_hvc)
+
+/* int __invoke_psci_fn_smc(u64 function_id, u64 arg0, u64 arg1, u64 arg2) */
+ENTRY(__invoke_psci_fn_smc)
+ smc #0
+ ret
+ENDPROC(__invoke_psci_fn_smc)
static int (*invoke_psci_fn)(u64, u64, u64, u64);
typedef int (*psci_initcall_t)(const struct device_node *);
+asmlinkage int __invoke_psci_fn_hvc(u64, u64, u64, u64);
+asmlinkage int __invoke_psci_fn_smc(u64, u64, u64, u64);
+
enum psci_function {
PSCI_FN_CPU_SUSPEND,
PSCI_FN_CPU_ON,
PSCI_0_2_POWER_STATE_AFFL_SHIFT;
}
-/*
- * The following two functions are invoked via the invoke_psci_fn pointer
- * and will not be inlined, allowing us to piggyback on the AAPCS.
- */
-static noinline int __invoke_psci_fn_hvc(u64 function_id, u64 arg0, u64 arg1,
- u64 arg2)
-{
- asm volatile(
- __asmeq("%0", "x0")
- __asmeq("%1", "x1")
- __asmeq("%2", "x2")
- __asmeq("%3", "x3")
- "hvc #0\n"
- : "+r" (function_id)
- : "r" (arg0), "r" (arg1), "r" (arg2));
-
- return function_id;
-}
-
-static noinline int __invoke_psci_fn_smc(u64 function_id, u64 arg0, u64 arg1,
- u64 arg2)
-{
- asm volatile(
- __asmeq("%0", "x0")
- __asmeq("%1", "x1")
- __asmeq("%2", "x2")
- __asmeq("%3", "x3")
- "smc #0\n"
- : "+r" (function_id)
- : "r" (arg0), "r" (arg1), "r" (arg2));
-
- return function_id;
-}
-
static int psci_get_version(void)
{
int err;
case __SI_TIMER:
err |= __put_user(from->si_tid, &to->si_tid);
err |= __put_user(from->si_overrun, &to->si_overrun);
- err |= __put_user((compat_uptr_t)(unsigned long)from->si_ptr,
- &to->si_ptr);
+ err |= __put_user(from->si_int, &to->si_int);
break;
case __SI_POLL:
err |= __put_user(from->si_band, &to->si_band);
case __SI_MESGQ: /* But this is */
err |= __put_user(from->si_pid, &to->si_pid);
err |= __put_user(from->si_uid, &to->si_uid);
- err |= __put_user((compat_uptr_t)(unsigned long)from->si_ptr, &to->si_ptr);
+ err |= __put_user(from->si_int, &to->si_int);
break;
case __SI_SYS:
err |= __put_user((compat_uptr_t)(unsigned long)
void update_vsyscall(struct timekeeper *tk)
{
struct timespec xtime_coarse;
- u32 use_syscall = strcmp(tk->tkr.clock->name, "arch_sys_counter");
+ u32 use_syscall = strcmp(tk->tkr_mono.clock->name, "arch_sys_counter");
++vdso_data->tb_seq_count;
smp_wmb();
vdso_data->wtm_clock_nsec = tk->wall_to_monotonic.tv_nsec;
if (!use_syscall) {
- vdso_data->cs_cycle_last = tk->tkr.cycle_last;
+ vdso_data->cs_cycle_last = tk->tkr_mono.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr.xtime_nsec;
- vdso_data->cs_mult = tk->tkr.mult;
- vdso_data->cs_shift = tk->tkr.shift;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
+ vdso_data->cs_mult = tk->tkr_mono.mult;
+ vdso_data->cs_shift = tk->tkr_mono.shift;
}
smp_wmb();
/* int __kernel_clock_getres(clockid_t clock_id, struct timespec *res); */
ENTRY(__kernel_clock_getres)
.cfi_startproc
- cbz w1, 3f
-
cmp w0, #CLOCK_REALTIME
ccmp w0, #CLOCK_MONOTONIC, #0x4, ne
b.ne 1f
b.ne 4f
ldr x2, 6f
2:
+ cbz w1, 3f
stp xzr, x2, [x1]
3: /* res == NULL. */
config KVM
bool "Kernel-based Virtual Machine (KVM) support"
+ depends on OF
select MMU_NOTIFIER
select PREEMPT_NOTIFIERS
select ANON_INODES
select HAVE_KVM_ARCH_TLB_FLUSH_ALL
select KVM_MMIO
select KVM_ARM_HOST
- select KVM_ARM_VGIC
- select KVM_ARM_TIMER
select KVM_GENERIC_DIRTYLOG_READ_PROTECT
select SRCU
+ select HAVE_KVM_EVENTFD
+ select HAVE_KVM_IRQFD
---help---
Support hosting virtualized guest machines.
large, so only choose a reasonable number that you expect to
actually use.
-config KVM_ARM_VGIC
- bool
- depends on KVM_ARM_HOST && OF
- select HAVE_KVM_IRQCHIP
- ---help---
- Adds support for a hardware assisted, in-kernel GIC emulation.
-
-config KVM_ARM_TIMER
- bool
- depends on KVM_ARM_VGIC
- ---help---
- Adds support for the Architected Timers in virtual machines.
-
endif # VIRTUALIZATION
# Makefile for Kernel-based Virtual Machine module
#
-ccflags-y += -Ivirt/kvm -Iarch/arm64/kvm
+ccflags-y += -Iarch/arm64/kvm
CFLAGS_arm.o := -I.
CFLAGS_mmu.o := -I.
obj-$(CONFIG_KVM_ARM_HOST) += kvm.o
-kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/kvm_main.o $(KVM)/coalesced_mmio.o $(KVM)/eventfd.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/arm.o $(ARM)/mmu.o $(ARM)/mmio.o
kvm-$(CONFIG_KVM_ARM_HOST) += $(ARM)/psci.o $(ARM)/perf.o
kvm-$(CONFIG_KVM_ARM_HOST) += hyp.o hyp-init.o handle_exit.o
kvm-$(CONFIG_KVM_ARM_HOST) += guest.o reset.o sys_regs.o sys_regs_generic_v8.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v2-emul.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += vgic-v2-switch.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v3.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += $(KVM)/arm/vgic-v3-emul.o
-kvm-$(CONFIG_KVM_ARM_VGIC) += vgic-v3-switch.o
-kvm-$(CONFIG_KVM_ARM_TIMER) += $(KVM)/arm/arch_timer.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v2-emul.o
+kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v2-switch.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/vgic-v3-emul.o
+kvm-$(CONFIG_KVM_ARM_HOST) += vgic-v3-switch.o
+kvm-$(CONFIG_KVM_ARM_HOST) += $(KVM)/arm/arch_timer.o
}
early_param("coherent_pool", early_coherent_pool);
-static void *__alloc_from_pool(size_t size, struct page **ret_page)
+static void *__alloc_from_pool(size_t size, struct page **ret_page, gfp_t flags)
{
unsigned long val;
void *ptr = NULL;
*ret_page = phys_to_page(phys);
ptr = (void *)val;
+ if (flags & __GFP_ZERO)
+ memset(ptr, 0, size);
}
return ptr;
flags |= GFP_DMA;
if (IS_ENABLED(CONFIG_DMA_CMA) && (flags & __GFP_WAIT)) {
struct page *page;
+ void *addr;
size = PAGE_ALIGN(size);
page = dma_alloc_from_contiguous(dev, size >> PAGE_SHIFT,
return NULL;
*dma_handle = phys_to_dma(dev, page_to_phys(page));
- return page_address(page);
+ addr = page_address(page);
+ if (flags & __GFP_ZERO)
+ memset(addr, 0, size);
+ return addr;
} else {
return swiotlb_alloc_coherent(dev, size, dma_handle, flags);
}
if (!coherent && !(flags & __GFP_WAIT)) {
struct page *page = NULL;
- void *addr = __alloc_from_pool(size, &page);
+ void *addr = __alloc_from_pool(size, &page, flags);
if (addr)
*dma_handle = phys_to_dma(dev, page_to_phys(page));
.mapping_error = swiotlb_dma_mapping_error,
};
-extern int swiotlb_late_init_with_default_size(size_t default_size);
-
static int __init atomic_pool_init(void)
{
pgprot_t prot = __pgprot(PROT_NORMAL_NC);
return -ENOMEM;
}
-static int __init swiotlb_late_init(void)
+static int __init arm64_dma_init(void)
{
- size_t swiotlb_size = min(SZ_64M, MAX_ORDER_NR_PAGES << PAGE_SHIFT);
+ int ret;
dma_ops = &swiotlb_dma_ops;
- return swiotlb_late_init_with_default_size(swiotlb_size);
-}
-
-static int __init arm64_dma_init(void)
-{
- int ret = 0;
-
- ret |= swiotlb_late_init();
- ret |= atomic_pool_init();
+ ret = atomic_pool_init();
return ret;
}
#include <linux/dma-mapping.h>
#include <linux/dma-contiguous.h>
#include <linux/efi.h>
+#include <linux/swiotlb.h>
#include <asm/fixmap.h>
#include <asm/memory.h>
#include "mm.h"
phys_addr_t memstart_addr __read_mostly = 0;
+phys_addr_t arm64_dma_phys_limit __read_mostly;
#ifdef CONFIG_BLK_DEV_INITRD
static int __init early_initrd(char *p)
/* 4GB maximum for 32-bit only capable devices */
if (IS_ENABLED(CONFIG_ZONE_DMA)) {
- max_dma = PFN_DOWN(max_zone_dma_phys());
+ max_dma = PFN_DOWN(arm64_dma_phys_limit);
zone_size[ZONE_DMA] = max_dma - min;
}
zone_size[ZONE_NORMAL] = max - max_dma;
void __init arm64_memblock_init(void)
{
- phys_addr_t dma_phys_limit = 0;
-
memblock_enforce_memory_limit(memory_limit);
/*
/* 4GB maximum for 32-bit only capable devices */
if (IS_ENABLED(CONFIG_ZONE_DMA))
- dma_phys_limit = max_zone_dma_phys();
- dma_contiguous_reserve(dma_phys_limit);
+ arm64_dma_phys_limit = max_zone_dma_phys();
+ else
+ arm64_dma_phys_limit = PHYS_MASK + 1;
+ dma_contiguous_reserve(arm64_dma_phys_limit);
memblock_allow_resize();
memblock_dump_all();
*/
void __init mem_init(void)
{
+ swiotlb_init(1);
+
set_max_mapnr(pfn_to_page(max_pfn) - mem_map);
#ifndef CONFIG_SPARSEMEM_VMEMMAP
WARN_ON_ONCE(1);
}
- if (!is_module_address(start) || !is_module_address(end - 1))
+ if (start < MODULES_VADDR || start >= MODULES_END)
+ return -EINVAL;
+
+ if (end < MODULES_VADDR || end >= MODULES_END)
return -EINVAL;
data.set_mask = set_mask;
the loader. We need to make sure that it is out of the way of the program
that it will "exec", and that there is sufficient room for the brk. */
-#define ELF_ET_DYN_BASE (2 * TASK_SIZE / 3)
+#define ELF_ET_DYN_BASE (TASK_SIZE / 3 * 2)
/* This yields a mask that user programs can use to figure out what
*/
#define pgtable_cache_init() do { } while (0)
+/*
+ * c6x is !MMU, so define the simpliest implementation
+ */
+#define pgprot_writecombine pgprot_noncached
+
#include <asm-generic/pgtable.h>
#endif /* _ASM_C6X_PGTABLE_H */
#define PGDIR_MASK (~(PGDIR_SIZE - 1))
#define PTRS_PER_PGD 64
+#define __PAGETABLE_PUD_FOLDED
#define PUD_SHIFT 26
#define PTRS_PER_PUD 1
#define PUD_SIZE (1UL << PUD_SHIFT)
#define PUD_MASK (~(PUD_SIZE - 1))
#define PUE_SIZE 256
+#define __PAGETABLE_PMD_FOLDED
#define PMD_SHIFT 26
#define PMD_SIZE (1UL << PMD_SHIFT)
#define PMD_MASK (~(PMD_SIZE - 1))
int __init pcibios_init(void)
{
+ struct pci_bus *bus;
struct pci_ops *dir = NULL;
LIST_HEAD(resources);
printk("PCI: Probing PCI hardware\n");
pci_add_resource(&resources, &pci_ioport_resource);
pci_add_resource(&resources, &pci_iomem_resource);
- pci_scan_root_bus(NULL, 0, pci_root_ops, NULL, &resources);
+ bus = pci_scan_root_bus(NULL, 0, pci_root_ops, NULL, &resources);
pcibios_irq_init();
pcibios_fixup_irqs();
pcibios_resource_survey();
+ if (!bus)
+ return 0;
+ pci_bus_add_devices(bus);
return 0;
}
if (bus == NULL) {
kfree(res);
kfree(controller);
+ return;
}
+ pci_bus_add_devices(bus);
}
/*
* the M32R is two-level, so we don't really have any
* PMD directory physically.
*/
+#define __PAGETABLE_PMD_FOLDED
#define PMD_SHIFT 22
#define PTRS_PER_PMD 1
schedule_timeout(msecs_to_jiffies(200));
rootbus = pci_scan_bus(0, &mcf_pci_ops, NULL);
+ if (!rootbus)
+ return -ENODEV;
+
rootbus->resource[0] = &mcf_pci_io;
rootbus->resource[1] = &mcf_pci_mem;
pci_fixup_irqs(pci_common_swizzle, mcf_pci_map_irq);
pci_bus_size_bridges(rootbus);
pci_bus_assign_resources(rootbus);
+ pci_bus_add_devices(rootbus);
return 0;
}
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
# CONFIG_NET_VENDOR_INTEL is not set
# CONFIG_NET_VENDOR_MARVELL is not set
# CONFIG_NET_VENDOR_MICREL is not set
-CONFIG_NE2000=m
+CONFIG_NE2000=y
# CONFIG_NET_VENDOR_QUALCOMM is not set
# CONFIG_NET_VENDOR_ROCKER is not set
# CONFIG_NET_VENDOR_SAMSUNG is not set
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
CONFIG_DLM=m
CONFIG_MAGIC_SYSRQ=y
CONFIG_ASYNC_RAID6_TEST=m
+CONFIG_TEST_HEXDUMP=m
CONFIG_TEST_STRING_HELPERS=m
CONFIG_TEST_KSTRTOX=m
+CONFIG_TEST_RHASHTABLE=m
CONFIG_TEST_LKM=m
CONFIG_TEST_USER_COPY=m
CONFIG_TEST_BPF=m
CONFIG_CRYPTO_DRBG_CTR=y
CONFIG_CRYPTO_USER_API_HASH=m
CONFIG_CRYPTO_USER_API_SKCIPHER=m
+CONFIG_CRYPTO_USER_API_RNG=m
# CONFIG_CRYPTO_HW is not set
CONFIG_XZ_DEC_TEST=m
* but WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- *
*/
#ifndef mcfqspi_h
*/
#ifdef CONFIG_SUN3
#define PTRS_PER_PTE 16
+#define __PAGETABLE_PMD_FOLDED
#define PTRS_PER_PMD 1
#define PTRS_PER_PGD 2048
#elif defined(CONFIG_COLDFIRE)
#define PTRS_PER_PTE 512
+#define __PAGETABLE_PMD_FOLDED
#define PTRS_PER_PMD 1
#define PTRS_PER_PGD 1024
#else
r = dev->resource + idx;
if (!r->start && r->end) {
- pr_err(KERN_ERR "PCI: Device %s not available because of resource collisions\n",
+ pr_err("PCI: Device %s not available because of resource collisions\n",
pci_name(dev));
return -EINVAL;
}
GNU CC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with GNU CC; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+GNU General Public License for more details. */
#define BITS_PER_UNIT 8
GNU CC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with GNU CC; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+GNU General Public License for more details. */
#define BITS_PER_UNIT 8
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+General Public License for more details. */
/* As a special exception, if you link this library with files
compiled with GCC to produce an executable, this does not cause
GNU CC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with GNU CC; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+GNU General Public License for more details. */
#define BITS_PER_UNIT 8
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+General Public License for more details. */
/* As a special exception, if you link this library with files
compiled with GCC to produce an executable, this does not cause
GNU CC is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
-GNU General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with GNU CC; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+GNU General Public License for more details. */
#ifdef CONFIG_CPU_HAS_NO_MULDIV64
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+General Public License for more details. */
/* As a special exception, if you link this library with files
compiled with GCC to produce an executable, this does not cause
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+General Public License for more details. */
/* As a special exception, if you link this library with files
compiled with GCC to produce an executable, this does not cause
This file is distributed in the hope that it will be useful, but
WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
-General Public License for more details.
-
-You should have received a copy of the GNU General Public License
-along with this program; see the file COPYING. If not, write to
-the Free Software Foundation, 59 Temple Place - Suite 330,
-Boston, MA 02111-1307, USA. */
+General Public License for more details. */
/* As a special exception, if you link this library with files
compiled with GCC to produce an executable, this does not cause
/* Disable all interrupts. Unlike a VIA it looks like we */
/* do this by setting the source's interrupt level to zero. */
- for (i = 0; i <= OSS_NUM_SOURCES; i++) {
+ for (i = 0; i < OSS_NUM_SOURCES; i++)
oss->irq_level[i] = 0;
- }
}
/*
#define _ASM_METAG_IO_H
#include <linux/types.h>
+#include <asm/pgtable-bits.h>
#define IO_SPACE_LIMIT 0
--- /dev/null
+/*
+ * Meta page table definitions.
+ */
+
+#ifndef _METAG_PGTABLE_BITS_H
+#define _METAG_PGTABLE_BITS_H
+
+#include <asm/metag_mem.h>
+
+/*
+ * Definitions for MMU descriptors
+ *
+ * These are the hardware bits in the MMCU pte entries.
+ * Derived from the Meta toolkit headers.
+ */
+#define _PAGE_PRESENT MMCU_ENTRY_VAL_BIT
+#define _PAGE_WRITE MMCU_ENTRY_WR_BIT
+#define _PAGE_PRIV MMCU_ENTRY_PRIV_BIT
+/* Write combine bit - this can cause writes to occur out of order */
+#define _PAGE_WR_COMBINE MMCU_ENTRY_WRC_BIT
+/* Sys coherent bit - this bit is never used by Linux */
+#define _PAGE_SYS_COHERENT MMCU_ENTRY_SYS_BIT
+#define _PAGE_ALWAYS_ZERO_1 0x020
+#define _PAGE_CACHE_CTRL0 0x040
+#define _PAGE_CACHE_CTRL1 0x080
+#define _PAGE_ALWAYS_ZERO_2 0x100
+#define _PAGE_ALWAYS_ZERO_3 0x200
+#define _PAGE_ALWAYS_ZERO_4 0x400
+#define _PAGE_ALWAYS_ZERO_5 0x800
+
+/* These are software bits that we stuff into the gaps in the hardware
+ * pte entries that are not used. Note, these DO get stored in the actual
+ * hardware, but the hardware just does not use them.
+ */
+#define _PAGE_ACCESSED _PAGE_ALWAYS_ZERO_1
+#define _PAGE_DIRTY _PAGE_ALWAYS_ZERO_2
+
+/* Pages owned, and protected by, the kernel. */
+#define _PAGE_KERNEL _PAGE_PRIV
+
+/* No cacheing of this page */
+#define _PAGE_CACHE_WIN0 (MMCU_CWIN_UNCACHED << MMCU_ENTRY_CWIN_S)
+/* burst cacheing - good for data streaming */
+#define _PAGE_CACHE_WIN1 (MMCU_CWIN_BURST << MMCU_ENTRY_CWIN_S)
+/* One cache way per thread */
+#define _PAGE_CACHE_WIN2 (MMCU_CWIN_C1SET << MMCU_ENTRY_CWIN_S)
+/* Full on cacheing */
+#define _PAGE_CACHE_WIN3 (MMCU_CWIN_CACHED << MMCU_ENTRY_CWIN_S)
+
+#define _PAGE_CACHEABLE (_PAGE_CACHE_WIN3 | _PAGE_WR_COMBINE)
+
+/* which bits are used for cache control ... */
+#define _PAGE_CACHE_MASK (_PAGE_CACHE_CTRL0 | _PAGE_CACHE_CTRL1 | \
+ _PAGE_WR_COMBINE)
+
+/* This is a mask of the bits that pte_modify is allowed to change. */
+#define _PAGE_CHG_MASK (PAGE_MASK)
+
+#define _PAGE_SZ_SHIFT 1
+#define _PAGE_SZ_4K (0x0)
+#define _PAGE_SZ_8K (0x1 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_16K (0x2 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_32K (0x3 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_64K (0x4 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_128K (0x5 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_256K (0x6 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_512K (0x7 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_1M (0x8 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_2M (0x9 << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_4M (0xa << _PAGE_SZ_SHIFT)
+#define _PAGE_SZ_MASK (0xf << _PAGE_SZ_SHIFT)
+
+#if defined(CONFIG_PAGE_SIZE_4K)
+#define _PAGE_SZ (_PAGE_SZ_4K)
+#elif defined(CONFIG_PAGE_SIZE_8K)
+#define _PAGE_SZ (_PAGE_SZ_8K)
+#elif defined(CONFIG_PAGE_SIZE_16K)
+#define _PAGE_SZ (_PAGE_SZ_16K)
+#endif
+#define _PAGE_TABLE (_PAGE_SZ | _PAGE_PRESENT)
+
+#if defined(CONFIG_HUGETLB_PAGE_SIZE_8K)
+# define _PAGE_SZHUGE (_PAGE_SZ_8K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_16K)
+# define _PAGE_SZHUGE (_PAGE_SZ_16K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_32K)
+# define _PAGE_SZHUGE (_PAGE_SZ_32K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_64K)
+# define _PAGE_SZHUGE (_PAGE_SZ_64K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_128K)
+# define _PAGE_SZHUGE (_PAGE_SZ_128K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_256K)
+# define _PAGE_SZHUGE (_PAGE_SZ_256K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_512K)
+# define _PAGE_SZHUGE (_PAGE_SZ_512K)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_1M)
+# define _PAGE_SZHUGE (_PAGE_SZ_1M)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_2M)
+# define _PAGE_SZHUGE (_PAGE_SZ_2M)
+#elif defined(CONFIG_HUGETLB_PAGE_SIZE_4M)
+# define _PAGE_SZHUGE (_PAGE_SZ_4M)
+#endif
+
+#endif /* _METAG_PGTABLE_BITS_H */
#ifndef _METAG_PGTABLE_H
#define _METAG_PGTABLE_H
+#include <asm/pgtable-bits.h>
#include <asm-generic/pgtable-nopmd.h>
/* Invalid regions on Meta: 0x00000000-0x001FFFFF and 0xFFFF0000-0xFFFFFFFF */
#define VMALLOC_END 0x7FFFFFFF
#endif
-/*
- * Definitions for MMU descriptors
- *
- * These are the hardware bits in the MMCU pte entries.
- * Derived from the Meta toolkit headers.
- */
-#define _PAGE_PRESENT MMCU_ENTRY_VAL_BIT
-#define _PAGE_WRITE MMCU_ENTRY_WR_BIT
-#define _PAGE_PRIV MMCU_ENTRY_PRIV_BIT
-/* Write combine bit - this can cause writes to occur out of order */
-#define _PAGE_WR_COMBINE MMCU_ENTRY_WRC_BIT
-/* Sys coherent bit - this bit is never used by Linux */
-#define _PAGE_SYS_COHERENT MMCU_ENTRY_SYS_BIT
-#define _PAGE_ALWAYS_ZERO_1 0x020
-#define _PAGE_CACHE_CTRL0 0x040
-#define _PAGE_CACHE_CTRL1 0x080
-#define _PAGE_ALWAYS_ZERO_2 0x100
-#define _PAGE_ALWAYS_ZERO_3 0x200
-#define _PAGE_ALWAYS_ZERO_4 0x400
-#define _PAGE_ALWAYS_ZERO_5 0x800
-
-/* These are software bits that we stuff into the gaps in the hardware
- * pte entries that are not used. Note, these DO get stored in the actual
- * hardware, but the hardware just does not use them.
- */
-#define _PAGE_ACCESSED _PAGE_ALWAYS_ZERO_1
-#define _PAGE_DIRTY _PAGE_ALWAYS_ZERO_2
-
-/* Pages owned, and protected by, the kernel. */
-#define _PAGE_KERNEL _PAGE_PRIV
-
-/* No cacheing of this page */
-#define _PAGE_CACHE_WIN0 (MMCU_CWIN_UNCACHED << MMCU_ENTRY_CWIN_S)
-/* burst cacheing - good for data streaming */
-#define _PAGE_CACHE_WIN1 (MMCU_CWIN_BURST << MMCU_ENTRY_CWIN_S)
-/* One cache way per thread */
-#define _PAGE_CACHE_WIN2 (MMCU_CWIN_C1SET << MMCU_ENTRY_CWIN_S)
-/* Full on cacheing */
-#define _PAGE_CACHE_WIN3 (MMCU_CWIN_CACHED << MMCU_ENTRY_CWIN_S)
-
-#define _PAGE_CACHEABLE (_PAGE_CACHE_WIN3 | _PAGE_WR_COMBINE)
-
-/* which bits are used for cache control ... */
-#define _PAGE_CACHE_MASK (_PAGE_CACHE_CTRL0 | _PAGE_CACHE_CTRL1 | \
- _PAGE_WR_COMBINE)
-
-/* This is a mask of the bits that pte_modify is allowed to change. */
-#define _PAGE_CHG_MASK (PAGE_MASK)
-
-#define _PAGE_SZ_SHIFT 1
-#define _PAGE_SZ_4K (0x0)
-#define _PAGE_SZ_8K (0x1 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_16K (0x2 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_32K (0x3 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_64K (0x4 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_128K (0x5 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_256K (0x6 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_512K (0x7 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_1M (0x8 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_2M (0x9 << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_4M (0xa << _PAGE_SZ_SHIFT)
-#define _PAGE_SZ_MASK (0xf << _PAGE_SZ_SHIFT)
-
-#if defined(CONFIG_PAGE_SIZE_4K)
-#define _PAGE_SZ (_PAGE_SZ_4K)
-#elif defined(CONFIG_PAGE_SIZE_8K)
-#define _PAGE_SZ (_PAGE_SZ_8K)
-#elif defined(CONFIG_PAGE_SIZE_16K)
-#define _PAGE_SZ (_PAGE_SZ_16K)
-#endif
-#define _PAGE_TABLE (_PAGE_SZ | _PAGE_PRESENT)
-
-#if defined(CONFIG_HUGETLB_PAGE_SIZE_8K)
-# define _PAGE_SZHUGE (_PAGE_SZ_8K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_16K)
-# define _PAGE_SZHUGE (_PAGE_SZ_16K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_32K)
-# define _PAGE_SZHUGE (_PAGE_SZ_32K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_64K)
-# define _PAGE_SZHUGE (_PAGE_SZ_64K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_128K)
-# define _PAGE_SZHUGE (_PAGE_SZ_128K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_256K)
-# define _PAGE_SZHUGE (_PAGE_SZ_256K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_512K)
-# define _PAGE_SZHUGE (_PAGE_SZ_512K)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_1M)
-# define _PAGE_SZHUGE (_PAGE_SZ_1M)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_2M)
-# define _PAGE_SZHUGE (_PAGE_SZ_2M)
-#elif defined(CONFIG_HUGETLB_PAGE_SIZE_4M)
-# define _PAGE_SZHUGE (_PAGE_SZ_4M)
-#endif
-
/*
* The Linux memory management assumes a three-level page table setup. On
* Meta, we use that, but "fold" the mid level into the top-level page
unsigned long get_wchan(struct task_struct *p);
-#define KSTK_EIP(tsk) ((tsk)->thread.kernel_context->CurrPC)
-#define KSTK_ESP(tsk) ((tsk)->thread.kernel_context->AX[0].U0)
+#define KSTK_EIP(tsk) (task_pt_regs(tsk)->ctx.CurrPC)
+#define KSTK_ESP(tsk) (task_pt_regs(tsk)->ctx.AX[0].U0)
#define user_stack_pointer(regs) ((regs)->ctx.AX[0].U0)
* The LP register should point to the location where the called function
* should return. [note that MAKE_SYS_CALL uses label 1] */
/* See if the system call number is valid */
+ blti r12, 5f
addi r11, r12, -__NR_syscalls;
- bgei r11,5f;
+ bgei r11, 5f;
/* Figure out which function to use for this system call. */
/* Note Microblaze barrel shift is optional, so don't rely on it */
add r12, r12, r12; /* convert num -> ptr */
/* The syscall number is invalid, return an error. */
5:
- rtsd r15, 8; /* looks like a normal subroutine return */
+ braid ret_from_trap
addi r3, r0, -ENOSYS;
/* Entry point used to return from a syscall/trap */
bri 1b
/* Maybe handle a signal */
-5:
+5:
andi r11, r19, _TIF_SIGPENDING | _TIF_NOTIFY_RESUME;
beqi r11, 4f; /* Signals to handle, handle them */
/* Call common code to handle resource allocation */
pcibios_resource_survey();
+ list_for_each_entry_safe(hose, tmp, &hose_list, list_node) {
+ if (hose->bus)
+ pci_bus_add_devices(hose->bus);
+ }
return 0;
}
.set push
SET_HARDFLOAT
cfc1 \tmp, fcr31
- swc1 $f0, THREAD_FPR0_LS64(\thread)
- swc1 $f1, THREAD_FPR1_LS64(\thread)
- swc1 $f2, THREAD_FPR2_LS64(\thread)
- swc1 $f3, THREAD_FPR3_LS64(\thread)
- swc1 $f4, THREAD_FPR4_LS64(\thread)
- swc1 $f5, THREAD_FPR5_LS64(\thread)
- swc1 $f6, THREAD_FPR6_LS64(\thread)
- swc1 $f7, THREAD_FPR7_LS64(\thread)
- swc1 $f8, THREAD_FPR8_LS64(\thread)
- swc1 $f9, THREAD_FPR9_LS64(\thread)
- swc1 $f10, THREAD_FPR10_LS64(\thread)
- swc1 $f11, THREAD_FPR11_LS64(\thread)
- swc1 $f12, THREAD_FPR12_LS64(\thread)
- swc1 $f13, THREAD_FPR13_LS64(\thread)
- swc1 $f14, THREAD_FPR14_LS64(\thread)
- swc1 $f15, THREAD_FPR15_LS64(\thread)
- swc1 $f16, THREAD_FPR16_LS64(\thread)
- swc1 $f17, THREAD_FPR17_LS64(\thread)
- swc1 $f18, THREAD_FPR18_LS64(\thread)
- swc1 $f19, THREAD_FPR19_LS64(\thread)
- swc1 $f20, THREAD_FPR20_LS64(\thread)
- swc1 $f21, THREAD_FPR21_LS64(\thread)
- swc1 $f22, THREAD_FPR22_LS64(\thread)
- swc1 $f23, THREAD_FPR23_LS64(\thread)
- swc1 $f24, THREAD_FPR24_LS64(\thread)
- swc1 $f25, THREAD_FPR25_LS64(\thread)
- swc1 $f26, THREAD_FPR26_LS64(\thread)
- swc1 $f27, THREAD_FPR27_LS64(\thread)
- swc1 $f28, THREAD_FPR28_LS64(\thread)
- swc1 $f29, THREAD_FPR29_LS64(\thread)
- swc1 $f30, THREAD_FPR30_LS64(\thread)
- swc1 $f31, THREAD_FPR31_LS64(\thread)
+ swc1 $f0, THREAD_FPR0(\thread)
+ swc1 $f1, THREAD_FPR1(\thread)
+ swc1 $f2, THREAD_FPR2(\thread)
+ swc1 $f3, THREAD_FPR3(\thread)
+ swc1 $f4, THREAD_FPR4(\thread)
+ swc1 $f5, THREAD_FPR5(\thread)
+ swc1 $f6, THREAD_FPR6(\thread)
+ swc1 $f7, THREAD_FPR7(\thread)
+ swc1 $f8, THREAD_FPR8(\thread)
+ swc1 $f9, THREAD_FPR9(\thread)
+ swc1 $f10, THREAD_FPR10(\thread)
+ swc1 $f11, THREAD_FPR11(\thread)
+ swc1 $f12, THREAD_FPR12(\thread)
+ swc1 $f13, THREAD_FPR13(\thread)
+ swc1 $f14, THREAD_FPR14(\thread)
+ swc1 $f15, THREAD_FPR15(\thread)
+ swc1 $f16, THREAD_FPR16(\thread)
+ swc1 $f17, THREAD_FPR17(\thread)
+ swc1 $f18, THREAD_FPR18(\thread)
+ swc1 $f19, THREAD_FPR19(\thread)
+ swc1 $f20, THREAD_FPR20(\thread)
+ swc1 $f21, THREAD_FPR21(\thread)
+ swc1 $f22, THREAD_FPR22(\thread)
+ swc1 $f23, THREAD_FPR23(\thread)
+ swc1 $f24, THREAD_FPR24(\thread)
+ swc1 $f25, THREAD_FPR25(\thread)
+ swc1 $f26, THREAD_FPR26(\thread)
+ swc1 $f27, THREAD_FPR27(\thread)
+ swc1 $f28, THREAD_FPR28(\thread)
+ swc1 $f29, THREAD_FPR29(\thread)
+ swc1 $f30, THREAD_FPR30(\thread)
+ swc1 $f31, THREAD_FPR31(\thread)
sw \tmp, THREAD_FCR31(\thread)
.set pop
.endm
.set push
SET_HARDFLOAT
lw \tmp, THREAD_FCR31(\thread)
- lwc1 $f0, THREAD_FPR0_LS64(\thread)
- lwc1 $f1, THREAD_FPR1_LS64(\thread)
- lwc1 $f2, THREAD_FPR2_LS64(\thread)
- lwc1 $f3, THREAD_FPR3_LS64(\thread)
- lwc1 $f4, THREAD_FPR4_LS64(\thread)
- lwc1 $f5, THREAD_FPR5_LS64(\thread)
- lwc1 $f6, THREAD_FPR6_LS64(\thread)
- lwc1 $f7, THREAD_FPR7_LS64(\thread)
- lwc1 $f8, THREAD_FPR8_LS64(\thread)
- lwc1 $f9, THREAD_FPR9_LS64(\thread)
- lwc1 $f10, THREAD_FPR10_LS64(\thread)
- lwc1 $f11, THREAD_FPR11_LS64(\thread)
- lwc1 $f12, THREAD_FPR12_LS64(\thread)
- lwc1 $f13, THREAD_FPR13_LS64(\thread)
- lwc1 $f14, THREAD_FPR14_LS64(\thread)
- lwc1 $f15, THREAD_FPR15_LS64(\thread)
- lwc1 $f16, THREAD_FPR16_LS64(\thread)
- lwc1 $f17, THREAD_FPR17_LS64(\thread)
- lwc1 $f18, THREAD_FPR18_LS64(\thread)
- lwc1 $f19, THREAD_FPR19_LS64(\thread)
- lwc1 $f20, THREAD_FPR20_LS64(\thread)
- lwc1 $f21, THREAD_FPR21_LS64(\thread)
- lwc1 $f22, THREAD_FPR22_LS64(\thread)
- lwc1 $f23, THREAD_FPR23_LS64(\thread)
- lwc1 $f24, THREAD_FPR24_LS64(\thread)
- lwc1 $f25, THREAD_FPR25_LS64(\thread)
- lwc1 $f26, THREAD_FPR26_LS64(\thread)
- lwc1 $f27, THREAD_FPR27_LS64(\thread)
- lwc1 $f28, THREAD_FPR28_LS64(\thread)
- lwc1 $f29, THREAD_FPR29_LS64(\thread)
- lwc1 $f30, THREAD_FPR30_LS64(\thread)
- lwc1 $f31, THREAD_FPR31_LS64(\thread)
+ lwc1 $f0, THREAD_FPR0(\thread)
+ lwc1 $f1, THREAD_FPR1(\thread)
+ lwc1 $f2, THREAD_FPR2(\thread)
+ lwc1 $f3, THREAD_FPR3(\thread)
+ lwc1 $f4, THREAD_FPR4(\thread)
+ lwc1 $f5, THREAD_FPR5(\thread)
+ lwc1 $f6, THREAD_FPR6(\thread)
+ lwc1 $f7, THREAD_FPR7(\thread)
+ lwc1 $f8, THREAD_FPR8(\thread)
+ lwc1 $f9, THREAD_FPR9(\thread)
+ lwc1 $f10, THREAD_FPR10(\thread)
+ lwc1 $f11, THREAD_FPR11(\thread)
+ lwc1 $f12, THREAD_FPR12(\thread)
+ lwc1 $f13, THREAD_FPR13(\thread)
+ lwc1 $f14, THREAD_FPR14(\thread)
+ lwc1 $f15, THREAD_FPR15(\thread)
+ lwc1 $f16, THREAD_FPR16(\thread)
+ lwc1 $f17, THREAD_FPR17(\thread)
+ lwc1 $f18, THREAD_FPR18(\thread)
+ lwc1 $f19, THREAD_FPR19(\thread)
+ lwc1 $f20, THREAD_FPR20(\thread)
+ lwc1 $f21, THREAD_FPR21(\thread)
+ lwc1 $f22, THREAD_FPR22(\thread)
+ lwc1 $f23, THREAD_FPR23(\thread)
+ lwc1 $f24, THREAD_FPR24(\thread)
+ lwc1 $f25, THREAD_FPR25(\thread)
+ lwc1 $f26, THREAD_FPR26(\thread)
+ lwc1 $f27, THREAD_FPR27(\thread)
+ lwc1 $f28, THREAD_FPR28(\thread)
+ lwc1 $f29, THREAD_FPR29(\thread)
+ lwc1 $f30, THREAD_FPR30(\thread)
+ lwc1 $f31, THREAD_FPR31(\thread)
ctc1 \tmp, fcr31
.set pop
.endm
.set push
SET_HARDFLOAT
cfc1 \tmp, fcr31
- sdc1 $f0, THREAD_FPR0_LS64(\thread)
- sdc1 $f2, THREAD_FPR2_LS64(\thread)
- sdc1 $f4, THREAD_FPR4_LS64(\thread)
- sdc1 $f6, THREAD_FPR6_LS64(\thread)
- sdc1 $f8, THREAD_FPR8_LS64(\thread)
- sdc1 $f10, THREAD_FPR10_LS64(\thread)
- sdc1 $f12, THREAD_FPR12_LS64(\thread)
- sdc1 $f14, THREAD_FPR14_LS64(\thread)
- sdc1 $f16, THREAD_FPR16_LS64(\thread)
- sdc1 $f18, THREAD_FPR18_LS64(\thread)
- sdc1 $f20, THREAD_FPR20_LS64(\thread)
- sdc1 $f22, THREAD_FPR22_LS64(\thread)
- sdc1 $f24, THREAD_FPR24_LS64(\thread)
- sdc1 $f26, THREAD_FPR26_LS64(\thread)
- sdc1 $f28, THREAD_FPR28_LS64(\thread)
- sdc1 $f30, THREAD_FPR30_LS64(\thread)
+ sdc1 $f0, THREAD_FPR0(\thread)
+ sdc1 $f2, THREAD_FPR2(\thread)
+ sdc1 $f4, THREAD_FPR4(\thread)
+ sdc1 $f6, THREAD_FPR6(\thread)
+ sdc1 $f8, THREAD_FPR8(\thread)
+ sdc1 $f10, THREAD_FPR10(\thread)
+ sdc1 $f12, THREAD_FPR12(\thread)
+ sdc1 $f14, THREAD_FPR14(\thread)
+ sdc1 $f16, THREAD_FPR16(\thread)
+ sdc1 $f18, THREAD_FPR18(\thread)
+ sdc1 $f20, THREAD_FPR20(\thread)
+ sdc1 $f22, THREAD_FPR22(\thread)
+ sdc1 $f24, THREAD_FPR24(\thread)
+ sdc1 $f26, THREAD_FPR26(\thread)
+ sdc1 $f28, THREAD_FPR28(\thread)
+ sdc1 $f30, THREAD_FPR30(\thread)
sw \tmp, THREAD_FCR31(\thread)
.set pop
.endm
.set push
.set mips64r2
SET_HARDFLOAT
- sdc1 $f1, THREAD_FPR1_LS64(\thread)
- sdc1 $f3, THREAD_FPR3_LS64(\thread)
- sdc1 $f5, THREAD_FPR5_LS64(\thread)
- sdc1 $f7, THREAD_FPR7_LS64(\thread)
- sdc1 $f9, THREAD_FPR9_LS64(\thread)
- sdc1 $f11, THREAD_FPR11_LS64(\thread)
- sdc1 $f13, THREAD_FPR13_LS64(\thread)
- sdc1 $f15, THREAD_FPR15_LS64(\thread)
- sdc1 $f17, THREAD_FPR17_LS64(\thread)
- sdc1 $f19, THREAD_FPR19_LS64(\thread)
- sdc1 $f21, THREAD_FPR21_LS64(\thread)
- sdc1 $f23, THREAD_FPR23_LS64(\thread)
- sdc1 $f25, THREAD_FPR25_LS64(\thread)
- sdc1 $f27, THREAD_FPR27_LS64(\thread)
- sdc1 $f29, THREAD_FPR29_LS64(\thread)
- sdc1 $f31, THREAD_FPR31_LS64(\thread)
+ sdc1 $f1, THREAD_FPR1(\thread)
+ sdc1 $f3, THREAD_FPR3(\thread)
+ sdc1 $f5, THREAD_FPR5(\thread)
+ sdc1 $f7, THREAD_FPR7(\thread)
+ sdc1 $f9, THREAD_FPR9(\thread)
+ sdc1 $f11, THREAD_FPR11(\thread)
+ sdc1 $f13, THREAD_FPR13(\thread)
+ sdc1 $f15, THREAD_FPR15(\thread)
+ sdc1 $f17, THREAD_FPR17(\thread)
+ sdc1 $f19, THREAD_FPR19(\thread)
+ sdc1 $f21, THREAD_FPR21(\thread)
+ sdc1 $f23, THREAD_FPR23(\thread)
+ sdc1 $f25, THREAD_FPR25(\thread)
+ sdc1 $f27, THREAD_FPR27(\thread)
+ sdc1 $f29, THREAD_FPR29(\thread)
+ sdc1 $f31, THREAD_FPR31(\thread)
.set pop
.endm
.set push
SET_HARDFLOAT
lw \tmp, THREAD_FCR31(\thread)
- ldc1 $f0, THREAD_FPR0_LS64(\thread)
- ldc1 $f2, THREAD_FPR2_LS64(\thread)
- ldc1 $f4, THREAD_FPR4_LS64(\thread)
- ldc1 $f6, THREAD_FPR6_LS64(\thread)
- ldc1 $f8, THREAD_FPR8_LS64(\thread)
- ldc1 $f10, THREAD_FPR10_LS64(\thread)
- ldc1 $f12, THREAD_FPR12_LS64(\thread)
- ldc1 $f14, THREAD_FPR14_LS64(\thread)
- ldc1 $f16, THREAD_FPR16_LS64(\thread)
- ldc1 $f18, THREAD_FPR18_LS64(\thread)
- ldc1 $f20, THREAD_FPR20_LS64(\thread)
- ldc1 $f22, THREAD_FPR22_LS64(\thread)
- ldc1 $f24, THREAD_FPR24_LS64(\thread)
- ldc1 $f26, THREAD_FPR26_LS64(\thread)
- ldc1 $f28, THREAD_FPR28_LS64(\thread)
- ldc1 $f30, THREAD_FPR30_LS64(\thread)
+ ldc1 $f0, THREAD_FPR0(\thread)
+ ldc1 $f2, THREAD_FPR2(\thread)
+ ldc1 $f4, THREAD_FPR4(\thread)
+ ldc1 $f6, THREAD_FPR6(\thread)
+ ldc1 $f8, THREAD_FPR8(\thread)
+ ldc1 $f10, THREAD_FPR10(\thread)
+ ldc1 $f12, THREAD_FPR12(\thread)
+ ldc1 $f14, THREAD_FPR14(\thread)
+ ldc1 $f16, THREAD_FPR16(\thread)
+ ldc1 $f18, THREAD_FPR18(\thread)
+ ldc1 $f20, THREAD_FPR20(\thread)
+ ldc1 $f22, THREAD_FPR22(\thread)
+ ldc1 $f24, THREAD_FPR24(\thread)
+ ldc1 $f26, THREAD_FPR26(\thread)
+ ldc1 $f28, THREAD_FPR28(\thread)
+ ldc1 $f30, THREAD_FPR30(\thread)
ctc1 \tmp, fcr31
.endm
.set push
.set mips64r2
SET_HARDFLOAT
- ldc1 $f1, THREAD_FPR1_LS64(\thread)
- ldc1 $f3, THREAD_FPR3_LS64(\thread)
- ldc1 $f5, THREAD_FPR5_LS64(\thread)
- ldc1 $f7, THREAD_FPR7_LS64(\thread)
- ldc1 $f9, THREAD_FPR9_LS64(\thread)
- ldc1 $f11, THREAD_FPR11_LS64(\thread)
- ldc1 $f13, THREAD_FPR13_LS64(\thread)
- ldc1 $f15, THREAD_FPR15_LS64(\thread)
- ldc1 $f17, THREAD_FPR17_LS64(\thread)
- ldc1 $f19, THREAD_FPR19_LS64(\thread)
- ldc1 $f21, THREAD_FPR21_LS64(\thread)
- ldc1 $f23, THREAD_FPR23_LS64(\thread)
- ldc1 $f25, THREAD_FPR25_LS64(\thread)
- ldc1 $f27, THREAD_FPR27_LS64(\thread)
- ldc1 $f29, THREAD_FPR29_LS64(\thread)
- ldc1 $f31, THREAD_FPR31_LS64(\thread)
+ ldc1 $f1, THREAD_FPR1(\thread)
+ ldc1 $f3, THREAD_FPR3(\thread)
+ ldc1 $f5, THREAD_FPR5(\thread)
+ ldc1 $f7, THREAD_FPR7(\thread)
+ ldc1 $f9, THREAD_FPR9(\thread)
+ ldc1 $f11, THREAD_FPR11(\thread)
+ ldc1 $f13, THREAD_FPR13(\thread)
+ ldc1 $f15, THREAD_FPR15(\thread)
+ ldc1 $f17, THREAD_FPR17(\thread)
+ ldc1 $f19, THREAD_FPR19(\thread)
+ ldc1 $f21, THREAD_FPR21(\thread)
+ ldc1 $f23, THREAD_FPR23(\thread)
+ ldc1 $f25, THREAD_FPR25(\thread)
+ ldc1 $f27, THREAD_FPR27(\thread)
+ ldc1 $f29, THREAD_FPR29(\thread)
+ ldc1 $f31, THREAD_FPR31(\thread)
.set pop
.endm
.endm
#ifdef TOOLCHAIN_SUPPORTS_MSA
+ .macro _cfcmsa rd, cs
+ .set push
+ .set mips32r2
+ .set msa
+ cfcmsa \rd, $\cs
+ .set pop
+ .endm
+
+ .macro _ctcmsa cd, rs
+ .set push
+ .set mips32r2
+ .set msa
+ ctcmsa $\cd, \rs
+ .set pop
+ .endm
+
.macro ld_d wd, off, base
.set push
.set mips32r2
.set pop
.endm
- .macro copy_u_w rd, ws, n
+ .macro copy_u_w ws, n
.set push
.set mips32r2
.set msa
- copy_u.w \rd, $w\ws[\n]
+ copy_u.w $1, $w\ws[\n]
.set pop
.endm
- .macro copy_u_d rd, ws, n
+ .macro copy_u_d ws, n
.set push
.set mips64r2
.set msa
- copy_u.d \rd, $w\ws[\n]
+ copy_u.d $1, $w\ws[\n]
.set pop
.endm
- .macro insert_w wd, n, rs
+ .macro insert_w wd, n
.set push
.set mips32r2
.set msa
- insert.w $w\wd[\n], \rs
+ insert.w $w\wd[\n], $1
.set pop
.endm
- .macro insert_d wd, n, rs
+ .macro insert_d wd, n
.set push
.set mips64r2
.set msa
- insert.d $w\wd[\n], \rs
+ insert.d $w\wd[\n], $1
.set pop
.endm
#else
/*
* Temporary until all toolchains in use include MSA support.
*/
- .macro cfcmsa rd, cs
+ .macro _cfcmsa rd, cs
.set push
.set noat
SET_HARDFLOAT
.set pop
.endm
- .macro ctcmsa cd, rs
+ .macro _ctcmsa cd, rs
.set push
.set noat
SET_HARDFLOAT
.set pop
.endm
- .macro copy_u_w rd, ws, n
+ .macro copy_u_w ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
.word COPY_UW_MSA_INSN | (\n << 16) | (\ws << 11)
- /* move triggers an assembler bug... */
- or \rd, $1, zero
.set pop
.endm
- .macro copy_u_d rd, ws, n
+ .macro copy_u_d ws, n
.set push
.set noat
SET_HARDFLOAT
.insn
.word COPY_UD_MSA_INSN | (\n << 16) | (\ws << 11)
- /* move triggers an assembler bug... */
- or \rd, $1, zero
.set pop
.endm
- .macro insert_w wd, n, rs
+ .macro insert_w wd, n
.set push
.set noat
SET_HARDFLOAT
- /* move triggers an assembler bug... */
- or $1, \rs, zero
.word INSERT_W_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
- .macro insert_d wd, n, rs
+ .macro insert_d wd, n
.set push
.set noat
SET_HARDFLOAT
- /* move triggers an assembler bug... */
- or $1, \rs, zero
.word INSERT_D_MSA_INSN | (\n << 16) | (\wd << 6)
.set pop
.endm
.set push
.set noat
SET_HARDFLOAT
- cfcmsa $1, MSA_CSR
+ _cfcmsa $1, MSA_CSR
sw $1, THREAD_MSA_CSR(\thread)
.set pop
.endm
.set noat
SET_HARDFLOAT
lw $1, THREAD_MSA_CSR(\thread)
- ctcmsa MSA_CSR, $1
+ _ctcmsa MSA_CSR, $1
.set pop
ld_d 0, THREAD_FPR0, \thread
ld_d 1, THREAD_FPR1, \thread
insert_w \wd, 2
insert_w \wd, 3
#endif
- .if 31-\wd
- msa_init_upper (\wd+1)
- .endif
.endm
.macro msa_init_all_upper
SET_HARDFLOAT
not $1, zero
msa_init_upper 0
+ msa_init_upper 1
+ msa_init_upper 2
+ msa_init_upper 3
+ msa_init_upper 4
+ msa_init_upper 5
+ msa_init_upper 6
+ msa_init_upper 7
+ msa_init_upper 8
+ msa_init_upper 9
+ msa_init_upper 10
+ msa_init_upper 11
+ msa_init_upper 12
+ msa_init_upper 13
+ msa_init_upper 14
+ msa_init_upper 15
+ msa_init_upper 16
+ msa_init_upper 17
+ msa_init_upper 18
+ msa_init_upper 19
+ msa_init_upper 20
+ msa_init_upper 21
+ msa_init_upper 22
+ msa_init_upper 23
+ msa_init_upper 24
+ msa_init_upper 25
+ msa_init_upper 26
+ msa_init_upper 27
+ msa_init_upper 28
+ msa_init_upper 29
+ msa_init_upper 30
+ msa_init_upper 31
.set pop
.endm
#define FPU_FR_MASK 0x1
};
+#define __disable_fpu() \
+do { \
+ clear_c0_status(ST0_CU1); \
+ disable_fpu_hazard(); \
+} while (0)
+
static inline int __enable_fpu(enum fpu_mode mode)
{
int fr;
enable_fpu_hazard();
/* check FR has the desired value */
- return (!!(read_c0_status() & ST0_FR) == !!fr) ? 0 : SIGFPE;
+ if (!!(read_c0_status() & ST0_FR) == !!fr)
+ return 0;
+
+ /* unsupported FR value */
+ __disable_fpu();
+ return SIGFPE;
default:
BUG();
return SIGFPE;
}
-#define __disable_fpu() \
-do { \
- clear_c0_status(ST0_CU1); \
- disable_fpu_hazard(); \
-} while (0)
-
#define clear_fpu_owner() clear_thread_flag(TIF_USEDFPU)
static inline int __is_fpu_owner(void)
}
disable_msa();
clear_thread_flag(TIF_USEDMSA);
+ __disable_fpu();
} else if (is_fpu_owner()) {
if (save)
_save_fp(current);
#ifndef _ASM_MIPS_JUMP_LABEL_H
#define _ASM_MIPS_JUMP_LABEL_H
-#include <linux/types.h>
+#ifndef __ASSEMBLY__
-#ifdef __KERNEL__
+#include <linux/types.h>
#define JUMP_LABEL_NOP_SIZE 4
return true;
}
-#endif /* __KERNEL__ */
-
#ifdef CONFIG_64BIT
typedef u64 jump_label_t;
#else
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif /* _ASM_MIPS_JUMP_LABEL_H */
DIE_RI,
DIE_PAGE_FAULT,
DIE_BREAK,
- DIE_SSTEPBP
+ DIE_SSTEPBP,
+ DIE_MSAFP
};
#endif /* _ASM_MIPS_KDEBUG_H */
/* MIPS KVM register ids */
#define MIPS_CP0_32(_R, _S) \
- (KVM_REG_MIPS | KVM_REG_SIZE_U32 | 0x10000 | (8 * (_R) + (_S)))
+ (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U32 | (8 * (_R) + (_S)))
#define MIPS_CP0_64(_R, _S) \
- (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 0x10000 | (8 * (_R) + (_S)))
+ (KVM_REG_MIPS_CP0 | KVM_REG_SIZE_U64 | (8 * (_R) + (_S)))
#define KVM_REG_MIPS_CP0_INDEX MIPS_CP0_32(0, 0)
#define KVM_REG_MIPS_CP0_ENTRYLO0 MIPS_CP0_64(2, 0)
#define KVM_REG_MIPS_CP0_STATUS MIPS_CP0_32(12, 0)
#define KVM_REG_MIPS_CP0_CAUSE MIPS_CP0_32(13, 0)
#define KVM_REG_MIPS_CP0_EPC MIPS_CP0_64(14, 0)
+#define KVM_REG_MIPS_CP0_PRID MIPS_CP0_32(15, 0)
#define KVM_REG_MIPS_CP0_EBASE MIPS_CP0_64(15, 1)
#define KVM_REG_MIPS_CP0_CONFIG MIPS_CP0_32(16, 0)
#define KVM_REG_MIPS_CP0_CONFIG1 MIPS_CP0_32(16, 1)
#define KVM_REG_MIPS_CP0_CONFIG2 MIPS_CP0_32(16, 2)
#define KVM_REG_MIPS_CP0_CONFIG3 MIPS_CP0_32(16, 3)
+#define KVM_REG_MIPS_CP0_CONFIG4 MIPS_CP0_32(16, 4)
+#define KVM_REG_MIPS_CP0_CONFIG5 MIPS_CP0_32(16, 5)
#define KVM_REG_MIPS_CP0_CONFIG7 MIPS_CP0_32(16, 7)
#define KVM_REG_MIPS_CP0_XCONTEXT MIPS_CP0_64(20, 0)
#define KVM_REG_MIPS_CP0_ERROREPC MIPS_CP0_64(30, 0)
u32 syscall_exits;
u32 resvd_inst_exits;
u32 break_inst_exits;
+ u32 trap_inst_exits;
+ u32 msa_fpe_exits;
+ u32 fpe_exits;
+ u32 msa_disabled_exits;
u32 flush_dcache_exits;
u32 halt_successful_poll;
u32 halt_wakeup;
SYSCALL_EXITS,
RESVD_INST_EXITS,
BREAK_INST_EXITS,
+ TRAP_INST_EXITS,
+ MSA_FPE_EXITS,
+ FPE_EXITS,
+ MSA_DISABLED_EXITS,
FLUSH_DCACHE_EXITS,
MAX_KVM_MIPS_EXIT_TYPES
};
#define MIPS_CP0_CONFIG1_SEL 1
#define MIPS_CP0_CONFIG2_SEL 2
#define MIPS_CP0_CONFIG3_SEL 3
+#define MIPS_CP0_CONFIG4_SEL 4
+#define MIPS_CP0_CONFIG5_SEL 5
/* Config0 register bits */
#define CP0C0_M 31
#define CP0C3_SM 1
#define CP0C3_TL 0
-/* Have config1, Cacheable, noncoherent, write-back, write allocate*/
-#define MIPS_CONFIG0 \
- ((1 << CP0C0_M) | (0x3 << CP0C0_K0))
-
-/* Have config2, no coprocessor2 attached, no MDMX support attached,
- no performance counters, watch registers present,
- no code compression, EJTAG present, no FPU, no watch registers */
-#define MIPS_CONFIG1 \
-((1 << CP0C1_M) | \
- (0 << CP0C1_C2) | (0 << CP0C1_MD) | (0 << CP0C1_PC) | \
- (0 << CP0C1_WR) | (0 << CP0C1_CA) | (1 << CP0C1_EP) | \
- (0 << CP0C1_FP))
-
-/* Have config3, no tertiary/secondary caches implemented */
-#define MIPS_CONFIG2 \
-((1 << CP0C2_M))
-
-/* No config4, no DSP ASE, no large physaddr (PABITS),
- no external interrupt controller, no vectored interrupts,
- no 1kb pages, no SmartMIPS ASE, no trace logic */
-#define MIPS_CONFIG3 \
-((0 << CP0C3_M) | (0 << CP0C3_DSPP) | (0 << CP0C3_LPA) | \
- (0 << CP0C3_VEIC) | (0 << CP0C3_VInt) | (0 << CP0C3_SP) | \
- (0 << CP0C3_SM) | (0 << CP0C3_TL))
-
/* MMU types, the first four entries have the same layout as the
CP0C0_MT field. */
enum mips_mmu_types {
*/
#define T_TRAP 13 /* Trap instruction */
#define T_VCEI 14 /* Virtual coherency exception */
+#define T_MSAFPE 14 /* MSA floating point exception */
#define T_FPE 15 /* Floating point exception */
+#define T_MSADIS 21 /* MSA disabled exception */
#define T_WATCH 23 /* Watch address reference */
#define T_VCED 31 /* Virtual coherency data */
long tlb_lo1;
};
+#define KVM_MIPS_FPU_FPU 0x1
+#define KVM_MIPS_FPU_MSA 0x2
+
#define KVM_MIPS_GUEST_TLB_SIZE 64
struct kvm_vcpu_arch {
void *host_ebase, *guest_ebase;
/* FPU State */
struct mips_fpu_struct fpu;
+ /* Which FPU state is loaded (KVM_MIPS_FPU_*) */
+ unsigned int fpu_inuse;
/* COP0 State */
struct mips_coproc *cop0;
/* WAIT executed */
int wait;
+
+ u8 fpu_enabled;
+ u8 msa_enabled;
};
#define kvm_read_c0_guest_config1(cop0) (cop0->reg[MIPS_CP0_CONFIG][1])
#define kvm_read_c0_guest_config2(cop0) (cop0->reg[MIPS_CP0_CONFIG][2])
#define kvm_read_c0_guest_config3(cop0) (cop0->reg[MIPS_CP0_CONFIG][3])
+#define kvm_read_c0_guest_config4(cop0) (cop0->reg[MIPS_CP0_CONFIG][4])
+#define kvm_read_c0_guest_config5(cop0) (cop0->reg[MIPS_CP0_CONFIG][5])
#define kvm_read_c0_guest_config7(cop0) (cop0->reg[MIPS_CP0_CONFIG][7])
#define kvm_write_c0_guest_config(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][0] = (val))
#define kvm_write_c0_guest_config1(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][1] = (val))
#define kvm_write_c0_guest_config2(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][2] = (val))
#define kvm_write_c0_guest_config3(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][3] = (val))
+#define kvm_write_c0_guest_config4(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][4] = (val))
+#define kvm_write_c0_guest_config5(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][5] = (val))
#define kvm_write_c0_guest_config7(cop0, val) (cop0->reg[MIPS_CP0_CONFIG][7] = (val))
#define kvm_read_c0_guest_errorepc(cop0) (cop0->reg[MIPS_CP0_ERROR_PC][0])
#define kvm_write_c0_guest_errorepc(cop0, val) (cop0->reg[MIPS_CP0_ERROR_PC][0] = (val))
kvm_set_c0_guest_ebase(cop0, ((val) & (change))); \
}
+/* Helpers */
+
+static inline bool kvm_mips_guest_can_have_fpu(struct kvm_vcpu_arch *vcpu)
+{
+ return (!__builtin_constant_p(cpu_has_fpu) || cpu_has_fpu) &&
+ vcpu->fpu_enabled;
+}
+
+static inline bool kvm_mips_guest_has_fpu(struct kvm_vcpu_arch *vcpu)
+{
+ return kvm_mips_guest_can_have_fpu(vcpu) &&
+ kvm_read_c0_guest_config1(vcpu->cop0) & MIPS_CONF1_FP;
+}
+
+static inline bool kvm_mips_guest_can_have_msa(struct kvm_vcpu_arch *vcpu)
+{
+ return (!__builtin_constant_p(cpu_has_msa) || cpu_has_msa) &&
+ vcpu->msa_enabled;
+}
+
+static inline bool kvm_mips_guest_has_msa(struct kvm_vcpu_arch *vcpu)
+{
+ return kvm_mips_guest_can_have_msa(vcpu) &&
+ kvm_read_c0_guest_config3(vcpu->cop0) & MIPS_CONF3_MSA;
+}
struct kvm_mips_callbacks {
int (*handle_cop_unusable)(struct kvm_vcpu *vcpu);
int (*handle_syscall)(struct kvm_vcpu *vcpu);
int (*handle_res_inst)(struct kvm_vcpu *vcpu);
int (*handle_break)(struct kvm_vcpu *vcpu);
+ int (*handle_trap)(struct kvm_vcpu *vcpu);
+ int (*handle_msa_fpe)(struct kvm_vcpu *vcpu);
+ int (*handle_fpe)(struct kvm_vcpu *vcpu);
+ int (*handle_msa_disabled)(struct kvm_vcpu *vcpu);
int (*vm_init)(struct kvm *kvm);
int (*vcpu_init)(struct kvm_vcpu *vcpu);
int (*vcpu_setup)(struct kvm_vcpu *vcpu);
const struct kvm_one_reg *reg, s64 *v);
int (*set_one_reg)(struct kvm_vcpu *vcpu,
const struct kvm_one_reg *reg, s64 v);
+ int (*vcpu_get_regs)(struct kvm_vcpu *vcpu);
+ int (*vcpu_set_regs)(struct kvm_vcpu *vcpu);
};
extern struct kvm_mips_callbacks *kvm_mips_callbacks;
int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks);
/* Trampoline ASM routine to start running in "Guest" context */
extern int __kvm_mips_vcpu_run(struct kvm_run *run, struct kvm_vcpu *vcpu);
+/* FPU/MSA context management */
+void __kvm_save_fpu(struct kvm_vcpu_arch *vcpu);
+void __kvm_restore_fpu(struct kvm_vcpu_arch *vcpu);
+void __kvm_restore_fcsr(struct kvm_vcpu_arch *vcpu);
+void __kvm_save_msa(struct kvm_vcpu_arch *vcpu);
+void __kvm_restore_msa(struct kvm_vcpu_arch *vcpu);
+void __kvm_restore_msa_upper(struct kvm_vcpu_arch *vcpu);
+void __kvm_restore_msacsr(struct kvm_vcpu_arch *vcpu);
+void kvm_own_fpu(struct kvm_vcpu *vcpu);
+void kvm_own_msa(struct kvm_vcpu *vcpu);
+void kvm_drop_fpu(struct kvm_vcpu *vcpu);
+void kvm_lose_fpu(struct kvm_vcpu *vcpu);
+
/* TLB handling */
uint32_t kvm_get_kernel_asid(struct kvm_vcpu *vcpu);
struct kvm_run *run,
struct kvm_vcpu *vcpu);
+extern enum emulation_result kvm_mips_emulate_trap_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu);
+
+extern enum emulation_result kvm_mips_emulate_msafpe_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu);
+
+extern enum emulation_result kvm_mips_emulate_fpe_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu);
+
+extern enum emulation_result kvm_mips_emulate_msadis_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu);
+
extern enum emulation_result kvm_mips_complete_mmio_load(struct kvm_vcpu *vcpu,
struct kvm_run *run);
struct kvm_run *run,
struct kvm_vcpu *vcpu);
+unsigned int kvm_mips_config1_wrmask(struct kvm_vcpu *vcpu);
+unsigned int kvm_mips_config3_wrmask(struct kvm_vcpu *vcpu);
+unsigned int kvm_mips_config4_wrmask(struct kvm_vcpu *vcpu);
+unsigned int kvm_mips_config5_wrmask(struct kvm_vcpu *vcpu);
+
/* Dynamic binary translation */
extern int kvm_mips_trans_cache_index(uint32_t inst, uint32_t *opc,
struct kvm_vcpu *vcpu);
#ifdef CONFIG_CPU_LITTLE_ENDIAN
# define FPR_IDX(width, idx) (idx)
#else
-# define FPR_IDX(width, idx) ((FPU_REG_WIDTH / (width)) - 1 - (idx))
+# define FPR_IDX(width, idx) ((idx) ^ ((64 / (width)) - 1))
#endif
#define BUILD_FPR_ACCESS(width) \
/*
* for KVM_GET_FPU and KVM_SET_FPU
- *
- * If Status[FR] is zero (32-bit FPU), the upper 32-bits of the FPRs
- * are zero filled.
*/
struct kvm_fpu {
- __u64 fpr[32];
- __u32 fir;
- __u32 fccr;
- __u32 fexr;
- __u32 fenr;
- __u32 fcsr;
- __u32 pad;
};
/*
- * For MIPS, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access CP0
+ * For MIPS, we use KVM_SET_ONE_REG and KVM_GET_ONE_REG to access various
* registers. The id field is broken down as follows:
*
- * bits[2..0] - Register 'sel' index.
- * bits[7..3] - Register 'rd' index.
- * bits[15..8] - Must be zero.
- * bits[31..16] - 1 -> CP0 registers.
- * bits[51..32] - Must be zero.
* bits[63..52] - As per linux/kvm.h
+ * bits[51..32] - Must be zero.
+ * bits[31..16] - Register set.
+ *
+ * Register set = 0: GP registers from kvm_regs (see definitions below).
+ *
+ * Register set = 1: CP0 registers.
+ * bits[15..8] - Must be zero.
+ * bits[7..3] - Register 'rd' index.
+ * bits[2..0] - Register 'sel' index.
+ *
+ * Register set = 2: KVM specific registers (see definitions below).
+ *
+ * Register set = 3: FPU / MSA registers (see definitions below).
*
* Other sets registers may be added in the future. Each set would
* have its own identifier in bits[31..16].
- *
- * The registers defined in struct kvm_regs are also accessible, the
- * id values for these are below.
*/
-#define KVM_REG_MIPS_R0 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 0)
-#define KVM_REG_MIPS_R1 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 1)
-#define KVM_REG_MIPS_R2 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 2)
-#define KVM_REG_MIPS_R3 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 3)
-#define KVM_REG_MIPS_R4 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 4)
-#define KVM_REG_MIPS_R5 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 5)
-#define KVM_REG_MIPS_R6 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 6)
-#define KVM_REG_MIPS_R7 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 7)
-#define KVM_REG_MIPS_R8 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 8)
-#define KVM_REG_MIPS_R9 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 9)
-#define KVM_REG_MIPS_R10 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 10)
-#define KVM_REG_MIPS_R11 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 11)
-#define KVM_REG_MIPS_R12 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 12)
-#define KVM_REG_MIPS_R13 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 13)
-#define KVM_REG_MIPS_R14 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 14)
-#define KVM_REG_MIPS_R15 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 15)
-#define KVM_REG_MIPS_R16 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 16)
-#define KVM_REG_MIPS_R17 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 17)
-#define KVM_REG_MIPS_R18 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 18)
-#define KVM_REG_MIPS_R19 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 19)
-#define KVM_REG_MIPS_R20 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 20)
-#define KVM_REG_MIPS_R21 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 21)
-#define KVM_REG_MIPS_R22 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 22)
-#define KVM_REG_MIPS_R23 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 23)
-#define KVM_REG_MIPS_R24 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 24)
-#define KVM_REG_MIPS_R25 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 25)
-#define KVM_REG_MIPS_R26 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 26)
-#define KVM_REG_MIPS_R27 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 27)
-#define KVM_REG_MIPS_R28 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 28)
-#define KVM_REG_MIPS_R29 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 29)
-#define KVM_REG_MIPS_R30 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 30)
-#define KVM_REG_MIPS_R31 (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 31)
-
-#define KVM_REG_MIPS_HI (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 32)
-#define KVM_REG_MIPS_LO (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 33)
-#define KVM_REG_MIPS_PC (KVM_REG_MIPS | KVM_REG_SIZE_U64 | 34)
-
-/* KVM specific control registers */
+#define KVM_REG_MIPS_GP (KVM_REG_MIPS | 0x0000000000000000ULL)
+#define KVM_REG_MIPS_CP0 (KVM_REG_MIPS | 0x0000000000010000ULL)
+#define KVM_REG_MIPS_KVM (KVM_REG_MIPS | 0x0000000000020000ULL)
+#define KVM_REG_MIPS_FPU (KVM_REG_MIPS | 0x0000000000030000ULL)
+
+
+/*
+ * KVM_REG_MIPS_GP - General purpose registers from kvm_regs.
+ */
+
+#define KVM_REG_MIPS_R0 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 0)
+#define KVM_REG_MIPS_R1 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 1)
+#define KVM_REG_MIPS_R2 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 2)
+#define KVM_REG_MIPS_R3 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 3)
+#define KVM_REG_MIPS_R4 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 4)
+#define KVM_REG_MIPS_R5 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 5)
+#define KVM_REG_MIPS_R6 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 6)
+#define KVM_REG_MIPS_R7 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 7)
+#define KVM_REG_MIPS_R8 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 8)
+#define KVM_REG_MIPS_R9 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 9)
+#define KVM_REG_MIPS_R10 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 10)
+#define KVM_REG_MIPS_R11 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 11)
+#define KVM_REG_MIPS_R12 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 12)
+#define KVM_REG_MIPS_R13 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 13)
+#define KVM_REG_MIPS_R14 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 14)
+#define KVM_REG_MIPS_R15 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 15)
+#define KVM_REG_MIPS_R16 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 16)
+#define KVM_REG_MIPS_R17 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 17)
+#define KVM_REG_MIPS_R18 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 18)
+#define KVM_REG_MIPS_R19 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 19)
+#define KVM_REG_MIPS_R20 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 20)
+#define KVM_REG_MIPS_R21 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 21)
+#define KVM_REG_MIPS_R22 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 22)
+#define KVM_REG_MIPS_R23 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 23)
+#define KVM_REG_MIPS_R24 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 24)
+#define KVM_REG_MIPS_R25 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 25)
+#define KVM_REG_MIPS_R26 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 26)
+#define KVM_REG_MIPS_R27 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 27)
+#define KVM_REG_MIPS_R28 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 28)
+#define KVM_REG_MIPS_R29 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 29)
+#define KVM_REG_MIPS_R30 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 30)
+#define KVM_REG_MIPS_R31 (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 31)
+
+#define KVM_REG_MIPS_HI (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 32)
+#define KVM_REG_MIPS_LO (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 33)
+#define KVM_REG_MIPS_PC (KVM_REG_MIPS_GP | KVM_REG_SIZE_U64 | 34)
+
+
+/*
+ * KVM_REG_MIPS_KVM - KVM specific control registers.
+ */
/*
* CP0_Count control
* safely without losing time or guest timer interrupts.
* Other: Reserved, do not change.
*/
-#define KVM_REG_MIPS_COUNT_CTL (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
- 0x20000 | 0)
+#define KVM_REG_MIPS_COUNT_CTL (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 0)
#define KVM_REG_MIPS_COUNT_CTL_DC 0x00000001
/*
* emulated.
* Modifications to times in the future are rejected.
*/
-#define KVM_REG_MIPS_COUNT_RESUME (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
- 0x20000 | 1)
+#define KVM_REG_MIPS_COUNT_RESUME (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 1)
/*
* CP0_Count rate in Hz
* Specifies the rate of the CP0_Count timer in Hz. Modifications occur without
* discontinuities in CP0_Count.
*/
-#define KVM_REG_MIPS_COUNT_HZ (KVM_REG_MIPS | KVM_REG_SIZE_U64 | \
- 0x20000 | 2)
+#define KVM_REG_MIPS_COUNT_HZ (KVM_REG_MIPS_KVM | KVM_REG_SIZE_U64 | 2)
+
+
+/*
+ * KVM_REG_MIPS_FPU - Floating Point and MIPS SIMD Architecture (MSA) registers.
+ *
+ * bits[15..8] - Register subset (see definitions below).
+ * bits[7..5] - Must be zero.
+ * bits[4..0] - Register number within register subset.
+ */
+
+#define KVM_REG_MIPS_FPR (KVM_REG_MIPS_FPU | 0x0000000000000000ULL)
+#define KVM_REG_MIPS_FCR (KVM_REG_MIPS_FPU | 0x0000000000000100ULL)
+#define KVM_REG_MIPS_MSACR (KVM_REG_MIPS_FPU | 0x0000000000000200ULL)
+
+/*
+ * KVM_REG_MIPS_FPR - Floating point / Vector registers.
+ */
+#define KVM_REG_MIPS_FPR_32(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U32 | (n))
+#define KVM_REG_MIPS_FPR_64(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U64 | (n))
+#define KVM_REG_MIPS_VEC_128(n) (KVM_REG_MIPS_FPR | KVM_REG_SIZE_U128 | (n))
+
+/*
+ * KVM_REG_MIPS_FCR - Floating point control registers.
+ */
+#define KVM_REG_MIPS_FCR_IR (KVM_REG_MIPS_FCR | KVM_REG_SIZE_U32 | 0)
+#define KVM_REG_MIPS_FCR_CSR (KVM_REG_MIPS_FCR | KVM_REG_SIZE_U32 | 31)
+
+/*
+ * KVM_REG_MIPS_MSACR - MIPS SIMD Architecture (MSA) control registers.
+ */
+#define KVM_REG_MIPS_MSA_IR (KVM_REG_MIPS_MSACR | KVM_REG_SIZE_U32 | 0)
+#define KVM_REG_MIPS_MSA_CSR (KVM_REG_MIPS_MSACR | KVM_REG_SIZE_U32 | 1)
+
/*
* KVM MIPS specific structures and definitions
OFFSET(THREAD_FPR30, task_struct, thread.fpu.fpr[30]);
OFFSET(THREAD_FPR31, task_struct, thread.fpu.fpr[31]);
- /* the least significant 64 bits of each FP register */
- OFFSET(THREAD_FPR0_LS64, task_struct,
- thread.fpu.fpr[0].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR1_LS64, task_struct,
- thread.fpu.fpr[1].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR2_LS64, task_struct,
- thread.fpu.fpr[2].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR3_LS64, task_struct,
- thread.fpu.fpr[3].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR4_LS64, task_struct,
- thread.fpu.fpr[4].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR5_LS64, task_struct,
- thread.fpu.fpr[5].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR6_LS64, task_struct,
- thread.fpu.fpr[6].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR7_LS64, task_struct,
- thread.fpu.fpr[7].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR8_LS64, task_struct,
- thread.fpu.fpr[8].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR9_LS64, task_struct,
- thread.fpu.fpr[9].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR10_LS64, task_struct,
- thread.fpu.fpr[10].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR11_LS64, task_struct,
- thread.fpu.fpr[11].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR12_LS64, task_struct,
- thread.fpu.fpr[12].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR13_LS64, task_struct,
- thread.fpu.fpr[13].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR14_LS64, task_struct,
- thread.fpu.fpr[14].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR15_LS64, task_struct,
- thread.fpu.fpr[15].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR16_LS64, task_struct,
- thread.fpu.fpr[16].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR17_LS64, task_struct,
- thread.fpu.fpr[17].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR18_LS64, task_struct,
- thread.fpu.fpr[18].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR19_LS64, task_struct,
- thread.fpu.fpr[19].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR20_LS64, task_struct,
- thread.fpu.fpr[20].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR21_LS64, task_struct,
- thread.fpu.fpr[21].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR22_LS64, task_struct,
- thread.fpu.fpr[22].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR23_LS64, task_struct,
- thread.fpu.fpr[23].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR24_LS64, task_struct,
- thread.fpu.fpr[24].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR25_LS64, task_struct,
- thread.fpu.fpr[25].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR26_LS64, task_struct,
- thread.fpu.fpr[26].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR27_LS64, task_struct,
- thread.fpu.fpr[27].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR28_LS64, task_struct,
- thread.fpu.fpr[28].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR29_LS64, task_struct,
- thread.fpu.fpr[29].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR30_LS64, task_struct,
- thread.fpu.fpr[30].val64[FPR_IDX(64, 0)]);
- OFFSET(THREAD_FPR31_LS64, task_struct,
- thread.fpu.fpr[31].val64[FPR_IDX(64, 0)]);
-
OFFSET(THREAD_FCR31, task_struct, thread.fpu.fcr31);
OFFSET(THREAD_MSA_CSR, task_struct, thread.fpu.msacsr);
BLANK();
OFFSET(VCPU_LO, kvm_vcpu_arch, lo);
OFFSET(VCPU_HI, kvm_vcpu_arch, hi);
OFFSET(VCPU_PC, kvm_vcpu_arch, pc);
+ BLANK();
+
+ OFFSET(VCPU_FPR0, kvm_vcpu_arch, fpu.fpr[0]);
+ OFFSET(VCPU_FPR1, kvm_vcpu_arch, fpu.fpr[1]);
+ OFFSET(VCPU_FPR2, kvm_vcpu_arch, fpu.fpr[2]);
+ OFFSET(VCPU_FPR3, kvm_vcpu_arch, fpu.fpr[3]);
+ OFFSET(VCPU_FPR4, kvm_vcpu_arch, fpu.fpr[4]);
+ OFFSET(VCPU_FPR5, kvm_vcpu_arch, fpu.fpr[5]);
+ OFFSET(VCPU_FPR6, kvm_vcpu_arch, fpu.fpr[6]);
+ OFFSET(VCPU_FPR7, kvm_vcpu_arch, fpu.fpr[7]);
+ OFFSET(VCPU_FPR8, kvm_vcpu_arch, fpu.fpr[8]);
+ OFFSET(VCPU_FPR9, kvm_vcpu_arch, fpu.fpr[9]);
+ OFFSET(VCPU_FPR10, kvm_vcpu_arch, fpu.fpr[10]);
+ OFFSET(VCPU_FPR11, kvm_vcpu_arch, fpu.fpr[11]);
+ OFFSET(VCPU_FPR12, kvm_vcpu_arch, fpu.fpr[12]);
+ OFFSET(VCPU_FPR13, kvm_vcpu_arch, fpu.fpr[13]);
+ OFFSET(VCPU_FPR14, kvm_vcpu_arch, fpu.fpr[14]);
+ OFFSET(VCPU_FPR15, kvm_vcpu_arch, fpu.fpr[15]);
+ OFFSET(VCPU_FPR16, kvm_vcpu_arch, fpu.fpr[16]);
+ OFFSET(VCPU_FPR17, kvm_vcpu_arch, fpu.fpr[17]);
+ OFFSET(VCPU_FPR18, kvm_vcpu_arch, fpu.fpr[18]);
+ OFFSET(VCPU_FPR19, kvm_vcpu_arch, fpu.fpr[19]);
+ OFFSET(VCPU_FPR20, kvm_vcpu_arch, fpu.fpr[20]);
+ OFFSET(VCPU_FPR21, kvm_vcpu_arch, fpu.fpr[21]);
+ OFFSET(VCPU_FPR22, kvm_vcpu_arch, fpu.fpr[22]);
+ OFFSET(VCPU_FPR23, kvm_vcpu_arch, fpu.fpr[23]);
+ OFFSET(VCPU_FPR24, kvm_vcpu_arch, fpu.fpr[24]);
+ OFFSET(VCPU_FPR25, kvm_vcpu_arch, fpu.fpr[25]);
+ OFFSET(VCPU_FPR26, kvm_vcpu_arch, fpu.fpr[26]);
+ OFFSET(VCPU_FPR27, kvm_vcpu_arch, fpu.fpr[27]);
+ OFFSET(VCPU_FPR28, kvm_vcpu_arch, fpu.fpr[28]);
+ OFFSET(VCPU_FPR29, kvm_vcpu_arch, fpu.fpr[29]);
+ OFFSET(VCPU_FPR30, kvm_vcpu_arch, fpu.fpr[30]);
+ OFFSET(VCPU_FPR31, kvm_vcpu_arch, fpu.fpr[31]);
+
+ OFFSET(VCPU_FCR31, kvm_vcpu_arch, fpu.fcr31);
+ OFFSET(VCPU_MSA_CSR, kvm_vcpu_arch, fpu.msacsr);
+ BLANK();
+
OFFSET(VCPU_COP0, kvm_vcpu_arch, cop0);
OFFSET(VCPU_GUEST_KERNEL_ASID, kvm_vcpu_arch, guest_kernel_asid);
OFFSET(VCPU_GUEST_USER_ASID, kvm_vcpu_arch, guest_user_asid);
.set mips1
SET_HARDFLOAT
cfc1 a1, fcr31
- li a2, ~(0x3f << 12)
- and a2, a1
- ctc1 a2, fcr31
.set pop
- TRACE_IRQS_ON
- STI
+ CLI
+ TRACE_IRQS_OFF
+ .endm
+
+ .macro __build_clear_msa_fpe
+ _cfcmsa a1, MSA_CSR
+ CLI
+ TRACE_IRQS_OFF
.endm
.macro __build_clear_ade
BUILD_HANDLER cpu cpu sti silent /* #11 */
BUILD_HANDLER ov ov sti silent /* #12 */
BUILD_HANDLER tr tr sti silent /* #13 */
- BUILD_HANDLER msa_fpe msa_fpe sti silent /* #14 */
+ BUILD_HANDLER msa_fpe msa_fpe msa_fpe silent /* #14 */
BUILD_HANDLER fpe fpe fpe silent /* #15 */
BUILD_HANDLER ftlb ftlb none silent /* #16 */
BUILD_HANDLER msa msa sti silent /* #21 */
#define CREATE_TRACE_POINTS
#include <trace/events/syscalls.h>
+static void init_fp_ctx(struct task_struct *target)
+{
+ /* If FP has been used then the target already has context */
+ if (tsk_used_math(target))
+ return;
+
+ /* Begin with data registers set to all 1s... */
+ memset(&target->thread.fpu.fpr, ~0, sizeof(target->thread.fpu.fpr));
+
+ /* ...and FCSR zeroed */
+ target->thread.fpu.fcr31 = 0;
+
+ /*
+ * Record that the target has "used" math, such that the context
+ * just initialised, and any modifications made by the caller,
+ * aren't discarded.
+ */
+ set_stopped_child_used_math(target);
+}
+
/*
* Called by kernel/ptrace.c when detaching..
*
if (!access_ok(VERIFY_READ, data, 33 * 8))
return -EIO;
+ init_fp_ctx(child);
fregs = get_fpu_regs(child);
for (i = 0; i < 32; i++) {
/* XXX fcr31 */
+ init_fp_ctx(target);
+
if (sizeof(target->thread.fpu.fpr[i]) == sizeof(elf_fpreg_t))
return user_regset_copyin(&pos, &count, &kbuf, &ubuf,
&target->thread.fpu,
case FPR_BASE ... FPR_BASE + 31: {
union fpureg *fregs = get_fpu_regs(child);
- if (!tsk_used_math(child)) {
- /* FP not yet used */
- memset(&child->thread.fpu, ~0,
- sizeof(child->thread.fpu));
- child->thread.fpu.fcr31 = 0;
- }
+ init_fp_ctx(child);
#ifdef CONFIG_32BIT
if (test_thread_flag(TIF_32BIT_FPREGS)) {
/*
.endm
.set noreorder
- .set MIPS_ISA_ARCH_LEVEL_RAW
LEAF(_save_fp_context)
.set push
/* Save 32-bit process floating point context */
LEAF(_save_fp_context32)
.set push
+ .set MIPS_ISA_ARCH_LEVEL_RAW
SET_HARDFLOAT
cfc1 t1, fcr31
int process_fpemu_return(int sig, void __user *fault_addr)
{
+ /*
+ * We can't allow the emulated instruction to leave any of the cause
+ * bits set in FCSR. If they were then the kernel would take an FP
+ * exception when restoring FP context.
+ */
+ current->thread.fpu.fcr31 &= ~FPU_CSR_ALL_X;
+
if (sig == SIGSEGV || sig == SIGBUS) {
struct siginfo si = {0};
si.si_addr = fault_addr;
if (notify_die(DIE_FP, "FP exception", regs, 0, regs_to_trapnr(regs),
SIGFPE) == NOTIFY_STOP)
goto out;
+
+ /* Clear FCSR.Cause before enabling interrupts */
+ write_32bit_cp1_register(CP1_STATUS, fcr31 & ~FPU_CSR_ALL_X);
+ local_irq_enable();
+
die_if_kernel("FP exception in kernel code", regs);
if (fcr31 & FPU_CSR_UNI_X) {
sig = fpu_emulator_cop1Handler(regs, ¤t->thread.fpu, 1,
&fault_addr);
- /*
- * We can't allow the emulated instruction to leave any of
- * the cause bit set in $fcr31.
- */
- current->thread.fpu.fcr31 &= ~FPU_CSR_ALL_X;
+ /* If something went wrong, signal */
+ process_fpemu_return(sig, fault_addr);
/* Restore the hardware register state */
own_fpu(1); /* Using the FPU again. */
- /* If something went wrong, signal */
- process_fpemu_return(sig, fault_addr);
-
goto out;
} else if (fcr31 & FPU_CSR_INV_X)
info.si_code = FPE_FLTINV;
exception_exit(prev_state);
}
-asmlinkage void do_msa_fpe(struct pt_regs *regs)
+asmlinkage void do_msa_fpe(struct pt_regs *regs, unsigned int msacsr)
{
enum ctx_state prev_state;
prev_state = exception_enter();
+ if (notify_die(DIE_MSAFP, "MSA FP exception", regs, 0,
+ regs_to_trapnr(regs), SIGFPE) == NOTIFY_STOP)
+ goto out;
+
+ /* Clear MSACSR.Cause before enabling interrupts */
+ write_msa_csr(msacsr & ~MSA_CSR_CAUSEF);
+ local_irq_enable();
+
die_if_kernel("do_msa_fpe invoked from kernel context!", regs);
force_sig(SIGFPE, current);
+out:
exception_exit(prev_state);
}
# Makefile for KVM support for MIPS
#
-common-objs = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
+common-objs-y = $(addprefix ../../../virt/kvm/, kvm_main.o coalesced_mmio.o)
EXTRA_CFLAGS += -Ivirt/kvm -Iarch/mips/kvm
-kvm-objs := $(common-objs) mips.o emulate.o locore.o \
+common-objs-$(CONFIG_CPU_HAS_MSA) += msa.o
+
+kvm-objs := $(common-objs-y) mips.o emulate.o locore.o \
interrupt.o stats.o commpage.o \
- dyntrans.o trap_emul.o
+ dyntrans.o trap_emul.o fpu.o
obj-$(CONFIG_KVM) += kvm.o
obj-y += callback.o tlb.o
return EMULATE_DONE;
}
+/**
+ * kvm_mips_config1_wrmask() - Find mask of writable bits in guest Config1
+ * @vcpu: Virtual CPU.
+ *
+ * Finds the mask of bits which are writable in the guest's Config1 CP0
+ * register, by userland (currently read-only to the guest).
+ */
+unsigned int kvm_mips_config1_wrmask(struct kvm_vcpu *vcpu)
+{
+ unsigned int mask = 0;
+
+ /* Permit FPU to be present if FPU is supported */
+ if (kvm_mips_guest_can_have_fpu(&vcpu->arch))
+ mask |= MIPS_CONF1_FP;
+
+ return mask;
+}
+
+/**
+ * kvm_mips_config3_wrmask() - Find mask of writable bits in guest Config3
+ * @vcpu: Virtual CPU.
+ *
+ * Finds the mask of bits which are writable in the guest's Config3 CP0
+ * register, by userland (currently read-only to the guest).
+ */
+unsigned int kvm_mips_config3_wrmask(struct kvm_vcpu *vcpu)
+{
+ /* Config4 is optional */
+ unsigned int mask = MIPS_CONF_M;
+
+ /* Permit MSA to be present if MSA is supported */
+ if (kvm_mips_guest_can_have_msa(&vcpu->arch))
+ mask |= MIPS_CONF3_MSA;
+
+ return mask;
+}
+
+/**
+ * kvm_mips_config4_wrmask() - Find mask of writable bits in guest Config4
+ * @vcpu: Virtual CPU.
+ *
+ * Finds the mask of bits which are writable in the guest's Config4 CP0
+ * register, by userland (currently read-only to the guest).
+ */
+unsigned int kvm_mips_config4_wrmask(struct kvm_vcpu *vcpu)
+{
+ /* Config5 is optional */
+ return MIPS_CONF_M;
+}
+
+/**
+ * kvm_mips_config5_wrmask() - Find mask of writable bits in guest Config5
+ * @vcpu: Virtual CPU.
+ *
+ * Finds the mask of bits which are writable in the guest's Config5 CP0
+ * register, by the guest itself.
+ */
+unsigned int kvm_mips_config5_wrmask(struct kvm_vcpu *vcpu)
+{
+ unsigned int mask = 0;
+
+ /* Permit MSAEn changes if MSA supported and enabled */
+ if (kvm_mips_guest_has_msa(&vcpu->arch))
+ mask |= MIPS_CONF5_MSAEN;
+
+ /*
+ * Permit guest FPU mode changes if FPU is enabled and the relevant
+ * feature exists according to FIR register.
+ */
+ if (kvm_mips_guest_has_fpu(&vcpu->arch)) {
+ if (cpu_has_fre)
+ mask |= MIPS_CONF5_FRE;
+ /* We don't support UFR or UFE */
+ }
+
+ return mask;
+}
+
enum emulation_result kvm_mips_emulate_CP0(uint32_t inst, uint32_t *opc,
uint32_t cause, struct kvm_run *run,
struct kvm_vcpu *vcpu)
kvm_mips_write_compare(vcpu,
vcpu->arch.gprs[rt]);
} else if ((rd == MIPS_CP0_STATUS) && (sel == 0)) {
- kvm_write_c0_guest_status(cop0,
- vcpu->arch.gprs[rt]);
+ unsigned int old_val, val, change;
+
+ old_val = kvm_read_c0_guest_status(cop0);
+ val = vcpu->arch.gprs[rt];
+ change = val ^ old_val;
+
+ /* Make sure that the NMI bit is never set */
+ val &= ~ST0_NMI;
+
+ /*
+ * Don't allow CU1 or FR to be set unless FPU
+ * capability enabled and exists in guest
+ * configuration.
+ */
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ val &= ~(ST0_CU1 | ST0_FR);
+
+ /*
+ * Also don't allow FR to be set if host doesn't
+ * support it.
+ */
+ if (!(current_cpu_data.fpu_id & MIPS_FPIR_F64))
+ val &= ~ST0_FR;
+
+
+ /* Handle changes in FPU mode */
+ preempt_disable();
+
+ /*
+ * FPU and Vector register state is made
+ * UNPREDICTABLE by a change of FR, so don't
+ * even bother saving it.
+ */
+ if (change & ST0_FR)
+ kvm_drop_fpu(vcpu);
+
+ /*
+ * If MSA state is already live, it is undefined
+ * how it interacts with FR=0 FPU state, and we
+ * don't want to hit reserved instruction
+ * exceptions trying to save the MSA state later
+ * when CU=1 && FR=1, so play it safe and save
+ * it first.
+ */
+ if (change & ST0_CU1 && !(val & ST0_FR) &&
+ vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
+ kvm_lose_fpu(vcpu);
+
/*
- * Make sure that CU1 and NMI bits are
- * never set
+ * Propagate CU1 (FPU enable) changes
+ * immediately if the FPU context is already
+ * loaded. When disabling we leave the context
+ * loaded so it can be quickly enabled again in
+ * the near future.
*/
- kvm_clear_c0_guest_status(cop0,
- (ST0_CU1 | ST0_NMI));
+ if (change & ST0_CU1 &&
+ vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
+ change_c0_status(ST0_CU1, val);
+
+ preempt_enable();
+
+ kvm_write_c0_guest_status(cop0, val);
#ifdef CONFIG_KVM_MIPS_DYN_TRANS
- kvm_mips_trans_mtc0(inst, opc, vcpu);
+ /*
+ * If FPU present, we need CU1/FR bits to take
+ * effect fairly soon.
+ */
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ kvm_mips_trans_mtc0(inst, opc, vcpu);
#endif
+ } else if ((rd == MIPS_CP0_CONFIG) && (sel == 5)) {
+ unsigned int old_val, val, change, wrmask;
+
+ old_val = kvm_read_c0_guest_config5(cop0);
+ val = vcpu->arch.gprs[rt];
+
+ /* Only a few bits are writable in Config5 */
+ wrmask = kvm_mips_config5_wrmask(vcpu);
+ change = (val ^ old_val) & wrmask;
+ val = old_val ^ change;
+
+
+ /* Handle changes in FPU/MSA modes */
+ preempt_disable();
+
+ /*
+ * Propagate FRE changes immediately if the FPU
+ * context is already loaded.
+ */
+ if (change & MIPS_CONF5_FRE &&
+ vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
+ change_c0_config5(MIPS_CONF5_FRE, val);
+
+ /*
+ * Propagate MSAEn changes immediately if the
+ * MSA context is already loaded. When disabling
+ * we leave the context loaded so it can be
+ * quickly enabled again in the near future.
+ */
+ if (change & MIPS_CONF5_MSAEN &&
+ vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
+ change_c0_config5(MIPS_CONF5_MSAEN,
+ val);
+
+ preempt_enable();
+
+ kvm_write_c0_guest_config5(cop0, val);
} else if ((rd == MIPS_CP0_CAUSE) && (sel == 0)) {
uint32_t old_cause, new_cause;
return er;
}
+enum emulation_result kvm_mips_emulate_trap_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct kvm_vcpu_arch *arch = &vcpu->arch;
+ enum emulation_result er = EMULATE_DONE;
+
+ if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
+ /* save old pc */
+ kvm_write_c0_guest_epc(cop0, arch->pc);
+ kvm_set_c0_guest_status(cop0, ST0_EXL);
+
+ if (cause & CAUSEF_BD)
+ kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
+ else
+ kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
+
+ kvm_debug("Delivering TRAP @ pc %#lx\n", arch->pc);
+
+ kvm_change_c0_guest_cause(cop0, (0xff),
+ (T_TRAP << CAUSEB_EXCCODE));
+
+ /* Set PC to the exception entry point */
+ arch->pc = KVM_GUEST_KSEG0 + 0x180;
+
+ } else {
+ kvm_err("Trying to deliver TRAP when EXL is already set\n");
+ er = EMULATE_FAIL;
+ }
+
+ return er;
+}
+
+enum emulation_result kvm_mips_emulate_msafpe_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct kvm_vcpu_arch *arch = &vcpu->arch;
+ enum emulation_result er = EMULATE_DONE;
+
+ if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
+ /* save old pc */
+ kvm_write_c0_guest_epc(cop0, arch->pc);
+ kvm_set_c0_guest_status(cop0, ST0_EXL);
+
+ if (cause & CAUSEF_BD)
+ kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
+ else
+ kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
+
+ kvm_debug("Delivering MSAFPE @ pc %#lx\n", arch->pc);
+
+ kvm_change_c0_guest_cause(cop0, (0xff),
+ (T_MSAFPE << CAUSEB_EXCCODE));
+
+ /* Set PC to the exception entry point */
+ arch->pc = KVM_GUEST_KSEG0 + 0x180;
+
+ } else {
+ kvm_err("Trying to deliver MSAFPE when EXL is already set\n");
+ er = EMULATE_FAIL;
+ }
+
+ return er;
+}
+
+enum emulation_result kvm_mips_emulate_fpe_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct kvm_vcpu_arch *arch = &vcpu->arch;
+ enum emulation_result er = EMULATE_DONE;
+
+ if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
+ /* save old pc */
+ kvm_write_c0_guest_epc(cop0, arch->pc);
+ kvm_set_c0_guest_status(cop0, ST0_EXL);
+
+ if (cause & CAUSEF_BD)
+ kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
+ else
+ kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
+
+ kvm_debug("Delivering FPE @ pc %#lx\n", arch->pc);
+
+ kvm_change_c0_guest_cause(cop0, (0xff),
+ (T_FPE << CAUSEB_EXCCODE));
+
+ /* Set PC to the exception entry point */
+ arch->pc = KVM_GUEST_KSEG0 + 0x180;
+
+ } else {
+ kvm_err("Trying to deliver FPE when EXL is already set\n");
+ er = EMULATE_FAIL;
+ }
+
+ return er;
+}
+
+enum emulation_result kvm_mips_emulate_msadis_exc(unsigned long cause,
+ uint32_t *opc,
+ struct kvm_run *run,
+ struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct kvm_vcpu_arch *arch = &vcpu->arch;
+ enum emulation_result er = EMULATE_DONE;
+
+ if ((kvm_read_c0_guest_status(cop0) & ST0_EXL) == 0) {
+ /* save old pc */
+ kvm_write_c0_guest_epc(cop0, arch->pc);
+ kvm_set_c0_guest_status(cop0, ST0_EXL);
+
+ if (cause & CAUSEF_BD)
+ kvm_set_c0_guest_cause(cop0, CAUSEF_BD);
+ else
+ kvm_clear_c0_guest_cause(cop0, CAUSEF_BD);
+
+ kvm_debug("Delivering MSADIS @ pc %#lx\n", arch->pc);
+
+ kvm_change_c0_guest_cause(cop0, (0xff),
+ (T_MSADIS << CAUSEB_EXCCODE));
+
+ /* Set PC to the exception entry point */
+ arch->pc = KVM_GUEST_KSEG0 + 0x180;
+
+ } else {
+ kvm_err("Trying to deliver MSADIS when EXL is already set\n");
+ er = EMULATE_FAIL;
+ }
+
+ return er;
+}
+
/* ll/sc, rdhwr, sync emulation */
#define OPCODE 0xfc000000
case T_SYSCALL:
case T_BREAK:
case T_RES_INST:
+ case T_TRAP:
+ case T_MSAFPE:
+ case T_FPE:
+ case T_MSADIS:
break;
case T_COP_UNUSABLE:
--- /dev/null
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * FPU context handling code for KVM.
+ *
+ * Copyright (C) 2015 Imagination Technologies Ltd.
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/fpregdef.h>
+#include <asm/mipsregs.h>
+#include <asm/regdef.h>
+
+ .set noreorder
+ .set noat
+
+LEAF(__kvm_save_fpu)
+ .set push
+ .set mips64r2
+ SET_HARDFLOAT
+ mfc0 t0, CP0_STATUS
+ sll t0, t0, 5 # is Status.FR set?
+ bgez t0, 1f # no: skip odd doubles
+ nop
+ sdc1 $f1, VCPU_FPR1(a0)
+ sdc1 $f3, VCPU_FPR3(a0)
+ sdc1 $f5, VCPU_FPR5(a0)
+ sdc1 $f7, VCPU_FPR7(a0)
+ sdc1 $f9, VCPU_FPR9(a0)
+ sdc1 $f11, VCPU_FPR11(a0)
+ sdc1 $f13, VCPU_FPR13(a0)
+ sdc1 $f15, VCPU_FPR15(a0)
+ sdc1 $f17, VCPU_FPR17(a0)
+ sdc1 $f19, VCPU_FPR19(a0)
+ sdc1 $f21, VCPU_FPR21(a0)
+ sdc1 $f23, VCPU_FPR23(a0)
+ sdc1 $f25, VCPU_FPR25(a0)
+ sdc1 $f27, VCPU_FPR27(a0)
+ sdc1 $f29, VCPU_FPR29(a0)
+ sdc1 $f31, VCPU_FPR31(a0)
+1: sdc1 $f0, VCPU_FPR0(a0)
+ sdc1 $f2, VCPU_FPR2(a0)
+ sdc1 $f4, VCPU_FPR4(a0)
+ sdc1 $f6, VCPU_FPR6(a0)
+ sdc1 $f8, VCPU_FPR8(a0)
+ sdc1 $f10, VCPU_FPR10(a0)
+ sdc1 $f12, VCPU_FPR12(a0)
+ sdc1 $f14, VCPU_FPR14(a0)
+ sdc1 $f16, VCPU_FPR16(a0)
+ sdc1 $f18, VCPU_FPR18(a0)
+ sdc1 $f20, VCPU_FPR20(a0)
+ sdc1 $f22, VCPU_FPR22(a0)
+ sdc1 $f24, VCPU_FPR24(a0)
+ sdc1 $f26, VCPU_FPR26(a0)
+ sdc1 $f28, VCPU_FPR28(a0)
+ jr ra
+ sdc1 $f30, VCPU_FPR30(a0)
+ .set pop
+ END(__kvm_save_fpu)
+
+LEAF(__kvm_restore_fpu)
+ .set push
+ .set mips64r2
+ SET_HARDFLOAT
+ mfc0 t0, CP0_STATUS
+ sll t0, t0, 5 # is Status.FR set?
+ bgez t0, 1f # no: skip odd doubles
+ nop
+ ldc1 $f1, VCPU_FPR1(a0)
+ ldc1 $f3, VCPU_FPR3(a0)
+ ldc1 $f5, VCPU_FPR5(a0)
+ ldc1 $f7, VCPU_FPR7(a0)
+ ldc1 $f9, VCPU_FPR9(a0)
+ ldc1 $f11, VCPU_FPR11(a0)
+ ldc1 $f13, VCPU_FPR13(a0)
+ ldc1 $f15, VCPU_FPR15(a0)
+ ldc1 $f17, VCPU_FPR17(a0)
+ ldc1 $f19, VCPU_FPR19(a0)
+ ldc1 $f21, VCPU_FPR21(a0)
+ ldc1 $f23, VCPU_FPR23(a0)
+ ldc1 $f25, VCPU_FPR25(a0)
+ ldc1 $f27, VCPU_FPR27(a0)
+ ldc1 $f29, VCPU_FPR29(a0)
+ ldc1 $f31, VCPU_FPR31(a0)
+1: ldc1 $f0, VCPU_FPR0(a0)
+ ldc1 $f2, VCPU_FPR2(a0)
+ ldc1 $f4, VCPU_FPR4(a0)
+ ldc1 $f6, VCPU_FPR6(a0)
+ ldc1 $f8, VCPU_FPR8(a0)
+ ldc1 $f10, VCPU_FPR10(a0)
+ ldc1 $f12, VCPU_FPR12(a0)
+ ldc1 $f14, VCPU_FPR14(a0)
+ ldc1 $f16, VCPU_FPR16(a0)
+ ldc1 $f18, VCPU_FPR18(a0)
+ ldc1 $f20, VCPU_FPR20(a0)
+ ldc1 $f22, VCPU_FPR22(a0)
+ ldc1 $f24, VCPU_FPR24(a0)
+ ldc1 $f26, VCPU_FPR26(a0)
+ ldc1 $f28, VCPU_FPR28(a0)
+ jr ra
+ ldc1 $f30, VCPU_FPR30(a0)
+ .set pop
+ END(__kvm_restore_fpu)
+
+LEAF(__kvm_restore_fcsr)
+ .set push
+ SET_HARDFLOAT
+ lw t0, VCPU_FCR31(a0)
+ /*
+ * The ctc1 must stay at this offset in __kvm_restore_fcsr.
+ * See kvm_mips_csr_die_notify() which handles t0 containing a value
+ * which triggers an FP Exception, which must be stepped over and
+ * ignored since the set cause bits must remain there for the guest.
+ */
+ ctc1 t0, fcr31
+ jr ra
+ nop
+ .set pop
+ END(__kvm_restore_fcsr)
#define PT_HOST_USERLOCAL PT_EPC
#define CP0_DDATA_LO $28,3
+#define CP0_CONFIG3 $16,3
+#define CP0_CONFIG5 $16,5
#define CP0_EBASE $15,1
#define CP0_INTCTL $12,1
LONG_L k0, VCPU_HOST_EBASE(k1)
mtc0 k0,CP0_EBASE
+ /*
+ * If FPU is enabled, save FCR31 and clear it so that later ctc1's don't
+ * trigger FPE for pending exceptions.
+ */
+ .set at
+ and v1, v0, ST0_CU1
+ beqz v1, 1f
+ nop
+ .set push
+ SET_HARDFLOAT
+ cfc1 t0, fcr31
+ sw t0, VCPU_FCR31(k1)
+ ctc1 zero,fcr31
+ .set pop
+ .set noat
+1:
+
+#ifdef CONFIG_CPU_HAS_MSA
+ /*
+ * If MSA is enabled, save MSACSR and clear it so that later
+ * instructions don't trigger MSAFPE for pending exceptions.
+ */
+ mfc0 t0, CP0_CONFIG3
+ ext t0, t0, 28, 1 /* MIPS_CONF3_MSAP */
+ beqz t0, 1f
+ nop
+ mfc0 t0, CP0_CONFIG5
+ ext t0, t0, 27, 1 /* MIPS_CONF5_MSAEN */
+ beqz t0, 1f
+ nop
+ _cfcmsa t0, MSA_CSR
+ sw t0, VCPU_MSA_CSR(k1)
+ _ctcmsa MSA_CSR, zero
+1:
+#endif
+
/* Now that the new EBASE has been loaded, unset BEV and KSU_USER */
.set at
and v0, v0, ~(ST0_EXL | KSU_USER | ST0_IE)
#include <linux/errno.h>
#include <linux/err.h>
+#include <linux/kdebug.h>
#include <linux/module.h>
#include <linux/vmalloc.h>
#include <linux/fs.h>
{ "syscall", VCPU_STAT(syscall_exits), KVM_STAT_VCPU },
{ "resvd_inst", VCPU_STAT(resvd_inst_exits), KVM_STAT_VCPU },
{ "break_inst", VCPU_STAT(break_inst_exits), KVM_STAT_VCPU },
+ { "trap_inst", VCPU_STAT(trap_inst_exits), KVM_STAT_VCPU },
+ { "msa_fpe", VCPU_STAT(msa_fpe_exits), KVM_STAT_VCPU },
+ { "fpe", VCPU_STAT(fpe_exits), KVM_STAT_VCPU },
+ { "msa_disabled", VCPU_STAT(msa_disabled_exits), KVM_STAT_VCPU },
{ "flush_dcache", VCPU_STAT(flush_dcache_exits), KVM_STAT_VCPU },
{ "halt_successful_poll", VCPU_STAT(halt_successful_poll), KVM_STAT_VCPU },
{ "halt_wakeup", VCPU_STAT(halt_wakeup), KVM_STAT_VCPU },
KVM_REG_MIPS_CP0_STATUS,
KVM_REG_MIPS_CP0_CAUSE,
KVM_REG_MIPS_CP0_EPC,
+ KVM_REG_MIPS_CP0_PRID,
KVM_REG_MIPS_CP0_CONFIG,
KVM_REG_MIPS_CP0_CONFIG1,
KVM_REG_MIPS_CP0_CONFIG2,
KVM_REG_MIPS_CP0_CONFIG3,
+ KVM_REG_MIPS_CP0_CONFIG4,
+ KVM_REG_MIPS_CP0_CONFIG5,
KVM_REG_MIPS_CP0_CONFIG7,
KVM_REG_MIPS_CP0_ERROREPC,
const struct kvm_one_reg *reg)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct mips_fpu_struct *fpu = &vcpu->arch.fpu;
int ret;
s64 v;
+ s64 vs[2];
+ unsigned int idx;
switch (reg->id) {
+ /* General purpose registers */
case KVM_REG_MIPS_R0 ... KVM_REG_MIPS_R31:
v = (long)vcpu->arch.gprs[reg->id - KVM_REG_MIPS_R0];
break;
v = (long)vcpu->arch.pc;
break;
+ /* Floating point registers */
+ case KVM_REG_MIPS_FPR_32(0) ... KVM_REG_MIPS_FPR_32(31):
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_FPR_32(0);
+ /* Odd singles in top of even double when FR=0 */
+ if (kvm_read_c0_guest_status(cop0) & ST0_FR)
+ v = get_fpr32(&fpu->fpr[idx], 0);
+ else
+ v = get_fpr32(&fpu->fpr[idx & ~1], idx & 1);
+ break;
+ case KVM_REG_MIPS_FPR_64(0) ... KVM_REG_MIPS_FPR_64(31):
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_FPR_64(0);
+ /* Can't access odd doubles in FR=0 mode */
+ if (idx & 1 && !(kvm_read_c0_guest_status(cop0) & ST0_FR))
+ return -EINVAL;
+ v = get_fpr64(&fpu->fpr[idx], 0);
+ break;
+ case KVM_REG_MIPS_FCR_IR:
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ v = boot_cpu_data.fpu_id;
+ break;
+ case KVM_REG_MIPS_FCR_CSR:
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ v = fpu->fcr31;
+ break;
+
+ /* MIPS SIMD Architecture (MSA) registers */
+ case KVM_REG_MIPS_VEC_128(0) ... KVM_REG_MIPS_VEC_128(31):
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ /* Can't access MSA registers in FR=0 mode */
+ if (!(kvm_read_c0_guest_status(cop0) & ST0_FR))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_VEC_128(0);
+#ifdef CONFIG_CPU_LITTLE_ENDIAN
+ /* least significant byte first */
+ vs[0] = get_fpr64(&fpu->fpr[idx], 0);
+ vs[1] = get_fpr64(&fpu->fpr[idx], 1);
+#else
+ /* most significant byte first */
+ vs[0] = get_fpr64(&fpu->fpr[idx], 1);
+ vs[1] = get_fpr64(&fpu->fpr[idx], 0);
+#endif
+ break;
+ case KVM_REG_MIPS_MSA_IR:
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ v = boot_cpu_data.msa_id;
+ break;
+ case KVM_REG_MIPS_MSA_CSR:
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ v = fpu->msacsr;
+ break;
+
+ /* Co-processor 0 registers */
case KVM_REG_MIPS_CP0_INDEX:
v = (long)kvm_read_c0_guest_index(cop0);
break;
case KVM_REG_MIPS_CP0_EPC:
v = (long)kvm_read_c0_guest_epc(cop0);
break;
- case KVM_REG_MIPS_CP0_ERROREPC:
- v = (long)kvm_read_c0_guest_errorepc(cop0);
+ case KVM_REG_MIPS_CP0_PRID:
+ v = (long)kvm_read_c0_guest_prid(cop0);
break;
case KVM_REG_MIPS_CP0_CONFIG:
v = (long)kvm_read_c0_guest_config(cop0);
case KVM_REG_MIPS_CP0_CONFIG3:
v = (long)kvm_read_c0_guest_config3(cop0);
break;
+ case KVM_REG_MIPS_CP0_CONFIG4:
+ v = (long)kvm_read_c0_guest_config4(cop0);
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG5:
+ v = (long)kvm_read_c0_guest_config5(cop0);
+ break;
case KVM_REG_MIPS_CP0_CONFIG7:
v = (long)kvm_read_c0_guest_config7(cop0);
break;
+ case KVM_REG_MIPS_CP0_ERROREPC:
+ v = (long)kvm_read_c0_guest_errorepc(cop0);
+ break;
/* registers to be handled specially */
case KVM_REG_MIPS_CP0_COUNT:
case KVM_REG_MIPS_COUNT_CTL:
u32 v32 = (u32)v;
return put_user(v32, uaddr32);
+ } else if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U128) {
+ void __user *uaddr = (void __user *)(long)reg->addr;
+
+ return copy_to_user(uaddr, vs, 16);
} else {
return -EINVAL;
}
const struct kvm_one_reg *reg)
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
- u64 v;
+ struct mips_fpu_struct *fpu = &vcpu->arch.fpu;
+ s64 v;
+ s64 vs[2];
+ unsigned int idx;
if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U64) {
u64 __user *uaddr64 = (u64 __user *)(long)reg->addr;
if (get_user(v32, uaddr32) != 0)
return -EFAULT;
v = (s64)v32;
+ } else if ((reg->id & KVM_REG_SIZE_MASK) == KVM_REG_SIZE_U128) {
+ void __user *uaddr = (void __user *)(long)reg->addr;
+
+ return copy_from_user(vs, uaddr, 16);
} else {
return -EINVAL;
}
switch (reg->id) {
+ /* General purpose registers */
case KVM_REG_MIPS_R0:
/* Silently ignore requests to set $0 */
break;
vcpu->arch.pc = v;
break;
+ /* Floating point registers */
+ case KVM_REG_MIPS_FPR_32(0) ... KVM_REG_MIPS_FPR_32(31):
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_FPR_32(0);
+ /* Odd singles in top of even double when FR=0 */
+ if (kvm_read_c0_guest_status(cop0) & ST0_FR)
+ set_fpr32(&fpu->fpr[idx], 0, v);
+ else
+ set_fpr32(&fpu->fpr[idx & ~1], idx & 1, v);
+ break;
+ case KVM_REG_MIPS_FPR_64(0) ... KVM_REG_MIPS_FPR_64(31):
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_FPR_64(0);
+ /* Can't access odd doubles in FR=0 mode */
+ if (idx & 1 && !(kvm_read_c0_guest_status(cop0) & ST0_FR))
+ return -EINVAL;
+ set_fpr64(&fpu->fpr[idx], 0, v);
+ break;
+ case KVM_REG_MIPS_FCR_IR:
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ /* Read-only */
+ break;
+ case KVM_REG_MIPS_FCR_CSR:
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch))
+ return -EINVAL;
+ fpu->fcr31 = v;
+ break;
+
+ /* MIPS SIMD Architecture (MSA) registers */
+ case KVM_REG_MIPS_VEC_128(0) ... KVM_REG_MIPS_VEC_128(31):
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ idx = reg->id - KVM_REG_MIPS_VEC_128(0);
+#ifdef CONFIG_CPU_LITTLE_ENDIAN
+ /* least significant byte first */
+ set_fpr64(&fpu->fpr[idx], 0, vs[0]);
+ set_fpr64(&fpu->fpr[idx], 1, vs[1]);
+#else
+ /* most significant byte first */
+ set_fpr64(&fpu->fpr[idx], 1, vs[0]);
+ set_fpr64(&fpu->fpr[idx], 0, vs[1]);
+#endif
+ break;
+ case KVM_REG_MIPS_MSA_IR:
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ /* Read-only */
+ break;
+ case KVM_REG_MIPS_MSA_CSR:
+ if (!kvm_mips_guest_has_msa(&vcpu->arch))
+ return -EINVAL;
+ fpu->msacsr = v;
+ break;
+
+ /* Co-processor 0 registers */
case KVM_REG_MIPS_CP0_INDEX:
kvm_write_c0_guest_index(cop0, v);
break;
case KVM_REG_MIPS_CP0_EPC:
kvm_write_c0_guest_epc(cop0, v);
break;
+ case KVM_REG_MIPS_CP0_PRID:
+ kvm_write_c0_guest_prid(cop0, v);
+ break;
case KVM_REG_MIPS_CP0_ERROREPC:
kvm_write_c0_guest_errorepc(cop0, v);
break;
case KVM_REG_MIPS_CP0_COUNT:
case KVM_REG_MIPS_CP0_COMPARE:
case KVM_REG_MIPS_CP0_CAUSE:
+ case KVM_REG_MIPS_CP0_CONFIG:
+ case KVM_REG_MIPS_CP0_CONFIG1:
+ case KVM_REG_MIPS_CP0_CONFIG2:
+ case KVM_REG_MIPS_CP0_CONFIG3:
+ case KVM_REG_MIPS_CP0_CONFIG4:
+ case KVM_REG_MIPS_CP0_CONFIG5:
case KVM_REG_MIPS_COUNT_CTL:
case KVM_REG_MIPS_COUNT_RESUME:
case KVM_REG_MIPS_COUNT_HZ:
return 0;
}
+static int kvm_vcpu_ioctl_enable_cap(struct kvm_vcpu *vcpu,
+ struct kvm_enable_cap *cap)
+{
+ int r = 0;
+
+ if (!kvm_vm_ioctl_check_extension(vcpu->kvm, cap->cap))
+ return -EINVAL;
+ if (cap->flags)
+ return -EINVAL;
+ if (cap->args[0])
+ return -EINVAL;
+
+ switch (cap->cap) {
+ case KVM_CAP_MIPS_FPU:
+ vcpu->arch.fpu_enabled = true;
+ break;
+ case KVM_CAP_MIPS_MSA:
+ vcpu->arch.msa_enabled = true;
+ break;
+ default:
+ r = -EINVAL;
+ break;
+ }
+
+ return r;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp, unsigned int ioctl,
unsigned long arg)
{
r = kvm_vcpu_ioctl_interrupt(vcpu, &irq);
break;
}
+ case KVM_ENABLE_CAP: {
+ struct kvm_enable_cap cap;
+
+ r = -EFAULT;
+ if (copy_from_user(&cap, argp, sizeof(cap)))
+ goto out;
+ r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
+ break;
+ }
default:
r = -ENOIOCTLCMD;
}
switch (ext) {
case KVM_CAP_ONE_REG:
+ case KVM_CAP_ENABLE_CAP:
r = 1;
break;
case KVM_CAP_COALESCED_MMIO:
r = KVM_COALESCED_MMIO_PAGE_OFFSET;
break;
+ case KVM_CAP_MIPS_FPU:
+ r = !!cpu_has_fpu;
+ break;
+ case KVM_CAP_MIPS_MSA:
+ /*
+ * We don't support MSA vector partitioning yet:
+ * 1) It would require explicit support which can't be tested
+ * yet due to lack of support in current hardware.
+ * 2) It extends the state that would need to be saved/restored
+ * by e.g. QEMU for migration.
+ *
+ * When vector partitioning hardware becomes available, support
+ * could be added by requiring a flag when enabling
+ * KVM_CAP_MIPS_MSA capability to indicate that userland knows
+ * to save/restore the appropriate extra state.
+ */
+ r = cpu_has_msa && !(boot_cpu_data.msa_id & MSA_IR_WRPF);
+ break;
default:
r = 0;
break;
ret = kvm_mips_callbacks->handle_break(vcpu);
break;
+ case T_TRAP:
+ ++vcpu->stat.trap_inst_exits;
+ trace_kvm_exit(vcpu, TRAP_INST_EXITS);
+ ret = kvm_mips_callbacks->handle_trap(vcpu);
+ break;
+
+ case T_MSAFPE:
+ ++vcpu->stat.msa_fpe_exits;
+ trace_kvm_exit(vcpu, MSA_FPE_EXITS);
+ ret = kvm_mips_callbacks->handle_msa_fpe(vcpu);
+ break;
+
+ case T_FPE:
+ ++vcpu->stat.fpe_exits;
+ trace_kvm_exit(vcpu, FPE_EXITS);
+ ret = kvm_mips_callbacks->handle_fpe(vcpu);
+ break;
+
+ case T_MSADIS:
+ ++vcpu->stat.msa_disabled_exits;
+ trace_kvm_exit(vcpu, MSA_DISABLED_EXITS);
+ ret = kvm_mips_callbacks->handle_msa_disabled(vcpu);
+ break;
+
default:
kvm_err("Exception Code: %d, not yet handled, @ PC: %p, inst: 0x%08x BadVaddr: %#lx Status: %#lx\n",
exccode, opc, kvm_get_inst(opc, vcpu), badvaddr,
}
}
+ if (ret == RESUME_GUEST) {
+ /*
+ * If FPU / MSA are enabled (i.e. the guest's FPU / MSA context
+ * is live), restore FCR31 / MSACSR.
+ *
+ * This should be before returning to the guest exception
+ * vector, as it may well cause an [MSA] FP exception if there
+ * are pending exception bits unmasked. (see
+ * kvm_mips_csr_die_notifier() for how that is handled).
+ */
+ if (kvm_mips_guest_has_fpu(&vcpu->arch) &&
+ read_c0_status() & ST0_CU1)
+ __kvm_restore_fcsr(&vcpu->arch);
+
+ if (kvm_mips_guest_has_msa(&vcpu->arch) &&
+ read_c0_config5() & MIPS_CONF5_MSAEN)
+ __kvm_restore_msacsr(&vcpu->arch);
+ }
+
/* Disable HTW before returning to guest or host */
htw_stop();
return ret;
}
+/* Enable FPU for guest and restore context */
+void kvm_own_fpu(struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ unsigned int sr, cfg5;
+
+ preempt_disable();
+
+ sr = kvm_read_c0_guest_status(cop0);
+
+ /*
+ * If MSA state is already live, it is undefined how it interacts with
+ * FR=0 FPU state, and we don't want to hit reserved instruction
+ * exceptions trying to save the MSA state later when CU=1 && FR=1, so
+ * play it safe and save it first.
+ *
+ * In theory we shouldn't ever hit this case since kvm_lose_fpu() should
+ * get called when guest CU1 is set, however we can't trust the guest
+ * not to clobber the status register directly via the commpage.
+ */
+ if (cpu_has_msa && sr & ST0_CU1 && !(sr & ST0_FR) &&
+ vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA)
+ kvm_lose_fpu(vcpu);
+
+ /*
+ * Enable FPU for guest
+ * We set FR and FRE according to guest context
+ */
+ change_c0_status(ST0_CU1 | ST0_FR, sr);
+ if (cpu_has_fre) {
+ cfg5 = kvm_read_c0_guest_config5(cop0);
+ change_c0_config5(MIPS_CONF5_FRE, cfg5);
+ }
+ enable_fpu_hazard();
+
+ /* If guest FPU state not active, restore it now */
+ if (!(vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)) {
+ __kvm_restore_fpu(&vcpu->arch);
+ vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_FPU;
+ }
+
+ preempt_enable();
+}
+
+#ifdef CONFIG_CPU_HAS_MSA
+/* Enable MSA for guest and restore context */
+void kvm_own_msa(struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ unsigned int sr, cfg5;
+
+ preempt_disable();
+
+ /*
+ * Enable FPU if enabled in guest, since we're restoring FPU context
+ * anyway. We set FR and FRE according to guest context.
+ */
+ if (kvm_mips_guest_has_fpu(&vcpu->arch)) {
+ sr = kvm_read_c0_guest_status(cop0);
+
+ /*
+ * If FR=0 FPU state is already live, it is undefined how it
+ * interacts with MSA state, so play it safe and save it first.
+ */
+ if (!(sr & ST0_FR) &&
+ (vcpu->arch.fpu_inuse & (KVM_MIPS_FPU_FPU |
+ KVM_MIPS_FPU_MSA)) == KVM_MIPS_FPU_FPU)
+ kvm_lose_fpu(vcpu);
+
+ change_c0_status(ST0_CU1 | ST0_FR, sr);
+ if (sr & ST0_CU1 && cpu_has_fre) {
+ cfg5 = kvm_read_c0_guest_config5(cop0);
+ change_c0_config5(MIPS_CONF5_FRE, cfg5);
+ }
+ }
+
+ /* Enable MSA for guest */
+ set_c0_config5(MIPS_CONF5_MSAEN);
+ enable_fpu_hazard();
+
+ switch (vcpu->arch.fpu_inuse & (KVM_MIPS_FPU_FPU | KVM_MIPS_FPU_MSA)) {
+ case KVM_MIPS_FPU_FPU:
+ /*
+ * Guest FPU state already loaded, only restore upper MSA state
+ */
+ __kvm_restore_msa_upper(&vcpu->arch);
+ vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_MSA;
+ break;
+ case 0:
+ /* Neither FPU or MSA already active, restore full MSA state */
+ __kvm_restore_msa(&vcpu->arch);
+ vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_MSA;
+ if (kvm_mips_guest_has_fpu(&vcpu->arch))
+ vcpu->arch.fpu_inuse |= KVM_MIPS_FPU_FPU;
+ break;
+ default:
+ break;
+ }
+
+ preempt_enable();
+}
+#endif
+
+/* Drop FPU & MSA without saving it */
+void kvm_drop_fpu(struct kvm_vcpu *vcpu)
+{
+ preempt_disable();
+ if (cpu_has_msa && vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA) {
+ disable_msa();
+ vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_MSA;
+ }
+ if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
+ clear_c0_status(ST0_CU1 | ST0_FR);
+ vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_FPU;
+ }
+ preempt_enable();
+}
+
+/* Save and disable FPU & MSA */
+void kvm_lose_fpu(struct kvm_vcpu *vcpu)
+{
+ /*
+ * FPU & MSA get disabled in root context (hardware) when it is disabled
+ * in guest context (software), but the register state in the hardware
+ * may still be in use. This is why we explicitly re-enable the hardware
+ * before saving.
+ */
+
+ preempt_disable();
+ if (cpu_has_msa && vcpu->arch.fpu_inuse & KVM_MIPS_FPU_MSA) {
+ set_c0_config5(MIPS_CONF5_MSAEN);
+ enable_fpu_hazard();
+
+ __kvm_save_msa(&vcpu->arch);
+
+ /* Disable MSA & FPU */
+ disable_msa();
+ if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU)
+ clear_c0_status(ST0_CU1 | ST0_FR);
+ vcpu->arch.fpu_inuse &= ~(KVM_MIPS_FPU_FPU | KVM_MIPS_FPU_MSA);
+ } else if (vcpu->arch.fpu_inuse & KVM_MIPS_FPU_FPU) {
+ set_c0_status(ST0_CU1);
+ enable_fpu_hazard();
+
+ __kvm_save_fpu(&vcpu->arch);
+ vcpu->arch.fpu_inuse &= ~KVM_MIPS_FPU_FPU;
+
+ /* Disable FPU */
+ clear_c0_status(ST0_CU1 | ST0_FR);
+ }
+ preempt_enable();
+}
+
+/*
+ * Step over a specific ctc1 to FCSR and a specific ctcmsa to MSACSR which are
+ * used to restore guest FCSR/MSACSR state and may trigger a "harmless" FP/MSAFP
+ * exception if cause bits are set in the value being written.
+ */
+static int kvm_mips_csr_die_notify(struct notifier_block *self,
+ unsigned long cmd, void *ptr)
+{
+ struct die_args *args = (struct die_args *)ptr;
+ struct pt_regs *regs = args->regs;
+ unsigned long pc;
+
+ /* Only interested in FPE and MSAFPE */
+ if (cmd != DIE_FP && cmd != DIE_MSAFP)
+ return NOTIFY_DONE;
+
+ /* Return immediately if guest context isn't active */
+ if (!(current->flags & PF_VCPU))
+ return NOTIFY_DONE;
+
+ /* Should never get here from user mode */
+ BUG_ON(user_mode(regs));
+
+ pc = instruction_pointer(regs);
+ switch (cmd) {
+ case DIE_FP:
+ /* match 2nd instruction in __kvm_restore_fcsr */
+ if (pc != (unsigned long)&__kvm_restore_fcsr + 4)
+ return NOTIFY_DONE;
+ break;
+ case DIE_MSAFP:
+ /* match 2nd/3rd instruction in __kvm_restore_msacsr */
+ if (!cpu_has_msa ||
+ pc < (unsigned long)&__kvm_restore_msacsr + 4 ||
+ pc > (unsigned long)&__kvm_restore_msacsr + 8)
+ return NOTIFY_DONE;
+ break;
+ }
+
+ /* Move PC forward a little and continue executing */
+ instruction_pointer(regs) += 4;
+
+ return NOTIFY_STOP;
+}
+
+static struct notifier_block kvm_mips_csr_die_notifier = {
+ .notifier_call = kvm_mips_csr_die_notify,
+};
+
int __init kvm_mips_init(void)
{
int ret;
if (ret)
return ret;
+ register_die_notifier(&kvm_mips_csr_die_notifier);
+
/*
* On MIPS, kernel modules are executed from "mapped space", which
* requires TLBs. The TLB handling code is statically linked with
kvm_mips_release_pfn_clean = kvm_release_pfn_clean;
kvm_mips_is_error_pfn = is_error_pfn;
- pr_info("KVM/MIPS Initialized\n");
return 0;
}
kvm_mips_release_pfn_clean = NULL;
kvm_mips_is_error_pfn = NULL;
- pr_info("KVM/MIPS unloaded\n");
+ unregister_die_notifier(&kvm_mips_csr_die_notifier);
}
module_init(kvm_mips_init);
--- /dev/null
+/*
+ * This file is subject to the terms and conditions of the GNU General Public
+ * License. See the file "COPYING" in the main directory of this archive
+ * for more details.
+ *
+ * MIPS SIMD Architecture (MSA) context handling code for KVM.
+ *
+ * Copyright (C) 2015 Imagination Technologies Ltd.
+ */
+
+#include <asm/asm.h>
+#include <asm/asm-offsets.h>
+#include <asm/asmmacro.h>
+#include <asm/regdef.h>
+
+ .set noreorder
+ .set noat
+
+LEAF(__kvm_save_msa)
+ st_d 0, VCPU_FPR0, a0
+ st_d 1, VCPU_FPR1, a0
+ st_d 2, VCPU_FPR2, a0
+ st_d 3, VCPU_FPR3, a0
+ st_d 4, VCPU_FPR4, a0
+ st_d 5, VCPU_FPR5, a0
+ st_d 6, VCPU_FPR6, a0
+ st_d 7, VCPU_FPR7, a0
+ st_d 8, VCPU_FPR8, a0
+ st_d 9, VCPU_FPR9, a0
+ st_d 10, VCPU_FPR10, a0
+ st_d 11, VCPU_FPR11, a0
+ st_d 12, VCPU_FPR12, a0
+ st_d 13, VCPU_FPR13, a0
+ st_d 14, VCPU_FPR14, a0
+ st_d 15, VCPU_FPR15, a0
+ st_d 16, VCPU_FPR16, a0
+ st_d 17, VCPU_FPR17, a0
+ st_d 18, VCPU_FPR18, a0
+ st_d 19, VCPU_FPR19, a0
+ st_d 20, VCPU_FPR20, a0
+ st_d 21, VCPU_FPR21, a0
+ st_d 22, VCPU_FPR22, a0
+ st_d 23, VCPU_FPR23, a0
+ st_d 24, VCPU_FPR24, a0
+ st_d 25, VCPU_FPR25, a0
+ st_d 26, VCPU_FPR26, a0
+ st_d 27, VCPU_FPR27, a0
+ st_d 28, VCPU_FPR28, a0
+ st_d 29, VCPU_FPR29, a0
+ st_d 30, VCPU_FPR30, a0
+ st_d 31, VCPU_FPR31, a0
+ jr ra
+ nop
+ END(__kvm_save_msa)
+
+LEAF(__kvm_restore_msa)
+ ld_d 0, VCPU_FPR0, a0
+ ld_d 1, VCPU_FPR1, a0
+ ld_d 2, VCPU_FPR2, a0
+ ld_d 3, VCPU_FPR3, a0
+ ld_d 4, VCPU_FPR4, a0
+ ld_d 5, VCPU_FPR5, a0
+ ld_d 6, VCPU_FPR6, a0
+ ld_d 7, VCPU_FPR7, a0
+ ld_d 8, VCPU_FPR8, a0
+ ld_d 9, VCPU_FPR9, a0
+ ld_d 10, VCPU_FPR10, a0
+ ld_d 11, VCPU_FPR11, a0
+ ld_d 12, VCPU_FPR12, a0
+ ld_d 13, VCPU_FPR13, a0
+ ld_d 14, VCPU_FPR14, a0
+ ld_d 15, VCPU_FPR15, a0
+ ld_d 16, VCPU_FPR16, a0
+ ld_d 17, VCPU_FPR17, a0
+ ld_d 18, VCPU_FPR18, a0
+ ld_d 19, VCPU_FPR19, a0
+ ld_d 20, VCPU_FPR20, a0
+ ld_d 21, VCPU_FPR21, a0
+ ld_d 22, VCPU_FPR22, a0
+ ld_d 23, VCPU_FPR23, a0
+ ld_d 24, VCPU_FPR24, a0
+ ld_d 25, VCPU_FPR25, a0
+ ld_d 26, VCPU_FPR26, a0
+ ld_d 27, VCPU_FPR27, a0
+ ld_d 28, VCPU_FPR28, a0
+ ld_d 29, VCPU_FPR29, a0
+ ld_d 30, VCPU_FPR30, a0
+ ld_d 31, VCPU_FPR31, a0
+ jr ra
+ nop
+ END(__kvm_restore_msa)
+
+ .macro kvm_restore_msa_upper wr, off, base
+ .set push
+ .set noat
+#ifdef CONFIG_64BIT
+ ld $1, \off(\base)
+ insert_d \wr, 1
+#elif defined(CONFIG_CPU_LITTLE_ENDIAN)
+ lw $1, \off(\base)
+ insert_w \wr, 2
+ lw $1, (\off+4)(\base)
+ insert_w \wr, 3
+#else /* CONFIG_CPU_BIG_ENDIAN */
+ lw $1, (\off+4)(\base)
+ insert_w \wr, 2
+ lw $1, \off(\base)
+ insert_w \wr, 3
+#endif
+ .set pop
+ .endm
+
+LEAF(__kvm_restore_msa_upper)
+ kvm_restore_msa_upper 0, VCPU_FPR0 +8, a0
+ kvm_restore_msa_upper 1, VCPU_FPR1 +8, a0
+ kvm_restore_msa_upper 2, VCPU_FPR2 +8, a0
+ kvm_restore_msa_upper 3, VCPU_FPR3 +8, a0
+ kvm_restore_msa_upper 4, VCPU_FPR4 +8, a0
+ kvm_restore_msa_upper 5, VCPU_FPR5 +8, a0
+ kvm_restore_msa_upper 6, VCPU_FPR6 +8, a0
+ kvm_restore_msa_upper 7, VCPU_FPR7 +8, a0
+ kvm_restore_msa_upper 8, VCPU_FPR8 +8, a0
+ kvm_restore_msa_upper 9, VCPU_FPR9 +8, a0
+ kvm_restore_msa_upper 10, VCPU_FPR10+8, a0
+ kvm_restore_msa_upper 11, VCPU_FPR11+8, a0
+ kvm_restore_msa_upper 12, VCPU_FPR12+8, a0
+ kvm_restore_msa_upper 13, VCPU_FPR13+8, a0
+ kvm_restore_msa_upper 14, VCPU_FPR14+8, a0
+ kvm_restore_msa_upper 15, VCPU_FPR15+8, a0
+ kvm_restore_msa_upper 16, VCPU_FPR16+8, a0
+ kvm_restore_msa_upper 17, VCPU_FPR17+8, a0
+ kvm_restore_msa_upper 18, VCPU_FPR18+8, a0
+ kvm_restore_msa_upper 19, VCPU_FPR19+8, a0
+ kvm_restore_msa_upper 20, VCPU_FPR20+8, a0
+ kvm_restore_msa_upper 21, VCPU_FPR21+8, a0
+ kvm_restore_msa_upper 22, VCPU_FPR22+8, a0
+ kvm_restore_msa_upper 23, VCPU_FPR23+8, a0
+ kvm_restore_msa_upper 24, VCPU_FPR24+8, a0
+ kvm_restore_msa_upper 25, VCPU_FPR25+8, a0
+ kvm_restore_msa_upper 26, VCPU_FPR26+8, a0
+ kvm_restore_msa_upper 27, VCPU_FPR27+8, a0
+ kvm_restore_msa_upper 28, VCPU_FPR28+8, a0
+ kvm_restore_msa_upper 29, VCPU_FPR29+8, a0
+ kvm_restore_msa_upper 30, VCPU_FPR30+8, a0
+ kvm_restore_msa_upper 31, VCPU_FPR31+8, a0
+ jr ra
+ nop
+ END(__kvm_restore_msa_upper)
+
+LEAF(__kvm_restore_msacsr)
+ lw t0, VCPU_MSA_CSR(a0)
+ /*
+ * The ctcmsa must stay at this offset in __kvm_restore_msacsr.
+ * See kvm_mips_csr_die_notify() which handles t0 containing a value
+ * which triggers an MSA FP Exception, which must be stepped over and
+ * ignored since the set cause bits must remain there for the guest.
+ */
+ _ctcmsa MSA_CSR, t0
+ jr ra
+ nop
+ END(__kvm_restore_msacsr)
"System Call",
"Reserved Inst",
"Break Inst",
+ "Trap Inst",
+ "MSA FPE",
+ "FPE",
+ "MSA Disabled",
"D-Cache Flushes",
};
if (idx > current_cpu_data.tlbsize) {
kvm_err("%s: Invalid Index: %d\n", __func__, idx);
kvm_mips_dump_host_tlbs();
+ local_irq_restore(flags);
return -1;
}
}
}
+ /* restore guest state to registers */
+ kvm_mips_callbacks->vcpu_set_regs(vcpu);
+
local_irq_restore(flags);
}
vcpu->arch.preempt_entryhi = read_c0_entryhi();
vcpu->arch.last_sched_cpu = cpu;
+ /* save guest state in registers */
+ kvm_mips_callbacks->vcpu_get_regs(vcpu);
+
if (((cpu_context(cpu, current->mm) ^ asid_cache(cpu)) &
ASID_VERSION_MASK)) {
kvm_debug("%s: Dropping MMU Context: %#lx\n", __func__,
TP_PROTO(struct kvm_vcpu *vcpu, unsigned int reason),
TP_ARGS(vcpu, reason),
TP_STRUCT__entry(
- __field(struct kvm_vcpu *, vcpu)
+ __field(unsigned long, pc)
__field(unsigned int, reason)
),
TP_fast_assign(
- __entry->vcpu = vcpu;
+ __entry->pc = vcpu->arch.pc;
__entry->reason = reason;
),
TP_printk("[%s]PC: 0x%08lx",
kvm_mips_exit_types_str[__entry->reason],
- __entry->vcpu->arch.pc)
+ __entry->pc)
);
#endif /* _TRACE_KVM_H */
static int kvm_trap_emul_handle_cop_unusable(struct kvm_vcpu *vcpu)
{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
struct kvm_run *run = vcpu->run;
uint32_t __user *opc = (uint32_t __user *) vcpu->arch.pc;
unsigned long cause = vcpu->arch.host_cp0_cause;
enum emulation_result er = EMULATE_DONE;
int ret = RESUME_GUEST;
- if (((cause & CAUSEF_CE) >> CAUSEB_CE) == 1)
- er = kvm_mips_emulate_fpu_exc(cause, opc, run, vcpu);
- else
+ if (((cause & CAUSEF_CE) >> CAUSEB_CE) == 1) {
+ /* FPU Unusable */
+ if (!kvm_mips_guest_has_fpu(&vcpu->arch) ||
+ (kvm_read_c0_guest_status(cop0) & ST0_CU1) == 0) {
+ /*
+ * Unusable/no FPU in guest:
+ * deliver guest COP1 Unusable Exception
+ */
+ er = kvm_mips_emulate_fpu_exc(cause, opc, run, vcpu);
+ } else {
+ /* Restore FPU state */
+ kvm_own_fpu(vcpu);
+ er = EMULATE_DONE;
+ }
+ } else {
er = kvm_mips_emulate_inst(cause, opc, run, vcpu);
+ }
switch (er) {
case EMULATE_DONE:
return ret;
}
+static int kvm_trap_emul_handle_trap(struct kvm_vcpu *vcpu)
+{
+ struct kvm_run *run = vcpu->run;
+ uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
+ unsigned long cause = vcpu->arch.host_cp0_cause;
+ enum emulation_result er = EMULATE_DONE;
+ int ret = RESUME_GUEST;
+
+ er = kvm_mips_emulate_trap_exc(cause, opc, run, vcpu);
+ if (er == EMULATE_DONE) {
+ ret = RESUME_GUEST;
+ } else {
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ }
+ return ret;
+}
+
+static int kvm_trap_emul_handle_msa_fpe(struct kvm_vcpu *vcpu)
+{
+ struct kvm_run *run = vcpu->run;
+ uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
+ unsigned long cause = vcpu->arch.host_cp0_cause;
+ enum emulation_result er = EMULATE_DONE;
+ int ret = RESUME_GUEST;
+
+ er = kvm_mips_emulate_msafpe_exc(cause, opc, run, vcpu);
+ if (er == EMULATE_DONE) {
+ ret = RESUME_GUEST;
+ } else {
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ }
+ return ret;
+}
+
+static int kvm_trap_emul_handle_fpe(struct kvm_vcpu *vcpu)
+{
+ struct kvm_run *run = vcpu->run;
+ uint32_t __user *opc = (uint32_t __user *)vcpu->arch.pc;
+ unsigned long cause = vcpu->arch.host_cp0_cause;
+ enum emulation_result er = EMULATE_DONE;
+ int ret = RESUME_GUEST;
+
+ er = kvm_mips_emulate_fpe_exc(cause, opc, run, vcpu);
+ if (er == EMULATE_DONE) {
+ ret = RESUME_GUEST;
+ } else {
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ }
+ return ret;
+}
+
+/**
+ * kvm_trap_emul_handle_msa_disabled() - Guest used MSA while disabled in root.
+ * @vcpu: Virtual CPU context.
+ *
+ * Handle when the guest attempts to use MSA when it is disabled.
+ */
+static int kvm_trap_emul_handle_msa_disabled(struct kvm_vcpu *vcpu)
+{
+ struct mips_coproc *cop0 = vcpu->arch.cop0;
+ struct kvm_run *run = vcpu->run;
+ uint32_t __user *opc = (uint32_t __user *) vcpu->arch.pc;
+ unsigned long cause = vcpu->arch.host_cp0_cause;
+ enum emulation_result er = EMULATE_DONE;
+ int ret = RESUME_GUEST;
+
+ if (!kvm_mips_guest_has_msa(&vcpu->arch) ||
+ (kvm_read_c0_guest_status(cop0) & (ST0_CU1 | ST0_FR)) == ST0_CU1) {
+ /*
+ * No MSA in guest, or FPU enabled and not in FR=1 mode,
+ * guest reserved instruction exception
+ */
+ er = kvm_mips_emulate_ri_exc(cause, opc, run, vcpu);
+ } else if (!(kvm_read_c0_guest_config5(cop0) & MIPS_CONF5_MSAEN)) {
+ /* MSA disabled by guest, guest MSA disabled exception */
+ er = kvm_mips_emulate_msadis_exc(cause, opc, run, vcpu);
+ } else {
+ /* Restore MSA/FPU state */
+ kvm_own_msa(vcpu);
+ er = EMULATE_DONE;
+ }
+
+ switch (er) {
+ case EMULATE_DONE:
+ ret = RESUME_GUEST;
+ break;
+
+ case EMULATE_FAIL:
+ run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
+ ret = RESUME_HOST;
+ break;
+
+ default:
+ BUG();
+ }
+ return ret;
+}
+
static int kvm_trap_emul_vm_init(struct kvm *kvm)
{
return 0;
* guest will come up as expected, for now we simulate a MIPS 24kc
*/
kvm_write_c0_guest_prid(cop0, 0x00019300);
- kvm_write_c0_guest_config(cop0,
- MIPS_CONFIG0 | (0x1 << CP0C0_AR) |
+ /* Have config1, Cacheable, noncoherent, write-back, write allocate */
+ kvm_write_c0_guest_config(cop0, MIPS_CONF_M | (0x3 << CP0C0_K0) |
+ (0x1 << CP0C0_AR) |
(MMU_TYPE_R4000 << CP0C0_MT));
/* Read the cache characteristics from the host Config1 Register */
(1 << CP0C1_WR) | (1 << CP0C1_CA));
kvm_write_c0_guest_config1(cop0, config1);
- kvm_write_c0_guest_config2(cop0, MIPS_CONFIG2);
- /* MIPS_CONFIG2 | (read_c0_config2() & 0xfff) */
- kvm_write_c0_guest_config3(cop0, MIPS_CONFIG3 | (0 << CP0C3_VInt) |
- (1 << CP0C3_ULRI));
+ /* Have config3, no tertiary/secondary caches implemented */
+ kvm_write_c0_guest_config2(cop0, MIPS_CONF_M);
+ /* MIPS_CONF_M | (read_c0_config2() & 0xfff) */
+
+ /* Have config4, UserLocal */
+ kvm_write_c0_guest_config3(cop0, MIPS_CONF_M | MIPS_CONF3_ULRI);
+
+ /* Have config5 */
+ kvm_write_c0_guest_config4(cop0, MIPS_CONF_M);
+
+ /* No config6 */
+ kvm_write_c0_guest_config5(cop0, 0);
/* Set Wait IE/IXMT Ignore in Config7, IAR, AR */
kvm_write_c0_guest_config7(cop0, (MIPS_CONF7_WII) | (1 << 10));
{
struct mips_coproc *cop0 = vcpu->arch.cop0;
int ret = 0;
+ unsigned int cur, change;
switch (reg->id) {
case KVM_REG_MIPS_CP0_COUNT:
kvm_write_c0_guest_cause(cop0, v);
}
break;
+ case KVM_REG_MIPS_CP0_CONFIG:
+ /* read-only for now */
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG1:
+ cur = kvm_read_c0_guest_config1(cop0);
+ change = (cur ^ v) & kvm_mips_config1_wrmask(vcpu);
+ if (change) {
+ v = cur ^ change;
+ kvm_write_c0_guest_config1(cop0, v);
+ }
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG2:
+ /* read-only for now */
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG3:
+ cur = kvm_read_c0_guest_config3(cop0);
+ change = (cur ^ v) & kvm_mips_config3_wrmask(vcpu);
+ if (change) {
+ v = cur ^ change;
+ kvm_write_c0_guest_config3(cop0, v);
+ }
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG4:
+ cur = kvm_read_c0_guest_config4(cop0);
+ change = (cur ^ v) & kvm_mips_config4_wrmask(vcpu);
+ if (change) {
+ v = cur ^ change;
+ kvm_write_c0_guest_config4(cop0, v);
+ }
+ break;
+ case KVM_REG_MIPS_CP0_CONFIG5:
+ cur = kvm_read_c0_guest_config5(cop0);
+ change = (cur ^ v) & kvm_mips_config5_wrmask(vcpu);
+ if (change) {
+ v = cur ^ change;
+ kvm_write_c0_guest_config5(cop0, v);
+ }
+ break;
case KVM_REG_MIPS_COUNT_CTL:
ret = kvm_mips_set_count_ctl(vcpu, v);
break;
return ret;
}
+static int kvm_trap_emul_vcpu_get_regs(struct kvm_vcpu *vcpu)
+{
+ kvm_lose_fpu(vcpu);
+
+ return 0;
+}
+
+static int kvm_trap_emul_vcpu_set_regs(struct kvm_vcpu *vcpu)
+{
+ return 0;
+}
+
static struct kvm_mips_callbacks kvm_trap_emul_callbacks = {
/* exit handlers */
.handle_cop_unusable = kvm_trap_emul_handle_cop_unusable,
.handle_syscall = kvm_trap_emul_handle_syscall,
.handle_res_inst = kvm_trap_emul_handle_res_inst,
.handle_break = kvm_trap_emul_handle_break,
+ .handle_trap = kvm_trap_emul_handle_trap,
+ .handle_msa_fpe = kvm_trap_emul_handle_msa_fpe,
+ .handle_fpe = kvm_trap_emul_handle_fpe,
+ .handle_msa_disabled = kvm_trap_emul_handle_msa_disabled,
.vm_init = kvm_trap_emul_vm_init,
.vcpu_init = kvm_trap_emul_vcpu_init,
.irq_clear = kvm_mips_irq_clear_cb,
.get_one_reg = kvm_trap_emul_get_one_reg,
.set_one_reg = kvm_trap_emul_set_one_reg,
+ .vcpu_get_regs = kvm_trap_emul_vcpu_get_regs,
+ .vcpu_set_regs = kvm_trap_emul_vcpu_set_regs,
};
int kvm_mips_emulation_init(struct kvm_mips_callbacks **install_callbacks)
int proc_dolasatrtc(struct ctl_table *table, int write,
void *buffer, size_t *lenp, loff_t *ppos)
{
- struct timespec ts;
+ struct timespec64 ts;
int r;
if (!write) {
- read_persistent_clock(&ts);
+ read_persistent_clock64(&ts);
rtctmp = ts.tv_sec;
/* check for time < 0 and set to 0 */
if (rtctmp < 0)
static struct irqaction hpet_irq = {
.handler = hpet_irq_handler,
- .flags = IRQF_DISABLED | IRQF_NOBALANCING | IRQF_TIMER,
+ .flags = IRQF_NOBALANCING | IRQF_TIMER,
.name = "hpet",
};
pci_add_resource_offset(&resources, hose->io_resource, hose->io_offset);
bus = pci_scan_root_bus(NULL, next_busno, hose->pci_ops, hose,
&resources);
- if (!bus)
- pci_free_resource_list(&resources);
-
hose->bus = bus;
need_domain_info = need_domain_info || hose->index;
hose->need_domain_info = need_domain_info;
- if (bus) {
- next_busno = bus->busn_res.end + 1;
- /* Don't allow 8-bit bus number overflow inside the hose -
- reserve some space for bridges. */
- if (next_busno > 224) {
- next_busno = 0;
- need_domain_info = 1;
- }
- if (!pci_has_flag(PCI_PROBE_ONLY)) {
- pci_bus_size_bridges(bus);
- pci_bus_assign_resources(bus);
- }
+ if (!bus) {
+ pci_free_resource_list(&resources);
+ return;
+ }
+
+ next_busno = bus->busn_res.end + 1;
+ /* Don't allow 8-bit bus number overflow inside the hose -
+ reserve some space for bridges. */
+ if (next_busno > 224) {
+ next_busno = 0;
+ need_domain_info = 1;
+ }
+
+ if (!pci_has_flag(PCI_PROBE_ONLY)) {
+ pci_bus_size_bridges(bus);
+ pci_bus_assign_resources(bus);
}
+ pci_bus_add_devices(bus);
}
#ifdef CONFIG_OF
#define PGDIR_SHIFT 22
#define PTRS_PER_PGD 1024
#define PTRS_PER_PUD 1 /* we don't really have any PUD physically */
+#define __PAGETABLE_PUD_FOLDED
#define PTRS_PER_PMD 1 /* we don't really have any PMD physically */
+#define __PAGETABLE_PMD_FOLDED
#define PTRS_PER_PTE 1024
#define PGD_SIZE PAGE_SIZE
{
resource_size_t io_offset, mem_offset;
LIST_HEAD(resources);
+ struct pci_bus *bus;
ioport_resource.start = 0xA0000000;
ioport_resource.end = 0xDFFFFFFF;
pci_add_resource_offset(&resources, &pci_ioport_resource, io_offset);
pci_add_resource_offset(&resources, &pci_iomem_resource, mem_offset);
- pci_scan_root_bus(NULL, 0, &pci_direct_ampci, NULL, &resources);
+ bus = pci_scan_root_bus(NULL, 0, &pci_direct_ampci, NULL, &resources);
+ if (!bus)
+ return 0;
pcibios_irq_init();
pcibios_fixup_irqs();
pcibios_resource_survey();
+ pci_bus_add_devices(bus);
return 0;
}
#include <uapi/asm/ptrace.h>
+/* This struct defines the way the registers are stored on the
+ stack during a system call. */
+
#ifndef __ASSEMBLY__
+struct pt_regs {
+ unsigned long r8; /* r8-r15 Caller-saved GP registers */
+ unsigned long r9;
+ unsigned long r10;
+ unsigned long r11;
+ unsigned long r12;
+ unsigned long r13;
+ unsigned long r14;
+ unsigned long r15;
+ unsigned long r1; /* Assembler temporary */
+ unsigned long r2; /* Retval LS 32bits */
+ unsigned long r3; /* Retval MS 32bits */
+ unsigned long r4; /* r4-r7 Register arguments */
+ unsigned long r5;
+ unsigned long r6;
+ unsigned long r7;
+ unsigned long orig_r2; /* Copy of r2 ?? */
+ unsigned long ra; /* Return address */
+ unsigned long fp; /* Frame pointer */
+ unsigned long sp; /* Stack pointer */
+ unsigned long gp; /* Global pointer */
+ unsigned long estatus;
+ unsigned long ea; /* Exception return address (pc) */
+ unsigned long orig_r7;
+};
+
+/*
+ * This is the extended stack used by signal handlers and the context
+ * switcher: it's pushed after the normal "struct pt_regs".
+ */
+struct switch_stack {
+ unsigned long r16; /* r16-r23 Callee-saved GP registers */
+ unsigned long r17;
+ unsigned long r18;
+ unsigned long r19;
+ unsigned long r20;
+ unsigned long r21;
+ unsigned long r22;
+ unsigned long r23;
+ unsigned long fp;
+ unsigned long gp;
+ unsigned long ra;
+};
+
#define user_mode(regs) (((regs)->estatus & ESTATUS_EU))
#define instruction_pointer(regs) ((regs)->ra)
0-0x7FFFFFFF for user-thead
0-0xFFFFFFFF for kernel-thread
*/
- struct restart_block restart_block;
struct pt_regs *regs;
};
.cpu = 0, \
.preempt_count = INIT_PREEMPT_COUNT, \
.addr_limit = KERNEL_DS, \
- .restart_block = { \
- .fn = do_no_restart_syscall, \
- }, \
}
#define init_thread_info (init_thread_union.thread_info)
+++ /dev/null
-/*
- * Copyright (C) 2010 Tobias Klauser <tklauser@distanz.ch>
- * Copyright (C) 2004 Microtronix Datacom Ltd
- *
- * This file is subject to the terms and conditions of the GNU General Public
- * License. See the file "COPYING" in the main directory of this archive
- * for more details.
- */
-
-#ifndef _ASM_NIOS2_UCONTEXT_H
-#define _ASM_NIOS2_UCONTEXT_H
-
-typedef int greg_t;
-#define NGREG 32
-typedef greg_t gregset_t[NGREG];
-
-struct mcontext {
- int version;
- gregset_t gregs;
-};
-
-#define MCONTEXT_VERSION 2
-
-struct ucontext {
- unsigned long uc_flags;
- struct ucontext *uc_link;
- stack_t uc_stack;
- struct mcontext uc_mcontext;
- sigset_t uc_sigmask; /* mask last for extensibility */
-};
-
-#endif
include include/uapi/asm-generic/Kbuild.asm
header-y += elf.h
-header-y += ucontext.h
+
+generic-y += ucontext.h
typedef unsigned long elf_greg_t;
-#define ELF_NGREG \
- ((sizeof(struct pt_regs) + sizeof(struct switch_stack)) / \
- sizeof(elf_greg_t))
+#define ELF_NGREG 49
typedef elf_greg_t elf_gregset_t[ELF_NGREG];
typedef unsigned long elf_fpregset_t;
#define PTR_IPENDING 37
#define PTR_CPUID 38
#define PTR_CTL6 39
-#define PTR_CTL7 40
+#define PTR_EXCEPTION 40
#define PTR_PTEADDR 41
#define PTR_TLBACC 42
#define PTR_TLBMISC 43
+#define PTR_ECCINJ 44
+#define PTR_BADADDR 45
+#define PTR_CONFIG 46
+#define PTR_MPUBASE 47
+#define PTR_MPUACC 48
-#define NUM_PTRACE_REG (PTR_TLBMISC + 1)
+#define NUM_PTRACE_REG (PTR_MPUACC + 1)
-/* this struct defines the way the registers are stored on the
- stack during a system call.
-
- There is a fake_regs in setup.c that has to match pt_regs.*/
-
-struct pt_regs {
- unsigned long r8; /* r8-r15 Caller-saved GP registers */
- unsigned long r9;
- unsigned long r10;
- unsigned long r11;
- unsigned long r12;
- unsigned long r13;
- unsigned long r14;
- unsigned long r15;
- unsigned long r1; /* Assembler temporary */
- unsigned long r2; /* Retval LS 32bits */
- unsigned long r3; /* Retval MS 32bits */
- unsigned long r4; /* r4-r7 Register arguments */
- unsigned long r5;
- unsigned long r6;
- unsigned long r7;
- unsigned long orig_r2; /* Copy of r2 ?? */
- unsigned long ra; /* Return address */
- unsigned long fp; /* Frame pointer */
- unsigned long sp; /* Stack pointer */
- unsigned long gp; /* Global pointer */
- unsigned long estatus;
- unsigned long ea; /* Exception return address (pc) */
- unsigned long orig_r7;
-};
-
-/*
- * This is the extended stack used by signal handlers and the context
- * switcher: it's pushed after the normal "struct pt_regs".
- */
-struct switch_stack {
- unsigned long r16; /* r16-r23 Callee-saved GP registers */
- unsigned long r17;
- unsigned long r18;
- unsigned long r19;
- unsigned long r20;
- unsigned long r21;
- unsigned long r22;
- unsigned long r23;
- unsigned long fp;
- unsigned long gp;
- unsigned long ra;
+/* User structures for general purpose registers. */
+struct user_pt_regs {
+ __u32 regs[49];
};
#endif /* __ASSEMBLY__ */
* details.
*/
-#ifndef _ASM_NIOS2_SIGCONTEXT_H
-#define _ASM_NIOS2_SIGCONTEXT_H
+#ifndef _UAPI__ASM_SIGCONTEXT_H
+#define _UAPI__ASM_SIGCONTEXT_H
-#include <asm/ptrace.h>
+#include <linux/types.h>
+
+#define MCONTEXT_VERSION 2
struct sigcontext {
- struct pt_regs regs;
- unsigned long sc_mask; /* old sigmask */
+ int version;
+ unsigned long gregs[32];
};
#endif
***********************************************************************
*/
ENTRY(handle_trap)
- ldw r24, -4(ea) /* instruction that caused the exception */
+ ldwio r24, -4(ea) /* instruction that caused the exception */
srli r24, r24, 4
andi r24, r24, 0x7c
movia r9,trap_table
struct ucontext *uc, int *pr2)
{
int temp;
- greg_t *gregs = uc->uc_mcontext.gregs;
+ unsigned long *gregs = uc->uc_mcontext.gregs;
int err;
/* Always make any pending restarted system calls return -EINTR */
- current_thread_info()->restart_block.fn = do_no_restart_syscall;
+ current->restart_block.fn = do_no_restart_syscall;
err = __get_user(temp, &uc->uc_mcontext.version);
if (temp != MCONTEXT_VERSION)
static inline int rt_setup_ucontext(struct ucontext *uc, struct pt_regs *regs)
{
struct switch_stack *sw = (struct switch_stack *)regs - 1;
- greg_t *gregs = uc->uc_mcontext.gregs;
+ unsigned long *gregs = uc->uc_mcontext.gregs;
int err = 0;
err |= __put_user(MCONTEXT_VERSION, &uc->uc_mcontext.version);
end += (cpuinfo.dcache_line_size - 1);
end &= ~(cpuinfo.dcache_line_size - 1);
- if (end > start + cpuinfo.dcache_size)
- end = start + cpuinfo.dcache_size;
-
for (addr = start; addr < end; addr += cpuinfo.dcache_line_size) {
__asm__ __volatile__ (" flushda 0(%0)\n"
: /* Outputs */
break;
}
-survive:
/*
* If for any reason at all we couldn't handle the fault,
* make sure we exit gracefully rather than endlessly redo
*/
out_of_memory:
up_read(&mm->mmap_sem);
- if (is_global_init(tsk)) {
- yield();
- down_read(&mm->mmap_sem);
- goto survive;
- }
if (!user_mode(regs))
goto no_context;
pagefault_out_of_memory();
if (likely(pgd != NULL)) {
memset(pgd, 0, PAGE_SIZE<<PGD_ALLOC_ORDER);
-#ifdef CONFIG_64BIT
+#if PT_NLEVELS == 3
actual_pgd += PTRS_PER_PGD;
/* Populate first pmd with allocated memory. We mark it
* with PxD_FLAG_ATTACHED as a signal to the system that this
static inline void pgd_free(struct mm_struct *mm, pgd_t *pgd)
{
-#ifdef CONFIG_64BIT
+#if PT_NLEVELS == 3
pgd -= PTRS_PER_PGD;
#endif
free_pages((unsigned long)pgd, PGD_ALLOC_ORDER);
static inline void pmd_free(struct mm_struct *mm, pmd_t *pmd)
{
-#ifdef CONFIG_64BIT
if(pmd_flag(*pmd) & PxD_FLAG_ATTACHED)
- /* This is the permanent pmd attached to the pgd;
- * cannot free it */
+ /*
+ * This is the permanent pmd attached to the pgd;
+ * cannot free it.
+ * Increment the counter to compensate for the decrement
+ * done by generic mm code.
+ */
+ mm_inc_nr_pmds(mm);
return;
-#endif
free_pages((unsigned long)pmd, PMD_ORDER);
}
static inline void
pmd_populate_kernel(struct mm_struct *mm, pmd_t *pmd, pte_t *pte)
{
-#ifdef CONFIG_64BIT
+#if PT_NLEVELS == 3
/* preserve the gateway marker if this is the beginning of
* the permanent pmd */
if(pmd_flag(*pmd) & PxD_FLAG_ATTACHED)
#if PT_NLEVELS == 3
#define BITS_PER_PMD (PAGE_SHIFT + PMD_ORDER - BITS_PER_PMD_ENTRY)
#else
+#define __PAGETABLE_PMD_FOLDED
#define BITS_PER_PMD 0
#endif
#define PTRS_PER_PMD (1UL << BITS_PER_PMD)
#define ENTRY_COMP(_name_) .word sys_##_name_
#endif
- ENTRY_SAME(restart_syscall) /* 0 */
- ENTRY_SAME(exit)
+90: ENTRY_SAME(restart_syscall) /* 0 */
+91: ENTRY_SAME(exit)
ENTRY_SAME(fork_wrapper)
ENTRY_SAME(read)
ENTRY_SAME(write)
ENTRY_SAME(bpf)
ENTRY_COMP(execveat)
- /* Nothing yet */
+
+.ifne (. - 90b) - (__NR_Linux_syscalls * (91b - 90b))
+.error "size of syscall table does not fit value of __NR_Linux_syscalls"
+.endif
#undef ENTRY_SAME
#undef ENTRY_DIFF
static inline int cpu_nr_cores(void)
{
- return NR_CPUS >> threads_shift;
+ return nr_cpu_ids >> threads_shift;
}
static inline cpumask_t cpu_online_cores_map(void)
int pci_domain_number, unsigned long pe_num);
extern int iommu_add_device(struct device *dev);
extern void iommu_del_device(struct device *dev);
+extern int __init tce_iommu_bus_notifier_init(void);
#else
static inline void iommu_register_group(struct iommu_table *tbl,
int pci_domain_number,
static inline void iommu_del_device(struct device *dev)
{
}
+
+static inline int __init tce_iommu_bus_notifier_init(void)
+{
+ return 0;
+}
#endif /* !CONFIG_IOMMU_API */
static inline void set_iommu_table_base_and_group(struct device *dev,
--- /dev/null
+#ifndef _ASM_POWERPC_IRQ_WORK_H
+#define _ASM_POWERPC_IRQ_WORK_H
+
+static inline bool arch_irq_work_has_interrupt(void)
+{
+ return true;
+}
+
+#endif /* _ASM_POWERPC_IRQ_WORK_H */
#define PPC_INST_MFSPR_PVR_MASK 0xfc1fffff
#define PPC_INST_MFTMR 0x7c0002dc
#define PPC_INST_MSGSND 0x7c00019c
+#define PPC_INST_MSGCLR 0x7c0001dc
#define PPC_INST_MSGSNDP 0x7c00011c
#define PPC_INST_MTTMR 0x7c0003dc
#define PPC_INST_NOP 0x60000000
___PPC_RB(b) | __PPC_EH(eh))
#define PPC_MSGSND(b) stringify_in_c(.long PPC_INST_MSGSND | \
___PPC_RB(b))
+#define PPC_MSGCLR(b) stringify_in_c(.long PPC_INST_MSGCLR | \
+ ___PPC_RB(b))
#define PPC_MSGSNDP(b) stringify_in_c(.long PPC_INST_MSGSNDP | \
___PPC_RB(b))
#define PPC_POPCNTB(a, s) stringify_in_c(.long PPC_INST_POPCNTB | \
#define SRR1_ISI_N_OR_G 0x10000000 /* ISI: Access is no-exec or G */
#define SRR1_ISI_PROT 0x08000000 /* ISI: Other protection fault */
#define SRR1_WAKEMASK 0x00380000 /* reason for wakeup */
+#define SRR1_WAKEMASK_P8 0x003c0000 /* reason for wakeup on POWER8 */
#define SRR1_WAKESYSERR 0x00300000 /* System error */
#define SRR1_WAKEEE 0x00200000 /* External interrupt */
#define SRR1_WAKEMT 0x00280000 /* mtctrl */
#define SRR1_WAKEHMI 0x00280000 /* Hypervisor maintenance */
#define SRR1_WAKEDEC 0x00180000 /* Decrementer interrupt */
+#define SRR1_WAKEDBELL 0x00140000 /* Privileged doorbell on P8 */
#define SRR1_WAKETHERM 0x00100000 /* Thermal management interrupt */
#define SRR1_WAKERESET 0x00100000 /* System reset */
+#define SRR1_WAKEHDBELL 0x000c0000 /* Hypervisor doorbell on P8 */
#define SRR1_WAKESTATE 0x00030000 /* Powersave exit mask [46:47] */
#define SRR1_WS_DEEPEST 0x00030000 /* Some resources not maintained,
* may not be recoverable */
.machine_check_early = __machine_check_early_realmode_p8,
.platform = "power8",
},
+ { /* Power8NVL */
+ .pvr_mask = 0xffff0000,
+ .pvr_value = 0x004c0000,
+ .cpu_name = "POWER8NVL (raw)",
+ .cpu_features = CPU_FTRS_POWER8,
+ .cpu_user_features = COMMON_USER_POWER8,
+ .cpu_user_features2 = COMMON_USER2_POWER8,
+ .mmu_features = MMU_FTRS_POWER8,
+ .icache_bsize = 128,
+ .dcache_bsize = 128,
+ .num_pmcs = 6,
+ .pmc_type = PPC_PMC_IBM,
+ .oprofile_cpu_type = "ppc64/power8",
+ .oprofile_type = PPC_OPROFILE_INVALID,
+ .cpu_setup = __setup_cpu_power8,
+ .cpu_restore = __restore_cpu_power8,
+ .flush_tlb = __flush_tlb_power8,
+ .machine_check_early = __machine_check_early_realmode_p8,
+ .platform = "power8",
+ },
{ /* Power8 DD1: Does not support doorbell IPIs */
.pvr_mask = 0xffffff00,
.pvr_value = 0x004d0100,
#include <asm/dbell.h>
#include <asm/irq_regs.h>
+#include <asm/kvm_ppc.h>
#ifdef CONFIG_SMP
void doorbell_setup_this_cpu(void)
may_hard_irq_enable();
+ kvmppc_set_host_ipi(smp_processor_id(), 0);
__this_cpu_inc(irq_stat.doorbell_irqs);
smp_ipi_demux();
bne 9f /* continue in V mode if we are. */
5:
-#ifdef CONFIG_KVM_BOOK3S_64_HV
+#ifdef CONFIG_KVM_BOOK3S_64_HANDLER
/*
* We are coming from kernel context. Check if we are coming from
* guest. if yes, then we can continue. We will fall through
}
EXPORT_SYMBOL_GPL(iommu_del_device);
+static int tce_iommu_bus_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ struct device *dev = data;
+
+ switch (action) {
+ case BUS_NOTIFY_ADD_DEVICE:
+ return iommu_add_device(dev);
+ case BUS_NOTIFY_DEL_DEVICE:
+ if (dev->iommu_group)
+ iommu_del_device(dev);
+ return 0;
+ default:
+ return 0;
+ }
+}
+
+static struct notifier_block tce_iommu_bus_nb = {
+ .notifier_call = tce_iommu_bus_notifier,
+};
+
+int __init tce_iommu_bus_notifier_init(void)
+{
+ bus_register_notifier(&pci_bus_type, &tce_iommu_bus_nb);
+ return 0;
+}
#endif /* CONFIG_IOMMU_API */
if (smp_ops->give_timebase)
smp_ops->give_timebase();
- /* Wait until cpu puts itself in the online map */
- while (!cpu_online(cpu))
+ /* Wait until cpu puts itself in the online & active maps */
+ while (!cpu_online(cpu) || !cpu_active(cpu))
cpu_relax();
return 0;
spin_lock(&vcpu->arch.vpa_update_lock);
lppaca = (struct lppaca *)vcpu->arch.vpa.pinned_addr;
if (lppaca)
- yield_count = lppaca->yield_count;
+ yield_count = be32_to_cpu(lppaca->yield_count);
spin_unlock(&vcpu->arch.vpa_update_lock);
return yield_count;
}
static void kvmppc_set_lpcr(struct kvm_vcpu *vcpu, u64 new_lpcr,
bool preserve_top32)
{
+ struct kvm *kvm = vcpu->kvm;
struct kvmppc_vcore *vc = vcpu->arch.vcore;
u64 mask;
+ mutex_lock(&kvm->lock);
spin_lock(&vc->lock);
/*
* If ILE (interrupt little-endian) has changed, update the
* MSR_LE bit in the intr_msr for each vcpu in this vcore.
*/
if ((new_lpcr & LPCR_ILE) != (vc->lpcr & LPCR_ILE)) {
- struct kvm *kvm = vcpu->kvm;
struct kvm_vcpu *vcpu;
int i;
- mutex_lock(&kvm->lock);
kvm_for_each_vcpu(i, vcpu, kvm) {
if (vcpu->arch.vcore != vc)
continue;
else
vcpu->arch.intr_msr &= ~MSR_LE;
}
- mutex_unlock(&kvm->lock);
}
/*
mask &= 0xFFFFFFFF;
vc->lpcr = (vc->lpcr & ~mask) | (new_lpcr & mask);
spin_unlock(&vc->lock);
+ mutex_unlock(&kvm->lock);
}
static int kvmppc_get_one_reg_hv(struct kvm_vcpu *vcpu, u64 id,
/* Save HEIR (HV emulation assist reg) in emul_inst
if this is an HEI (HV emulation interrupt, e40) */
li r3,KVM_INST_FETCH_FAILED
+ stw r3,VCPU_LAST_INST(r9)
cmpwi r12,BOOK3S_INTERRUPT_H_EMUL_ASSIST
bne 11f
mfspr r3,SPRN_HEIR
#include <asm/kvm_para.h>
#include <asm/kvm_host.h>
#include <asm/kvm_ppc.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
#define MAX_CPU 32
#define MAX_SRC 256
clear_bit(n_IRQ, q->queue);
}
-static inline int IRQ_testbit(struct irq_queue *q, int n_IRQ)
-{
- return test_bit(n_IRQ, q->queue);
-}
-
static void IRQ_check(struct openpic *opp, struct irq_queue *q)
{
int irq = -1;
return -ENXIO;
}
-static int kvm_mpic_read(struct kvm_io_device *this, gpa_t addr,
- int len, void *ptr)
+static int kvm_mpic_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
return ret;
}
-static int kvm_mpic_write(struct kvm_io_device *this, gpa_t addr,
- int len, const void *ptr)
+static int kvm_mpic_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, const void *ptr)
{
struct openpic *opp = container_of(this, struct openpic, mmio);
int ret;
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_read(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
idx = srcu_read_lock(&vcpu->kvm->srcu);
- ret = kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, run->mmio.phys_addr,
+ ret = kvm_io_bus_write(vcpu, KVM_MMIO_BUS, run->mmio.phys_addr,
bytes, &run->mmio.data);
srcu_read_unlock(&vcpu->kvm->srcu, idx);
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/jump_label.h>
#include <asm/ppc_asm.h>
#include <asm/hvcall.h>
#include <asm/asm-offsets.h>
#include <asm/opal.h>
-#include <asm/jump_label.h>
.section ".text"
#endif
}
-static int tce_iommu_bus_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
-{
- struct device *dev = data;
-
- switch (action) {
- case BUS_NOTIFY_ADD_DEVICE:
- return iommu_add_device(dev);
- case BUS_NOTIFY_DEL_DEVICE:
- if (dev->iommu_group)
- iommu_del_device(dev);
- return 0;
- default:
- return 0;
- }
-}
-
-static struct notifier_block tce_iommu_bus_nb = {
- .notifier_call = tce_iommu_bus_notifier,
-};
-
-static int __init tce_iommu_bus_notifier_init(void)
-{
- bus_register_notifier(&pci_bus_type, &tce_iommu_bus_nb);
- return 0;
-}
machine_subsys_initcall_sync(powernv, tce_iommu_bus_notifier_init);
#include <asm/runlatch.h>
#include <asm/code-patching.h>
#include <asm/dbell.h>
+#include <asm/kvm_ppc.h>
+#include <asm/ppc-opcode.h>
#include "powernv.h"
static void pnv_smp_cpu_kill_self(void)
{
unsigned int cpu;
- unsigned long srr1;
+ unsigned long srr1, wmask;
u32 idle_states;
/* Standard hot unplug procedure */
generic_set_cpu_dead(cpu);
smp_wmb();
+ wmask = SRR1_WAKEMASK;
+ if (cpu_has_feature(CPU_FTR_ARCH_207S))
+ wmask = SRR1_WAKEMASK_P8;
+
idle_states = pnv_get_supported_cpuidle_states();
/* We don't want to take decrementer interrupts while we are offline,
* so clear LPCR:PECE1. We keep PECE2 enabled.
* having finished executing in a KVM guest, then srr1
* contains 0.
*/
- if ((srr1 & SRR1_WAKEMASK) == SRR1_WAKEEE) {
+ if ((srr1 & wmask) == SRR1_WAKEEE) {
icp_native_flush_interrupt();
local_paca->irq_happened &= PACA_IRQ_HARD_DIS;
smp_mb();
+ } else if ((srr1 & wmask) == SRR1_WAKEHDBELL) {
+ unsigned long msg = PPC_DBELL_TYPE(PPC_DBELL_SERVER);
+ asm volatile(PPC_MSGCLR(%0) : : "r" (msg));
+ kvmppc_set_host_ipi(cpu, 0);
}
if (cpu_core_split_required())
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*/
+#include <linux/jump_label.h>
#include <asm/hvcall.h>
#include <asm/processor.h>
#include <asm/ppc_asm.h>
#include <asm/asm-offsets.h>
#include <asm/ptrace.h>
-#include <asm/jump_label.h>
.section ".text"
}
__setup("multitce=", disable_multitce);
+
+machine_subsys_initcall_sync(pseries, tce_iommu_bus_notifier_init);
#include <linux/dma-mapping.h>
#include <linux/console.h>
#include <linux/export.h>
-#include <linux/static_key.h>
+#include <linux/jump_label.h>
#include <asm/processor.h>
#include <asm/mmu.h>
#include <asm/page.h>
static struct kobject *mobility_kobj;
struct update_props_workarea {
- u32 phandle;
- u32 state;
- u64 reserved;
- u32 nprops;
+ __be32 phandle;
+ __be32 state;
+ __be64 reserved;
+ __be32 nprops;
} __packed;
#define NODE_ACTION_MASK 0xff000000
return rc;
}
-static int delete_dt_node(u32 phandle)
+static int delete_dt_node(__be32 phandle)
{
struct device_node *dn;
- dn = of_find_node_by_phandle(phandle);
+ dn = of_find_node_by_phandle(be32_to_cpu(phandle));
if (!dn)
return -ENOENT;
return 0;
}
-static int update_dt_node(u32 phandle, s32 scope)
+static int update_dt_node(__be32 phandle, s32 scope)
{
struct update_props_workarea *upwa;
struct device_node *dn;
char *prop_data;
char *rtas_buf;
int update_properties_token;
+ u32 nprops;
u32 vd;
update_properties_token = rtas_token("ibm,update-properties");
if (!rtas_buf)
return -ENOMEM;
- dn = of_find_node_by_phandle(phandle);
+ dn = of_find_node_by_phandle(be32_to_cpu(phandle));
if (!dn) {
kfree(rtas_buf);
return -ENOENT;
break;
prop_data = rtas_buf + sizeof(*upwa);
+ nprops = be32_to_cpu(upwa->nprops);
/* On the first call to ibm,update-properties for a node the
* the first property value descriptor contains an empty
*/
if (*prop_data == 0) {
prop_data++;
- vd = *(u32 *)prop_data;
+ vd = be32_to_cpu(*(__be32 *)prop_data);
prop_data += vd + sizeof(vd);
- upwa->nprops--;
+ nprops--;
}
- for (i = 0; i < upwa->nprops; i++) {
+ for (i = 0; i < nprops; i++) {
char *prop_name;
prop_name = prop_data;
prop_data += strlen(prop_name) + 1;
- vd = *(u32 *)prop_data;
+ vd = be32_to_cpu(*(__be32 *)prop_data);
prop_data += sizeof(vd);
switch (vd) {
return 0;
}
-static int add_dt_node(u32 parent_phandle, u32 drc_index)
+static int add_dt_node(__be32 parent_phandle, __be32 drc_index)
{
struct device_node *dn;
struct device_node *parent_dn;
int rc;
- parent_dn = of_find_node_by_phandle(parent_phandle);
+ parent_dn = of_find_node_by_phandle(be32_to_cpu(parent_phandle));
if (!parent_dn)
return -ENOENT;
int pseries_devicetree_update(s32 scope)
{
char *rtas_buf;
- u32 *data;
+ __be32 *data;
int update_nodes_token;
int rc;
if (rc && rc != 1)
break;
- data = (u32 *)rtas_buf + 4;
- while (*data & NODE_ACTION_MASK) {
+ data = (__be32 *)rtas_buf + 4;
+ while (be32_to_cpu(*data) & NODE_ACTION_MASK) {
int i;
- u32 action = *data & NODE_ACTION_MASK;
- int node_count = *data & NODE_COUNT_MASK;
+ u32 action = be32_to_cpu(*data) & NODE_ACTION_MASK;
+ u32 node_count = be32_to_cpu(*data) & NODE_COUNT_MASK;
data++;
for (i = 0; i < node_count; i++) {
- u32 phandle = *data++;
- u32 drc_index;
+ __be32 phandle = *data++;
+ __be32 drc_index;
switch (action) {
case DELETE_DT_NODE:
extern unsigned long mmap_rnd_mask;
-#define STACK_RND_MASK (mmap_rnd_mask)
+#define STACK_RND_MASK (test_thread_flag(TIF_31BIT) ? 0x7ff : mmap_rnd_mask)
#define ARCH_DLINFO \
do { \
#ifndef _ASM_S390_JUMP_LABEL_H
#define _ASM_S390_JUMP_LABEL_H
+#ifndef __ASSEMBLY__
+
#include <linux/types.h>
#define JUMP_LABEL_NOP_SIZE 6
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif
__u32 fac; /* 0x01a0 */
__u8 reserved1a4[20]; /* 0x01a4 */
__u64 cbrlo; /* 0x01b8 */
- __u8 reserved1c0[30]; /* 0x01c0 */
+ __u8 reserved1c0[8]; /* 0x01c0 */
+ __u32 ecd; /* 0x01c8 */
+ __u8 reserved1cc[18]; /* 0x01cc */
__u64 pp; /* 0x01de */
__u8 reserved1e6[2]; /* 0x01e6 */
__u64 itdba; /* 0x01e8 */
__u8 data[256];
} __packed;
+struct kvm_s390_vregs {
+ __vector128 vrs[32];
+ __u8 reserved200[512]; /* for future vector expansion */
+} __packed;
+
struct sie_page {
struct kvm_s390_sie_block sie_block;
__u8 reserved200[1024]; /* 0x0200 */
struct kvm_s390_itdb itdb; /* 0x0600 */
- __u8 reserved700[2304]; /* 0x0700 */
+ __u8 reserved700[1280]; /* 0x0700 */
+ struct kvm_s390_vregs vregs; /* 0x0c00 */
} __packed;
struct kvm_vcpu_stat {
u32 instruction_sigp_stop;
u32 instruction_sigp_stop_store_status;
u32 instruction_sigp_store_status;
+ u32 instruction_sigp_store_adtl_status;
u32 instruction_sigp_arch;
u32 instruction_sigp_prefix;
u32 instruction_sigp_restart;
#define PGM_SPECIAL_OPERATION 0x13
#define PGM_OPERAND 0x15
#define PGM_TRACE_TABEL 0x16
+#define PGM_VECTOR_PROCESSING 0x1b
#define PGM_SPACE_SWITCH 0x1c
#define PGM_HFP_SQUARE_ROOT 0x1d
#define PGM_PC_TRANSLATION_SPEC 0x1f
IRQ_PEND_COUNT
};
+/* We have 2M for virtio device descriptor pages. Smallest amount of
+ * memory per page is 24 bytes (1 queue), so (2048*1024) / 24 = 87381
+ */
+#define KVM_S390_MAX_VIRTIO_IRQS 87381
+
/*
* Repressible (non-floating) machine check interrupts
* subclass bits in MCIC
unsigned long pending_irqs;
};
+#define FIRQ_LIST_IO_ISC_0 0
+#define FIRQ_LIST_IO_ISC_1 1
+#define FIRQ_LIST_IO_ISC_2 2
+#define FIRQ_LIST_IO_ISC_3 3
+#define FIRQ_LIST_IO_ISC_4 4
+#define FIRQ_LIST_IO_ISC_5 5
+#define FIRQ_LIST_IO_ISC_6 6
+#define FIRQ_LIST_IO_ISC_7 7
+#define FIRQ_LIST_PFAULT 8
+#define FIRQ_LIST_VIRTIO 9
+#define FIRQ_LIST_COUNT 10
+#define FIRQ_CNTR_IO 0
+#define FIRQ_CNTR_SERVICE 1
+#define FIRQ_CNTR_VIRTIO 2
+#define FIRQ_CNTR_PFAULT 3
+#define FIRQ_MAX_COUNT 4
+
struct kvm_s390_float_interrupt {
+ unsigned long pending_irqs;
spinlock_t lock;
- struct list_head list;
- atomic_t active;
+ struct list_head lists[FIRQ_LIST_COUNT];
+ int counters[FIRQ_MAX_COUNT];
+ struct kvm_s390_mchk_info mchk;
+ struct kvm_s390_ext_info srv_signal;
int next_rr_cpu;
unsigned long idle_mask[BITS_TO_LONGS(KVM_MAX_VCPUS)];
- unsigned int irq_count;
};
struct kvm_hw_wp_info_arch {
s390_fp_regs host_fpregs;
unsigned int host_acrs[NUM_ACRS];
s390_fp_regs guest_fpregs;
+ struct kvm_s390_vregs *host_vregs;
struct kvm_s390_local_interrupt local_int;
struct hrtimer ckc_timer;
struct kvm_s390_pgm_info pgm;
#define S390_ARCH_FAC_MASK_SIZE_U64 \
(S390_ARCH_FAC_MASK_SIZE_BYTE / sizeof(u64))
-struct s390_model_fac {
- /* facilities used in SIE context */
- __u64 sie[S390_ARCH_FAC_LIST_SIZE_U64];
- /* subset enabled by kvm */
- __u64 kvm[S390_ARCH_FAC_LIST_SIZE_U64];
+struct kvm_s390_fac {
+ /* facility list requested by guest */
+ __u64 list[S390_ARCH_FAC_LIST_SIZE_U64];
+ /* facility mask supported by kvm & hosting machine */
+ __u64 mask[S390_ARCH_FAC_LIST_SIZE_U64];
};
struct kvm_s390_cpu_model {
- struct s390_model_fac *fac;
+ struct kvm_s390_fac *fac;
struct cpuid cpu_id;
unsigned short ibc;
};
int use_cmma;
int user_cpu_state_ctrl;
int user_sigp;
+ int user_stsi;
struct s390_io_adapter *adapters[MAX_S390_IO_ADAPTERS];
wait_queue_head_t ipte_wq;
int ipte_lock_count;
{
int cpu = smp_processor_id();
+ S390_lowcore.user_asce = next->context.asce_bits | __pa(next->pgd);
if (prev == next)
return;
if (MACHINE_HAS_TLB_LC)
atomic_dec(&prev->context.attach_count);
if (MACHINE_HAS_TLB_LC)
cpumask_clear_cpu(cpu, &prev->context.cpu_attach_mask);
- S390_lowcore.user_asce = next->context.asce_bits | __pa(next->pgd);
}
#define finish_arch_post_lock_switch finish_arch_post_lock_switch
#endif
}
-static inline void clear_page(void *page)
-{
- register unsigned long reg1 asm ("1") = 0;
- register void *reg2 asm ("2") = page;
- register unsigned long reg3 asm ("3") = 4096;
- asm volatile(
- " mvcl 2,0"
- : "+d" (reg2), "+d" (reg3) : "d" (reg1)
- : "memory", "cc");
-}
+#define clear_page(page) memset((page), 0, PAGE_SIZE)
/*
* copy_page uses the mvcl instruction with 0xb0 padding byte in order to
*/
#define PTRS_PER_PTE 256
#ifndef CONFIG_64BIT
+#define __PAGETABLE_PUD_FOLDED
#define PTRS_PER_PMD 1
+#define __PAGETABLE_PMD_FOLDED
#define PTRS_PER_PUD 1
#else /* CONFIG_64BIT */
#define PTRS_PER_PMD 2048
#define KVM_SYNC_CRS (1UL << 3)
#define KVM_SYNC_ARCH0 (1UL << 4)
#define KVM_SYNC_PFAULT (1UL << 5)
+#define KVM_SYNC_VRS (1UL << 6)
/* definition of registers in kvm_run */
struct kvm_sync_regs {
__u64 prefix; /* prefix register */
__u64 pft; /* pfault token [PFAULT] */
__u64 pfs; /* pfault select [PFAULT] */
__u64 pfc; /* pfault compare [PFAULT] */
+ __u64 vrs[32][2]; /* vector registers */
+ __u8 reserved[512]; /* for future vector expansion */
+ __u32 fpc; /* only valid with vector registers */
};
#define KVM_REG_S390_TODPR (KVM_REG_S390 | KVM_REG_SIZE_U32 | 0x1)
* and returns a key, which can be used to find a mnemonic name
* of the instruction in the icpt_insn_codes table.
*/
-#define icpt_insn_decoder(insn) \
+#define icpt_insn_decoder(insn) ( \
INSN_DECODE_IPA0(0x01, insn, 48, 0xff) \
INSN_DECODE_IPA0(0xaa, insn, 48, 0x0f) \
INSN_DECODE_IPA0(0xb2, insn, 48, 0xff) \
INSN_DECODE_IPA0(0xe5, insn, 48, 0xff) \
INSN_DECODE_IPA0(0xeb, insn, 16, 0xff) \
INSN_DECODE_IPA0(0xc8, insn, 48, 0x0f) \
- INSN_DECODE(insn)
+ INSN_DECODE(insn))
#endif /* _UAPI_ASM_S390_SIE_H */
#else /* CONFIG_32BIT */
DEFINE(__LC_DATA_EXC_CODE, offsetof(struct _lowcore, data_exc_code));
DEFINE(__LC_MCCK_FAIL_STOR_ADDR, offsetof(struct _lowcore, failing_storage_address));
+ DEFINE(__LC_VX_SAVE_AREA_ADDR, offsetof(struct _lowcore, vector_save_area_addr));
DEFINE(__LC_EXT_PARAMS2, offsetof(struct _lowcore, ext_params2));
DEFINE(SAVE_AREA_BASE, offsetof(struct _lowcore, floating_pt_save_area));
DEFINE(__LC_PASTE, offsetof(struct _lowcore, paste));
unsigned long ftrace_plt;
+static inline void ftrace_generate_orig_insn(struct ftrace_insn *insn)
+{
+#ifdef CC_USING_HOTPATCH
+ /* brcl 0,0 */
+ insn->opc = 0xc004;
+ insn->disp = 0;
+#else
+ /* stg r14,8(r15) */
+ insn->opc = 0xe3e0;
+ insn->disp = 0xf0080024;
+#endif
+}
+
+static inline int is_kprobe_on_ftrace(struct ftrace_insn *insn)
+{
+#ifdef CONFIG_KPROBES
+ if (insn->opc == BREAKPOINT_INSTRUCTION)
+ return 1;
+#endif
+ return 0;
+}
+
+static inline void ftrace_generate_kprobe_nop_insn(struct ftrace_insn *insn)
+{
+#ifdef CONFIG_KPROBES
+ insn->opc = BREAKPOINT_INSTRUCTION;
+ insn->disp = KPROBE_ON_FTRACE_NOP;
+#endif
+}
+
+static inline void ftrace_generate_kprobe_call_insn(struct ftrace_insn *insn)
+{
+#ifdef CONFIG_KPROBES
+ insn->opc = BREAKPOINT_INSTRUCTION;
+ insn->disp = KPROBE_ON_FTRACE_CALL;
+#endif
+}
+
int ftrace_modify_call(struct dyn_ftrace *rec, unsigned long old_addr,
unsigned long addr)
{
return -EFAULT;
if (addr == MCOUNT_ADDR) {
/* Initial code replacement */
-#ifdef CC_USING_HOTPATCH
- /* We expect to see brcl 0,0 */
- ftrace_generate_nop_insn(&orig);
-#else
- /* We expect to see stg r14,8(r15) */
- orig.opc = 0xe3e0;
- orig.disp = 0xf0080024;
-#endif
+ ftrace_generate_orig_insn(&orig);
ftrace_generate_nop_insn(&new);
- } else if (old.opc == BREAKPOINT_INSTRUCTION) {
+ } else if (is_kprobe_on_ftrace(&old)) {
/*
* If we find a breakpoint instruction, a kprobe has been
* placed at the beginning of the function. We write the
* bytes of the original instruction so that the kprobes
* handler can execute a nop, if it reaches this breakpoint.
*/
- new.opc = orig.opc = BREAKPOINT_INSTRUCTION;
- orig.disp = KPROBE_ON_FTRACE_CALL;
- new.disp = KPROBE_ON_FTRACE_NOP;
+ ftrace_generate_kprobe_call_insn(&orig);
+ ftrace_generate_kprobe_nop_insn(&new);
} else {
/* Replace ftrace call with a nop. */
ftrace_generate_call_insn(&orig, rec->ip);
if (probe_kernel_read(&old, (void *) rec->ip, sizeof(old)))
return -EFAULT;
- if (old.opc == BREAKPOINT_INSTRUCTION) {
+ if (is_kprobe_on_ftrace(&old)) {
/*
* If we find a breakpoint instruction, a kprobe has been
* placed at the beginning of the function. We write the
* bytes of the original instruction so that the kprobes
* handler can execute a brasl if it reaches this breakpoint.
*/
- new.opc = orig.opc = BREAKPOINT_INSTRUCTION;
- orig.disp = KPROBE_ON_FTRACE_NOP;
- new.disp = KPROBE_ON_FTRACE_CALL;
+ ftrace_generate_kprobe_nop_insn(&orig);
+ ftrace_generate_kprobe_call_insn(&new);
} else {
/* Replace nop with an ftrace call. */
ftrace_generate_nop_insn(&orig);
insn->offset = (entry->target - entry->code) >> 1;
}
-static void jump_label_bug(struct jump_entry *entry, struct insn *insn)
+static void jump_label_bug(struct jump_entry *entry, struct insn *expected,
+ struct insn *new)
{
unsigned char *ipc = (unsigned char *)entry->code;
- unsigned char *ipe = (unsigned char *)insn;
+ unsigned char *ipe = (unsigned char *)expected;
+ unsigned char *ipn = (unsigned char *)new;
pr_emerg("Jump label code mismatch at %pS [%p]\n", ipc, ipc);
pr_emerg("Found: %02x %02x %02x %02x %02x %02x\n",
ipc[0], ipc[1], ipc[2], ipc[3], ipc[4], ipc[5]);
pr_emerg("Expected: %02x %02x %02x %02x %02x %02x\n",
ipe[0], ipe[1], ipe[2], ipe[3], ipe[4], ipe[5]);
+ pr_emerg("New: %02x %02x %02x %02x %02x %02x\n",
+ ipn[0], ipn[1], ipn[2], ipn[3], ipn[4], ipn[5]);
panic("Corrupted kernel text");
}
}
if (init) {
if (memcmp((void *)entry->code, &orignop, sizeof(orignop)))
- jump_label_bug(entry, &old);
+ jump_label_bug(entry, &orignop, &new);
} else {
if (memcmp((void *)entry->code, &old, sizeof(old)))
- jump_label_bug(entry, &old);
+ jump_label_bug(entry, &old, &new);
}
probe_kernel_write((void *)entry->code, &new, sizeof(new));
}
const Elf_Shdr *sechdrs,
struct module *me)
{
+ jump_label_apply_nops(me);
vfree(me->arch.syminfo);
me->arch.syminfo = NULL;
return 0;
static struct attribute *cpumsf_pmu_events_attr[] = {
CPUMF_EVENT_PTR(SF, SF_CYCLES_BASIC),
- CPUMF_EVENT_PTR(SF, SF_CYCLES_BASIC_DIAG),
+ NULL,
NULL,
};
return -EINVAL;
}
- if (si.ad)
+ if (si.ad) {
sfb_set_limits(CPUM_SF_MIN_SDB, CPUM_SF_MAX_SDB);
+ cpumsf_pmu_events_attr[1] =
+ CPUMF_EVENT_PTR(SF, SF_CYCLES_BASIC_DIAG);
+ }
sfdbg = debug_register(KMSG_COMPONENT, 2, 1, 80);
if (!sfdbg)
static DEFINE_PER_CPU(struct cpuid, cpu_id);
-void cpu_relax(void)
+void notrace cpu_relax(void)
{
if (!smp_cpu_mtid && MACHINE_HAS_DIAG44)
asm volatile("diag 0,0,0x44");
lhi %r1,1
sigp %r1,%r0,SIGP_SET_ARCHITECTURE
sam64
+#ifdef CONFIG_SMP
+ larl %r1,smp_cpu_mt_shift
+ icm %r1,15,0(%r1)
+ jz smt_done
+ llgfr %r1,%r1
+smt_loop:
+ sigp %r1,%r0,SIGP_SET_MULTI_THREADING
+ brc 8,smt_done /* accepted */
+ brc 2,smt_loop /* busy, try again */
+smt_done:
+#endif
larl %r1,.Lnew_pgm_check_psw
lpswe 0(%r1)
pgm_check_entry:
{
u64 nsecps;
- if (tk->tkr.clock != &clocksource_tod)
+ if (tk->tkr_mono.clock != &clocksource_tod)
return;
/* Make userspace gettimeofday spin until we're done. */
++vdso_data->tb_update_count;
smp_wmb();
- vdso_data->xtime_tod_stamp = tk->tkr.cycle_last;
+ vdso_data->xtime_tod_stamp = tk->tkr_mono.cycle_last;
vdso_data->xtime_clock_sec = tk->xtime_sec;
- vdso_data->xtime_clock_nsec = tk->tkr.xtime_nsec;
+ vdso_data->xtime_clock_nsec = tk->tkr_mono.xtime_nsec;
vdso_data->wtom_clock_sec =
tk->xtime_sec + tk->wall_to_monotonic.tv_sec;
- vdso_data->wtom_clock_nsec = tk->tkr.xtime_nsec +
- + ((u64) tk->wall_to_monotonic.tv_nsec << tk->tkr.shift);
- nsecps = (u64) NSEC_PER_SEC << tk->tkr.shift;
+ vdso_data->wtom_clock_nsec = tk->tkr_mono.xtime_nsec +
+ + ((u64) tk->wall_to_monotonic.tv_nsec << tk->tkr_mono.shift);
+ nsecps = (u64) NSEC_PER_SEC << tk->tkr_mono.shift;
while (vdso_data->wtom_clock_nsec >= nsecps) {
vdso_data->wtom_clock_nsec -= nsecps;
vdso_data->wtom_clock_sec++;
vdso_data->xtime_coarse_sec = tk->xtime_sec;
vdso_data->xtime_coarse_nsec =
- (long)(tk->tkr.xtime_nsec >> tk->tkr.shift);
+ (long)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);
vdso_data->wtom_coarse_sec =
vdso_data->xtime_coarse_sec + tk->wall_to_monotonic.tv_sec;
vdso_data->wtom_coarse_nsec =
vdso_data->wtom_coarse_sec++;
}
- vdso_data->tk_mult = tk->tkr.mult;
- vdso_data->tk_shift = tk->tkr.shift;
+ vdso_data->tk_mult = tk->tkr_mono.mult;
+ vdso_data->tk_shift = tk->tkr_mono.shift;
smp_wmb();
++vdso_data->tb_update_count;
}
if (register_external_irq(EXT_IRQ_TIMING_ALERT, timing_alert_interrupt))
panic("Couldn't request external interrupt 0x1406");
- if (clocksource_register(&clocksource_tod) != 0)
+ if (__clocksource_register(&clocksource_tod) != 0)
panic("Could not register TOD clock source");
/* Enable TOD clock interrupts on the boot cpu. */
if (vcpu->run->s.regs.gprs[rx] & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = read_guest(vcpu, vcpu->run->s.regs.gprs[rx], &parm, sizeof(parm));
+ rc = read_guest(vcpu, vcpu->run->s.regs.gprs[rx], rx, &parm, sizeof(parm));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
if (parm.parm_version != 2 || parm.parm_len < 5 || parm.code != 0x258)
* - gpr 3 contains the virtqueue index (passed as datamatch)
* - gpr 4 contains the index on the bus (optionally)
*/
- ret = kvm_io_bus_write_cookie(vcpu->kvm, KVM_VIRTIO_CCW_NOTIFY_BUS,
+ ret = kvm_io_bus_write_cookie(vcpu, KVM_VIRTIO_CCW_NOTIFY_BUS,
vcpu->run->s.regs.gprs[2] & 0xffffffff,
8, &vcpu->run->s.regs.gprs[3],
vcpu->run->s.regs.gprs[4]);
int kvm_s390_handle_diag(struct kvm_vcpu *vcpu)
{
- int code = kvm_s390_get_base_disp_rs(vcpu) & 0xffff;
+ int code = kvm_s390_get_base_disp_rs(vcpu, NULL) & 0xffff;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
#include <asm/pgtable.h>
#include "kvm-s390.h"
#include "gaccess.h"
+#include <asm/switch_to.h>
union asce {
unsigned long val;
unsigned long pfra : 52; /* Page-Frame Real Address */
};
+union alet {
+ u32 val;
+ struct {
+ u32 reserved : 7;
+ u32 p : 1;
+ u32 alesn : 8;
+ u32 alen : 16;
+ };
+};
+
+union ald {
+ u32 val;
+ struct {
+ u32 : 1;
+ u32 alo : 24;
+ u32 all : 7;
+ };
+};
+
+struct ale {
+ unsigned long i : 1; /* ALEN-Invalid Bit */
+ unsigned long : 5;
+ unsigned long fo : 1; /* Fetch-Only Bit */
+ unsigned long p : 1; /* Private Bit */
+ unsigned long alesn : 8; /* Access-List-Entry Sequence Number */
+ unsigned long aleax : 16; /* Access-List-Entry Authorization Index */
+ unsigned long : 32;
+ unsigned long : 1;
+ unsigned long asteo : 25; /* ASN-Second-Table-Entry Origin */
+ unsigned long : 6;
+ unsigned long astesn : 32; /* ASTE Sequence Number */
+} __packed;
+
+struct aste {
+ unsigned long i : 1; /* ASX-Invalid Bit */
+ unsigned long ato : 29; /* Authority-Table Origin */
+ unsigned long : 1;
+ unsigned long b : 1; /* Base-Space Bit */
+ unsigned long ax : 16; /* Authorization Index */
+ unsigned long atl : 12; /* Authority-Table Length */
+ unsigned long : 2;
+ unsigned long ca : 1; /* Controlled-ASN Bit */
+ unsigned long ra : 1; /* Reusable-ASN Bit */
+ unsigned long asce : 64; /* Address-Space-Control Element */
+ unsigned long ald : 32;
+ unsigned long astesn : 32;
+ /* .. more fields there */
+} __packed;
int ipte_lock_held(struct kvm_vcpu *vcpu)
{
ipte_unlock_simple(vcpu);
}
-static unsigned long get_vcpu_asce(struct kvm_vcpu *vcpu)
+static int ar_translation(struct kvm_vcpu *vcpu, union asce *asce, ar_t ar,
+ int write)
+{
+ union alet alet;
+ struct ale ale;
+ struct aste aste;
+ unsigned long ald_addr, authority_table_addr;
+ union ald ald;
+ int eax, rc;
+ u8 authority_table;
+
+ if (ar >= NUM_ACRS)
+ return -EINVAL;
+
+ save_access_regs(vcpu->run->s.regs.acrs);
+ alet.val = vcpu->run->s.regs.acrs[ar];
+
+ if (ar == 0 || alet.val == 0) {
+ asce->val = vcpu->arch.sie_block->gcr[1];
+ return 0;
+ } else if (alet.val == 1) {
+ asce->val = vcpu->arch.sie_block->gcr[7];
+ return 0;
+ }
+
+ if (alet.reserved)
+ return PGM_ALET_SPECIFICATION;
+
+ if (alet.p)
+ ald_addr = vcpu->arch.sie_block->gcr[5];
+ else
+ ald_addr = vcpu->arch.sie_block->gcr[2];
+ ald_addr &= 0x7fffffc0;
+
+ rc = read_guest_real(vcpu, ald_addr + 16, &ald.val, sizeof(union ald));
+ if (rc)
+ return rc;
+
+ if (alet.alen / 8 > ald.all)
+ return PGM_ALEN_TRANSLATION;
+
+ if (0x7fffffff - ald.alo * 128 < alet.alen * 16)
+ return PGM_ADDRESSING;
+
+ rc = read_guest_real(vcpu, ald.alo * 128 + alet.alen * 16, &ale,
+ sizeof(struct ale));
+ if (rc)
+ return rc;
+
+ if (ale.i == 1)
+ return PGM_ALEN_TRANSLATION;
+ if (ale.alesn != alet.alesn)
+ return PGM_ALE_SEQUENCE;
+
+ rc = read_guest_real(vcpu, ale.asteo * 64, &aste, sizeof(struct aste));
+ if (rc)
+ return rc;
+
+ if (aste.i)
+ return PGM_ASTE_VALIDITY;
+ if (aste.astesn != ale.astesn)
+ return PGM_ASTE_SEQUENCE;
+
+ if (ale.p == 1) {
+ eax = (vcpu->arch.sie_block->gcr[8] >> 16) & 0xffff;
+ if (ale.aleax != eax) {
+ if (eax / 16 > aste.atl)
+ return PGM_EXTENDED_AUTHORITY;
+
+ authority_table_addr = aste.ato * 4 + eax / 4;
+
+ rc = read_guest_real(vcpu, authority_table_addr,
+ &authority_table,
+ sizeof(u8));
+ if (rc)
+ return rc;
+
+ if ((authority_table & (0x40 >> ((eax & 3) * 2))) == 0)
+ return PGM_EXTENDED_AUTHORITY;
+ }
+ }
+
+ if (ale.fo == 1 && write)
+ return PGM_PROTECTION;
+
+ asce->val = aste.asce;
+ return 0;
+}
+
+struct trans_exc_code_bits {
+ unsigned long addr : 52; /* Translation-exception Address */
+ unsigned long fsi : 2; /* Access Exception Fetch/Store Indication */
+ unsigned long : 6;
+ unsigned long b60 : 1;
+ unsigned long b61 : 1;
+ unsigned long as : 2; /* ASCE Identifier */
+};
+
+enum {
+ FSI_UNKNOWN = 0, /* Unknown wether fetch or store */
+ FSI_STORE = 1, /* Exception was due to store operation */
+ FSI_FETCH = 2 /* Exception was due to fetch operation */
+};
+
+static int get_vcpu_asce(struct kvm_vcpu *vcpu, union asce *asce,
+ ar_t ar, int write)
{
+ int rc;
+ psw_t *psw = &vcpu->arch.sie_block->gpsw;
+ struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
+ struct trans_exc_code_bits *tec_bits;
+
+ memset(pgm, 0, sizeof(*pgm));
+ tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
+ tec_bits->fsi = write ? FSI_STORE : FSI_FETCH;
+ tec_bits->as = psw_bits(*psw).as;
+
+ if (!psw_bits(*psw).t) {
+ asce->val = 0;
+ asce->r = 1;
+ return 0;
+ }
+
switch (psw_bits(vcpu->arch.sie_block->gpsw).as) {
case PSW_AS_PRIMARY:
- return vcpu->arch.sie_block->gcr[1];
+ asce->val = vcpu->arch.sie_block->gcr[1];
+ return 0;
case PSW_AS_SECONDARY:
- return vcpu->arch.sie_block->gcr[7];
+ asce->val = vcpu->arch.sie_block->gcr[7];
+ return 0;
case PSW_AS_HOME:
- return vcpu->arch.sie_block->gcr[13];
+ asce->val = vcpu->arch.sie_block->gcr[13];
+ return 0;
+ case PSW_AS_ACCREG:
+ rc = ar_translation(vcpu, asce, ar, write);
+ switch (rc) {
+ case PGM_ALEN_TRANSLATION:
+ case PGM_ALE_SEQUENCE:
+ case PGM_ASTE_VALIDITY:
+ case PGM_ASTE_SEQUENCE:
+ case PGM_EXTENDED_AUTHORITY:
+ vcpu->arch.pgm.exc_access_id = ar;
+ break;
+ case PGM_PROTECTION:
+ tec_bits->b60 = 1;
+ tec_bits->b61 = 1;
+ break;
+ }
+ if (rc > 0)
+ pgm->code = rc;
+ return rc;
}
return 0;
}
* @vcpu: virtual cpu
* @gva: guest virtual address
* @gpa: points to where guest physical (absolute) address should be stored
+ * @asce: effective asce
* @write: indicates if access is a write access
*
* Translate a guest virtual address into a guest absolute address by means
- * of dynamic address translation as specified by the architecuture.
+ * of dynamic address translation as specified by the architecture.
* If the resulting absolute address is not available in the configuration
* an addressing exception is indicated and @gpa will not be changed.
*
* by the architecture
*/
static unsigned long guest_translate(struct kvm_vcpu *vcpu, unsigned long gva,
- unsigned long *gpa, int write)
+ unsigned long *gpa, const union asce asce,
+ int write)
{
union vaddress vaddr = {.addr = gva};
union raddress raddr = {.addr = gva};
union ctlreg0 ctlreg0;
unsigned long ptr;
int edat1, edat2;
- union asce asce;
ctlreg0.val = vcpu->arch.sie_block->gcr[0];
edat1 = ctlreg0.edat && test_kvm_facility(vcpu->kvm, 8);
edat2 = edat1 && test_kvm_facility(vcpu->kvm, 78);
- asce.val = get_vcpu_asce(vcpu);
if (asce.r)
goto real_address;
ptr = asce.origin * 4096;
return (ga & ~0x11fful) == 0;
}
-static int low_address_protection_enabled(struct kvm_vcpu *vcpu)
+static int low_address_protection_enabled(struct kvm_vcpu *vcpu,
+ const union asce asce)
{
union ctlreg0 ctlreg0 = {.val = vcpu->arch.sie_block->gcr[0]};
psw_t *psw = &vcpu->arch.sie_block->gpsw;
- union asce asce;
if (!ctlreg0.lap)
return 0;
- asce.val = get_vcpu_asce(vcpu);
if (psw_bits(*psw).t && asce.p)
return 0;
return 1;
}
-struct trans_exc_code_bits {
- unsigned long addr : 52; /* Translation-exception Address */
- unsigned long fsi : 2; /* Access Exception Fetch/Store Indication */
- unsigned long : 7;
- unsigned long b61 : 1;
- unsigned long as : 2; /* ASCE Identifier */
-};
-
-enum {
- FSI_UNKNOWN = 0, /* Unknown wether fetch or store */
- FSI_STORE = 1, /* Exception was due to store operation */
- FSI_FETCH = 2 /* Exception was due to fetch operation */
-};
-
static int guest_page_range(struct kvm_vcpu *vcpu, unsigned long ga,
unsigned long *pages, unsigned long nr_pages,
- int write)
+ const union asce asce, int write)
{
struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
psw_t *psw = &vcpu->arch.sie_block->gpsw;
struct trans_exc_code_bits *tec_bits;
int lap_enabled, rc;
- memset(pgm, 0, sizeof(*pgm));
tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
- tec_bits->fsi = write ? FSI_STORE : FSI_FETCH;
- tec_bits->as = psw_bits(*psw).as;
- lap_enabled = low_address_protection_enabled(vcpu);
+ lap_enabled = low_address_protection_enabled(vcpu, asce);
while (nr_pages) {
ga = kvm_s390_logical_to_effective(vcpu, ga);
tec_bits->addr = ga >> PAGE_SHIFT;
}
ga &= PAGE_MASK;
if (psw_bits(*psw).t) {
- rc = guest_translate(vcpu, ga, pages, write);
+ rc = guest_translate(vcpu, ga, pages, asce, write);
if (rc < 0)
return rc;
if (rc == PGM_PROTECTION)
return 0;
}
-int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
unsigned long len, int write)
{
psw_t *psw = &vcpu->arch.sie_block->gpsw;
if (!len)
return 0;
- /* Access register mode is not supported yet. */
- if (psw_bits(*psw).t && psw_bits(*psw).as == PSW_AS_ACCREG)
- return -EOPNOTSUPP;
+ rc = get_vcpu_asce(vcpu, &asce, ar, write);
+ if (rc)
+ return rc;
nr_pages = (((ga & ~PAGE_MASK) + len - 1) >> PAGE_SHIFT) + 1;
pages = pages_array;
if (nr_pages > ARRAY_SIZE(pages_array))
pages = vmalloc(nr_pages * sizeof(unsigned long));
if (!pages)
return -ENOMEM;
- asce.val = get_vcpu_asce(vcpu);
need_ipte_lock = psw_bits(*psw).t && !asce.r;
if (need_ipte_lock)
ipte_lock(vcpu);
- rc = guest_page_range(vcpu, ga, pages, nr_pages, write);
+ rc = guest_page_range(vcpu, ga, pages, nr_pages, asce, write);
for (idx = 0; idx < nr_pages && !rc; idx++) {
gpa = *(pages + idx) + (ga & ~PAGE_MASK);
_len = min(PAGE_SIZE - (gpa & ~PAGE_MASK), len);
* Note: The IPTE lock is not taken during this function, so the caller
* has to take care of this.
*/
-int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
+int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
unsigned long *gpa, int write)
{
struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
union asce asce;
int rc;
- /* Access register mode is not supported yet. */
- if (psw_bits(*psw).t && psw_bits(*psw).as == PSW_AS_ACCREG)
- return -EOPNOTSUPP;
-
gva = kvm_s390_logical_to_effective(vcpu, gva);
- memset(pgm, 0, sizeof(*pgm));
tec = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
- tec->as = psw_bits(*psw).as;
- tec->fsi = write ? FSI_STORE : FSI_FETCH;
+ rc = get_vcpu_asce(vcpu, &asce, ar, write);
tec->addr = gva >> PAGE_SHIFT;
- if (is_low_address(gva) && low_address_protection_enabled(vcpu)) {
+ if (rc)
+ return rc;
+ if (is_low_address(gva) && low_address_protection_enabled(vcpu, asce)) {
if (write) {
rc = pgm->code = PGM_PROTECTION;
return rc;
}
}
- asce.val = get_vcpu_asce(vcpu);
if (psw_bits(*psw).t && !asce.r) { /* Use DAT? */
- rc = guest_translate(vcpu, gva, gpa, write);
+ rc = guest_translate(vcpu, gva, gpa, asce, write);
if (rc > 0) {
if (rc == PGM_PROTECTION)
tec->b61 = 1;
}
/**
- * kvm_s390_check_low_addr_protection - check for low-address protection
- * @ga: Guest address
+ * check_gva_range - test a range of guest virtual addresses for accessibility
+ */
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
+ unsigned long length, int is_write)
+{
+ unsigned long gpa;
+ unsigned long currlen;
+ int rc = 0;
+
+ ipte_lock(vcpu);
+ while (length > 0 && !rc) {
+ currlen = min(length, PAGE_SIZE - (gva % PAGE_SIZE));
+ rc = guest_translate_address(vcpu, gva, ar, &gpa, is_write);
+ gva += currlen;
+ length -= currlen;
+ }
+ ipte_unlock(vcpu);
+
+ return rc;
+}
+
+/**
+ * kvm_s390_check_low_addr_prot_real - check for low-address protection
+ * @gra: Guest real address
*
* Checks whether an address is subject to low-address protection and set
* up vcpu->arch.pgm accordingly if necessary.
*
* Return: 0 if no protection exception, or PGM_PROTECTION if protected.
*/
-int kvm_s390_check_low_addr_protection(struct kvm_vcpu *vcpu, unsigned long ga)
+int kvm_s390_check_low_addr_prot_real(struct kvm_vcpu *vcpu, unsigned long gra)
{
struct kvm_s390_pgm_info *pgm = &vcpu->arch.pgm;
psw_t *psw = &vcpu->arch.sie_block->gpsw;
struct trans_exc_code_bits *tec_bits;
+ union ctlreg0 ctlreg0 = {.val = vcpu->arch.sie_block->gcr[0]};
- if (!is_low_address(ga) || !low_address_protection_enabled(vcpu))
+ if (!ctlreg0.lap || !is_low_address(gra))
return 0;
memset(pgm, 0, sizeof(*pgm));
tec_bits = (struct trans_exc_code_bits *)&pgm->trans_exc_code;
tec_bits->fsi = FSI_STORE;
tec_bits->as = psw_bits(*psw).as;
- tec_bits->addr = ga >> PAGE_SHIFT;
+ tec_bits->addr = gra >> PAGE_SHIFT;
pgm->code = PGM_PROTECTION;
return pgm->code;
}
int guest_translate_address(struct kvm_vcpu *vcpu, unsigned long gva,
- unsigned long *gpa, int write);
+ ar_t ar, unsigned long *gpa, int write);
+int check_gva_range(struct kvm_vcpu *vcpu, unsigned long gva, ar_t ar,
+ unsigned long length, int is_write);
-int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int access_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
unsigned long len, int write);
int access_guest_real(struct kvm_vcpu *vcpu, unsigned long gra,
* write_guest - copy data from kernel space to guest space
* @vcpu: virtual cpu
* @ga: guest address
+ * @ar: access register
* @data: source address in kernel space
* @len: number of bytes to copy
*
* If DAT is off data will be copied to guest real or absolute memory.
* If DAT is on data will be copied to the address space as specified by
* the address space bits of the PSW:
- * Primary, secondory or home space (access register mode is currently not
- * implemented).
+ * Primary, secondary, home space or access register mode.
* The addressing mode of the PSW is also inspected, so that address wrap
* around is taken into account for 24-, 31- and 64-bit addressing mode,
* if the to be copied data crosses page boundaries in guest address space.
* if data has been changed in guest space in case of an exception.
*/
static inline __must_check
-int write_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int write_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
unsigned long len)
{
- return access_guest(vcpu, ga, data, len, 1);
+ return access_guest(vcpu, ga, ar, data, len, 1);
}
/**
* read_guest - copy data from guest space to kernel space
* @vcpu: virtual cpu
* @ga: guest address
+ * @ar: access register
* @data: destination address in kernel space
* @len: number of bytes to copy
*
* data will be copied from guest space to kernel space.
*/
static inline __must_check
-int read_guest(struct kvm_vcpu *vcpu, unsigned long ga, void *data,
+int read_guest(struct kvm_vcpu *vcpu, unsigned long ga, ar_t ar, void *data,
unsigned long len)
{
- return access_guest(vcpu, ga, data, len, 0);
+ return access_guest(vcpu, ga, ar, data, len, 0);
}
/**
void ipte_lock(struct kvm_vcpu *vcpu);
void ipte_unlock(struct kvm_vcpu *vcpu);
int ipte_lock_held(struct kvm_vcpu *vcpu);
-int kvm_s390_check_low_addr_protection(struct kvm_vcpu *vcpu, unsigned long ga);
+int kvm_s390_check_low_addr_prot_real(struct kvm_vcpu *vcpu, unsigned long gra);
#endif /* __KVM_S390_GACCESS_H */
if (!wp_info->old_data)
return -ENOMEM;
/* try to backup the original value */
- ret = read_guest(vcpu, wp_info->phys_addr, wp_info->old_data,
- wp_info->len);
+ ret = read_guest_abs(vcpu, wp_info->phys_addr, wp_info->old_data,
+ wp_info->len);
if (ret) {
kfree(wp_info->old_data);
wp_info->old_data = NULL;
continue;
/* refetch the wp data and compare it to the old value */
- if (!read_guest(vcpu, wp_info->phys_addr, temp,
- wp_info->len)) {
+ if (!read_guest_abs(vcpu, wp_info->phys_addr, temp,
+ wp_info->len)) {
if (memcmp(temp, wp_info->old_data, wp_info->len)) {
kfree(temp);
return wp_info;
pgm_info->mon_class_nr = vcpu->arch.sie_block->mcn;
pgm_info->mon_code = vcpu->arch.sie_block->tecmc;
break;
+ case PGM_VECTOR_PROCESSING:
case PGM_DATA:
pgm_info->data_exc_code = vcpu->arch.sie_block->dxc;
break;
/* Make sure that the source is paged-in */
rc = guest_translate_address(vcpu, vcpu->run->s.regs.gprs[reg2],
- &srcaddr, 0);
+ reg2, &srcaddr, 0);
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
rc = kvm_arch_fault_in_page(vcpu, srcaddr, 0);
/* Make sure that the destination is paged-in */
rc = guest_translate_address(vcpu, vcpu->run->s.regs.gprs[reg1],
- &dstaddr, 1);
+ reg1, &dstaddr, 1);
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
rc = kvm_arch_fault_in_page(vcpu, dstaddr, 1);
/*
* handling kvm guest interrupts
*
- * Copyright IBM Corp. 2008,2014
+ * Copyright IBM Corp. 2008, 2015
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License (version 2 only)
#include <linux/signal.h>
#include <linux/slab.h>
#include <linux/bitmap.h>
+#include <linux/vmalloc.h>
#include <asm/asm-offsets.h>
+#include <asm/dis.h>
#include <asm/uaccess.h>
#include <asm/sclp.h>
+#include <asm/isc.h>
#include "kvm-s390.h"
#include "gaccess.h"
#include "trace-s390.h"
#define PFAULT_DONE 0x0680
#define VIRTIO_PARAM 0x0d00
-static int is_ioint(u64 type)
-{
- return ((type & 0xfffe0000u) != 0xfffe0000u);
-}
-
int psw_extint_disabled(struct kvm_vcpu *vcpu)
{
return !(vcpu->arch.sie_block->gpsw.mask & PSW_MASK_EXT);
return 1;
}
-static u64 int_word_to_isc_bits(u32 int_word)
+static int ckc_irq_pending(struct kvm_vcpu *vcpu)
+{
+ if (!(vcpu->arch.sie_block->ckc <
+ get_tod_clock_fast() + vcpu->arch.sie_block->epoch))
+ return 0;
+ return ckc_interrupts_enabled(vcpu);
+}
+
+static int cpu_timer_interrupts_enabled(struct kvm_vcpu *vcpu)
+{
+ return !psw_extint_disabled(vcpu) &&
+ (vcpu->arch.sie_block->gcr[0] & 0x400ul);
+}
+
+static int cpu_timer_irq_pending(struct kvm_vcpu *vcpu)
+{
+ return (vcpu->arch.sie_block->cputm >> 63) &&
+ cpu_timer_interrupts_enabled(vcpu);
+}
+
+static inline int is_ioirq(unsigned long irq_type)
{
- u8 isc = (int_word & 0x38000000) >> 27;
+ return ((irq_type >= IRQ_PEND_IO_ISC_0) &&
+ (irq_type <= IRQ_PEND_IO_ISC_7));
+}
+static uint64_t isc_to_isc_bits(int isc)
+{
return (0x80 >> isc) << 24;
}
-static int __must_check __interrupt_is_deliverable(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
+static inline u8 int_word_to_isc(u32 int_word)
{
- switch (inti->type) {
- case KVM_S390_INT_EXTERNAL_CALL:
- if (psw_extint_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[0] & 0x2000ul)
- return 1;
- return 0;
- case KVM_S390_INT_EMERGENCY:
- if (psw_extint_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[0] & 0x4000ul)
- return 1;
- return 0;
- case KVM_S390_INT_CLOCK_COMP:
- return ckc_interrupts_enabled(vcpu);
- case KVM_S390_INT_CPU_TIMER:
- if (psw_extint_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[0] & 0x400ul)
- return 1;
- return 0;
- case KVM_S390_INT_SERVICE:
- case KVM_S390_INT_PFAULT_INIT:
- case KVM_S390_INT_PFAULT_DONE:
- case KVM_S390_INT_VIRTIO:
- if (psw_extint_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[0] & 0x200ul)
- return 1;
- return 0;
- case KVM_S390_PROGRAM_INT:
- case KVM_S390_SIGP_STOP:
- case KVM_S390_SIGP_SET_PREFIX:
- case KVM_S390_RESTART:
- return 1;
- case KVM_S390_MCHK:
- if (psw_mchk_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[14] & inti->mchk.cr14)
- return 1;
- return 0;
- case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
- if (psw_ioint_disabled(vcpu))
- return 0;
- if (vcpu->arch.sie_block->gcr[6] &
- int_word_to_isc_bits(inti->io.io_int_word))
- return 1;
- return 0;
- default:
- printk(KERN_WARNING "illegal interrupt type %llx\n",
- inti->type);
- BUG();
- }
- return 0;
+ return (int_word & 0x38000000) >> 27;
+}
+
+static inline unsigned long pending_floating_irqs(struct kvm_vcpu *vcpu)
+{
+ return vcpu->kvm->arch.float_int.pending_irqs;
}
static inline unsigned long pending_local_irqs(struct kvm_vcpu *vcpu)
return vcpu->arch.local_int.pending_irqs;
}
-static unsigned long deliverable_local_irqs(struct kvm_vcpu *vcpu)
+static unsigned long disable_iscs(struct kvm_vcpu *vcpu,
+ unsigned long active_mask)
+{
+ int i;
+
+ for (i = 0; i <= MAX_ISC; i++)
+ if (!(vcpu->arch.sie_block->gcr[6] & isc_to_isc_bits(i)))
+ active_mask &= ~(1UL << (IRQ_PEND_IO_ISC_0 + i));
+
+ return active_mask;
+}
+
+static unsigned long deliverable_irqs(struct kvm_vcpu *vcpu)
{
- unsigned long active_mask = pending_local_irqs(vcpu);
+ unsigned long active_mask;
+
+ active_mask = pending_local_irqs(vcpu);
+ active_mask |= pending_floating_irqs(vcpu);
if (psw_extint_disabled(vcpu))
active_mask &= ~IRQ_PEND_EXT_MASK;
+ if (psw_ioint_disabled(vcpu))
+ active_mask &= ~IRQ_PEND_IO_MASK;
+ else
+ active_mask = disable_iscs(vcpu, active_mask);
if (!(vcpu->arch.sie_block->gcr[0] & 0x2000ul))
__clear_bit(IRQ_PEND_EXT_EXTERNAL, &active_mask);
if (!(vcpu->arch.sie_block->gcr[0] & 0x4000ul))
__clear_bit(IRQ_PEND_EXT_CLOCK_COMP, &active_mask);
if (!(vcpu->arch.sie_block->gcr[0] & 0x400ul))
__clear_bit(IRQ_PEND_EXT_CPU_TIMER, &active_mask);
+ if (!(vcpu->arch.sie_block->gcr[0] & 0x200ul))
+ __clear_bit(IRQ_PEND_EXT_SERVICE, &active_mask);
if (psw_mchk_disabled(vcpu))
active_mask &= ~IRQ_PEND_MCHK_MASK;
+ if (!(vcpu->arch.sie_block->gcr[14] &
+ vcpu->kvm->arch.float_int.mchk.cr14))
+ __clear_bit(IRQ_PEND_MCHK_REP, &active_mask);
/*
* STOP irqs will never be actively delivered. They are triggered via
atomic_set_mask(flag, &vcpu->arch.sie_block->cpuflags);
}
+static void set_intercept_indicators_io(struct kvm_vcpu *vcpu)
+{
+ if (!(pending_floating_irqs(vcpu) & IRQ_PEND_IO_MASK))
+ return;
+ else if (psw_ioint_disabled(vcpu))
+ __set_cpuflag(vcpu, CPUSTAT_IO_INT);
+ else
+ vcpu->arch.sie_block->lctl |= LCTL_CR6;
+}
+
static void set_intercept_indicators_ext(struct kvm_vcpu *vcpu)
{
if (!(pending_local_irqs(vcpu) & IRQ_PEND_EXT_MASK))
__set_cpuflag(vcpu, CPUSTAT_STOP_INT);
}
-/* Set interception request for non-deliverable local interrupts */
-static void set_intercept_indicators_local(struct kvm_vcpu *vcpu)
+/* Set interception request for non-deliverable interrupts */
+static void set_intercept_indicators(struct kvm_vcpu *vcpu)
{
+ set_intercept_indicators_io(vcpu);
set_intercept_indicators_ext(vcpu);
set_intercept_indicators_mchk(vcpu);
set_intercept_indicators_stop(vcpu);
}
-static void __set_intercept_indicator(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
-{
- switch (inti->type) {
- case KVM_S390_INT_SERVICE:
- case KVM_S390_INT_PFAULT_DONE:
- case KVM_S390_INT_VIRTIO:
- if (psw_extint_disabled(vcpu))
- __set_cpuflag(vcpu, CPUSTAT_EXT_INT);
- else
- vcpu->arch.sie_block->lctl |= LCTL_CR0;
- break;
- case KVM_S390_MCHK:
- if (psw_mchk_disabled(vcpu))
- vcpu->arch.sie_block->ictl |= ICTL_LPSW;
- else
- vcpu->arch.sie_block->lctl |= LCTL_CR14;
- break;
- case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
- if (psw_ioint_disabled(vcpu))
- __set_cpuflag(vcpu, CPUSTAT_IO_INT);
- else
- vcpu->arch.sie_block->lctl |= LCTL_CR6;
- break;
- default:
- BUG();
- }
-}
-
static u16 get_ilc(struct kvm_vcpu *vcpu)
{
- const unsigned short table[] = { 2, 4, 4, 6 };
-
switch (vcpu->arch.sie_block->icptcode) {
case ICPT_INST:
case ICPT_INSTPROGI:
case ICPT_PARTEXEC:
case ICPT_IOINST:
/* last instruction only stored for these icptcodes */
- return table[vcpu->arch.sie_block->ipa >> 14];
+ return insn_length(vcpu->arch.sie_block->ipa >> 8);
case ICPT_PROGI:
return vcpu->arch.sie_block->pgmilc;
default:
static int __must_check __deliver_machine_check(struct kvm_vcpu *vcpu)
{
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- struct kvm_s390_mchk_info mchk;
- int rc;
+ struct kvm_s390_mchk_info mchk = {};
+ unsigned long adtl_status_addr;
+ int deliver = 0;
+ int rc = 0;
+ spin_lock(&fi->lock);
spin_lock(&li->lock);
- mchk = li->irq.mchk;
+ if (test_bit(IRQ_PEND_MCHK_EX, &li->pending_irqs) ||
+ test_bit(IRQ_PEND_MCHK_REP, &li->pending_irqs)) {
+ /*
+ * If there was an exigent machine check pending, then any
+ * repressible machine checks that might have been pending
+ * are indicated along with it, so always clear bits for
+ * repressible and exigent interrupts
+ */
+ mchk = li->irq.mchk;
+ clear_bit(IRQ_PEND_MCHK_EX, &li->pending_irqs);
+ clear_bit(IRQ_PEND_MCHK_REP, &li->pending_irqs);
+ memset(&li->irq.mchk, 0, sizeof(mchk));
+ deliver = 1;
+ }
/*
- * If there was an exigent machine check pending, then any repressible
- * machine checks that might have been pending are indicated along
- * with it, so always clear both bits
+ * We indicate floating repressible conditions along with
+ * other pending conditions. Channel Report Pending and Channel
+ * Subsystem damage are the only two and and are indicated by
+ * bits in mcic and masked in cr14.
*/
- clear_bit(IRQ_PEND_MCHK_EX, &li->pending_irqs);
- clear_bit(IRQ_PEND_MCHK_REP, &li->pending_irqs);
- memset(&li->irq.mchk, 0, sizeof(mchk));
+ if (test_and_clear_bit(IRQ_PEND_MCHK_REP, &fi->pending_irqs)) {
+ mchk.mcic |= fi->mchk.mcic;
+ mchk.cr14 |= fi->mchk.cr14;
+ memset(&fi->mchk, 0, sizeof(mchk));
+ deliver = 1;
+ }
spin_unlock(&li->lock);
+ spin_unlock(&fi->lock);
- VCPU_EVENT(vcpu, 4, "interrupt: machine check mcic=%llx",
- mchk.mcic);
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_MCHK,
- mchk.cr14, mchk.mcic);
-
- rc = kvm_s390_vcpu_store_status(vcpu, KVM_S390_STORE_STATUS_PREFIXED);
- rc |= put_guest_lc(vcpu, mchk.mcic,
- (u64 __user *) __LC_MCCK_CODE);
- rc |= put_guest_lc(vcpu, mchk.failing_storage_address,
- (u64 __user *) __LC_MCCK_FAIL_STOR_ADDR);
- rc |= write_guest_lc(vcpu, __LC_PSW_SAVE_AREA,
- &mchk.fixed_logout, sizeof(mchk.fixed_logout));
- rc |= write_guest_lc(vcpu, __LC_MCK_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_MCK_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
+ if (deliver) {
+ VCPU_EVENT(vcpu, 4, "interrupt: machine check mcic=%llx",
+ mchk.mcic);
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
+ KVM_S390_MCHK,
+ mchk.cr14, mchk.mcic);
+
+ rc = kvm_s390_vcpu_store_status(vcpu,
+ KVM_S390_STORE_STATUS_PREFIXED);
+ rc |= read_guest_lc(vcpu, __LC_VX_SAVE_AREA_ADDR,
+ &adtl_status_addr,
+ sizeof(unsigned long));
+ rc |= kvm_s390_vcpu_store_adtl_status(vcpu,
+ adtl_status_addr);
+ rc |= put_guest_lc(vcpu, mchk.mcic,
+ (u64 __user *) __LC_MCCK_CODE);
+ rc |= put_guest_lc(vcpu, mchk.failing_storage_address,
+ (u64 __user *) __LC_MCCK_FAIL_STOR_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_PSW_SAVE_AREA,
+ &mchk.fixed_logout,
+ sizeof(mchk.fixed_logout));
+ rc |= write_guest_lc(vcpu, __LC_MCK_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_MCK_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ }
return rc ? -EFAULT : 0;
}
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
struct kvm_s390_pgm_info pgm_info;
- int rc = 0;
+ int rc = 0, nullifying = false;
u16 ilc = get_ilc(vcpu);
spin_lock(&li->lock);
case PGM_LX_TRANSLATION:
case PGM_PRIMARY_AUTHORITY:
case PGM_SECONDARY_AUTHORITY:
+ nullifying = true;
+ /* fall through */
case PGM_SPACE_SWITCH:
rc = put_guest_lc(vcpu, pgm_info.trans_exc_code,
(u64 *)__LC_TRANS_EXC_CODE);
case PGM_EXTENDED_AUTHORITY:
rc = put_guest_lc(vcpu, pgm_info.exc_access_id,
(u8 *)__LC_EXC_ACCESS_ID);
+ nullifying = true;
break;
case PGM_ASCE_TYPE:
case PGM_PAGE_TRANSLATION:
(u8 *)__LC_EXC_ACCESS_ID);
rc |= put_guest_lc(vcpu, pgm_info.op_access_id,
(u8 *)__LC_OP_ACCESS_ID);
+ nullifying = true;
break;
case PGM_MONITOR:
rc = put_guest_lc(vcpu, pgm_info.mon_class_nr,
rc |= put_guest_lc(vcpu, pgm_info.mon_code,
(u64 *)__LC_MON_CODE);
break;
+ case PGM_VECTOR_PROCESSING:
case PGM_DATA:
rc = put_guest_lc(vcpu, pgm_info.data_exc_code,
(u32 *)__LC_DATA_EXC_CODE);
rc |= put_guest_lc(vcpu, pgm_info.exc_access_id,
(u8 *)__LC_EXC_ACCESS_ID);
break;
+ case PGM_STACK_FULL:
+ case PGM_STACK_EMPTY:
+ case PGM_STACK_SPECIFICATION:
+ case PGM_STACK_TYPE:
+ case PGM_STACK_OPERATION:
+ case PGM_TRACE_TABEL:
+ case PGM_CRYPTO_OPERATION:
+ nullifying = true;
+ break;
}
if (pgm_info.code & PGM_PER) {
(u8 *) __LC_PER_ACCESS_ID);
}
+ if (nullifying && vcpu->arch.sie_block->icptcode == ICPT_INST)
+ kvm_s390_rewind_psw(vcpu, ilc);
+
rc |= put_guest_lc(vcpu, ilc, (u16 *) __LC_PGM_ILC);
+ rc |= put_guest_lc(vcpu, vcpu->arch.sie_block->gbea,
+ (u64 *) __LC_LAST_BREAK);
rc |= put_guest_lc(vcpu, pgm_info.code,
(u16 *)__LC_PGM_INT_CODE);
rc |= write_guest_lc(vcpu, __LC_PGM_OLD_PSW,
return rc ? -EFAULT : 0;
}
-static int __must_check __deliver_service(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
+static int __must_check __deliver_service(struct kvm_vcpu *vcpu)
{
- int rc;
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
+ struct kvm_s390_ext_info ext;
+ int rc = 0;
+
+ spin_lock(&fi->lock);
+ if (!(test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs))) {
+ spin_unlock(&fi->lock);
+ return 0;
+ }
+ ext = fi->srv_signal;
+ memset(&fi->srv_signal, 0, sizeof(ext));
+ clear_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
VCPU_EVENT(vcpu, 4, "interrupt: sclp parm:%x",
- inti->ext.ext_params);
+ ext.ext_params);
vcpu->stat.deliver_service_signal++;
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, inti->type,
- inti->ext.ext_params, 0);
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_INT_SERVICE,
+ ext.ext_params, 0);
rc = put_guest_lc(vcpu, EXT_IRQ_SERVICE_SIG, (u16 *)__LC_EXT_INT_CODE);
rc |= put_guest_lc(vcpu, 0, (u16 *)__LC_EXT_CPU_ADDR);
&vcpu->arch.sie_block->gpsw, sizeof(psw_t));
rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
&vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= put_guest_lc(vcpu, inti->ext.ext_params,
+ rc |= put_guest_lc(vcpu, ext.ext_params,
(u32 *)__LC_EXT_PARAMS);
+
return rc ? -EFAULT : 0;
}
-static int __must_check __deliver_pfault_done(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
+static int __must_check __deliver_pfault_done(struct kvm_vcpu *vcpu)
{
- int rc;
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
+ struct kvm_s390_interrupt_info *inti;
+ int rc = 0;
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
- KVM_S390_INT_PFAULT_DONE, 0,
- inti->ext.ext_params2);
+ spin_lock(&fi->lock);
+ inti = list_first_entry_or_null(&fi->lists[FIRQ_LIST_PFAULT],
+ struct kvm_s390_interrupt_info,
+ list);
+ if (inti) {
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
+ KVM_S390_INT_PFAULT_DONE, 0,
+ inti->ext.ext_params2);
+ list_del(&inti->list);
+ fi->counters[FIRQ_CNTR_PFAULT] -= 1;
+ }
+ if (list_empty(&fi->lists[FIRQ_LIST_PFAULT]))
+ clear_bit(IRQ_PEND_PFAULT_DONE, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
- rc = put_guest_lc(vcpu, EXT_IRQ_CP_SERVICE, (u16 *)__LC_EXT_INT_CODE);
- rc |= put_guest_lc(vcpu, PFAULT_DONE, (u16 *)__LC_EXT_CPU_ADDR);
- rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= put_guest_lc(vcpu, inti->ext.ext_params2,
- (u64 *)__LC_EXT_PARAMS2);
+ if (inti) {
+ rc = put_guest_lc(vcpu, EXT_IRQ_CP_SERVICE,
+ (u16 *)__LC_EXT_INT_CODE);
+ rc |= put_guest_lc(vcpu, PFAULT_DONE,
+ (u16 *)__LC_EXT_CPU_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= put_guest_lc(vcpu, inti->ext.ext_params2,
+ (u64 *)__LC_EXT_PARAMS2);
+ kfree(inti);
+ }
return rc ? -EFAULT : 0;
}
-static int __must_check __deliver_virtio(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
+static int __must_check __deliver_virtio(struct kvm_vcpu *vcpu)
{
- int rc;
+ struct kvm_s390_float_interrupt *fi = &vcpu->kvm->arch.float_int;
+ struct kvm_s390_interrupt_info *inti;
+ int rc = 0;
- VCPU_EVENT(vcpu, 4, "interrupt: virtio parm:%x,parm64:%llx",
- inti->ext.ext_params, inti->ext.ext_params2);
- vcpu->stat.deliver_virtio_interrupt++;
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, inti->type,
- inti->ext.ext_params,
- inti->ext.ext_params2);
+ spin_lock(&fi->lock);
+ inti = list_first_entry_or_null(&fi->lists[FIRQ_LIST_VIRTIO],
+ struct kvm_s390_interrupt_info,
+ list);
+ if (inti) {
+ VCPU_EVENT(vcpu, 4,
+ "interrupt: virtio parm:%x,parm64:%llx",
+ inti->ext.ext_params, inti->ext.ext_params2);
+ vcpu->stat.deliver_virtio_interrupt++;
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
+ inti->type,
+ inti->ext.ext_params,
+ inti->ext.ext_params2);
+ list_del(&inti->list);
+ fi->counters[FIRQ_CNTR_VIRTIO] -= 1;
+ }
+ if (list_empty(&fi->lists[FIRQ_LIST_VIRTIO]))
+ clear_bit(IRQ_PEND_VIRTIO, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
- rc = put_guest_lc(vcpu, EXT_IRQ_CP_SERVICE, (u16 *)__LC_EXT_INT_CODE);
- rc |= put_guest_lc(vcpu, VIRTIO_PARAM, (u16 *)__LC_EXT_CPU_ADDR);
- rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= put_guest_lc(vcpu, inti->ext.ext_params,
- (u32 *)__LC_EXT_PARAMS);
- rc |= put_guest_lc(vcpu, inti->ext.ext_params2,
- (u64 *)__LC_EXT_PARAMS2);
+ if (inti) {
+ rc = put_guest_lc(vcpu, EXT_IRQ_CP_SERVICE,
+ (u16 *)__LC_EXT_INT_CODE);
+ rc |= put_guest_lc(vcpu, VIRTIO_PARAM,
+ (u16 *)__LC_EXT_CPU_ADDR);
+ rc |= write_guest_lc(vcpu, __LC_EXT_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_EXT_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= put_guest_lc(vcpu, inti->ext.ext_params,
+ (u32 *)__LC_EXT_PARAMS);
+ rc |= put_guest_lc(vcpu, inti->ext.ext_params2,
+ (u64 *)__LC_EXT_PARAMS2);
+ kfree(inti);
+ }
return rc ? -EFAULT : 0;
}
static int __must_check __deliver_io(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
+ unsigned long irq_type)
{
- int rc;
+ struct list_head *isc_list;
+ struct kvm_s390_float_interrupt *fi;
+ struct kvm_s390_interrupt_info *inti = NULL;
+ int rc = 0;
- VCPU_EVENT(vcpu, 4, "interrupt: I/O %llx", inti->type);
- vcpu->stat.deliver_io_int++;
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, inti->type,
- ((__u32)inti->io.subchannel_id << 16) |
- inti->io.subchannel_nr,
- ((__u64)inti->io.io_int_parm << 32) |
- inti->io.io_int_word);
-
- rc = put_guest_lc(vcpu, inti->io.subchannel_id,
- (u16 *)__LC_SUBCHANNEL_ID);
- rc |= put_guest_lc(vcpu, inti->io.subchannel_nr,
- (u16 *)__LC_SUBCHANNEL_NR);
- rc |= put_guest_lc(vcpu, inti->io.io_int_parm,
- (u32 *)__LC_IO_INT_PARM);
- rc |= put_guest_lc(vcpu, inti->io.io_int_word,
- (u32 *)__LC_IO_INT_WORD);
- rc |= write_guest_lc(vcpu, __LC_IO_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_IO_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- return rc ? -EFAULT : 0;
-}
+ fi = &vcpu->kvm->arch.float_int;
-static int __must_check __deliver_mchk_floating(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
-{
- struct kvm_s390_mchk_info *mchk = &inti->mchk;
- int rc;
+ spin_lock(&fi->lock);
+ isc_list = &fi->lists[irq_type - IRQ_PEND_IO_ISC_0];
+ inti = list_first_entry_or_null(isc_list,
+ struct kvm_s390_interrupt_info,
+ list);
+ if (inti) {
+ VCPU_EVENT(vcpu, 4, "interrupt: I/O %llx", inti->type);
+ vcpu->stat.deliver_io_int++;
+ trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id,
+ inti->type,
+ ((__u32)inti->io.subchannel_id << 16) |
+ inti->io.subchannel_nr,
+ ((__u64)inti->io.io_int_parm << 32) |
+ inti->io.io_int_word);
+ list_del(&inti->list);
+ fi->counters[FIRQ_CNTR_IO] -= 1;
+ }
+ if (list_empty(isc_list))
+ clear_bit(irq_type, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+
+ if (inti) {
+ rc = put_guest_lc(vcpu, inti->io.subchannel_id,
+ (u16 *)__LC_SUBCHANNEL_ID);
+ rc |= put_guest_lc(vcpu, inti->io.subchannel_nr,
+ (u16 *)__LC_SUBCHANNEL_NR);
+ rc |= put_guest_lc(vcpu, inti->io.io_int_parm,
+ (u32 *)__LC_IO_INT_PARM);
+ rc |= put_guest_lc(vcpu, inti->io.io_int_word,
+ (u32 *)__LC_IO_INT_WORD);
+ rc |= write_guest_lc(vcpu, __LC_IO_OLD_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ rc |= read_guest_lc(vcpu, __LC_IO_NEW_PSW,
+ &vcpu->arch.sie_block->gpsw,
+ sizeof(psw_t));
+ kfree(inti);
+ }
- VCPU_EVENT(vcpu, 4, "interrupt: machine check mcic=%llx",
- mchk->mcic);
- trace_kvm_s390_deliver_interrupt(vcpu->vcpu_id, KVM_S390_MCHK,
- mchk->cr14, mchk->mcic);
-
- rc = kvm_s390_vcpu_store_status(vcpu, KVM_S390_STORE_STATUS_PREFIXED);
- rc |= put_guest_lc(vcpu, mchk->mcic,
- (u64 __user *) __LC_MCCK_CODE);
- rc |= put_guest_lc(vcpu, mchk->failing_storage_address,
- (u64 __user *) __LC_MCCK_FAIL_STOR_ADDR);
- rc |= write_guest_lc(vcpu, __LC_PSW_SAVE_AREA,
- &mchk->fixed_logout, sizeof(mchk->fixed_logout));
- rc |= write_guest_lc(vcpu, __LC_MCK_OLD_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
- rc |= read_guest_lc(vcpu, __LC_MCK_NEW_PSW,
- &vcpu->arch.sie_block->gpsw, sizeof(psw_t));
return rc ? -EFAULT : 0;
}
static const deliver_irq_t deliver_irq_funcs[] = {
[IRQ_PEND_MCHK_EX] = __deliver_machine_check,
+ [IRQ_PEND_MCHK_REP] = __deliver_machine_check,
[IRQ_PEND_PROG] = __deliver_prog,
[IRQ_PEND_EXT_EMERGENCY] = __deliver_emergency_signal,
[IRQ_PEND_EXT_EXTERNAL] = __deliver_external_call,
[IRQ_PEND_RESTART] = __deliver_restart,
[IRQ_PEND_SET_PREFIX] = __deliver_set_prefix,
[IRQ_PEND_PFAULT_INIT] = __deliver_pfault_init,
+ [IRQ_PEND_EXT_SERVICE] = __deliver_service,
+ [IRQ_PEND_PFAULT_DONE] = __deliver_pfault_done,
+ [IRQ_PEND_VIRTIO] = __deliver_virtio,
};
-static int __must_check __deliver_floating_interrupt(struct kvm_vcpu *vcpu,
- struct kvm_s390_interrupt_info *inti)
-{
- int rc;
-
- switch (inti->type) {
- case KVM_S390_INT_SERVICE:
- rc = __deliver_service(vcpu, inti);
- break;
- case KVM_S390_INT_PFAULT_DONE:
- rc = __deliver_pfault_done(vcpu, inti);
- break;
- case KVM_S390_INT_VIRTIO:
- rc = __deliver_virtio(vcpu, inti);
- break;
- case KVM_S390_MCHK:
- rc = __deliver_mchk_floating(vcpu, inti);
- break;
- case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
- rc = __deliver_io(vcpu, inti);
- break;
- default:
- BUG();
- }
-
- return rc;
-}
-
/* Check whether an external call is pending (deliverable or not) */
int kvm_s390_ext_call_pending(struct kvm_vcpu *vcpu)
{
int kvm_s390_vcpu_has_irq(struct kvm_vcpu *vcpu, int exclude_stop)
{
- struct kvm_s390_float_interrupt *fi = vcpu->arch.local_int.float_int;
- struct kvm_s390_interrupt_info *inti;
int rc;
- rc = !!deliverable_local_irqs(vcpu);
-
- if ((!rc) && atomic_read(&fi->active)) {
- spin_lock(&fi->lock);
- list_for_each_entry(inti, &fi->list, list)
- if (__interrupt_is_deliverable(vcpu, inti)) {
- rc = 1;
- break;
- }
- spin_unlock(&fi->lock);
- }
+ rc = !!deliverable_irqs(vcpu);
if (!rc && kvm_cpu_has_pending_timer(vcpu))
rc = 1;
int kvm_cpu_has_pending_timer(struct kvm_vcpu *vcpu)
{
- if (!(vcpu->arch.sie_block->ckc <
- get_tod_clock_fast() + vcpu->arch.sie_block->epoch))
- return 0;
- if (!ckc_interrupts_enabled(vcpu))
- return 0;
- return 1;
+ return ckc_irq_pending(vcpu) || cpu_timer_irq_pending(vcpu);
}
int kvm_s390_handle_wait(struct kvm_vcpu *vcpu)
int __must_check kvm_s390_deliver_pending_interrupts(struct kvm_vcpu *vcpu)
{
struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
- struct kvm_s390_float_interrupt *fi = vcpu->arch.local_int.float_int;
- struct kvm_s390_interrupt_info *n, *inti = NULL;
deliver_irq_t func;
- int deliver;
int rc = 0;
unsigned long irq_type;
- unsigned long deliverable_irqs;
+ unsigned long irqs;
__reset_intercept_indicators(vcpu);
/* pending ckc conditions might have been invalidated */
clear_bit(IRQ_PEND_EXT_CLOCK_COMP, &li->pending_irqs);
- if (kvm_cpu_has_pending_timer(vcpu))
+ if (ckc_irq_pending(vcpu))
set_bit(IRQ_PEND_EXT_CLOCK_COMP, &li->pending_irqs);
+ /* pending cpu timer conditions might have been invalidated */
+ clear_bit(IRQ_PEND_EXT_CPU_TIMER, &li->pending_irqs);
+ if (cpu_timer_irq_pending(vcpu))
+ set_bit(IRQ_PEND_EXT_CPU_TIMER, &li->pending_irqs);
+
do {
- deliverable_irqs = deliverable_local_irqs(vcpu);
+ irqs = deliverable_irqs(vcpu);
/* bits are in the order of interrupt priority */
- irq_type = find_first_bit(&deliverable_irqs, IRQ_PEND_COUNT);
+ irq_type = find_first_bit(&irqs, IRQ_PEND_COUNT);
if (irq_type == IRQ_PEND_COUNT)
break;
- func = deliver_irq_funcs[irq_type];
- if (!func) {
- WARN_ON_ONCE(func == NULL);
- clear_bit(irq_type, &li->pending_irqs);
- continue;
+ if (is_ioirq(irq_type)) {
+ rc = __deliver_io(vcpu, irq_type);
+ } else {
+ func = deliver_irq_funcs[irq_type];
+ if (!func) {
+ WARN_ON_ONCE(func == NULL);
+ clear_bit(irq_type, &li->pending_irqs);
+ continue;
+ }
+ rc = func(vcpu);
}
- rc = func(vcpu);
- } while (!rc && irq_type != IRQ_PEND_COUNT);
+ if (rc)
+ break;
+ } while (!rc);
- set_intercept_indicators_local(vcpu);
-
- if (!rc && atomic_read(&fi->active)) {
- do {
- deliver = 0;
- spin_lock(&fi->lock);
- list_for_each_entry_safe(inti, n, &fi->list, list) {
- if (__interrupt_is_deliverable(vcpu, inti)) {
- list_del(&inti->list);
- fi->irq_count--;
- deliver = 1;
- break;
- }
- __set_intercept_indicator(vcpu, inti);
- }
- if (list_empty(&fi->list))
- atomic_set(&fi->active, 0);
- spin_unlock(&fi->lock);
- if (deliver) {
- rc = __deliver_floating_interrupt(vcpu, inti);
- kfree(inti);
- }
- } while (!rc && deliver);
- }
+ set_intercept_indicators(vcpu);
return rc;
}
return 0;
}
+static struct kvm_s390_interrupt_info *get_io_int(struct kvm *kvm,
+ int isc, u32 schid)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+ struct list_head *isc_list = &fi->lists[FIRQ_LIST_IO_ISC_0 + isc];
+ struct kvm_s390_interrupt_info *iter;
+ u16 id = (schid & 0xffff0000U) >> 16;
+ u16 nr = schid & 0x0000ffffU;
+ spin_lock(&fi->lock);
+ list_for_each_entry(iter, isc_list, list) {
+ if (schid && (id != iter->io.subchannel_id ||
+ nr != iter->io.subchannel_nr))
+ continue;
+ /* found an appropriate entry */
+ list_del_init(&iter->list);
+ fi->counters[FIRQ_CNTR_IO] -= 1;
+ if (list_empty(isc_list))
+ clear_bit(IRQ_PEND_IO_ISC_0 + isc, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+ return iter;
+ }
+ spin_unlock(&fi->lock);
+ return NULL;
+}
+
+/*
+ * Dequeue and return an I/O interrupt matching any of the interruption
+ * subclasses as designated by the isc mask in cr6 and the schid (if != 0).
+ */
struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,
- u64 cr6, u64 schid)
+ u64 isc_mask, u32 schid)
+{
+ struct kvm_s390_interrupt_info *inti = NULL;
+ int isc;
+
+ for (isc = 0; isc <= MAX_ISC && !inti; isc++) {
+ if (isc_mask & isc_to_isc_bits(isc))
+ inti = get_io_int(kvm, isc, schid);
+ }
+ return inti;
+}
+
+#define SCCB_MASK 0xFFFFFFF8
+#define SCCB_EVENT_PENDING 0x3
+
+static int __inject_service(struct kvm *kvm,
+ struct kvm_s390_interrupt_info *inti)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+
+ spin_lock(&fi->lock);
+ fi->srv_signal.ext_params |= inti->ext.ext_params & SCCB_EVENT_PENDING;
+ /*
+ * Early versions of the QEMU s390 bios will inject several
+ * service interrupts after another without handling a
+ * condition code indicating busy.
+ * We will silently ignore those superfluous sccb values.
+ * A future version of QEMU will take care of serialization
+ * of servc requests
+ */
+ if (fi->srv_signal.ext_params & SCCB_MASK)
+ goto out;
+ fi->srv_signal.ext_params |= inti->ext.ext_params & SCCB_MASK;
+ set_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs);
+out:
+ spin_unlock(&fi->lock);
+ kfree(inti);
+ return 0;
+}
+
+static int __inject_virtio(struct kvm *kvm,
+ struct kvm_s390_interrupt_info *inti)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+
+ spin_lock(&fi->lock);
+ if (fi->counters[FIRQ_CNTR_VIRTIO] >= KVM_S390_MAX_VIRTIO_IRQS) {
+ spin_unlock(&fi->lock);
+ return -EBUSY;
+ }
+ fi->counters[FIRQ_CNTR_VIRTIO] += 1;
+ list_add_tail(&inti->list, &fi->lists[FIRQ_LIST_VIRTIO]);
+ set_bit(IRQ_PEND_VIRTIO, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+ return 0;
+}
+
+static int __inject_pfault_done(struct kvm *kvm,
+ struct kvm_s390_interrupt_info *inti)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+
+ spin_lock(&fi->lock);
+ if (fi->counters[FIRQ_CNTR_PFAULT] >=
+ (ASYNC_PF_PER_VCPU * KVM_MAX_VCPUS)) {
+ spin_unlock(&fi->lock);
+ return -EBUSY;
+ }
+ fi->counters[FIRQ_CNTR_PFAULT] += 1;
+ list_add_tail(&inti->list, &fi->lists[FIRQ_LIST_PFAULT]);
+ set_bit(IRQ_PEND_PFAULT_DONE, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+ return 0;
+}
+
+#define CR_PENDING_SUBCLASS 28
+static int __inject_float_mchk(struct kvm *kvm,
+ struct kvm_s390_interrupt_info *inti)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+
+ spin_lock(&fi->lock);
+ fi->mchk.cr14 |= inti->mchk.cr14 & (1UL << CR_PENDING_SUBCLASS);
+ fi->mchk.mcic |= inti->mchk.mcic;
+ set_bit(IRQ_PEND_MCHK_REP, &fi->pending_irqs);
+ spin_unlock(&fi->lock);
+ kfree(inti);
+ return 0;
+}
+
+static int __inject_io(struct kvm *kvm, struct kvm_s390_interrupt_info *inti)
{
struct kvm_s390_float_interrupt *fi;
- struct kvm_s390_interrupt_info *inti, *iter;
+ struct list_head *list;
+ int isc;
- if ((!schid && !cr6) || (schid && cr6))
- return NULL;
fi = &kvm->arch.float_int;
spin_lock(&fi->lock);
- inti = NULL;
- list_for_each_entry(iter, &fi->list, list) {
- if (!is_ioint(iter->type))
- continue;
- if (cr6 &&
- ((cr6 & int_word_to_isc_bits(iter->io.io_int_word)) == 0))
- continue;
- if (schid) {
- if (((schid & 0x00000000ffff0000) >> 16) !=
- iter->io.subchannel_id)
- continue;
- if ((schid & 0x000000000000ffff) !=
- iter->io.subchannel_nr)
- continue;
- }
- inti = iter;
- break;
- }
- if (inti) {
- list_del_init(&inti->list);
- fi->irq_count--;
+ if (fi->counters[FIRQ_CNTR_IO] >= KVM_S390_MAX_FLOAT_IRQS) {
+ spin_unlock(&fi->lock);
+ return -EBUSY;
}
- if (list_empty(&fi->list))
- atomic_set(&fi->active, 0);
+ fi->counters[FIRQ_CNTR_IO] += 1;
+
+ isc = int_word_to_isc(inti->io.io_int_word);
+ list = &fi->lists[FIRQ_LIST_IO_ISC_0 + isc];
+ list_add_tail(&inti->list, list);
+ set_bit(IRQ_PEND_IO_ISC_0 + isc, &fi->pending_irqs);
spin_unlock(&fi->lock);
- return inti;
+ return 0;
}
static int __inject_vm(struct kvm *kvm, struct kvm_s390_interrupt_info *inti)
{
struct kvm_s390_local_interrupt *li;
struct kvm_s390_float_interrupt *fi;
- struct kvm_s390_interrupt_info *iter;
struct kvm_vcpu *dst_vcpu = NULL;
int sigcpu;
- int rc = 0;
+ u64 type = READ_ONCE(inti->type);
+ int rc;
fi = &kvm->arch.float_int;
- spin_lock(&fi->lock);
- if (fi->irq_count >= KVM_S390_MAX_FLOAT_IRQS) {
+
+ switch (type) {
+ case KVM_S390_MCHK:
+ rc = __inject_float_mchk(kvm, inti);
+ break;
+ case KVM_S390_INT_VIRTIO:
+ rc = __inject_virtio(kvm, inti);
+ break;
+ case KVM_S390_INT_SERVICE:
+ rc = __inject_service(kvm, inti);
+ break;
+ case KVM_S390_INT_PFAULT_DONE:
+ rc = __inject_pfault_done(kvm, inti);
+ break;
+ case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
+ rc = __inject_io(kvm, inti);
+ break;
+ default:
rc = -EINVAL;
- goto unlock_fi;
}
- fi->irq_count++;
- if (!is_ioint(inti->type)) {
- list_add_tail(&inti->list, &fi->list);
- } else {
- u64 isc_bits = int_word_to_isc_bits(inti->io.io_int_word);
+ if (rc)
+ return rc;
- /* Keep I/O interrupts sorted in isc order. */
- list_for_each_entry(iter, &fi->list, list) {
- if (!is_ioint(iter->type))
- continue;
- if (int_word_to_isc_bits(iter->io.io_int_word)
- <= isc_bits)
- continue;
- break;
- }
- list_add_tail(&inti->list, &iter->list);
- }
- atomic_set(&fi->active, 1);
- if (atomic_read(&kvm->online_vcpus) == 0)
- goto unlock_fi;
sigcpu = find_first_bit(fi->idle_mask, KVM_MAX_VCPUS);
if (sigcpu == KVM_MAX_VCPUS) {
do {
dst_vcpu = kvm_get_vcpu(kvm, sigcpu);
li = &dst_vcpu->arch.local_int;
spin_lock(&li->lock);
- switch (inti->type) {
+ switch (type) {
case KVM_S390_MCHK:
atomic_set_mask(CPUSTAT_STOP_INT, li->cpuflags);
break;
}
spin_unlock(&li->lock);
kvm_s390_vcpu_wakeup(kvm_get_vcpu(kvm, sigcpu));
-unlock_fi:
- spin_unlock(&fi->lock);
- return rc;
+ return 0;
+
}
int kvm_s390_inject_vm(struct kvm *kvm,
return rc;
}
-void kvm_s390_reinject_io_int(struct kvm *kvm,
+int kvm_s390_reinject_io_int(struct kvm *kvm,
struct kvm_s390_interrupt_info *inti)
{
- __inject_vm(kvm, inti);
+ return __inject_vm(kvm, inti);
}
int s390int_to_s390irq(struct kvm_s390_interrupt *s390int,
spin_unlock(&li->lock);
}
-int kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
+static int do_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
{
- struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
int rc;
- spin_lock(&li->lock);
switch (irq->type) {
case KVM_S390_PROGRAM_INT:
VCPU_EVENT(vcpu, 3, "inject: program check %d (from user)",
default:
rc = -EINVAL;
}
+
+ return rc;
+}
+
+int kvm_s390_inject_vcpu(struct kvm_vcpu *vcpu, struct kvm_s390_irq *irq)
+{
+ struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
+ int rc;
+
+ spin_lock(&li->lock);
+ rc = do_inject_vcpu(vcpu, irq);
spin_unlock(&li->lock);
if (!rc)
kvm_s390_vcpu_wakeup(vcpu);
return rc;
}
-void kvm_s390_clear_float_irqs(struct kvm *kvm)
+static inline void clear_irq_list(struct list_head *_list)
{
- struct kvm_s390_float_interrupt *fi;
- struct kvm_s390_interrupt_info *n, *inti = NULL;
+ struct kvm_s390_interrupt_info *inti, *n;
- fi = &kvm->arch.float_int;
- spin_lock(&fi->lock);
- list_for_each_entry_safe(inti, n, &fi->list, list) {
+ list_for_each_entry_safe(inti, n, _list, list) {
list_del(&inti->list);
kfree(inti);
}
- fi->irq_count = 0;
- atomic_set(&fi->active, 0);
- spin_unlock(&fi->lock);
}
-static inline int copy_irq_to_user(struct kvm_s390_interrupt_info *inti,
- u8 *addr)
+static void inti_to_irq(struct kvm_s390_interrupt_info *inti,
+ struct kvm_s390_irq *irq)
{
- struct kvm_s390_irq __user *uptr = (struct kvm_s390_irq __user *) addr;
- struct kvm_s390_irq irq = {0};
-
- irq.type = inti->type;
+ irq->type = inti->type;
switch (inti->type) {
case KVM_S390_INT_PFAULT_INIT:
case KVM_S390_INT_PFAULT_DONE:
case KVM_S390_INT_VIRTIO:
- case KVM_S390_INT_SERVICE:
- irq.u.ext = inti->ext;
+ irq->u.ext = inti->ext;
break;
case KVM_S390_INT_IO_MIN...KVM_S390_INT_IO_MAX:
- irq.u.io = inti->io;
+ irq->u.io = inti->io;
break;
- case KVM_S390_MCHK:
- irq.u.mchk = inti->mchk;
- break;
- default:
- return -EINVAL;
}
+}
- if (copy_to_user(uptr, &irq, sizeof(irq)))
- return -EFAULT;
+void kvm_s390_clear_float_irqs(struct kvm *kvm)
+{
+ struct kvm_s390_float_interrupt *fi = &kvm->arch.float_int;
+ int i;
- return 0;
-}
+ spin_lock(&fi->lock);
+ for (i = 0; i < FIRQ_LIST_COUNT; i++)
+ clear_irq_list(&fi->lists[i]);
+ for (i = 0; i < FIRQ_MAX_COUNT; i++)
+ fi->counters[i] = 0;
+ spin_unlock(&fi->lock);
+};
-static int get_all_floating_irqs(struct kvm *kvm, __u8 *buf, __u64 len)
+static int get_all_floating_irqs(struct kvm *kvm, u8 __user *usrbuf, u64 len)
{
struct kvm_s390_interrupt_info *inti;
struct kvm_s390_float_interrupt *fi;
+ struct kvm_s390_irq *buf;
+ struct kvm_s390_irq *irq;
+ int max_irqs;
int ret = 0;
int n = 0;
+ int i;
+
+ if (len > KVM_S390_FLIC_MAX_BUFFER || len == 0)
+ return -EINVAL;
+
+ /*
+ * We are already using -ENOMEM to signal
+ * userspace it may retry with a bigger buffer,
+ * so we need to use something else for this case
+ */
+ buf = vzalloc(len);
+ if (!buf)
+ return -ENOBUFS;
+
+ max_irqs = len / sizeof(struct kvm_s390_irq);
fi = &kvm->arch.float_int;
spin_lock(&fi->lock);
-
- list_for_each_entry(inti, &fi->list, list) {
- if (len < sizeof(struct kvm_s390_irq)) {
+ for (i = 0; i < FIRQ_LIST_COUNT; i++) {
+ list_for_each_entry(inti, &fi->lists[i], list) {
+ if (n == max_irqs) {
+ /* signal userspace to try again */
+ ret = -ENOMEM;
+ goto out;
+ }
+ inti_to_irq(inti, &buf[n]);
+ n++;
+ }
+ }
+ if (test_bit(IRQ_PEND_EXT_SERVICE, &fi->pending_irqs)) {
+ if (n == max_irqs) {
/* signal userspace to try again */
ret = -ENOMEM;
- break;
+ goto out;
}
- ret = copy_irq_to_user(inti, buf);
- if (ret)
- break;
- buf += sizeof(struct kvm_s390_irq);
- len -= sizeof(struct kvm_s390_irq);
+ irq = (struct kvm_s390_irq *) &buf[n];
+ irq->type = KVM_S390_INT_SERVICE;
+ irq->u.ext = fi->srv_signal;
n++;
}
+ if (test_bit(IRQ_PEND_MCHK_REP, &fi->pending_irqs)) {
+ if (n == max_irqs) {
+ /* signal userspace to try again */
+ ret = -ENOMEM;
+ goto out;
+ }
+ irq = (struct kvm_s390_irq *) &buf[n];
+ irq->type = KVM_S390_MCHK;
+ irq->u.mchk = fi->mchk;
+ n++;
+}
+out:
spin_unlock(&fi->lock);
+ if (!ret && n > 0) {
+ if (copy_to_user(usrbuf, buf, sizeof(struct kvm_s390_irq) * n))
+ ret = -EFAULT;
+ }
+ vfree(buf);
return ret < 0 ? ret : n;
}
switch (attr->group) {
case KVM_DEV_FLIC_GET_ALL_IRQS:
- r = get_all_floating_irqs(dev->kvm, (u8 *) attr->addr,
+ r = get_all_floating_irqs(dev->kvm, (u8 __user *) attr->addr,
attr->attr);
break;
default:
{
return -EINVAL;
}
+
+int kvm_s390_set_irq_state(struct kvm_vcpu *vcpu, void __user *irqstate, int len)
+{
+ struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
+ struct kvm_s390_irq *buf;
+ int r = 0;
+ int n;
+
+ buf = vmalloc(len);
+ if (!buf)
+ return -ENOMEM;
+
+ if (copy_from_user((void *) buf, irqstate, len)) {
+ r = -EFAULT;
+ goto out_free;
+ }
+
+ /*
+ * Don't allow setting the interrupt state
+ * when there are already interrupts pending
+ */
+ spin_lock(&li->lock);
+ if (li->pending_irqs) {
+ r = -EBUSY;
+ goto out_unlock;
+ }
+
+ for (n = 0; n < len / sizeof(*buf); n++) {
+ r = do_inject_vcpu(vcpu, &buf[n]);
+ if (r)
+ break;
+ }
+
+out_unlock:
+ spin_unlock(&li->lock);
+out_free:
+ vfree(buf);
+
+ return r;
+}
+
+static void store_local_irq(struct kvm_s390_local_interrupt *li,
+ struct kvm_s390_irq *irq,
+ unsigned long irq_type)
+{
+ switch (irq_type) {
+ case IRQ_PEND_MCHK_EX:
+ case IRQ_PEND_MCHK_REP:
+ irq->type = KVM_S390_MCHK;
+ irq->u.mchk = li->irq.mchk;
+ break;
+ case IRQ_PEND_PROG:
+ irq->type = KVM_S390_PROGRAM_INT;
+ irq->u.pgm = li->irq.pgm;
+ break;
+ case IRQ_PEND_PFAULT_INIT:
+ irq->type = KVM_S390_INT_PFAULT_INIT;
+ irq->u.ext = li->irq.ext;
+ break;
+ case IRQ_PEND_EXT_EXTERNAL:
+ irq->type = KVM_S390_INT_EXTERNAL_CALL;
+ irq->u.extcall = li->irq.extcall;
+ break;
+ case IRQ_PEND_EXT_CLOCK_COMP:
+ irq->type = KVM_S390_INT_CLOCK_COMP;
+ break;
+ case IRQ_PEND_EXT_CPU_TIMER:
+ irq->type = KVM_S390_INT_CPU_TIMER;
+ break;
+ case IRQ_PEND_SIGP_STOP:
+ irq->type = KVM_S390_SIGP_STOP;
+ irq->u.stop = li->irq.stop;
+ break;
+ case IRQ_PEND_RESTART:
+ irq->type = KVM_S390_RESTART;
+ break;
+ case IRQ_PEND_SET_PREFIX:
+ irq->type = KVM_S390_SIGP_SET_PREFIX;
+ irq->u.prefix = li->irq.prefix;
+ break;
+ }
+}
+
+int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu, __u8 __user *buf, int len)
+{
+ uint8_t sigp_ctrl = vcpu->kvm->arch.sca->cpu[vcpu->vcpu_id].sigp_ctrl;
+ unsigned long sigp_emerg_pending[BITS_TO_LONGS(KVM_MAX_VCPUS)];
+ struct kvm_s390_local_interrupt *li = &vcpu->arch.local_int;
+ unsigned long pending_irqs;
+ struct kvm_s390_irq irq;
+ unsigned long irq_type;
+ int cpuaddr;
+ int n = 0;
+
+ spin_lock(&li->lock);
+ pending_irqs = li->pending_irqs;
+ memcpy(&sigp_emerg_pending, &li->sigp_emerg_pending,
+ sizeof(sigp_emerg_pending));
+ spin_unlock(&li->lock);
+
+ for_each_set_bit(irq_type, &pending_irqs, IRQ_PEND_COUNT) {
+ memset(&irq, 0, sizeof(irq));
+ if (irq_type == IRQ_PEND_EXT_EMERGENCY)
+ continue;
+ if (n + sizeof(irq) > len)
+ return -ENOBUFS;
+ store_local_irq(&vcpu->arch.local_int, &irq, irq_type);
+ if (copy_to_user(&buf[n], &irq, sizeof(irq)))
+ return -EFAULT;
+ n += sizeof(irq);
+ }
+
+ if (test_bit(IRQ_PEND_EXT_EMERGENCY, &pending_irqs)) {
+ for_each_set_bit(cpuaddr, sigp_emerg_pending, KVM_MAX_VCPUS) {
+ memset(&irq, 0, sizeof(irq));
+ if (n + sizeof(irq) > len)
+ return -ENOBUFS;
+ irq.type = KVM_S390_INT_EMERGENCY;
+ irq.u.emerg.code = cpuaddr;
+ if (copy_to_user(&buf[n], &irq, sizeof(irq)))
+ return -EFAULT;
+ n += sizeof(irq);
+ }
+ }
+
+ if ((sigp_ctrl & SIGP_CTRL_C) &&
+ (atomic_read(&vcpu->arch.sie_block->cpuflags) &
+ CPUSTAT_ECALL_PEND)) {
+ if (n + sizeof(irq) > len)
+ return -ENOBUFS;
+ memset(&irq, 0, sizeof(irq));
+ irq.type = KVM_S390_INT_EXTERNAL_CALL;
+ irq.u.extcall.code = sigp_ctrl & SIGP_CTRL_SCN_MASK;
+ if (copy_to_user(&buf[n], &irq, sizeof(irq)))
+ return -EFAULT;
+ n += sizeof(irq);
+ }
+
+ return n;
+}
#include <linux/random.h>
#include <linux/slab.h>
#include <linux/timer.h>
+#include <linux/vmalloc.h>
#include <asm/asm-offsets.h>
#include <asm/lowcore.h>
#include <asm/pgtable.h>
#include <asm/nmi.h>
#include <asm/switch_to.h>
+#include <asm/isc.h>
#include <asm/sclp.h>
#include "kvm-s390.h"
#include "gaccess.h"
#include "trace.h"
#include "trace-s390.h"
+#define MEM_OP_MAX_SIZE 65536 /* Maximum transfer size for KVM_S390_MEM_OP */
+#define LOCAL_IRQS 32
+#define VCPU_IRQS_MAX_BUF (sizeof(struct kvm_s390_irq) * \
+ (KVM_MAX_VCPUS + LOCAL_IRQS))
+
#define VCPU_STAT(x) offsetof(struct kvm_vcpu, stat.x), KVM_STAT_VCPU
struct kvm_stats_debugfs_item debugfs_entries[] = {
{ "instruction_sigp_stop", VCPU_STAT(instruction_sigp_stop) },
{ "instruction_sigp_stop_store_status", VCPU_STAT(instruction_sigp_stop_store_status) },
{ "instruction_sigp_store_status", VCPU_STAT(instruction_sigp_store_status) },
+ { "instruction_sigp_store_adtl_status", VCPU_STAT(instruction_sigp_store_adtl_status) },
{ "instruction_sigp_set_arch", VCPU_STAT(instruction_sigp_arch) },
{ "instruction_sigp_set_prefix", VCPU_STAT(instruction_sigp_prefix) },
{ "instruction_sigp_restart", VCPU_STAT(instruction_sigp_restart) },
/* upper facilities limit for kvm */
unsigned long kvm_s390_fac_list_mask[] = {
- 0xff82fffbf4fc2000UL,
- 0x005c000000000000UL,
+ 0xffe6fffbfcfdfc40UL,
+ 0x205c800000000000UL,
};
unsigned long kvm_s390_fac_list_mask_size(void)
case KVM_CAP_ONE_REG:
case KVM_CAP_ENABLE_CAP:
case KVM_CAP_S390_CSS_SUPPORT:
- case KVM_CAP_IRQFD:
case KVM_CAP_IOEVENTFD:
case KVM_CAP_DEVICE_CTRL:
case KVM_CAP_ENABLE_CAP_VM:
case KVM_CAP_S390_IRQCHIP:
case KVM_CAP_VM_ATTRIBUTES:
case KVM_CAP_MP_STATE:
+ case KVM_CAP_S390_INJECT_IRQ:
case KVM_CAP_S390_USER_SIGP:
+ case KVM_CAP_S390_USER_STSI:
+ case KVM_CAP_S390_SKEYS:
+ case KVM_CAP_S390_IRQ_STATE:
r = 1;
break;
+ case KVM_CAP_S390_MEM_OP:
+ r = MEM_OP_MAX_SIZE;
+ break;
case KVM_CAP_NR_VCPUS:
case KVM_CAP_MAX_VCPUS:
r = KVM_MAX_VCPUS;
case KVM_CAP_S390_COW:
r = MACHINE_HAS_ESOP;
break;
+ case KVM_CAP_S390_VECTOR_REGISTERS:
+ r = MACHINE_HAS_VX;
+ break;
default:
r = 0;
}
kvm->arch.user_sigp = 1;
r = 0;
break;
+ case KVM_CAP_S390_VECTOR_REGISTERS:
+ if (MACHINE_HAS_VX) {
+ set_kvm_facility(kvm->arch.model.fac->mask, 129);
+ set_kvm_facility(kvm->arch.model.fac->list, 129);
+ r = 0;
+ } else
+ r = -EINVAL;
+ break;
+ case KVM_CAP_S390_USER_STSI:
+ kvm->arch.user_stsi = 1;
+ r = 0;
+ break;
default:
r = -EINVAL;
break;
memcpy(&kvm->arch.model.cpu_id, &proc->cpuid,
sizeof(struct cpuid));
kvm->arch.model.ibc = proc->ibc;
- memcpy(kvm->arch.model.fac->kvm, proc->fac_list,
+ memcpy(kvm->arch.model.fac->list, proc->fac_list,
S390_ARCH_FAC_LIST_SIZE_BYTE);
} else
ret = -EFAULT;
}
memcpy(&proc->cpuid, &kvm->arch.model.cpu_id, sizeof(struct cpuid));
proc->ibc = kvm->arch.model.ibc;
- memcpy(&proc->fac_list, kvm->arch.model.fac->kvm, S390_ARCH_FAC_LIST_SIZE_BYTE);
+ memcpy(&proc->fac_list, kvm->arch.model.fac->list, S390_ARCH_FAC_LIST_SIZE_BYTE);
if (copy_to_user((void __user *)attr->addr, proc, sizeof(*proc)))
ret = -EFAULT;
kfree(proc);
}
get_cpu_id((struct cpuid *) &mach->cpuid);
mach->ibc = sclp_get_ibc();
- memcpy(&mach->fac_mask, kvm_s390_fac_list_mask,
- kvm_s390_fac_list_mask_size() * sizeof(u64));
+ memcpy(&mach->fac_mask, kvm->arch.model.fac->mask,
+ S390_ARCH_FAC_LIST_SIZE_BYTE);
memcpy((unsigned long *)&mach->fac_list, S390_lowcore.stfle_fac_list,
- S390_ARCH_FAC_LIST_SIZE_U64);
+ S390_ARCH_FAC_LIST_SIZE_BYTE);
if (copy_to_user((void __user *)attr->addr, mach, sizeof(*mach)))
ret = -EFAULT;
kfree(mach);
return ret;
}
+static long kvm_s390_get_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
+{
+ uint8_t *keys;
+ uint64_t hva;
+ unsigned long curkey;
+ int i, r = 0;
+
+ if (args->flags != 0)
+ return -EINVAL;
+
+ /* Is this guest using storage keys? */
+ if (!mm_use_skey(current->mm))
+ return KVM_S390_GET_SKEYS_NONE;
+
+ /* Enforce sane limit on memory allocation */
+ if (args->count < 1 || args->count > KVM_S390_SKEYS_MAX)
+ return -EINVAL;
+
+ keys = kmalloc_array(args->count, sizeof(uint8_t),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!keys)
+ keys = vmalloc(sizeof(uint8_t) * args->count);
+ if (!keys)
+ return -ENOMEM;
+
+ for (i = 0; i < args->count; i++) {
+ hva = gfn_to_hva(kvm, args->start_gfn + i);
+ if (kvm_is_error_hva(hva)) {
+ r = -EFAULT;
+ goto out;
+ }
+
+ curkey = get_guest_storage_key(current->mm, hva);
+ if (IS_ERR_VALUE(curkey)) {
+ r = curkey;
+ goto out;
+ }
+ keys[i] = curkey;
+ }
+
+ r = copy_to_user((uint8_t __user *)args->skeydata_addr, keys,
+ sizeof(uint8_t) * args->count);
+ if (r)
+ r = -EFAULT;
+out:
+ kvfree(keys);
+ return r;
+}
+
+static long kvm_s390_set_skeys(struct kvm *kvm, struct kvm_s390_skeys *args)
+{
+ uint8_t *keys;
+ uint64_t hva;
+ int i, r = 0;
+
+ if (args->flags != 0)
+ return -EINVAL;
+
+ /* Enforce sane limit on memory allocation */
+ if (args->count < 1 || args->count > KVM_S390_SKEYS_MAX)
+ return -EINVAL;
+
+ keys = kmalloc_array(args->count, sizeof(uint8_t),
+ GFP_KERNEL | __GFP_NOWARN);
+ if (!keys)
+ keys = vmalloc(sizeof(uint8_t) * args->count);
+ if (!keys)
+ return -ENOMEM;
+
+ r = copy_from_user(keys, (uint8_t __user *)args->skeydata_addr,
+ sizeof(uint8_t) * args->count);
+ if (r) {
+ r = -EFAULT;
+ goto out;
+ }
+
+ /* Enable storage key handling for the guest */
+ s390_enable_skey();
+
+ for (i = 0; i < args->count; i++) {
+ hva = gfn_to_hva(kvm, args->start_gfn + i);
+ if (kvm_is_error_hva(hva)) {
+ r = -EFAULT;
+ goto out;
+ }
+
+ /* Lowest order bit is reserved */
+ if (keys[i] & 0x01) {
+ r = -EINVAL;
+ goto out;
+ }
+
+ r = set_guest_storage_key(current->mm, hva,
+ (unsigned long)keys[i], 0);
+ if (r)
+ goto out;
+ }
+out:
+ kvfree(keys);
+ return r;
+}
+
long kvm_arch_vm_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
r = kvm_s390_vm_has_attr(kvm, &attr);
break;
}
+ case KVM_S390_GET_SKEYS: {
+ struct kvm_s390_skeys args;
+
+ r = -EFAULT;
+ if (copy_from_user(&args, argp,
+ sizeof(struct kvm_s390_skeys)))
+ break;
+ r = kvm_s390_get_skeys(kvm, &args);
+ break;
+ }
+ case KVM_S390_SET_SKEYS: {
+ struct kvm_s390_skeys args;
+
+ r = -EFAULT;
+ if (copy_from_user(&args, argp,
+ sizeof(struct kvm_s390_skeys)))
+ break;
+ r = kvm_s390_set_skeys(kvm, &args);
+ break;
+ }
default:
r = -ENOTTY;
}
static int kvm_s390_query_ap_config(u8 *config)
{
u32 fcn_code = 0x04000000UL;
- u32 cc;
+ u32 cc = 0;
+ memset(config, 0, 128);
asm volatile(
"lgr 0,%1\n"
"lgr 2,%2\n"
".long 0xb2af0000\n" /* PQAP(QCI) */
- "ipm %0\n"
+ "0: ipm %0\n"
"srl %0,28\n"
- : "=r" (cc)
+ "1:\n"
+ EX_TABLE(0b, 1b)
+ : "+r" (cc)
: "r" (fcn_code), "r" (config)
: "cc", "0", "2", "memory"
);
kvm_s390_set_crycb_format(kvm);
- /* Disable AES/DEA protected key functions by default */
- kvm->arch.crypto.aes_kw = 0;
- kvm->arch.crypto.dea_kw = 0;
+ /* Enable AES/DEA protected key functions by default */
+ kvm->arch.crypto.aes_kw = 1;
+ kvm->arch.crypto.dea_kw = 1;
+ get_random_bytes(kvm->arch.crypto.crycb->aes_wrapping_key_mask,
+ sizeof(kvm->arch.crypto.crycb->aes_wrapping_key_mask));
+ get_random_bytes(kvm->arch.crypto.crycb->dea_wrapping_key_mask,
+ sizeof(kvm->arch.crypto.crycb->dea_wrapping_key_mask));
return 0;
}
kvm->arch.dbf = debug_register(debug_name, 8, 2, 8 * sizeof(long));
if (!kvm->arch.dbf)
- goto out_nodbf;
+ goto out_err;
/*
* The architectural maximum amount of facilities is 16 kbit. To store
* this amount, 2 kbyte of memory is required. Thus we need a full
- * page to hold the active copy (arch.model.fac->sie) and the current
- * facilities set (arch.model.fac->kvm). Its address size has to be
+ * page to hold the guest facility list (arch.model.fac->list) and the
+ * facility mask (arch.model.fac->mask). Its address size has to be
* 31 bits and word aligned.
*/
kvm->arch.model.fac =
- (struct s390_model_fac *) get_zeroed_page(GFP_KERNEL | GFP_DMA);
+ (struct kvm_s390_fac *) get_zeroed_page(GFP_KERNEL | GFP_DMA);
if (!kvm->arch.model.fac)
- goto out_nofac;
-
- memcpy(kvm->arch.model.fac->kvm, S390_lowcore.stfle_fac_list,
- S390_ARCH_FAC_LIST_SIZE_U64);
-
- /*
- * If this KVM host runs *not* in a LPAR, relax the facility bits
- * of the kvm facility mask by all missing facilities. This will allow
- * to determine the right CPU model by means of the remaining facilities.
- * Live guest migration must prohibit the migration of KVMs running in
- * a LPAR to non LPAR hosts.
- */
- if (!MACHINE_IS_LPAR)
- for (i = 0; i < kvm_s390_fac_list_mask_size(); i++)
- kvm_s390_fac_list_mask[i] &= kvm->arch.model.fac->kvm[i];
+ goto out_err;
- /*
- * Apply the kvm facility mask to limit the kvm supported/tolerated
- * facility list.
- */
+ /* Populate the facility mask initially. */
+ memcpy(kvm->arch.model.fac->mask, S390_lowcore.stfle_fac_list,
+ S390_ARCH_FAC_LIST_SIZE_BYTE);
for (i = 0; i < S390_ARCH_FAC_LIST_SIZE_U64; i++) {
if (i < kvm_s390_fac_list_mask_size())
- kvm->arch.model.fac->kvm[i] &= kvm_s390_fac_list_mask[i];
+ kvm->arch.model.fac->mask[i] &= kvm_s390_fac_list_mask[i];
else
- kvm->arch.model.fac->kvm[i] = 0UL;
+ kvm->arch.model.fac->mask[i] = 0UL;
}
+ /* Populate the facility list initially. */
+ memcpy(kvm->arch.model.fac->list, kvm->arch.model.fac->mask,
+ S390_ARCH_FAC_LIST_SIZE_BYTE);
+
kvm_s390_get_cpu_id(&kvm->arch.model.cpu_id);
kvm->arch.model.ibc = sclp_get_ibc() & 0x0fff;
if (kvm_s390_crypto_init(kvm) < 0)
- goto out_crypto;
+ goto out_err;
spin_lock_init(&kvm->arch.float_int.lock);
- INIT_LIST_HEAD(&kvm->arch.float_int.list);
+ for (i = 0; i < FIRQ_LIST_COUNT; i++)
+ INIT_LIST_HEAD(&kvm->arch.float_int.lists[i]);
init_waitqueue_head(&kvm->arch.ipte_wq);
mutex_init(&kvm->arch.ipte_mutex);
} else {
kvm->arch.gmap = gmap_alloc(current->mm, (1UL << 44) - 1);
if (!kvm->arch.gmap)
- goto out_nogmap;
+ goto out_err;
kvm->arch.gmap->private = kvm;
kvm->arch.gmap->pfault_enabled = 0;
}
spin_lock_init(&kvm->arch.start_stop_lock);
return 0;
-out_nogmap:
+out_err:
kfree(kvm->arch.crypto.crycb);
-out_crypto:
free_page((unsigned long)kvm->arch.model.fac);
-out_nofac:
debug_unregister(kvm->arch.dbf);
-out_nodbf:
free_page((unsigned long)(kvm->arch.sca));
-out_err:
return rc;
}
KVM_SYNC_CRS |
KVM_SYNC_ARCH0 |
KVM_SYNC_PFAULT;
+ if (test_kvm_facility(vcpu->kvm, 129))
+ vcpu->run->kvm_valid_regs |= KVM_SYNC_VRS;
if (kvm_is_ucontrol(vcpu->kvm))
return __kvm_ucontrol_vcpu_init(vcpu);
void kvm_arch_vcpu_load(struct kvm_vcpu *vcpu, int cpu)
{
save_fp_ctl(&vcpu->arch.host_fpregs.fpc);
- save_fp_regs(vcpu->arch.host_fpregs.fprs);
+ if (test_kvm_facility(vcpu->kvm, 129))
+ save_vx_regs((__vector128 *)&vcpu->arch.host_vregs->vrs);
+ else
+ save_fp_regs(vcpu->arch.host_fpregs.fprs);
save_access_regs(vcpu->arch.host_acrs);
- restore_fp_ctl(&vcpu->arch.guest_fpregs.fpc);
- restore_fp_regs(vcpu->arch.guest_fpregs.fprs);
+ if (test_kvm_facility(vcpu->kvm, 129)) {
+ restore_fp_ctl(&vcpu->run->s.regs.fpc);
+ restore_vx_regs((__vector128 *)&vcpu->run->s.regs.vrs);
+ } else {
+ restore_fp_ctl(&vcpu->arch.guest_fpregs.fpc);
+ restore_fp_regs(vcpu->arch.guest_fpregs.fprs);
+ }
restore_access_regs(vcpu->run->s.regs.acrs);
gmap_enable(vcpu->arch.gmap);
atomic_set_mask(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags);
{
atomic_clear_mask(CPUSTAT_RUNNING, &vcpu->arch.sie_block->cpuflags);
gmap_disable(vcpu->arch.gmap);
- save_fp_ctl(&vcpu->arch.guest_fpregs.fpc);
- save_fp_regs(vcpu->arch.guest_fpregs.fprs);
+ if (test_kvm_facility(vcpu->kvm, 129)) {
+ save_fp_ctl(&vcpu->run->s.regs.fpc);
+ save_vx_regs((__vector128 *)&vcpu->run->s.regs.vrs);
+ } else {
+ save_fp_ctl(&vcpu->arch.guest_fpregs.fpc);
+ save_fp_regs(vcpu->arch.guest_fpregs.fprs);
+ }
save_access_regs(vcpu->run->s.regs.acrs);
restore_fp_ctl(&vcpu->arch.host_fpregs.fpc);
- restore_fp_regs(vcpu->arch.host_fpregs.fprs);
+ if (test_kvm_facility(vcpu->kvm, 129))
+ restore_vx_regs((__vector128 *)&vcpu->arch.host_vregs->vrs);
+ else
+ restore_fp_regs(vcpu->arch.host_fpregs.fprs);
restore_access_regs(vcpu->arch.host_acrs);
}
return 0;
}
+static void kvm_s390_vcpu_setup_model(struct kvm_vcpu *vcpu)
+{
+ struct kvm_s390_cpu_model *model = &vcpu->kvm->arch.model;
+
+ vcpu->arch.cpu_id = model->cpu_id;
+ vcpu->arch.sie_block->ibc = model->ibc;
+ vcpu->arch.sie_block->fac = (int) (long) model->fac->list;
+}
+
int kvm_arch_vcpu_setup(struct kvm_vcpu *vcpu)
{
int rc = 0;
CPUSTAT_SM |
CPUSTAT_STOPPED |
CPUSTAT_GED);
+ kvm_s390_vcpu_setup_model(vcpu);
+
vcpu->arch.sie_block->ecb = 6;
if (test_kvm_facility(vcpu->kvm, 50) && test_kvm_facility(vcpu->kvm, 73))
vcpu->arch.sie_block->ecb |= 0x10;
vcpu->arch.sie_block->eca |= 1;
if (sclp_has_sigpif())
vcpu->arch.sie_block->eca |= 0x10000000U;
- vcpu->arch.sie_block->ictl |= ICTL_ISKE | ICTL_SSKE | ICTL_RRBE |
- ICTL_TPROT;
+ if (test_kvm_facility(vcpu->kvm, 129)) {
+ vcpu->arch.sie_block->eca |= 0x00020000;
+ vcpu->arch.sie_block->ecd |= 0x20000000;
+ }
+ vcpu->arch.sie_block->ictl |= ICTL_ISKE | ICTL_SSKE | ICTL_RRBE;
if (kvm_s390_cmma_enabled(vcpu->kvm)) {
rc = kvm_s390_vcpu_setup_cmma(vcpu);
hrtimer_init(&vcpu->arch.ckc_timer, CLOCK_MONOTONIC, HRTIMER_MODE_REL);
vcpu->arch.ckc_timer.function = kvm_s390_idle_wakeup;
- mutex_lock(&vcpu->kvm->lock);
- vcpu->arch.cpu_id = vcpu->kvm->arch.model.cpu_id;
- memcpy(vcpu->kvm->arch.model.fac->sie, vcpu->kvm->arch.model.fac->kvm,
- S390_ARCH_FAC_LIST_SIZE_BYTE);
- vcpu->arch.sie_block->ibc = vcpu->kvm->arch.model.ibc;
- mutex_unlock(&vcpu->kvm->lock);
-
kvm_s390_vcpu_crypto_setup(vcpu);
return rc;
vcpu->arch.sie_block = &sie_page->sie_block;
vcpu->arch.sie_block->itdba = (unsigned long) &sie_page->itdb;
+ vcpu->arch.host_vregs = &sie_page->vregs;
vcpu->arch.sie_block->icpua = id;
if (!kvm_is_ucontrol(kvm)) {
vcpu->arch.sie_block->scaol = (__u32)(__u64)kvm->arch.sca;
set_bit(63 - id, (unsigned long *) &kvm->arch.sca->mcn);
}
- vcpu->arch.sie_block->fac = (int) (long) kvm->arch.model.fac->sie;
spin_lock_init(&vcpu->arch.local_int.lock);
vcpu->arch.local_int.float_int = &kvm->arch.float_int;
return 0;
}
+static int vcpu_post_run_fault_in_sie(struct kvm_vcpu *vcpu)
+{
+ psw_t *psw = &vcpu->arch.sie_block->gpsw;
+ u8 opcode;
+ int rc;
+
+ VCPU_EVENT(vcpu, 3, "%s", "fault in sie instruction");
+ trace_kvm_s390_sie_fault(vcpu);
+
+ /*
+ * We want to inject an addressing exception, which is defined as a
+ * suppressing or terminating exception. However, since we came here
+ * by a DAT access exception, the PSW still points to the faulting
+ * instruction since DAT exceptions are nullifying. So we've got
+ * to look up the current opcode to get the length of the instruction
+ * to be able to forward the PSW.
+ */
+ rc = read_guest(vcpu, psw->addr, 0, &opcode, 1);
+ if (rc)
+ return kvm_s390_inject_prog_cond(vcpu, rc);
+ psw->addr = __rewind_psw(*psw, -insn_length(opcode));
+
+ return kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING);
+}
+
static int vcpu_post_run(struct kvm_vcpu *vcpu, int exit_reason)
{
int rc = -1;
}
}
- if (rc == -1) {
- VCPU_EVENT(vcpu, 3, "%s", "fault in sie instruction");
- trace_kvm_s390_sie_fault(vcpu);
- rc = kvm_s390_inject_program_int(vcpu, PGM_ADDRESSING);
- }
+ if (rc == -1)
+ rc = vcpu_post_run_fault_in_sie(vcpu);
memcpy(&vcpu->run->s.regs.gprs[14], &vcpu->arch.sie_block->gg14, 16);
return kvm_s390_store_status_unloaded(vcpu, addr);
}
+/*
+ * store additional status at address
+ */
+int kvm_s390_store_adtl_status_unloaded(struct kvm_vcpu *vcpu,
+ unsigned long gpa)
+{
+ /* Only bits 0-53 are used for address formation */
+ if (!(gpa & ~0x3ff))
+ return 0;
+
+ return write_guest_abs(vcpu, gpa & ~0x3ff,
+ (void *)&vcpu->run->s.regs.vrs, 512);
+}
+
+int kvm_s390_vcpu_store_adtl_status(struct kvm_vcpu *vcpu, unsigned long addr)
+{
+ if (!test_kvm_facility(vcpu->kvm, 129))
+ return 0;
+
+ /*
+ * The guest VXRS are in the host VXRs due to the lazy
+ * copying in vcpu load/put. Let's update our copies before we save
+ * it into the save area.
+ */
+ save_vx_regs((__vector128 *)&vcpu->run->s.regs.vrs);
+
+ return kvm_s390_store_adtl_status_unloaded(vcpu, addr);
+}
+
static void __disable_ibs_on_vcpu(struct kvm_vcpu *vcpu)
{
kvm_check_request(KVM_REQ_ENABLE_IBS, vcpu);
return r;
}
+static long kvm_s390_guest_mem_op(struct kvm_vcpu *vcpu,
+ struct kvm_s390_mem_op *mop)
+{
+ void __user *uaddr = (void __user *)mop->buf;
+ void *tmpbuf = NULL;
+ int r, srcu_idx;
+ const u64 supported_flags = KVM_S390_MEMOP_F_INJECT_EXCEPTION
+ | KVM_S390_MEMOP_F_CHECK_ONLY;
+
+ if (mop->flags & ~supported_flags)
+ return -EINVAL;
+
+ if (mop->size > MEM_OP_MAX_SIZE)
+ return -E2BIG;
+
+ if (!(mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY)) {
+ tmpbuf = vmalloc(mop->size);
+ if (!tmpbuf)
+ return -ENOMEM;
+ }
+
+ srcu_idx = srcu_read_lock(&vcpu->kvm->srcu);
+
+ switch (mop->op) {
+ case KVM_S390_MEMOP_LOGICAL_READ:
+ if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
+ r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size, false);
+ break;
+ }
+ r = read_guest(vcpu, mop->gaddr, mop->ar, tmpbuf, mop->size);
+ if (r == 0) {
+ if (copy_to_user(uaddr, tmpbuf, mop->size))
+ r = -EFAULT;
+ }
+ break;
+ case KVM_S390_MEMOP_LOGICAL_WRITE:
+ if (mop->flags & KVM_S390_MEMOP_F_CHECK_ONLY) {
+ r = check_gva_range(vcpu, mop->gaddr, mop->ar, mop->size, true);
+ break;
+ }
+ if (copy_from_user(tmpbuf, uaddr, mop->size)) {
+ r = -EFAULT;
+ break;
+ }
+ r = write_guest(vcpu, mop->gaddr, mop->ar, tmpbuf, mop->size);
+ break;
+ default:
+ r = -EINVAL;
+ }
+
+ srcu_read_unlock(&vcpu->kvm->srcu, srcu_idx);
+
+ if (r > 0 && (mop->flags & KVM_S390_MEMOP_F_INJECT_EXCEPTION) != 0)
+ kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
+
+ vfree(tmpbuf);
+ return r;
+}
+
long kvm_arch_vcpu_ioctl(struct file *filp,
unsigned int ioctl, unsigned long arg)
{
long r;
switch (ioctl) {
+ case KVM_S390_IRQ: {
+ struct kvm_s390_irq s390irq;
+
+ r = -EFAULT;
+ if (copy_from_user(&s390irq, argp, sizeof(s390irq)))
+ break;
+ r = kvm_s390_inject_vcpu(vcpu, &s390irq);
+ break;
+ }
case KVM_S390_INTERRUPT: {
struct kvm_s390_interrupt s390int;
struct kvm_s390_irq s390irq;
r = kvm_vcpu_ioctl_enable_cap(vcpu, &cap);
break;
}
+ case KVM_S390_MEM_OP: {
+ struct kvm_s390_mem_op mem_op;
+
+ if (copy_from_user(&mem_op, argp, sizeof(mem_op)) == 0)
+ r = kvm_s390_guest_mem_op(vcpu, &mem_op);
+ else
+ r = -EFAULT;
+ break;
+ }
+ case KVM_S390_SET_IRQ_STATE: {
+ struct kvm_s390_irq_state irq_state;
+
+ r = -EFAULT;
+ if (copy_from_user(&irq_state, argp, sizeof(irq_state)))
+ break;
+ if (irq_state.len > VCPU_IRQS_MAX_BUF ||
+ irq_state.len == 0 ||
+ irq_state.len % sizeof(struct kvm_s390_irq) > 0) {
+ r = -EINVAL;
+ break;
+ }
+ r = kvm_s390_set_irq_state(vcpu,
+ (void __user *) irq_state.buf,
+ irq_state.len);
+ break;
+ }
+ case KVM_S390_GET_IRQ_STATE: {
+ struct kvm_s390_irq_state irq_state;
+
+ r = -EFAULT;
+ if (copy_from_user(&irq_state, argp, sizeof(irq_state)))
+ break;
+ if (irq_state.len == 0) {
+ r = -EINVAL;
+ break;
+ }
+ r = kvm_s390_get_irq_state(vcpu,
+ (__u8 __user *) irq_state.buf,
+ irq_state.len);
+ break;
+ }
default:
r = -ENOTTY;
}
kvm_make_request(KVM_REQ_MMU_RELOAD, vcpu);
}
-static inline u64 kvm_s390_get_base_disp_s(struct kvm_vcpu *vcpu)
+typedef u8 __bitwise ar_t;
+
+static inline u64 kvm_s390_get_base_disp_s(struct kvm_vcpu *vcpu, ar_t *ar)
{
u32 base2 = vcpu->arch.sie_block->ipb >> 28;
u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16);
+ if (ar)
+ *ar = base2;
+
return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
}
static inline void kvm_s390_get_base_disp_sse(struct kvm_vcpu *vcpu,
- u64 *address1, u64 *address2)
+ u64 *address1, u64 *address2,
+ ar_t *ar_b1, ar_t *ar_b2)
{
u32 base1 = (vcpu->arch.sie_block->ipb & 0xf0000000) >> 28;
u32 disp1 = (vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16;
*address1 = (base1 ? vcpu->run->s.regs.gprs[base1] : 0) + disp1;
*address2 = (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
+
+ if (ar_b1)
+ *ar_b1 = base1;
+ if (ar_b2)
+ *ar_b2 = base2;
}
static inline void kvm_s390_get_regs_rre(struct kvm_vcpu *vcpu, int *r1, int *r2)
*r2 = (vcpu->arch.sie_block->ipb & 0x000f0000) >> 16;
}
-static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu)
+static inline u64 kvm_s390_get_base_disp_rsy(struct kvm_vcpu *vcpu, ar_t *ar)
{
u32 base2 = vcpu->arch.sie_block->ipb >> 28;
u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16) +
if (disp2 & 0x80000)
disp2+=0xfff00000;
+ if (ar)
+ *ar = base2;
+
return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + (long)(int)disp2;
}
-static inline u64 kvm_s390_get_base_disp_rs(struct kvm_vcpu *vcpu)
+static inline u64 kvm_s390_get_base_disp_rs(struct kvm_vcpu *vcpu, ar_t *ar)
{
u32 base2 = vcpu->arch.sie_block->ipb >> 28;
u32 disp2 = ((vcpu->arch.sie_block->ipb & 0x0fff0000) >> 16);
+ if (ar)
+ *ar = base2;
+
return (base2 ? vcpu->run->s.regs.gprs[base2] : 0) + disp2;
}
vcpu->arch.sie_block->gpsw.mask |= cc << 44;
}
-/* test availability of facility in a kvm intance */
+/* test availability of facility in a kvm instance */
static inline int test_kvm_facility(struct kvm *kvm, unsigned long nr)
{
- return __test_facility(nr, kvm->arch.model.fac->kvm);
+ return __test_facility(nr, kvm->arch.model.fac->mask) &&
+ __test_facility(nr, kvm->arch.model.fac->list);
+}
+
+static inline int set_kvm_facility(u64 *fac_list, unsigned long nr)
+{
+ unsigned char *ptr;
+
+ if (nr >= MAX_FACILITY_BIT)
+ return -EINVAL;
+ ptr = (unsigned char *) fac_list + (nr >> 3);
+ *ptr |= (0x80UL >> (nr & 7));
+ return 0;
}
/* are cpu states controlled by user space */
struct kvm_s390_irq *irq);
int __must_check kvm_s390_inject_program_int(struct kvm_vcpu *vcpu, u16 code);
struct kvm_s390_interrupt_info *kvm_s390_get_io_int(struct kvm *kvm,
- u64 cr6, u64 schid);
-void kvm_s390_reinject_io_int(struct kvm *kvm,
- struct kvm_s390_interrupt_info *inti);
+ u64 isc_mask, u32 schid);
+int kvm_s390_reinject_io_int(struct kvm *kvm,
+ struct kvm_s390_interrupt_info *inti);
int kvm_s390_mask_adapter(struct kvm *kvm, unsigned int id, bool masked);
/* implemented in intercept.c */
/* implemented in kvm-s390.c */
long kvm_arch_fault_in_page(struct kvm_vcpu *vcpu, gpa_t gpa, int writable);
int kvm_s390_store_status_unloaded(struct kvm_vcpu *vcpu, unsigned long addr);
+int kvm_s390_store_adtl_status_unloaded(struct kvm_vcpu *vcpu,
+ unsigned long addr);
int kvm_s390_vcpu_store_status(struct kvm_vcpu *vcpu, unsigned long addr);
+int kvm_s390_vcpu_store_adtl_status(struct kvm_vcpu *vcpu, unsigned long addr);
void kvm_s390_vcpu_start(struct kvm_vcpu *vcpu);
void kvm_s390_vcpu_stop(struct kvm_vcpu *vcpu);
void s390_vcpu_block(struct kvm_vcpu *vcpu);
extern struct kvm_device_ops kvm_flic_ops;
int kvm_s390_is_stop_irq_pending(struct kvm_vcpu *vcpu);
void kvm_s390_clear_stop_irq(struct kvm_vcpu *vcpu);
+int kvm_s390_set_irq_state(struct kvm_vcpu *vcpu,
+ void __user *buf, int len);
+int kvm_s390_get_irq_state(struct kvm_vcpu *vcpu,
+ __u8 __user *buf, int len);
/* implemented in guestdbg.c */
void kvm_s390_backup_guest_per_regs(struct kvm_vcpu *vcpu);
struct kvm_vcpu *cpup;
s64 hostclk, val;
int i, rc;
+ ar_t ar;
u64 op2;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- op2 = kvm_s390_get_base_disp_s(vcpu);
+ op2 = kvm_s390_get_base_disp_s(vcpu, &ar);
if (op2 & 7) /* Operand must be on a doubleword boundary */
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = read_guest(vcpu, op2, &val, sizeof(val));
+ rc = read_guest(vcpu, op2, ar, &val, sizeof(val));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
u64 operand2;
u32 address;
int rc;
+ ar_t ar;
vcpu->stat.instruction_spx++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- operand2 = kvm_s390_get_base_disp_s(vcpu);
+ operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
/* must be word boundary */
if (operand2 & 3)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
/* get the value */
- rc = read_guest(vcpu, operand2, &address, sizeof(address));
+ rc = read_guest(vcpu, operand2, ar, &address, sizeof(address));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
u64 operand2;
u32 address;
int rc;
+ ar_t ar;
vcpu->stat.instruction_stpx++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- operand2 = kvm_s390_get_base_disp_s(vcpu);
+ operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
/* must be word boundary */
if (operand2 & 3)
address = kvm_s390_get_prefix(vcpu);
/* get the value */
- rc = write_guest(vcpu, operand2, &address, sizeof(address));
+ rc = write_guest(vcpu, operand2, ar, &address, sizeof(address));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
u16 vcpu_id = vcpu->vcpu_id;
u64 ga;
int rc;
+ ar_t ar;
vcpu->stat.instruction_stap++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- ga = kvm_s390_get_base_disp_s(vcpu);
+ ga = kvm_s390_get_base_disp_s(vcpu, &ar);
if (ga & 1)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = write_guest(vcpu, ga, &vcpu_id, sizeof(vcpu_id));
+ rc = write_guest(vcpu, ga, ar, &vcpu_id, sizeof(vcpu_id));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
kvm_s390_get_regs_rre(vcpu, NULL, ®2);
addr = vcpu->run->s.regs.gprs[reg2] & PAGE_MASK;
addr = kvm_s390_logical_to_effective(vcpu, addr);
- if (kvm_s390_check_low_addr_protection(vcpu, addr))
+ if (kvm_s390_check_low_addr_prot_real(vcpu, addr))
return kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
addr = kvm_s390_real_to_abs(vcpu, addr);
struct kvm_s390_interrupt_info *inti;
unsigned long len;
u32 tpi_data[3];
- int cc, rc;
+ int rc;
u64 addr;
+ ar_t ar;
- rc = 0;
- addr = kvm_s390_get_base_disp_s(vcpu);
+ addr = kvm_s390_get_base_disp_s(vcpu, &ar);
if (addr & 3)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- cc = 0;
+
inti = kvm_s390_get_io_int(vcpu->kvm, vcpu->arch.sie_block->gcr[6], 0);
- if (!inti)
- goto no_interrupt;
- cc = 1;
+ if (!inti) {
+ kvm_s390_set_psw_cc(vcpu, 0);
+ return 0;
+ }
+
tpi_data[0] = inti->io.subchannel_id << 16 | inti->io.subchannel_nr;
tpi_data[1] = inti->io.io_int_parm;
tpi_data[2] = inti->io.io_int_word;
* provided area.
*/
len = sizeof(tpi_data) - 4;
- rc = write_guest(vcpu, addr, &tpi_data, len);
- if (rc)
- return kvm_s390_inject_prog_cond(vcpu, rc);
+ rc = write_guest(vcpu, addr, ar, &tpi_data, len);
+ if (rc) {
+ rc = kvm_s390_inject_prog_cond(vcpu, rc);
+ goto reinject_interrupt;
+ }
} else {
/*
* Store the three-word I/O interruption code into
* the appropriate lowcore area.
*/
len = sizeof(tpi_data);
- if (write_guest_lc(vcpu, __LC_SUBCHANNEL_ID, &tpi_data, len))
+ if (write_guest_lc(vcpu, __LC_SUBCHANNEL_ID, &tpi_data, len)) {
+ /* failed writes to the low core are not recoverable */
rc = -EFAULT;
+ goto reinject_interrupt;
+ }
}
+
+ /* irq was successfully handed to the guest */
+ kfree(inti);
+ kvm_s390_set_psw_cc(vcpu, 1);
+ return 0;
+reinject_interrupt:
/*
* If we encounter a problem storing the interruption code, the
* instruction is suppressed from the guest's view: reinject the
* interrupt.
*/
- if (!rc)
+ if (kvm_s390_reinject_io_int(vcpu->kvm, inti)) {
kfree(inti);
- else
- kvm_s390_reinject_io_int(vcpu->kvm, inti);
-no_interrupt:
- /* Set condition code and we're done. */
- if (!rc)
- kvm_s390_set_psw_cc(vcpu, cc);
+ rc = -EFAULT;
+ }
+ /* don't set the cc, a pgm irq was injected or we drop to user space */
return rc ? -EFAULT : 0;
}
static int handle_tsch(struct kvm_vcpu *vcpu)
{
- struct kvm_s390_interrupt_info *inti;
+ struct kvm_s390_interrupt_info *inti = NULL;
+ const u64 isc_mask = 0xffUL << 24; /* all iscs set */
- inti = kvm_s390_get_io_int(vcpu->kvm, 0,
- vcpu->run->s.regs.gprs[1]);
+ /* a valid schid has at least one bit set */
+ if (vcpu->run->s.regs.gprs[1])
+ inti = kvm_s390_get_io_int(vcpu->kvm, isc_mask,
+ vcpu->run->s.regs.gprs[1]);
/*
* Prepare exit to userspace.
* We need to shift the lower 32 facility bits (bit 0-31) from a u64
* into a u32 memory representation. They will remain bits 0-31.
*/
- fac = *vcpu->kvm->arch.model.fac->sie >> 32;
+ fac = *vcpu->kvm->arch.model.fac->list >> 32;
rc = write_guest_lc(vcpu, offsetof(struct _lowcore, stfl_fac_list),
&fac, sizeof(fac));
if (rc)
psw_compat_t new_psw;
u64 addr;
int rc;
+ ar_t ar;
if (gpsw->mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- addr = kvm_s390_get_base_disp_s(vcpu);
+ addr = kvm_s390_get_base_disp_s(vcpu, &ar);
if (addr & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = read_guest(vcpu, addr, &new_psw, sizeof(new_psw));
+ rc = read_guest(vcpu, addr, ar, &new_psw, sizeof(new_psw));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
if (!(new_psw.mask & PSW32_MASK_BASE))
psw_t new_psw;
u64 addr;
int rc;
+ ar_t ar;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- addr = kvm_s390_get_base_disp_s(vcpu);
+ addr = kvm_s390_get_base_disp_s(vcpu, &ar);
if (addr & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = read_guest(vcpu, addr, &new_psw, sizeof(new_psw));
+ rc = read_guest(vcpu, addr, ar, &new_psw, sizeof(new_psw));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
vcpu->arch.sie_block->gpsw = new_psw;
u64 stidp_data = vcpu->arch.stidp_data;
u64 operand2;
int rc;
+ ar_t ar;
vcpu->stat.instruction_stidp++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- operand2 = kvm_s390_get_base_disp_s(vcpu);
+ operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
if (operand2 & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
- rc = write_guest(vcpu, operand2, &stidp_data, sizeof(stidp_data));
+ rc = write_guest(vcpu, operand2, ar, &stidp_data, sizeof(stidp_data));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
for (n = mem->count - 1; n > 0 ; n--)
memcpy(&mem->vm[n], &mem->vm[n - 1], sizeof(mem->vm[0]));
+ memset(&mem->vm[0], 0, sizeof(mem->vm[0]));
mem->vm[0].cpus_total = cpus;
mem->vm[0].cpus_configured = cpus;
mem->vm[0].cpus_standby = 0;
ASCEBC(mem->vm[0].cpi, 16);
}
+static void insert_stsi_usr_data(struct kvm_vcpu *vcpu, u64 addr, ar_t ar,
+ u8 fc, u8 sel1, u16 sel2)
+{
+ vcpu->run->exit_reason = KVM_EXIT_S390_STSI;
+ vcpu->run->s390_stsi.addr = addr;
+ vcpu->run->s390_stsi.ar = ar;
+ vcpu->run->s390_stsi.fc = fc;
+ vcpu->run->s390_stsi.sel1 = sel1;
+ vcpu->run->s390_stsi.sel2 = sel2;
+}
+
static int handle_stsi(struct kvm_vcpu *vcpu)
{
int fc = (vcpu->run->s.regs.gprs[0] & 0xf0000000) >> 28;
unsigned long mem = 0;
u64 operand2;
int rc = 0;
+ ar_t ar;
vcpu->stat.instruction_stsi++;
VCPU_EVENT(vcpu, 4, "stsi: fc: %x sel1: %x sel2: %x", fc, sel1, sel2);
return 0;
}
- operand2 = kvm_s390_get_base_disp_s(vcpu);
+ operand2 = kvm_s390_get_base_disp_s(vcpu, &ar);
if (operand2 & 0xfff)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
break;
}
- rc = write_guest(vcpu, operand2, (void *)mem, PAGE_SIZE);
+ rc = write_guest(vcpu, operand2, ar, (void *)mem, PAGE_SIZE);
if (rc) {
rc = kvm_s390_inject_prog_cond(vcpu, rc);
goto out;
}
+ if (vcpu->kvm->arch.user_stsi) {
+ insert_stsi_usr_data(vcpu, operand2, ar, fc, sel1, sel2);
+ rc = -EREMOTE;
+ }
trace_kvm_s390_handle_stsi(vcpu, fc, sel1, sel2, operand2);
free_page(mem);
kvm_s390_set_psw_cc(vcpu, 0);
vcpu->run->s.regs.gprs[0] = 0;
- return 0;
+ return rc;
out_no_data:
kvm_s390_set_psw_cc(vcpu, 3);
out:
}
if (vcpu->run->s.regs.gprs[reg1] & PFMF_CF) {
- if (kvm_s390_check_low_addr_protection(vcpu, start))
+ if (kvm_s390_check_low_addr_prot_real(vcpu, start))
return kvm_s390_inject_prog_irq(vcpu, &vcpu->arch.pgm);
}
int reg, rc, nr_regs;
u32 ctl_array[16];
u64 ga;
+ ar_t ar;
vcpu->stat.instruction_lctl++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- ga = kvm_s390_get_base_disp_rs(vcpu);
+ ga = kvm_s390_get_base_disp_rs(vcpu, &ar);
if (ga & 3)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
trace_kvm_s390_handle_lctl(vcpu, 0, reg1, reg3, ga);
nr_regs = ((reg3 - reg1) & 0xf) + 1;
- rc = read_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u32));
+ rc = read_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u32));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
reg = reg1;
int reg, rc, nr_regs;
u32 ctl_array[16];
u64 ga;
+ ar_t ar;
vcpu->stat.instruction_stctl++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- ga = kvm_s390_get_base_disp_rs(vcpu);
+ ga = kvm_s390_get_base_disp_rs(vcpu, &ar);
if (ga & 3)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
break;
reg = (reg + 1) % 16;
} while (1);
- rc = write_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u32));
+ rc = write_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u32));
return rc ? kvm_s390_inject_prog_cond(vcpu, rc) : 0;
}
int reg, rc, nr_regs;
u64 ctl_array[16];
u64 ga;
+ ar_t ar;
vcpu->stat.instruction_lctlg++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- ga = kvm_s390_get_base_disp_rsy(vcpu);
+ ga = kvm_s390_get_base_disp_rsy(vcpu, &ar);
if (ga & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
trace_kvm_s390_handle_lctl(vcpu, 1, reg1, reg3, ga);
nr_regs = ((reg3 - reg1) & 0xf) + 1;
- rc = read_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u64));
+ rc = read_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u64));
if (rc)
return kvm_s390_inject_prog_cond(vcpu, rc);
reg = reg1;
int reg, rc, nr_regs;
u64 ctl_array[16];
u64 ga;
+ ar_t ar;
vcpu->stat.instruction_stctg++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- ga = kvm_s390_get_base_disp_rsy(vcpu);
+ ga = kvm_s390_get_base_disp_rsy(vcpu, &ar);
if (ga & 7)
return kvm_s390_inject_program_int(vcpu, PGM_SPECIFICATION);
break;
reg = (reg + 1) % 16;
} while (1);
- rc = write_guest(vcpu, ga, ctl_array, nr_regs * sizeof(u64));
+ rc = write_guest(vcpu, ga, ar, ctl_array, nr_regs * sizeof(u64));
return rc ? kvm_s390_inject_prog_cond(vcpu, rc) : 0;
}
unsigned long hva, gpa;
int ret = 0, cc = 0;
bool writable;
+ ar_t ar;
vcpu->stat.instruction_tprot++;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- kvm_s390_get_base_disp_sse(vcpu, &address1, &address2);
+ kvm_s390_get_base_disp_sse(vcpu, &address1, &address2, &ar, NULL);
/* we only handle the Linux memory detection case:
* access key == 0
return -EOPNOTSUPP;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_DAT)
ipte_lock(vcpu);
- ret = guest_translate_address(vcpu, address1, &gpa, 1);
+ ret = guest_translate_address(vcpu, address1, ar, &gpa, 1);
if (ret == PGM_PROTECTION) {
/* Write protected? Try again with read-only... */
cc = 1;
- ret = guest_translate_address(vcpu, address1, &gpa, 0);
+ ret = guest_translate_address(vcpu, address1, ar, &gpa, 0);
}
if (ret) {
if (ret == PGM_ADDRESSING || ret == PGM_TRANSLATION_SPEC) {
case SIGP_STORE_STATUS_AT_ADDRESS:
vcpu->stat.instruction_sigp_store_status++;
break;
+ case SIGP_STORE_ADDITIONAL_STATUS:
+ vcpu->stat.instruction_sigp_store_adtl_status++;
+ break;
case SIGP_SET_PREFIX:
vcpu->stat.instruction_sigp_prefix++;
break;
if (vcpu->arch.sie_block->gpsw.mask & PSW_MASK_PSTATE)
return kvm_s390_inject_program_int(vcpu, PGM_PRIVILEGED_OP);
- order_code = kvm_s390_get_base_disp_rs(vcpu);
+ order_code = kvm_s390_get_base_disp_rs(vcpu, NULL);
if (handle_sigp_order_in_user_space(vcpu, order_code))
return -EOPNOTSUPP;
int r3 = vcpu->arch.sie_block->ipa & 0x000f;
u16 cpu_addr = vcpu->run->s.regs.gprs[r3];
struct kvm_vcpu *dest_vcpu;
- u8 order_code = kvm_s390_get_base_disp_rs(vcpu);
+ u8 order_code = kvm_s390_get_base_disp_rs(vcpu, NULL);
trace_kvm_s390_handle_sigp_pei(vcpu, order_code, cpu_addr);
addr = ZPCI_IOMAP_ADDR_BASE | ((u64) idx << 48);
return (void __iomem *) addr + offset;
}
-EXPORT_SYMBOL_GPL(pci_iomap_range);
+EXPORT_SYMBOL(pci_iomap_range);
void __iomem *pci_iomap(struct pci_dev *dev, int bar, unsigned long maxlen)
{
}
spin_unlock(&zpci_iomap_lock);
}
-EXPORT_SYMBOL_GPL(pci_iounmap);
+EXPORT_SYMBOL(pci_iounmap);
static int pci_read(struct pci_bus *bus, unsigned int devfn, int where,
int size, u32 *val)
airq_iv_free_bit(zpci_aisb_iv, zdev->aisb);
}
-static void zpci_map_resources(struct zpci_dev *zdev)
+static void zpci_map_resources(struct pci_dev *pdev)
{
- struct pci_dev *pdev = zdev->pdev;
resource_size_t len;
int i;
}
}
-static void zpci_unmap_resources(struct zpci_dev *zdev)
+static void zpci_unmap_resources(struct pci_dev *pdev)
{
- struct pci_dev *pdev = zdev->pdev;
resource_size_t len;
int i;
zdev->pdev = pdev;
pdev->dev.groups = zpci_attr_groups;
- zpci_map_resources(zdev);
+ zpci_map_resources(pdev);
for (i = 0; i < PCI_BAR_COUNT; i++) {
res = &pdev->resource[i];
return 0;
}
+void pcibios_release_device(struct pci_dev *pdev)
+{
+ zpci_unmap_resources(pdev);
+}
+
int pcibios_enable_device(struct pci_dev *pdev, int mask)
{
struct zpci_dev *zdev = get_zdev(pdev);
zdev->pdev = pdev;
zpci_debug_init_device(zdev);
zpci_fmb_enable_device(zdev);
- zpci_map_resources(zdev);
return pci_enable_resources(pdev, mask);
}
{
struct zpci_dev *zdev = get_zdev(pdev);
- zpci_unmap_resources(zdev);
zpci_fmb_disable_device(zdev);
zpci_debug_exit_device(zdev);
zdev->pdev = NULL;
#ifdef CONFIG_HIBERNATE_CALLBACKS
static int zpci_restore(struct device *dev)
{
- struct zpci_dev *zdev = get_zdev(to_pci_dev(dev));
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct zpci_dev *zdev = get_zdev(pdev);
int ret = 0;
if (zdev->state != ZPCI_FN_STATE_ONLINE)
if (ret)
goto out;
- zpci_map_resources(zdev);
+ zpci_map_resources(pdev);
zpci_register_ioat(zdev, 0, zdev->start_dma + PAGE_OFFSET,
zdev->start_dma + zdev->iommu_size - 1,
(u64) zdev->dma_table);
static int zpci_freeze(struct device *dev)
{
- struct zpci_dev *zdev = get_zdev(to_pci_dev(dev));
+ struct pci_dev *pdev = to_pci_dev(dev);
+ struct zpci_dev *zdev = get_zdev(pdev);
if (zdev->state != ZPCI_FN_STATE_ONLINE)
return 0;
zpci_unregister_ioat(zdev, 0);
+ zpci_unmap_resources(pdev);
return clp_disable_fh(zdev);
}
zpci_cleanup_bus_resources(zdev);
return -EIO;
}
-
zdev->bus->max_bus_speed = zdev->max_bus_speed;
+ pci_bus_add_devices(zdev->bus);
return 0;
}
if (copy_from_user(buf, user_buffer, length))
goto out;
- memcpy_toio(io_addr, buf, length);
- ret = 0;
+ ret = zpci_memcpy_toio(io_addr, buf, length);
out:
if (buf != local_buf)
kfree(buf);
goto out;
io_addr = (void __iomem *)((pfn << PAGE_SHIFT) | (mmio_addr & ~PAGE_MASK));
- ret = -EFAULT;
- if ((unsigned long) io_addr < ZPCI_IOMAP_ADDR_BASE)
+ if ((unsigned long) io_addr < ZPCI_IOMAP_ADDR_BASE) {
+ ret = -EFAULT;
goto out;
-
- memcpy_fromio(buf, io_addr, length);
-
- if (copy_to_user(user_buffer, buf, length))
+ }
+ ret = zpci_memcpy_fromio(buf, io_addr, length);
+ if (ret)
goto out;
+ if (copy_to_user(user_buffer, buf, length))
+ ret = -EFAULT;
- ret = 0;
out:
if (buf != local_buf)
kfree(buf);
need_domain_info = need_domain_info || hose->index;
hose->need_domain_info = need_domain_info;
- if (bus) {
- next_busno = bus->busn_res.end + 1;
- /* Don't allow 8-bit bus number overflow inside the hose -
- reserve some space for bridges. */
- if (next_busno > 224) {
- next_busno = 0;
- need_domain_info = 1;
- }
- pci_bus_size_bridges(bus);
- pci_bus_assign_resources(bus);
- } else {
+ if (!bus) {
pci_free_resource_list(&resources);
+ return;
+ }
+
+ next_busno = bus->busn_res.end + 1;
+ /* Don't allow 8-bit bus number overflow inside the hose -
+ reserve some space for bridges. */
+ if (next_busno > 224) {
+ next_busno = 0;
+ need_domain_info = 1;
}
+
+ pci_bus_size_bridges(bus);
+ pci_bus_assign_resources(bus);
+ pci_bus_add_devices(bus);
}
/*
default "arch/sparc/configs/sparc32_defconfig" if SPARC32
default "arch/sparc/configs/sparc64_defconfig" if SPARC64
+config ARCH_PROC_KCORE_TEXT
+ def_bool y
+
config IOMMU_HELPER
bool
default y if SPARC64
unsigned long reg_val);
#endif
+
+#define HV_FAST_M7_GET_PERFREG 0x43
+#define HV_FAST_M7_SET_PERFREG 0x44
+
+#ifndef __ASSEMBLY__
+unsigned long sun4v_m7_get_perfreg(unsigned long reg_num,
+ unsigned long *reg_val);
+unsigned long sun4v_m7_set_perfreg(unsigned long reg_num,
+ unsigned long reg_val);
+#endif
+
/* Function numbers for HV_CORE_TRAP. */
#define HV_CORE_SET_VER 0x00
#define HV_CORE_PUTCHAR 0x01
#define HV_GRP_SDIO 0x0108
#define HV_GRP_SDIO_ERR 0x0109
#define HV_GRP_REBOOT_DATA 0x0110
+#define HV_GRP_M7_PERF 0x0114
#define HV_GRP_NIAG_PERF 0x0200
#define HV_GRP_FIRE_PERF 0x0201
#define HV_GRP_N2_CPU 0x0202
{
}
-#define ioread8(X) readb(X)
-#define ioread16(X) readw(X)
-#define ioread16be(X) __raw_readw(X)
-#define ioread32(X) readl(X)
-#define ioread32be(X) __raw_readl(X)
-#define iowrite8(val,X) writeb(val,X)
-#define iowrite16(val,X) writew(val,X)
-#define iowrite16be(val,X) __raw_writew(val,X)
-#define iowrite32(val,X) writel(val,X)
-#define iowrite32be(val,X) __raw_writel(val,X)
+#define ioread8 readb
+#define ioread16 readw
+#define ioread16be __raw_readw
+#define ioread32 readl
+#define ioread32be __raw_readl
+#define iowrite8 writeb
+#define iowrite16 writew
+#define iowrite16be __raw_writew
+#define iowrite32 writel
+#define iowrite32be __raw_writel
/* Create a virtual mapping cookie for an IO port range */
void __iomem *ioport_map(unsigned long port, unsigned int nr);
#ifndef _ASM_SPARC_JUMP_LABEL_H
#define _ASM_SPARC_JUMP_LABEL_H
-#ifdef __KERNEL__
+#ifndef __ASSEMBLY__
#include <linux/types.h>
return true;
}
-#endif /* __KERNEL__ */
-
typedef u32 jump_label_t;
struct jump_entry {
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif
extern int this_is_starfire;
void check_if_starfire(void);
-int starfire_hard_smp_processor_id(void);
void starfire_hookup(int);
unsigned int starfire_translate(unsigned long imap, unsigned int upaid);
void do_privop(struct pt_regs *regs);
void do_privact(struct pt_regs *regs);
void do_cee(struct pt_regs *regs);
-void do_cee_tl1(struct pt_regs *regs);
-void do_dae_tl1(struct pt_regs *regs);
-void do_iae_tl1(struct pt_regs *regs);
void do_div0_tl1(struct pt_regs *regs);
-void do_fpdis_tl1(struct pt_regs *regs);
void do_fpieee_tl1(struct pt_regs *regs);
void do_fpother_tl1(struct pt_regs *regs);
void do_ill_tl1(struct pt_regs *regs);
{ .group = HV_GRP_VT_CPU, },
{ .group = HV_GRP_T5_CPU, },
{ .group = HV_GRP_DIAG, .flags = FLAG_PRE_API },
+ { .group = HV_GRP_M7_PERF, },
};
static DEFINE_SPINLOCK(hvapi_lock);
retl
nop
ENDPROC(sun4v_t5_set_perfreg)
+
+ENTRY(sun4v_m7_get_perfreg)
+ mov %o1, %o4
+ mov HV_FAST_M7_GET_PERFREG, %o5
+ ta HV_FAST_TRAP
+ stx %o1, [%o4]
+ retl
+ nop
+ENDPROC(sun4v_m7_get_perfreg)
+
+ENTRY(sun4v_m7_set_perfreg)
+ mov HV_FAST_M7_SET_PERFREG, %o5
+ ta HV_FAST_TRAP
+ retl
+ nop
+ENDPROC(sun4v_m7_set_perfreg)
root_bus = pci_scan_root_bus(&ofdev->dev, 0, info->ops, info,
&resources);
- if (root_bus) {
- /* Setup IRQs of all devices using custom routines */
- pci_fixup_irqs(pci_common_swizzle, info->map_irq);
-
- /* Assign devices with resources */
- pci_assign_unassigned_resources();
- } else {
+ if (!root_bus) {
pci_free_resource_list(&resources);
+ return;
}
+
+ /* Setup IRQs of all devices using custom routines */
+ pci_fixup_irqs(pci_common_swizzle, info->map_irq);
+
+ /* Assign devices with resources */
+ pci_assign_unassigned_resources();
+ pci_bus_add_devices(root_bus);
}
void pcibios_fixup_bus(struct pci_bus *pbus)
(unsigned long long)r->end,
(unsigned int)r->flags);
- if (pci_claim_resource(dev, i) == 0)
- continue;
-
- pci_claim_bridge_resource(dev, i);
+ pci_claim_resource(dev, i);
}
}
}
pci_of_scan_bus(pbm, node, bus);
- pci_bus_add_devices(bus);
pci_bus_register_of_sysfs(bus);
pci_claim_bus_resources(bus);
-
+ pci_bus_add_devices(bus);
return bus;
}
struct linux_pbm_info *pbm = &pcic->pbm;
pbm->pci_bus = pci_scan_bus(pbm->pci_first_busno, &pcic_ops, pbm);
+ if (!pbm->pci_bus)
+ return;
+
#if 0 /* deadwood transplanted from sparc64 */
pci_fill_in_pbm_cookies(pbm->pci_bus, pbm, pbm->prom_node);
pci_record_assignments(pbm, pbm->pci_bus);
pci_assign_unassigned(pbm, pbm->pci_bus);
pci_fixup_irq(pbm, pbm->pci_bus);
#endif
+ pci_bus_add_devices(pbm->pci_bus);
}
/*
.pcr_nmi_disable = PCR_N4_PICNPT,
};
+static u64 m7_pcr_read(unsigned long reg_num)
+{
+ unsigned long val;
+
+ (void) sun4v_m7_get_perfreg(reg_num, &val);
+
+ return val;
+}
+
+static void m7_pcr_write(unsigned long reg_num, u64 val)
+{
+ (void) sun4v_m7_set_perfreg(reg_num, val);
+}
+
+static const struct pcr_ops m7_pcr_ops = {
+ .read_pcr = m7_pcr_read,
+ .write_pcr = m7_pcr_write,
+ .read_pic = n4_pic_read,
+ .write_pic = n4_pic_write,
+ .nmi_picl_value = n4_picl_value,
+ .pcr_nmi_enable = (PCR_N4_PICNPT | PCR_N4_STRACE |
+ PCR_N4_UTRACE | PCR_N4_TOE |
+ (26 << PCR_N4_SL_SHIFT)),
+ .pcr_nmi_disable = PCR_N4_PICNPT,
+};
static unsigned long perf_hsvc_group;
static unsigned long perf_hsvc_major;
perf_hsvc_group = HV_GRP_T5_CPU;
break;
+ case SUN4V_CHIP_SPARC_M7:
+ perf_hsvc_group = HV_GRP_M7_PERF;
+ break;
+
default:
return -ENODEV;
}
pcr_ops = &n5_pcr_ops;
break;
+ case SUN4V_CHIP_SPARC_M7:
+ pcr_ops = &m7_pcr_ops;
+ break;
+
default:
ret = -ENODEV;
break;
.num_pic_regs = 4,
};
+static void sparc_m7_write_pmc(int idx, u64 val)
+{
+ u64 pcr;
+
+ pcr = pcr_ops->read_pcr(idx);
+ /* ensure ov and ntc are reset */
+ pcr &= ~(PCR_N4_OV | PCR_N4_NTC);
+
+ pcr_ops->write_pic(idx, val & 0xffffffff);
+
+ pcr_ops->write_pcr(idx, pcr);
+}
+
+static const struct sparc_pmu sparc_m7_pmu = {
+ .event_map = niagara4_event_map,
+ .cache_map = &niagara4_cache_map,
+ .max_events = ARRAY_SIZE(niagara4_perfmon_event_map),
+ .read_pmc = sparc_vt_read_pmc,
+ .write_pmc = sparc_m7_write_pmc,
+ .upper_shift = 5,
+ .lower_shift = 5,
+ .event_mask = 0x7ff,
+ .user_bit = PCR_N4_UTRACE,
+ .priv_bit = PCR_N4_STRACE,
+
+ /* We explicitly don't support hypervisor tracing. */
+ .hv_bit = 0,
+
+ .irq_bit = PCR_N4_TOE,
+ .upper_nop = 0,
+ .lower_nop = 0,
+ .flags = 0,
+ .max_hw_events = 4,
+ .num_pcrs = 4,
+ .num_pic_regs = 4,
+};
static const struct sparc_pmu *sparc_pmu __read_mostly;
static u64 event_encoding(u64 event_id, int idx)
cpuc->pcr[0] |= cpuc->event[0]->hw.config_base;
}
+static void sparc_pmu_start(struct perf_event *event, int flags);
+
/* On this PMU each PIC has it's own PCR control register. */
static void calculate_multiple_pcrs(struct cpu_hw_events *cpuc)
{
struct perf_event *cp = cpuc->event[i];
struct hw_perf_event *hwc = &cp->hw;
int idx = hwc->idx;
- u64 enc;
if (cpuc->current_idx[i] != PIC_NO_INDEX)
continue;
- sparc_perf_event_set_period(cp, hwc, idx);
cpuc->current_idx[i] = idx;
- enc = perf_event_get_enc(cpuc->events[i]);
- cpuc->pcr[idx] &= ~mask_for_index(idx);
- if (hwc->state & PERF_HES_STOPPED)
- cpuc->pcr[idx] |= nop_for_index(idx);
- else
- cpuc->pcr[idx] |= event_encoding(enc, idx);
+ sparc_pmu_start(cp, PERF_EF_RELOAD);
}
out:
for (i = 0; i < cpuc->n_events; i++) {
int i;
local_irq_save(flags);
- perf_pmu_disable(event->pmu);
for (i = 0; i < cpuc->n_events; i++) {
if (event == cpuc->event[i]) {
}
}
- perf_pmu_enable(event->pmu);
local_irq_restore(flags);
}
unsigned long flags;
local_irq_save(flags);
- perf_pmu_disable(event->pmu);
n0 = cpuc->n_events;
if (n0 >= sparc_pmu->max_hw_events)
ret = 0;
out:
- perf_pmu_enable(event->pmu);
local_irq_restore(flags);
return ret;
}
sparc_pmu = &niagara4_pmu;
return true;
}
+ if (!strcmp(sparc_pmu_type, "sparc-m7")) {
+ sparc_pmu = &sparc_m7_pmu;
+ return true;
+ }
return false;
}
printk(" TPC[%lx] O7[%lx] I7[%lx] RPC[%lx]\n",
gp->tpc, gp->o7, gp->i7, gp->rpc);
}
+
+ touch_nmi_watchdog();
}
memset(global_cpu_snapshot, 0, sizeof(global_cpu_snapshot));
(cpu == this_cpu ? '*' : ' '), cpu,
pp->pcr[0], pp->pcr[1], pp->pcr[2], pp->pcr[3],
pp->pic[0], pp->pic[1], pp->pic[2], pp->pic[3]);
+
+ touch_nmi_watchdog();
}
memset(global_cpu_snapshot, 0, sizeof(global_cpu_snapshot));
scheduler_ipi();
}
-/* This is a nop because we capture all other cpus
- * anyways when making the PROM active.
- */
+static void stop_this_cpu(void *dummy)
+{
+ prom_stopself();
+}
+
void smp_send_stop(void)
{
+ int cpu;
+
+ if (tlb_type == hypervisor) {
+ for_each_online_cpu(cpu) {
+ if (cpu == smp_processor_id())
+ continue;
+#ifdef CONFIG_SUN_LDOMS
+ if (ldom_domaining_enabled) {
+ unsigned long hv_err;
+ hv_err = sun4v_cpu_stop(cpu);
+ if (hv_err)
+ printk(KERN_ERR "sun4v_cpu_stop() "
+ "failed err=%lu\n", hv_err);
+ } else
+#endif
+ prom_stopcpu_cpuid(cpu);
+ }
+ } else
+ smp_call_function(stop_this_cpu, NULL, 0);
}
/**
this_is_starfire = 1;
}
-int starfire_hard_smp_processor_id(void)
-{
- return upa_readl(0x1fff40000d0UL);
-}
-
/*
* Each Starfire board has 32 registers which perform translation
* and delivery of traditional interrupt packets into the extended
long err;
/* No need for backward compatibility. We can start fresh... */
- if (call <= SEMCTL) {
+ if (call <= SEMTIMEDOP) {
switch (call) {
case SEMOP:
err = sys_semtimedop(first, ptr,
.rating = 100,
.read = timer_cs_read,
.mask = CLOCKSOURCE_MASK(64),
- .shift = 2,
.flags = CLOCK_SOURCE_IS_CONTINUOUS,
};
static __init int setup_timer_cs(void)
{
timer_cs_enabled = 1;
- timer_cs.mult = clocksource_hz2mult(sparc_config.clock_rate,
- timer_cs.shift);
-
- return clocksource_register(&timer_cs);
+ return clocksource_register_hz(&timer_cs, sparc_config.clock_rate);
}
#ifdef CONFIG_SMP
}
user_instruction_dump ((unsigned int __user *) regs->tpc);
}
+ if (panic_on_oops)
+ panic("Fatal exception");
if (regs->tstate & TSTATE_PRIV)
do_exit(SIGKILL);
do_exit(SIGSEGV);
die_if_kernel("TL0: Cache Error Exception", regs);
}
-void do_cee_tl1(struct pt_regs *regs)
-{
- exception_enter();
- dump_tl1_traplog((struct tl1_traplog *)(regs + 1));
- die_if_kernel("TL1: Cache Error Exception", regs);
-}
-
-void do_dae_tl1(struct pt_regs *regs)
-{
- exception_enter();
- dump_tl1_traplog((struct tl1_traplog *)(regs + 1));
- die_if_kernel("TL1: Data Access Exception", regs);
-}
-
-void do_iae_tl1(struct pt_regs *regs)
-{
- exception_enter();
- dump_tl1_traplog((struct tl1_traplog *)(regs + 1));
- die_if_kernel("TL1: Instruction Access Exception", regs);
-}
-
void do_div0_tl1(struct pt_regs *regs)
{
exception_enter();
die_if_kernel("TL1: DIV0 Exception", regs);
}
-void do_fpdis_tl1(struct pt_regs *regs)
-{
- exception_enter();
- dump_tl1_traplog((struct tl1_traplog *)(regs + 1));
- die_if_kernel("TL1: FPU Disabled", regs);
-}
-
void do_fpieee_tl1(struct pt_regs *regs)
{
exception_enter();
.text
ENTRY(memmove) /* o0=dst o1=src o2=len */
- mov %o0, %g1
+ brz,pn %o2, 99f
+ mov %o0, %g1
+
cmp %o0, %o1
- bleu,pt %xcc, memcpy
+ bleu,pt %xcc, 2f
add %o1, %o2, %g7
cmp %g7, %o0
bleu,pt %xcc, memcpy
stb %g7, [%o0]
bne,pt %icc, 1b
sub %o0, 1, %o0
-
+99:
retl
mov %g1, %o0
+
+ /* We can't just call memcpy for these memmove cases. On some
+ * chips the memcpy uses cache initializing stores and when dst
+ * and src are close enough, those can clobber the source data
+ * before we've loaded it in.
+ */
+2: or %o0, %o1, %g7
+ or %o2, %g7, %g7
+ andcc %g7, 0x7, %g0
+ bne,pn %xcc, 4f
+ nop
+
+3: ldx [%o1], %g7
+ add %o1, 8, %o1
+ subcc %o2, 8, %o2
+ add %o0, 8, %o0
+ bne,pt %icc, 3b
+ stx %g7, [%o0 - 0x8]
+ ba,a,pt %xcc, 99b
+
+4: ldub [%o1], %g7
+ add %o1, 1, %o1
+ subcc %o2, 1, %o2
+ add %o0, 1, %o0
+ bne,pt %icc, 4b
+ stb %g7, [%o0 - 0x1]
+ ba,a,pt %xcc, 99b
ENDPROC(memmove)
return 0;
}
-device_initcall(report_memory);
+arch_initcall(report_memory);
#ifdef CONFIG_SMP
#define do_flush_tlb_kernel_range smp_flush_tlb_kernel_range
struct pci_bus *next_bus;
struct pci_dev *dev;
+ pci_bus_add_devices(root_bus);
+
list_for_each_entry(dev, &root_bus->devices, bus_list) {
/*
* Find the PCI host controller, ie. the 1st
alloc_mem_map_failed:
break;
}
+
+ pci_bus_add_devices(root_bus);
}
return 0;
void update_vsyscall(struct timekeeper *tk)
{
- if (tk->tkr.clock != &cycle_counter_cs)
+ if (tk->tkr_mono.clock != &cycle_counter_cs)
return;
write_seqcount_begin(&vdso_data->tb_seq);
- vdso_data->cycle_last = tk->tkr.cycle_last;
- vdso_data->mask = tk->tkr.mask;
- vdso_data->mult = tk->tkr.mult;
- vdso_data->shift = tk->tkr.shift;
+ vdso_data->cycle_last = tk->tkr_mono.cycle_last;
+ vdso_data->mask = tk->tkr_mono.mask;
+ vdso_data->mult = tk->tkr_mono.mult;
+ vdso_data->shift = tk->tkr_mono.shift;
vdso_data->wall_time_sec = tk->xtime_sec;
- vdso_data->wall_time_snsec = tk->tkr.xtime_nsec;
+ vdso_data->wall_time_snsec = tk->tkr_mono.xtime_nsec;
vdso_data->monotonic_time_sec = tk->xtime_sec
+ tk->wall_to_monotonic.tv_sec;
- vdso_data->monotonic_time_snsec = tk->tkr.xtime_nsec
+ vdso_data->monotonic_time_snsec = tk->tkr_mono.xtime_nsec
+ ((u64)tk->wall_to_monotonic.tv_nsec
- << tk->tkr.shift);
+ << tk->tkr_mono.shift);
while (vdso_data->monotonic_time_snsec >=
- (((u64)NSEC_PER_SEC) << tk->tkr.shift)) {
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
vdso_data->monotonic_time_snsec -=
- ((u64)NSEC_PER_SEC) << tk->tkr.shift;
+ ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
vdso_data->monotonic_time_sec++;
}
vdso_data->wall_time_coarse_sec = tk->xtime_sec;
- vdso_data->wall_time_coarse_nsec = (long)(tk->tkr.xtime_nsec >>
- tk->tkr.shift);
+ vdso_data->wall_time_coarse_nsec = (long)(tk->tkr_mono.xtime_nsec >>
+ tk->tkr_mono.shift);
vdso_data->monotonic_time_coarse_sec =
vdso_data->wall_time_coarse_sec + tk->wall_to_monotonic.tv_sec;
pci_fixup_irqs(pci_common_swizzle, pci_puv3_map_irq);
if (!pci_has_flag(PCI_PROBE_ONLY)) {
- /*
- * Size the bridge windows.
- */
pci_bus_size_bridges(puv3_bus);
-
- /*
- * Assign resources.
- */
pci_bus_assign_resources(puv3_bus);
}
-
+ pci_bus_add_devices(puv3_bus);
return 0;
}
subsys_initcall(pci_common_init);
def_bool y
config ZONE_DMA32
- bool
- default X86_64
+ def_bool y if X86_64
config AUDIT_ARCH
- bool
- default X86_64
+ def_bool y if X86_64
config ARCH_SUPPORTS_OPTIMIZED_INLINING
def_bool y
depends on X86_IO_APIC
select IOSF_MBI
select INTEL_IMR
+ select COMMON_CLK
---help---
Select to include support for Quark X1000 SoC.
Say Y here if you have a Quark based system such as the Arduino
depends on !SMP && X86_LOCAL_APIC
config X86_UP_APIC
- bool "Local APIC support on uniprocessors"
+ bool "Local APIC support on uniprocessors" if !PCI_MSI
+ default PCI_MSI
depends on X86_32 && !SMP && !X86_32_NON_STANDARD
---help---
A local APIC (Advanced Programmable Interrupt Controller) is an
performance counters), and the NMI watchdog which detects hard
lockups.
-config X86_UP_APIC_MSI
- def_bool y
- select X86_UP_APIC if X86_32 && !SMP && !X86_32_NON_STANDARD && PCI_MSI
-
config X86_UP_IOAPIC
bool "IO-APIC support on uniprocessors"
depends on X86_UP_APIC
select GENERIC_IRQ_LEGACY_ALLOC_HWIRQ
config X86_IO_APIC
- def_bool X86_64 || SMP || X86_32_NON_STANDARD || X86_UP_IOAPIC
- depends on X86_LOCAL_APIC
+ def_bool y
+ depends on X86_LOCAL_APIC || X86_UP_IOAPIC
select IRQ_DOMAIN
config X86_REROUTE_FOR_BROKEN_BOOT_IRQS
depends on MICROCODE
config MICROCODE_INTEL_EARLY
- def_bool n
+ bool
config MICROCODE_AMD_EARLY
- def_bool n
+ bool
config MICROCODE_EARLY
bool "Early load microcode"
def_bool y
depends on X86_64 || HIGHMEM64G
-config DIRECT_GBPAGES
- bool "Enable 1GB pages for kernel pagetables" if EXPERT
- default y
- depends on X86_64
+config X86_DIRECT_GBPAGES
+ def_bool y
+ depends on X86_64 && !DEBUG_PAGEALLOC && !KMEMCHECK
---help---
- Allow the kernel linear mapping to use 1GB pages on CPUs that
- support it. This can improve the kernel's performance a tiny bit by
- reducing TLB pressure. If in doubt, say "Y".
+ Certain kernel features effectively disable kernel
+ linear 1 GB mappings (even if the CPU otherwise
+ supports them), so don't confuse the user by printing
+ that we have them enabled.
# Common NUMA Features
config NUMA
depends on KEXEC_FILE
---help---
This option makes kernel signature verification mandatory for
- kexec_file_load() syscall. If kernel is signature can not be
- verified, kexec_file_load() will fail.
-
- This option enforces signature verification at generic level.
- One needs to enable signature verification for type of kernel
- image being loaded to make sure it works. For example, enable
- bzImage signature verification option to be able to load and
- verify signatures of bzImage. Otherwise kernel loading will fail.
+ the kexec_file_load() syscall.
+
+ In addition to that option, you need to enable signature
+ verification for the corresponding kernel image type being
+ loaded in order for this to work.
config KEXEC_BZIMAGE_VERIFY_SIG
bool "Enable bzImage signature verification support"
static const char build_str[] = UTS_RELEASE " (" LINUX_COMPILE_BY "@"
LINUX_COMPILE_HOST ") (" LINUX_COMPILER ") " UTS_VERSION;
-struct kaslr_setup_data {
- __u64 next;
- __u32 type;
- __u32 len;
- __u8 data[1];
-} kaslr_setup_data;
-
#define I8254_PORT_CONTROL 0x43
#define I8254_PORT_COUNTER0 0x40
#define I8254_CMD_READBACK 0xC0
return slots_fetch_random();
}
-static void add_kaslr_setup_data(struct boot_params *params, __u8 enabled)
-{
- struct setup_data *data;
-
- kaslr_setup_data.type = SETUP_KASLR;
- kaslr_setup_data.len = 1;
- kaslr_setup_data.next = 0;
- kaslr_setup_data.data[0] = enabled;
-
- data = (struct setup_data *)(unsigned long)params->hdr.setup_data;
-
- while (data && data->next)
- data = (struct setup_data *)(unsigned long)data->next;
-
- if (data)
- data->next = (unsigned long)&kaslr_setup_data;
- else
- params->hdr.setup_data = (unsigned long)&kaslr_setup_data;
-
-}
-
-unsigned char *choose_kernel_location(struct boot_params *params,
+unsigned char *choose_kernel_location(struct boot_params *boot_params,
unsigned char *input,
unsigned long input_size,
unsigned char *output,
#ifdef CONFIG_HIBERNATION
if (!cmdline_find_option_bool("kaslr")) {
debug_putstr("KASLR disabled by default...\n");
- add_kaslr_setup_data(params, 0);
goto out;
}
#else
if (cmdline_find_option_bool("nokaslr")) {
debug_putstr("KASLR disabled by cmdline...\n");
- add_kaslr_setup_data(params, 0);
goto out;
}
#endif
- add_kaslr_setup_data(params, 1);
+
+ boot_params->hdr.loadflags |= KASLR_FLAG;
/* Record the various known unsafe memory ranges. */
mem_avoid_init((unsigned long)input, input_size,
#include <asm/page_types.h>
#include <asm/boot.h>
#include <asm/asm-offsets.h>
+#include <asm/bootparam.h>
__HEAD
ENTRY(startup_32)
* Test KEEP_SEGMENTS flag to see if the bootloader is asking
* us to not reload segments
*/
- testb $(1<<6), BP_loadflags(%esi)
+ testb $KEEP_SEGMENTS, BP_loadflags(%esi)
jnz 1f
cli
#include <asm/msr.h>
#include <asm/processor-flags.h>
#include <asm/asm-offsets.h>
+#include <asm/bootparam.h>
__HEAD
.code32
* Test KEEP_SEGMENTS flag to see if the bootloader is asking
* us to not reload segments
*/
- testb $(1<<6), BP_loadflags(%esi)
+ testb $KEEP_SEGMENTS, BP_loadflags(%esi)
jnz 1f
cli
/* After gdt is loaded */
xorl %eax, %eax
lldt %ax
- movl $0x20, %eax
+ movl $__BOOT_TSS, %eax
ltr %ax
/*
real_mode = rmode;
+ /* Clear it for solely in-kernel use */
+ real_mode->hdr.loadflags &= ~KASLR_FLAG;
+
sanitize_boot_params(real_mode);
if (real_mode->screen_info.orig_video_mode == 7) {
* the entire decompressed kernel plus relocation table, or the
* entire decompressed kernel plus .bss and .brk sections.
*/
- output = choose_kernel_location(real_mode, input_data, input_len,
- output,
+ output = choose_kernel_location(real_mode, input_data, input_len, output,
output_len > run_size ? output_len
: run_size);
#if CONFIG_RANDOMIZE_BASE
/* aslr.c */
-unsigned char *choose_kernel_location(struct boot_params *params,
+unsigned char *choose_kernel_location(struct boot_params *boot_params,
unsigned char *input,
unsigned long input_size,
unsigned char *output,
bool has_cpuflag(int flag);
#else
static inline
-unsigned char *choose_kernel_location(struct boot_params *params,
+unsigned char *choose_kernel_location(struct boot_params *boot_params,
unsigned char *input,
unsigned long input_size,
unsigned char *output,
int delta = 0;
while (*s1 || *s2) {
- delta = *s2 - *s1;
+ delta = *s1 - *s2;
if (delta)
return delta;
s1++;
/*
* Common variables
*/
-int adapter; /* 0=CGA/MDA/HGC, 1=EGA, 2=VGA+ */
-u16 video_segment;
+int adapter; /* 0=CGA/MDA/HGC, 1=EGA, 2=VGA+ */
int force_x, force_y; /* Don't query the BIOS for cols/rows */
-
int do_restore; /* Screen contents changed during mode flip */
int graphic_mode; /* Graphic mode with linear frame buffer */
#include "video.h"
#include "vesa.h"
+static u16 video_segment;
+
static void store_cursor_position(void)
{
struct biosregs ireg, oreg;
#define ADAPTER_VGA 2
extern int adapter;
-extern u16 video_segment;
extern int force_x, force_y; /* Don't query the BIOS for cols/rows */
extern int do_restore; /* Restore screen contents */
extern int graphic_mode; /* Graphics mode with linear frame buffer */
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
-# CONFIG_USB_EHCI_TT_NEWSCHED is not set
+CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_PRINTER=y
CONFIG_USB_ANNOUNCE_NEW_DEVICES=y
CONFIG_USB_MON=y
CONFIG_USB_EHCI_HCD=y
-# CONFIG_USB_EHCI_TT_NEWSCHED is not set
+CONFIG_USB_EHCI_TT_NEWSCHED=y
CONFIG_USB_OHCI_HCD=y
CONFIG_USB_UHCI_HCD=y
CONFIG_USB_PRINTER=y
src = kmalloc(req->cryptlen + req->assoclen, GFP_ATOMIC);
if (!src)
return -ENOMEM;
- assoc = (src + req->cryptlen + auth_tag_len);
+ assoc = (src + req->cryptlen);
scatterwalk_map_and_copy(src, req->src, 0, req->cryptlen, 0);
scatterwalk_map_and_copy(assoc, req->assoc, 0,
req->assoclen, 0);
scatterwalk_done(&src_sg_walk, 0, 0);
scatterwalk_done(&assoc_sg_walk, 0, 0);
} else {
- scatterwalk_map_and_copy(dst, req->dst, 0, req->cryptlen, 1);
+ scatterwalk_map_and_copy(dst, req->dst, 0, tempCipherLen, 1);
kfree(src);
}
return retval;
## 2a) PROCESS FULL BLOCKS:
################################################################
full_block:
- movq $128,%rax
+ movl $128,%eax
lea 128*8*2(block_0), block_1
lea 128*8*3(block_0), block_2
add $128*8*1, block_0
movq R1, 8(%rsi)
popq R1
- movq $1,%rax
+ movl $1,%eax
ret
ENDPROC(twofish_enc_blk)
movq R1, 8(%rsi)
popq R1
- movq $1,%rax
+ movl $1,%eax
ret
ENDPROC(twofish_dec_blk)
#
obj-$(CONFIG_IA32_EMULATION) := ia32entry.o sys_ia32.o ia32_signal.o
-obj-$(CONFIG_IA32_EMULATION) += nosyscall.o syscall_ia32.o
obj-$(CONFIG_IA32_AOUT) += ia32_aout.o
}
static int ia32_restore_sigcontext(struct pt_regs *regs,
- struct sigcontext_ia32 __user *sc,
- unsigned int *pax)
+ struct sigcontext_ia32 __user *sc)
{
unsigned int tmpflags, err = 0;
void __user *buf;
RELOAD_SEG(es);
COPY(di); COPY(si); COPY(bp); COPY(sp); COPY(bx);
- COPY(dx); COPY(cx); COPY(ip);
+ COPY(dx); COPY(cx); COPY(ip); COPY(ax);
/* Don't touch extended registers */
COPY_SEG_CPL3(cs);
get_user_ex(tmp, &sc->fpstate);
buf = compat_ptr(tmp);
-
- get_user_ex(*pax, &sc->ax);
} get_user_catch(err);
err |= restore_xstate_sig(buf, 1);
+ force_iret();
+
return err;
}
struct pt_regs *regs = current_pt_regs();
struct sigframe_ia32 __user *frame = (struct sigframe_ia32 __user *)(regs->sp-8);
sigset_t set;
- unsigned int ax;
if (!access_ok(VERIFY_READ, frame, sizeof(*frame)))
goto badframe;
set_current_blocked(&set);
- if (ia32_restore_sigcontext(regs, &frame->sc, &ax))
+ if (ia32_restore_sigcontext(regs, &frame->sc))
goto badframe;
- return ax;
+ return regs->ax;
badframe:
signal_fault(regs, frame, "32bit sigreturn");
struct pt_regs *regs = current_pt_regs();
struct rt_sigframe_ia32 __user *frame;
sigset_t set;
- unsigned int ax;
frame = (struct rt_sigframe_ia32 __user *)(regs->sp - 4);
set_current_blocked(&set);
- if (ia32_restore_sigcontext(regs, &frame->uc.uc_mcontext, &ax))
+ if (ia32_restore_sigcontext(regs, &frame->uc.uc_mcontext))
goto badframe;
if (compat_restore_altstack(&frame->uc.uc_stack))
goto badframe;
- return ax;
+ return regs->ax;
badframe:
signal_fault(regs, frame, "32bit rt sigreturn");
.section .entry.text, "ax"
- .macro IA32_ARG_FIXUP noebp=0
- movl %edi,%r8d
- .if \noebp
- .else
- movl %ebp,%r9d
- .endif
- xchg %ecx,%esi
- movl %ebx,%edi
- movl %edx,%edx /* zero extension */
- .endm
-
- /* clobbers %eax */
- .macro CLEAR_RREGS offset=0, _r9=rax
+ /* clobbers %rax */
+ .macro CLEAR_RREGS _r9=rax
xorl %eax,%eax
- movq %rax,\offset+R11(%rsp)
- movq %rax,\offset+R10(%rsp)
- movq %\_r9,\offset+R9(%rsp)
- movq %rax,\offset+R8(%rsp)
+ movq %rax,R11(%rsp)
+ movq %rax,R10(%rsp)
+ movq %\_r9,R9(%rsp)
+ movq %rax,R8(%rsp)
.endm
/*
* If it's -1 to make us punt the syscall, then (u32)-1 is still
* an appropriately invalid value.
*/
- .macro LOAD_ARGS32 offset, _r9=0
+ .macro LOAD_ARGS32 _r9=0
.if \_r9
- movl \offset+16(%rsp),%r9d
+ movl R9(%rsp),%r9d
.endif
- movl \offset+40(%rsp),%ecx
- movl \offset+48(%rsp),%edx
- movl \offset+56(%rsp),%esi
- movl \offset+64(%rsp),%edi
+ movl RCX(%rsp),%ecx
+ movl RDX(%rsp),%edx
+ movl RSI(%rsp),%esi
+ movl RDI(%rsp),%edi
movl %eax,%eax /* zero extension */
.endm
/*
* 32bit SYSENTER instruction entry.
*
+ * SYSENTER loads ss, rsp, cs, and rip from previously programmed MSRs.
+ * IF and VM in rflags are cleared (IOW: interrupts are off).
+ * SYSENTER does not save anything on the stack,
+ * and does not save old rip (!!!) and rflags.
+ *
* Arguments:
- * %eax System call number.
- * %ebx Arg1
- * %ecx Arg2
- * %edx Arg3
- * %esi Arg4
- * %edi Arg5
- * %ebp user stack
- * 0(%ebp) Arg6
- *
- * Interrupts off.
- *
+ * eax system call number
+ * ebx arg1
+ * ecx arg2
+ * edx arg3
+ * esi arg4
+ * edi arg5
+ * ebp user stack
+ * 0(%ebp) arg6
+ *
* This is purely a fast path. For anything complicated we use the int 0x80
- * path below. Set up a complete hardware stack frame to share code
+ * path below. We set up a complete hardware stack frame to share code
* with the int 0x80 path.
- */
+ */
ENTRY(ia32_sysenter_target)
CFI_STARTPROC32 simple
CFI_SIGNAL_FRAME
CFI_DEF_CFA rsp,0
CFI_REGISTER rsp,rbp
- SWAPGS_UNSAFE_STACK
- movq PER_CPU_VAR(kernel_stack), %rsp
- addq $(KERNEL_STACK_OFFSET),%rsp
+
/*
- * No need to follow this irqs on/off section: the syscall
- * disabled irqs, here we enable it straight after entry:
+ * Interrupts are off on entry.
+ * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON,
+ * it is too small to ever cause noticeable irq latency.
*/
+ SWAPGS_UNSAFE_STACK
+ movq PER_CPU_VAR(cpu_tss + TSS_sp0), %rsp
ENABLE_INTERRUPTS(CLBR_NONE)
- movl %ebp,%ebp /* zero extension */
- pushq_cfi $__USER32_DS
- /*CFI_REL_OFFSET ss,0*/
- pushq_cfi %rbp
- CFI_REL_OFFSET rsp,0
- pushfq_cfi
- /*CFI_REL_OFFSET rflags,0*/
- movl TI_sysenter_return+THREAD_INFO(%rsp,3*8-KERNEL_STACK_OFFSET),%r10d
- CFI_REGISTER rip,r10
- pushq_cfi $__USER32_CS
- /*CFI_REL_OFFSET cs,0*/
+
+ /* Zero-extending 32-bit regs, do not remove */
+ movl %ebp, %ebp
movl %eax, %eax
- pushq_cfi %r10
- CFI_REL_OFFSET rip,0
- pushq_cfi %rax
+
+ movl ASM_THREAD_INFO(TI_sysenter_return, %rsp, 0), %r10d
+ CFI_REGISTER rip,r10
+
+ /* Construct struct pt_regs on stack */
+ pushq_cfi $__USER32_DS /* pt_regs->ss */
+ pushq_cfi %rbp /* pt_regs->sp */
+ CFI_REL_OFFSET rsp,0
+ pushfq_cfi /* pt_regs->flags */
+ pushq_cfi $__USER32_CS /* pt_regs->cs */
+ pushq_cfi %r10 /* pt_regs->ip = thread_info->sysenter_return */
+ CFI_REL_OFFSET rip,0
+ pushq_cfi_reg rax /* pt_regs->orig_ax */
+ pushq_cfi_reg rdi /* pt_regs->di */
+ pushq_cfi_reg rsi /* pt_regs->si */
+ pushq_cfi_reg rdx /* pt_regs->dx */
+ pushq_cfi_reg rcx /* pt_regs->cx */
+ pushq_cfi_reg rax /* pt_regs->ax */
cld
- SAVE_ARGS 0,1,0
- /* no need to do an access_ok check here because rbp has been
- 32bit zero extended */
+ sub $(10*8),%rsp /* pt_regs->r8-11,bp,bx,r12-15 not saved */
+ CFI_ADJUST_CFA_OFFSET 10*8
+
+ /*
+ * no need to do an access_ok check here because rbp has been
+ * 32bit zero extended
+ */
ASM_STAC
1: movl (%rbp),%ebp
_ASM_EXTABLE(1b,ia32_badarg)
* ourselves. To save a few cycles, we can check whether
* NT was set instead of doing an unconditional popfq.
*/
- testl $X86_EFLAGS_NT,EFLAGS-ARGOFFSET(%rsp)
+ testl $X86_EFLAGS_NT,EFLAGS(%rsp)
jnz sysenter_fix_flags
sysenter_flags_fixed:
- orl $TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ orl $TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
+ testl $_TIF_WORK_SYSCALL_ENTRY, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
CFI_REMEMBER_STATE
jnz sysenter_tracesys
cmpq $(IA32_NR_syscalls-1),%rax
ja ia32_badsys
sysenter_do_call:
- IA32_ARG_FIXUP
+ /* 32bit syscall -> 64bit C ABI argument conversion */
+ movl %edi,%r8d /* arg5 */
+ movl %ebp,%r9d /* arg6 */
+ xchg %ecx,%esi /* rsi:arg2, rcx:arg4 */
+ movl %ebx,%edi /* arg1 */
+ movl %edx,%edx /* arg3 (zero extension) */
sysenter_dispatch:
call *ia32_sys_call_table(,%rax,8)
- movq %rax,RAX-ARGOFFSET(%rsp)
+ movq %rax,RAX(%rsp)
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
- testl $_TIF_ALLWORK_MASK,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl $_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jnz sysexit_audit
sysexit_from_sys_call:
- andl $~TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- /* clear IF, that popfq doesn't enable interrupts early */
- andl $~0x200,EFLAGS-ARGOFFSET(%rsp)
- movl RIP-ARGOFFSET(%rsp),%edx /* User %eip */
- CFI_REGISTER rip,rdx
- RESTORE_ARGS 0,24,0,0,0,0
+ /*
+ * NB: SYSEXIT is not obviously safe for 64-bit kernels -- an
+ * NMI between STI and SYSEXIT has poorly specified behavior,
+ * and and NMI followed by an IRQ with usergs is fatal. So
+ * we just pretend we're using SYSEXIT but we really use
+ * SYSRETL instead.
+ *
+ * This code path is still called 'sysexit' because it pairs
+ * with 'sysenter' and it uses the SYSENTER calling convention.
+ */
+ andl $~TS_COMPAT,ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
+ movl RIP(%rsp),%ecx /* User %eip */
+ CFI_REGISTER rip,rcx
+ RESTORE_RSI_RDI
+ xorl %edx,%edx /* avoid info leaks */
xorq %r8,%r8
xorq %r9,%r9
xorq %r10,%r10
- xorq %r11,%r11
- popfq_cfi
+ movl EFLAGS(%rsp),%r11d /* User eflags */
/*CFI_RESTORE rflags*/
- popq_cfi %rcx /* User %esp */
- CFI_REGISTER rsp,rcx
TRACE_IRQS_ON
- ENABLE_INTERRUPTS_SYSEXIT32
+
+ /*
+ * SYSRETL works even on Intel CPUs. Use it in preference to SYSEXIT,
+ * since it avoids a dicey window with interrupts enabled.
+ */
+ movl RSP(%rsp),%esp
+
+ /*
+ * USERGS_SYSRET32 does:
+ * gsbase = user's gs base
+ * eip = ecx
+ * rflags = r11
+ * cs = __USER32_CS
+ * ss = __USER_DS
+ *
+ * The prologue set RIP(%rsp) to VDSO32_SYSENTER_RETURN, which does:
+ *
+ * pop %ebp
+ * pop %edx
+ * pop %ecx
+ *
+ * Therefore, we invoke SYSRETL with EDX and R8-R10 zeroed to
+ * avoid info leaks. R11 ends up with VDSO32_SYSENTER_RETURN's
+ * address (already known to user code), and R12-R15 are
+ * callee-saved and therefore don't contain any interesting
+ * kernel data.
+ */
+ USERGS_SYSRET32
CFI_RESTORE_STATE
movl %ebx,%esi /* 2nd arg: 1st syscall arg */
movl %eax,%edi /* 1st arg: syscall number */
call __audit_syscall_entry
- movl RAX-ARGOFFSET(%rsp),%eax /* reload syscall number */
+ movl RAX(%rsp),%eax /* reload syscall number */
cmpq $(IA32_NR_syscalls-1),%rax
ja ia32_badsys
movl %ebx,%edi /* reload 1st syscall arg */
- movl RCX-ARGOFFSET(%rsp),%esi /* reload 2nd syscall arg */
- movl RDX-ARGOFFSET(%rsp),%edx /* reload 3rd syscall arg */
- movl RSI-ARGOFFSET(%rsp),%ecx /* reload 4th syscall arg */
- movl RDI-ARGOFFSET(%rsp),%r8d /* reload 5th syscall arg */
+ movl RCX(%rsp),%esi /* reload 2nd syscall arg */
+ movl RDX(%rsp),%edx /* reload 3rd syscall arg */
+ movl RSI(%rsp),%ecx /* reload 4th syscall arg */
+ movl RDI(%rsp),%r8d /* reload 5th syscall arg */
.endm
.macro auditsys_exit exit
- testl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT),TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jnz ia32_ret_from_sys_call
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
1: setbe %al /* 1 if error, 0 if not */
movzbl %al,%edi /* zero-extend that into %edi */
call __audit_syscall_exit
- movq RAX-ARGOFFSET(%rsp),%rax /* reload syscall return value */
+ movq RAX(%rsp),%rax /* reload syscall return value */
movl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT),%edi
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
- testl %edi,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl %edi, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jz \exit
- CLEAR_RREGS -ARGOFFSET
+ CLEAR_RREGS
jmp int_with_check
.endm
sysenter_tracesys:
#ifdef CONFIG_AUDITSYSCALL
- testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT),TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jz sysenter_auditsys
#endif
- SAVE_REST
+ SAVE_EXTRA_REGS
CLEAR_RREGS
movq $-ENOSYS,RAX(%rsp)/* ptrace can change this for a bad syscall */
movq %rsp,%rdi /* &pt_regs -> arg1 */
call syscall_trace_enter
- LOAD_ARGS32 ARGOFFSET /* reload args from stack in case ptrace changed it */
- RESTORE_REST
+ LOAD_ARGS32 /* reload args from stack in case ptrace changed it */
+ RESTORE_EXTRA_REGS
cmpq $(IA32_NR_syscalls-1),%rax
ja int_ret_from_sys_call /* sysenter_tracesys has set RAX(%rsp) */
jmp sysenter_do_call
/*
* 32bit SYSCALL instruction entry.
*
+ * 32bit SYSCALL saves rip to rcx, clears rflags.RF, then saves rflags to r11,
+ * then loads new ss, cs, and rip from previously programmed MSRs.
+ * rflags gets masked by a value from another MSR (so CLD and CLAC
+ * are not needed). SYSCALL does not save anything on the stack
+ * and does not change rsp.
+ *
+ * Note: rflags saving+masking-with-MSR happens only in Long mode
+ * (in legacy 32bit mode, IF, RF and VM bits are cleared and that's it).
+ * Don't get confused: rflags saving+masking depends on Long Mode Active bit
+ * (EFER.LMA=1), NOT on bitness of userspace where SYSCALL executes
+ * or target CS descriptor's L bit (SYSCALL does not read segment descriptors).
+ *
* Arguments:
- * %eax System call number.
- * %ebx Arg1
- * %ecx return EIP
- * %edx Arg3
- * %esi Arg4
- * %edi Arg5
- * %ebp Arg2 [note: not saved in the stack frame, should not be touched]
- * %esp user stack
- * 0(%esp) Arg6
- *
- * Interrupts off.
- *
+ * eax system call number
+ * ecx return address
+ * ebx arg1
+ * ebp arg2 (note: not saved in the stack frame, should not be touched)
+ * edx arg3
+ * esi arg4
+ * edi arg5
+ * esp user stack
+ * 0(%esp) arg6
+ *
* This is purely a fast path. For anything complicated we use the int 0x80
- * path below. Set up a complete hardware stack frame to share code
- * with the int 0x80 path.
- */
+ * path below. We set up a complete hardware stack frame to share code
+ * with the int 0x80 path.
+ */
ENTRY(ia32_cstar_target)
CFI_STARTPROC32 simple
CFI_SIGNAL_FRAME
- CFI_DEF_CFA rsp,KERNEL_STACK_OFFSET
+ CFI_DEF_CFA rsp,0
CFI_REGISTER rip,rcx
/*CFI_REGISTER rflags,r11*/
+
+ /*
+ * Interrupts are off on entry.
+ * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON,
+ * it is too small to ever cause noticeable irq latency.
+ */
SWAPGS_UNSAFE_STACK
movl %esp,%r8d
CFI_REGISTER rsp,r8
movq PER_CPU_VAR(kernel_stack),%rsp
- /*
- * No need to follow this irqs on/off section: the syscall
- * disabled irqs and here we enable it straight after entry:
- */
ENABLE_INTERRUPTS(CLBR_NONE)
- SAVE_ARGS 8,0,0
- movl %eax,%eax /* zero extension */
- movq %rax,ORIG_RAX-ARGOFFSET(%rsp)
- movq %rcx,RIP-ARGOFFSET(%rsp)
- CFI_REL_OFFSET rip,RIP-ARGOFFSET
- movq %rbp,RCX-ARGOFFSET(%rsp) /* this lies slightly to ptrace */
+
+ /* Zero-extending 32-bit regs, do not remove */
+ movl %eax,%eax
+
+ /* Construct struct pt_regs on stack */
+ pushq_cfi $__USER32_DS /* pt_regs->ss */
+ pushq_cfi %r8 /* pt_regs->sp */
+ CFI_REL_OFFSET rsp,0
+ pushq_cfi %r11 /* pt_regs->flags */
+ pushq_cfi $__USER32_CS /* pt_regs->cs */
+ pushq_cfi %rcx /* pt_regs->ip */
+ CFI_REL_OFFSET rip,0
+ pushq_cfi_reg rax /* pt_regs->orig_ax */
+ pushq_cfi_reg rdi /* pt_regs->di */
+ pushq_cfi_reg rsi /* pt_regs->si */
+ pushq_cfi_reg rdx /* pt_regs->dx */
+ pushq_cfi_reg rbp /* pt_regs->cx */
movl %ebp,%ecx
- movq $__USER32_CS,CS-ARGOFFSET(%rsp)
- movq $__USER32_DS,SS-ARGOFFSET(%rsp)
- movq %r11,EFLAGS-ARGOFFSET(%rsp)
- /*CFI_REL_OFFSET rflags,EFLAGS-ARGOFFSET*/
- movq %r8,RSP-ARGOFFSET(%rsp)
- CFI_REL_OFFSET rsp,RSP-ARGOFFSET
- /* no need to do an access_ok check here because r8 has been
- 32bit zero extended */
- /* hardware stack frame is complete now */
+ pushq_cfi_reg rax /* pt_regs->ax */
+ sub $(10*8),%rsp /* pt_regs->r8-11,bp,bx,r12-15 not saved */
+ CFI_ADJUST_CFA_OFFSET 10*8
+
+ /*
+ * no need to do an access_ok check here because r8 has been
+ * 32bit zero extended
+ */
ASM_STAC
1: movl (%r8),%r9d
_ASM_EXTABLE(1b,ia32_badarg)
ASM_CLAC
- orl $TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ orl $TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
+ testl $_TIF_WORK_SYSCALL_ENTRY, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
CFI_REMEMBER_STATE
jnz cstar_tracesys
cmpq $IA32_NR_syscalls-1,%rax
ja ia32_badsys
cstar_do_call:
- IA32_ARG_FIXUP 1
+ /* 32bit syscall -> 64bit C ABI argument conversion */
+ movl %edi,%r8d /* arg5 */
+ /* r9 already loaded */ /* arg6 */
+ xchg %ecx,%esi /* rsi:arg2, rcx:arg4 */
+ movl %ebx,%edi /* arg1 */
+ movl %edx,%edx /* arg3 (zero extension) */
cstar_dispatch:
call *ia32_sys_call_table(,%rax,8)
- movq %rax,RAX-ARGOFFSET(%rsp)
+ movq %rax,RAX(%rsp)
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
- testl $_TIF_ALLWORK_MASK,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl $_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jnz sysretl_audit
sysretl_from_sys_call:
- andl $~TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- RESTORE_ARGS 0,-ARG_SKIP,0,0,0
- movl RIP-ARGOFFSET(%rsp),%ecx
+ andl $~TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
+ RESTORE_RSI_RDI_RDX
+ movl RIP(%rsp),%ecx
CFI_REGISTER rip,rcx
- movl EFLAGS-ARGOFFSET(%rsp),%r11d
+ movl EFLAGS(%rsp),%r11d
/*CFI_REGISTER rflags,r11*/
xorq %r10,%r10
xorq %r9,%r9
xorq %r8,%r8
TRACE_IRQS_ON
- movl RSP-ARGOFFSET(%rsp),%esp
+ movl RSP(%rsp),%esp
CFI_RESTORE rsp
+ /*
+ * 64bit->32bit SYSRET restores eip from ecx,
+ * eflags from r11 (but RF and VM bits are forced to 0),
+ * cs and ss are loaded from MSRs.
+ * (Note: 32bit->32bit SYSRET is different: since r11
+ * does not exist, it merely sets eflags.IF=1).
+ */
USERGS_SYSRET32
-
+
#ifdef CONFIG_AUDITSYSCALL
cstar_auditsys:
CFI_RESTORE_STATE
- movl %r9d,R9-ARGOFFSET(%rsp) /* register to be clobbered by call */
+ movl %r9d,R9(%rsp) /* register to be clobbered by call */
auditsys_entry_common
- movl R9-ARGOFFSET(%rsp),%r9d /* reload 6th syscall arg */
+ movl R9(%rsp),%r9d /* reload 6th syscall arg */
jmp cstar_dispatch
sysretl_audit:
cstar_tracesys:
#ifdef CONFIG_AUDITSYSCALL
- testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT),TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ testl $(_TIF_WORK_SYSCALL_ENTRY & ~_TIF_SYSCALL_AUDIT), ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jz cstar_auditsys
#endif
xchgl %r9d,%ebp
- SAVE_REST
- CLEAR_RREGS 0, r9
+ SAVE_EXTRA_REGS
+ CLEAR_RREGS r9
movq $-ENOSYS,RAX(%rsp) /* ptrace can change this for a bad syscall */
movq %rsp,%rdi /* &pt_regs -> arg1 */
call syscall_trace_enter
- LOAD_ARGS32 ARGOFFSET, 1 /* reload args from stack in case ptrace changed it */
- RESTORE_REST
+ LOAD_ARGS32 1 /* reload args from stack in case ptrace changed it */
+ RESTORE_EXTRA_REGS
xchgl %ebp,%r9d
cmpq $(IA32_NR_syscalls-1),%rax
ja int_ret_from_sys_call /* cstar_tracesys has set RAX(%rsp) */
jmp ia32_sysret
CFI_ENDPROC
-/*
- * Emulated IA32 system calls via int 0x80.
+/*
+ * Emulated IA32 system calls via int 0x80.
*
- * Arguments:
- * %eax System call number.
- * %ebx Arg1
- * %ecx Arg2
- * %edx Arg3
- * %esi Arg4
- * %edi Arg5
- * %ebp Arg6 [note: not saved in the stack frame, should not be touched]
+ * Arguments:
+ * eax system call number
+ * ebx arg1
+ * ecx arg2
+ * edx arg3
+ * esi arg4
+ * edi arg5
+ * ebp arg6 (note: not saved in the stack frame, should not be touched)
*
* Notes:
- * Uses the same stack frame as the x86-64 version.
- * All registers except %eax must be saved (but ptrace may violate that)
+ * Uses the same stack frame as the x86-64 version.
+ * All registers except eax must be saved (but ptrace may violate that).
* Arguments are zero extended. For system calls that want sign extension and
* take long arguments a wrapper is needed. Most calls can just be called
* directly.
- * Assumes it is only called from user space and entered with interrupts off.
- */
+ * Assumes it is only called from user space and entered with interrupts off.
+ */
ENTRY(ia32_syscall)
CFI_STARTPROC32 simple
CFI_SIGNAL_FRAME
- CFI_DEF_CFA rsp,SS+8-RIP
- /*CFI_REL_OFFSET ss,SS-RIP*/
- CFI_REL_OFFSET rsp,RSP-RIP
- /*CFI_REL_OFFSET rflags,EFLAGS-RIP*/
- /*CFI_REL_OFFSET cs,CS-RIP*/
- CFI_REL_OFFSET rip,RIP-RIP
- PARAVIRT_ADJUST_EXCEPTION_FRAME
- SWAPGS
+ CFI_DEF_CFA rsp,5*8
+ /*CFI_REL_OFFSET ss,4*8 */
+ CFI_REL_OFFSET rsp,3*8
+ /*CFI_REL_OFFSET rflags,2*8 */
+ /*CFI_REL_OFFSET cs,1*8 */
+ CFI_REL_OFFSET rip,0*8
+
/*
- * No need to follow this irqs on/off section: the syscall
- * disabled irqs and here we enable it straight after entry:
+ * Interrupts are off on entry.
+ * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON,
+ * it is too small to ever cause noticeable irq latency.
*/
+ PARAVIRT_ADJUST_EXCEPTION_FRAME
+ SWAPGS
ENABLE_INTERRUPTS(CLBR_NONE)
- movl %eax,%eax
- pushq_cfi %rax
+
+ /* Zero-extending 32-bit regs, do not remove */
+ movl %eax,%eax
+
+ /* Construct struct pt_regs on stack (iret frame is already on stack) */
+ pushq_cfi_reg rax /* pt_regs->orig_ax */
+ pushq_cfi_reg rdi /* pt_regs->di */
+ pushq_cfi_reg rsi /* pt_regs->si */
+ pushq_cfi_reg rdx /* pt_regs->dx */
+ pushq_cfi_reg rcx /* pt_regs->cx */
+ pushq_cfi_reg rax /* pt_regs->ax */
cld
- /* note the registers are not zero extended to the sf.
- this could be a problem. */
- SAVE_ARGS 0,1,0
- orl $TS_COMPAT,TI_status+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ sub $(10*8),%rsp /* pt_regs->r8-11,bp,bx,r12-15 not saved */
+ CFI_ADJUST_CFA_OFFSET 10*8
+
+ orl $TS_COMPAT, ASM_THREAD_INFO(TI_status, %rsp, SIZEOF_PTREGS)
+ testl $_TIF_WORK_SYSCALL_ENTRY, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jnz ia32_tracesys
cmpq $(IA32_NR_syscalls-1),%rax
ja ia32_badsys
ia32_do_call:
- IA32_ARG_FIXUP
+ /* 32bit syscall -> 64bit C ABI argument conversion */
+ movl %edi,%r8d /* arg5 */
+ movl %ebp,%r9d /* arg6 */
+ xchg %ecx,%esi /* rsi:arg2, rcx:arg4 */
+ movl %ebx,%edi /* arg1 */
+ movl %edx,%edx /* arg3 (zero extension) */
call *ia32_sys_call_table(,%rax,8) # xxx: rip relative
ia32_sysret:
- movq %rax,RAX-ARGOFFSET(%rsp)
+ movq %rax,RAX(%rsp)
ia32_ret_from_sys_call:
- CLEAR_RREGS -ARGOFFSET
- jmp int_ret_from_sys_call
+ CLEAR_RREGS
+ jmp int_ret_from_sys_call
-ia32_tracesys:
- SAVE_REST
+ia32_tracesys:
+ SAVE_EXTRA_REGS
CLEAR_RREGS
movq $-ENOSYS,RAX(%rsp) /* ptrace can change this for a bad syscall */
movq %rsp,%rdi /* &pt_regs -> arg1 */
call syscall_trace_enter
- LOAD_ARGS32 ARGOFFSET /* reload args from stack in case ptrace changed it */
- RESTORE_REST
+ LOAD_ARGS32 /* reload args from stack in case ptrace changed it */
+ RESTORE_EXTRA_REGS
cmpq $(IA32_NR_syscalls-1),%rax
ja int_ret_from_sys_call /* ia32_tracesys has set RAX(%rsp) */
jmp ia32_do_call
END(ia32_syscall)
ia32_badsys:
- movq $0,ORIG_RAX-ARGOFFSET(%rsp)
+ movq $0,ORIG_RAX(%rsp)
movq $-ENOSYS,%rax
jmp ia32_sysret
PTREGSCALL stub32_rt_sigreturn, sys32_rt_sigreturn
PTREGSCALL stub32_sigreturn, sys32_sigreturn
- PTREGSCALL stub32_execve, compat_sys_execve
- PTREGSCALL stub32_execveat, compat_sys_execveat
PTREGSCALL stub32_fork, sys_fork
PTREGSCALL stub32_vfork, sys_vfork
ALIGN
ia32_ptregs_common:
- popq %r11
CFI_ENDPROC
CFI_STARTPROC32 simple
CFI_SIGNAL_FRAME
- CFI_DEF_CFA rsp,SS+8-ARGOFFSET
- CFI_REL_OFFSET rax,RAX-ARGOFFSET
- CFI_REL_OFFSET rcx,RCX-ARGOFFSET
- CFI_REL_OFFSET rdx,RDX-ARGOFFSET
- CFI_REL_OFFSET rsi,RSI-ARGOFFSET
- CFI_REL_OFFSET rdi,RDI-ARGOFFSET
- CFI_REL_OFFSET rip,RIP-ARGOFFSET
-/* CFI_REL_OFFSET cs,CS-ARGOFFSET*/
-/* CFI_REL_OFFSET rflags,EFLAGS-ARGOFFSET*/
- CFI_REL_OFFSET rsp,RSP-ARGOFFSET
-/* CFI_REL_OFFSET ss,SS-ARGOFFSET*/
- SAVE_REST
+ CFI_DEF_CFA rsp,SIZEOF_PTREGS
+ CFI_REL_OFFSET rax,RAX
+ CFI_REL_OFFSET rcx,RCX
+ CFI_REL_OFFSET rdx,RDX
+ CFI_REL_OFFSET rsi,RSI
+ CFI_REL_OFFSET rdi,RDI
+ CFI_REL_OFFSET rip,RIP
+/* CFI_REL_OFFSET cs,CS*/
+/* CFI_REL_OFFSET rflags,EFLAGS*/
+ CFI_REL_OFFSET rsp,RSP
+/* CFI_REL_OFFSET ss,SS*/
+ SAVE_EXTRA_REGS 8
call *%rax
- RESTORE_REST
- jmp ia32_sysret /* misbalances the return cache */
+ RESTORE_EXTRA_REGS 8
+ ret
CFI_ENDPROC
END(ia32_ptregs_common)
+++ /dev/null
-#include <linux/kernel.h>
-#include <linux/errno.h>
-
-long compat_ni_syscall(void)
-{
- return -ENOSYS;
-}
advice);
}
-long sys32_vm86_warning(void)
-{
- struct task_struct *me = current;
- static char lastcomm[sizeof(me->comm)];
-
- if (strncmp(lastcomm, me->comm, sizeof(lastcomm))) {
- compat_printk(KERN_INFO
- "%s: vm86 mode not supported on 64 bit kernel\n",
- me->comm);
- strncpy(lastcomm, me->comm, sizeof(lastcomm));
- }
- return -ENOSYS;
-}
-
asmlinkage ssize_t sys32_readahead(int fd, unsigned off_lo, unsigned off_hi,
size_t count)
{
+++ /dev/null
-/* System call table for ia32 emulation. */
-
-#include <linux/linkage.h>
-#include <linux/sys.h>
-#include <linux/cache.h>
-#include <asm/asm-offsets.h>
-
-#define __SYSCALL_I386(nr, sym, compat) extern asmlinkage void compat(void) ;
-#include <asm/syscalls_32.h>
-#undef __SYSCALL_I386
-
-#define __SYSCALL_I386(nr, sym, compat) [nr] = compat,
-
-typedef void (*sys_call_ptr_t)(void);
-
-extern void compat_ni_syscall(void);
-
-const sys_call_ptr_t ia32_sys_call_table[__NR_ia32_syscall_max+1] = {
- /*
- * Smells like a compiler bug -- it doesn't work
- * when the & below is removed.
- */
- [0 ... __NR_ia32_syscall_max] = &compat_ni_syscall,
-#include <asm/syscalls_32.h>
-};
.endm
#endif
-.macro altinstruction_entry orig alt feature orig_len alt_len
+.macro altinstruction_entry orig alt feature orig_len alt_len pad_len
.long \orig - .
.long \alt - .
.word \feature
.byte \orig_len
.byte \alt_len
+ .byte \pad_len
+.endm
+
+.macro ALTERNATIVE oldinstr, newinstr, feature
+140:
+ \oldinstr
+141:
+ .skip -(((144f-143f)-(141b-140b)) > 0) * ((144f-143f)-(141b-140b)),0x90
+142:
+
+ .pushsection .altinstructions,"a"
+ altinstruction_entry 140b,143f,\feature,142b-140b,144f-143f,142b-141b
+ .popsection
+
+ .pushsection .altinstr_replacement,"ax"
+143:
+ \newinstr
+144:
+ .popsection
+.endm
+
+#define old_len 141b-140b
+#define new_len1 144f-143f
+#define new_len2 145f-144f
+
+/*
+ * max without conditionals. Idea adapted from:
+ * http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+ */
+#define alt_max_short(a, b) ((a) ^ (((a) ^ (b)) & -(-((a) < (b)))))
+
+.macro ALTERNATIVE_2 oldinstr, newinstr1, feature1, newinstr2, feature2
+140:
+ \oldinstr
+141:
+ .skip -((alt_max_short(new_len1, new_len2) - (old_len)) > 0) * \
+ (alt_max_short(new_len1, new_len2) - (old_len)),0x90
+142:
+
+ .pushsection .altinstructions,"a"
+ altinstruction_entry 140b,143f,\feature1,142b-140b,144f-143f,142b-141b
+ altinstruction_entry 140b,144f,\feature2,142b-140b,145f-144f,142b-141b
+ .popsection
+
+ .pushsection .altinstr_replacement,"ax"
+143:
+ \newinstr1
+144:
+ \newinstr2
+145:
+ .popsection
.endm
#endif /* __ASSEMBLY__ */
s32 repl_offset; /* offset to replacement instruction */
u16 cpuid; /* cpuid bit set for replacement */
u8 instrlen; /* length of original instruction */
- u8 replacementlen; /* length of new instruction, <= instrlen */
-};
+ u8 replacementlen; /* length of new instruction */
+ u8 padlen; /* length of build-time padding */
+} __packed;
extern void alternative_instructions(void);
extern void apply_alternatives(struct alt_instr *start, struct alt_instr *end);
}
#endif /* CONFIG_SMP */
-#define OLDINSTR(oldinstr) "661:\n\t" oldinstr "\n662:\n"
+#define b_replacement(num) "664"#num
+#define e_replacement(num) "665"#num
-#define b_replacement(number) "663"#number
-#define e_replacement(number) "664"#number
+#define alt_end_marker "663"
+#define alt_slen "662b-661b"
+#define alt_pad_len alt_end_marker"b-662b"
+#define alt_total_slen alt_end_marker"b-661b"
+#define alt_rlen(num) e_replacement(num)"f-"b_replacement(num)"f"
-#define alt_slen "662b-661b"
-#define alt_rlen(number) e_replacement(number)"f-"b_replacement(number)"f"
+#define __OLDINSTR(oldinstr, num) \
+ "661:\n\t" oldinstr "\n662:\n" \
+ ".skip -(((" alt_rlen(num) ")-(" alt_slen ")) > 0) * " \
+ "((" alt_rlen(num) ")-(" alt_slen ")),0x90\n"
-#define ALTINSTR_ENTRY(feature, number) \
+#define OLDINSTR(oldinstr, num) \
+ __OLDINSTR(oldinstr, num) \
+ alt_end_marker ":\n"
+
+/*
+ * max without conditionals. Idea adapted from:
+ * http://graphics.stanford.edu/~seander/bithacks.html#IntegerMinOrMax
+ *
+ * The additional "-" is needed because gas works with s32s.
+ */
+#define alt_max_short(a, b) "((" a ") ^ (((" a ") ^ (" b ")) & -(-((" a ") - (" b ")))))"
+
+/*
+ * Pad the second replacement alternative with additional NOPs if it is
+ * additionally longer than the first replacement alternative.
+ */
+#define OLDINSTR_2(oldinstr, num1, num2) \
+ "661:\n\t" oldinstr "\n662:\n" \
+ ".skip -((" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")) > 0) * " \
+ "(" alt_max_short(alt_rlen(num1), alt_rlen(num2)) " - (" alt_slen ")), 0x90\n" \
+ alt_end_marker ":\n"
+
+#define ALTINSTR_ENTRY(feature, num) \
" .long 661b - .\n" /* label */ \
- " .long " b_replacement(number)"f - .\n" /* new instruction */ \
+ " .long " b_replacement(num)"f - .\n" /* new instruction */ \
" .word " __stringify(feature) "\n" /* feature bit */ \
- " .byte " alt_slen "\n" /* source len */ \
- " .byte " alt_rlen(number) "\n" /* replacement len */
-
-#define DISCARD_ENTRY(number) /* rlen <= slen */ \
- " .byte 0xff + (" alt_rlen(number) ") - (" alt_slen ")\n"
+ " .byte " alt_total_slen "\n" /* source len */ \
+ " .byte " alt_rlen(num) "\n" /* replacement len */ \
+ " .byte " alt_pad_len "\n" /* pad len */
-#define ALTINSTR_REPLACEMENT(newinstr, feature, number) /* replacement */ \
- b_replacement(number)":\n\t" newinstr "\n" e_replacement(number) ":\n\t"
+#define ALTINSTR_REPLACEMENT(newinstr, feature, num) /* replacement */ \
+ b_replacement(num)":\n\t" newinstr "\n" e_replacement(num) ":\n\t"
/* alternative assembly primitive: */
#define ALTERNATIVE(oldinstr, newinstr, feature) \
- OLDINSTR(oldinstr) \
+ OLDINSTR(oldinstr, 1) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(feature, 1) \
".popsection\n" \
- ".pushsection .discard,\"aw\",@progbits\n" \
- DISCARD_ENTRY(1) \
- ".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinstr, feature, 1) \
".popsection"
#define ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2)\
- OLDINSTR(oldinstr) \
+ OLDINSTR_2(oldinstr, 1, 2) \
".pushsection .altinstructions,\"a\"\n" \
ALTINSTR_ENTRY(feature1, 1) \
ALTINSTR_ENTRY(feature2, 2) \
".popsection\n" \
- ".pushsection .discard,\"aw\",@progbits\n" \
- DISCARD_ENTRY(1) \
- DISCARD_ENTRY(2) \
- ".popsection\n" \
".pushsection .altinstr_replacement, \"ax\"\n" \
ALTINSTR_REPLACEMENT(newinstr1, feature1, 1) \
ALTINSTR_REPLACEMENT(newinstr2, feature2, 2) \
#define alternative(oldinstr, newinstr, feature) \
asm volatile (ALTERNATIVE(oldinstr, newinstr, feature) : : : "memory")
+#define alternative_2(oldinstr, newinstr1, feature1, newinstr2, feature2) \
+ asm volatile(ALTERNATIVE_2(oldinstr, newinstr1, feature1, newinstr2, feature2) ::: "memory")
+
/*
* Alternative inline assembly with input.
*
{
volatile u32 *addr = (volatile u32 *)(APIC_BASE + reg);
- alternative_io("movl %0, %1", "xchgl %0, %1", X86_BUG_11AP,
+ alternative_io("movl %0, %P1", "xchgl %0, %P1", X86_BUG_11AP,
ASM_OUTPUT2("=r" (v), "=m" (*addr)),
ASM_OUTPUT2("0" (v), "m" (*addr)));
}
extern void disconnect_bsp_APIC(int virt_wire_setup);
extern void disable_local_APIC(void);
extern void lapic_shutdown(void);
-extern int verify_local_APIC(void);
extern void sync_Arb_IDs(void);
extern void init_bsp_APIC(void);
extern void setup_local_APIC(void);
* Stop RDTSC speculation. This is needed when you need to use RDTSC
* (or get_cycles or vread that possibly accesses the TSC) in a defined
* code region.
- *
- * (Could use an alternative three way for this if there was one.)
*/
static __always_inline void rdtsc_barrier(void)
{
- alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
- alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
+ alternative_2("", "mfence", X86_FEATURE_MFENCE_RDTSC,
+ "lfence", X86_FEATURE_LFENCE_RDTSC);
}
#endif /* _ASM_X86_BARRIER_H */
* for assembly code:
*/
-#define R15 0
-#define R14 8
-#define R13 16
-#define R12 24
-#define RBP 32
-#define RBX 40
-
-/* arguments: interrupts/non tracing syscalls only save up to here: */
-#define R11 48
-#define R10 56
-#define R9 64
-#define R8 72
-#define RAX 80
-#define RCX 88
-#define RDX 96
-#define RSI 104
-#define RDI 112
-#define ORIG_RAX 120 /* + error_code */
-/* end of arguments */
-
-/* cpu exception frame or undefined in case of fast syscall: */
-#define RIP 128
-#define CS 136
-#define EFLAGS 144
-#define RSP 152
-#define SS 160
-
-#define ARGOFFSET R11
-
- .macro SAVE_ARGS addskip=0, save_rcx=1, save_r891011=1, rax_enosys=0
- subq $9*8+\addskip, %rsp
- CFI_ADJUST_CFA_OFFSET 9*8+\addskip
- movq_cfi rdi, 8*8
- movq_cfi rsi, 7*8
- movq_cfi rdx, 6*8
-
- .if \save_rcx
- movq_cfi rcx, 5*8
- .endif
+/* The layout forms the "struct pt_regs" on the stack: */
+/*
+ * C ABI says these regs are callee-preserved. They aren't saved on kernel entry
+ * unless syscall needs a complete, fully filled "struct pt_regs".
+ */
+#define R15 0*8
+#define R14 1*8
+#define R13 2*8
+#define R12 3*8
+#define RBP 4*8
+#define RBX 5*8
+/* These regs are callee-clobbered. Always saved on kernel entry. */
+#define R11 6*8
+#define R10 7*8
+#define R9 8*8
+#define R8 9*8
+#define RAX 10*8
+#define RCX 11*8
+#define RDX 12*8
+#define RSI 13*8
+#define RDI 14*8
+/*
+ * On syscall entry, this is syscall#. On CPU exception, this is error code.
+ * On hw interrupt, it's IRQ number:
+ */
+#define ORIG_RAX 15*8
+/* Return frame for iretq */
+#define RIP 16*8
+#define CS 17*8
+#define EFLAGS 18*8
+#define RSP 19*8
+#define SS 20*8
+
+#define SIZEOF_PTREGS 21*8
+
+ .macro ALLOC_PT_GPREGS_ON_STACK addskip=0
+ subq $15*8+\addskip, %rsp
+ CFI_ADJUST_CFA_OFFSET 15*8+\addskip
+ .endm
- .if \rax_enosys
- movq $-ENOSYS, 4*8(%rsp)
- .else
- movq_cfi rax, 4*8
+ .macro SAVE_C_REGS_HELPER offset=0 rax=1 rcx=1 r8910=1 r11=1
+ .if \r11
+ movq_cfi r11, 6*8+\offset
.endif
-
- .if \save_r891011
- movq_cfi r8, 3*8
- movq_cfi r9, 2*8
- movq_cfi r10, 1*8
- movq_cfi r11, 0*8
+ .if \r8910
+ movq_cfi r10, 7*8+\offset
+ movq_cfi r9, 8*8+\offset
+ movq_cfi r8, 9*8+\offset
+ .endif
+ .if \rax
+ movq_cfi rax, 10*8+\offset
+ .endif
+ .if \rcx
+ movq_cfi rcx, 11*8+\offset
.endif
+ movq_cfi rdx, 12*8+\offset
+ movq_cfi rsi, 13*8+\offset
+ movq_cfi rdi, 14*8+\offset
+ .endm
+ .macro SAVE_C_REGS offset=0
+ SAVE_C_REGS_HELPER \offset, 1, 1, 1, 1
+ .endm
+ .macro SAVE_C_REGS_EXCEPT_RAX_RCX offset=0
+ SAVE_C_REGS_HELPER \offset, 0, 0, 1, 1
+ .endm
+ .macro SAVE_C_REGS_EXCEPT_R891011
+ SAVE_C_REGS_HELPER 0, 1, 1, 0, 0
+ .endm
+ .macro SAVE_C_REGS_EXCEPT_RCX_R891011
+ SAVE_C_REGS_HELPER 0, 1, 0, 0, 0
+ .endm
+ .macro SAVE_C_REGS_EXCEPT_RAX_RCX_R11
+ SAVE_C_REGS_HELPER 0, 0, 0, 1, 0
+ .endm
+
+ .macro SAVE_EXTRA_REGS offset=0
+ movq_cfi r15, 0*8+\offset
+ movq_cfi r14, 1*8+\offset
+ movq_cfi r13, 2*8+\offset
+ movq_cfi r12, 3*8+\offset
+ movq_cfi rbp, 4*8+\offset
+ movq_cfi rbx, 5*8+\offset
+ .endm
+ .macro SAVE_EXTRA_REGS_RBP offset=0
+ movq_cfi rbp, 4*8+\offset
+ .endm
+ .macro RESTORE_EXTRA_REGS offset=0
+ movq_cfi_restore 0*8+\offset, r15
+ movq_cfi_restore 1*8+\offset, r14
+ movq_cfi_restore 2*8+\offset, r13
+ movq_cfi_restore 3*8+\offset, r12
+ movq_cfi_restore 4*8+\offset, rbp
+ movq_cfi_restore 5*8+\offset, rbx
.endm
-#define ARG_SKIP (9*8)
+ .macro ZERO_EXTRA_REGS
+ xorl %r15d, %r15d
+ xorl %r14d, %r14d
+ xorl %r13d, %r13d
+ xorl %r12d, %r12d
+ xorl %ebp, %ebp
+ xorl %ebx, %ebx
+ .endm
- .macro RESTORE_ARGS rstor_rax=1, addskip=0, rstor_rcx=1, rstor_r11=1, \
- rstor_r8910=1, rstor_rdx=1
+ .macro RESTORE_C_REGS_HELPER rstor_rax=1, rstor_rcx=1, rstor_r11=1, rstor_r8910=1, rstor_rdx=1
.if \rstor_r11
- movq_cfi_restore 0*8, r11
+ movq_cfi_restore 6*8, r11
.endif
-
.if \rstor_r8910
- movq_cfi_restore 1*8, r10
- movq_cfi_restore 2*8, r9
- movq_cfi_restore 3*8, r8
+ movq_cfi_restore 7*8, r10
+ movq_cfi_restore 8*8, r9
+ movq_cfi_restore 9*8, r8
.endif
-
.if \rstor_rax
- movq_cfi_restore 4*8, rax
+ movq_cfi_restore 10*8, rax
.endif
-
.if \rstor_rcx
- movq_cfi_restore 5*8, rcx
+ movq_cfi_restore 11*8, rcx
.endif
-
.if \rstor_rdx
- movq_cfi_restore 6*8, rdx
- .endif
-
- movq_cfi_restore 7*8, rsi
- movq_cfi_restore 8*8, rdi
-
- .if ARG_SKIP+\addskip > 0
- addq $ARG_SKIP+\addskip, %rsp
- CFI_ADJUST_CFA_OFFSET -(ARG_SKIP+\addskip)
+ movq_cfi_restore 12*8, rdx
.endif
+ movq_cfi_restore 13*8, rsi
+ movq_cfi_restore 14*8, rdi
.endm
-
- .macro LOAD_ARGS offset, skiprax=0
- movq \offset(%rsp), %r11
- movq \offset+8(%rsp), %r10
- movq \offset+16(%rsp), %r9
- movq \offset+24(%rsp), %r8
- movq \offset+40(%rsp), %rcx
- movq \offset+48(%rsp), %rdx
- movq \offset+56(%rsp), %rsi
- movq \offset+64(%rsp), %rdi
- .if \skiprax
- .else
- movq \offset+72(%rsp), %rax
- .endif
+ .macro RESTORE_C_REGS
+ RESTORE_C_REGS_HELPER 1,1,1,1,1
.endm
-
-#define REST_SKIP (6*8)
-
- .macro SAVE_REST
- subq $REST_SKIP, %rsp
- CFI_ADJUST_CFA_OFFSET REST_SKIP
- movq_cfi rbx, 5*8
- movq_cfi rbp, 4*8
- movq_cfi r12, 3*8
- movq_cfi r13, 2*8
- movq_cfi r14, 1*8
- movq_cfi r15, 0*8
+ .macro RESTORE_C_REGS_EXCEPT_RAX
+ RESTORE_C_REGS_HELPER 0,1,1,1,1
.endm
-
- .macro RESTORE_REST
- movq_cfi_restore 0*8, r15
- movq_cfi_restore 1*8, r14
- movq_cfi_restore 2*8, r13
- movq_cfi_restore 3*8, r12
- movq_cfi_restore 4*8, rbp
- movq_cfi_restore 5*8, rbx
- addq $REST_SKIP, %rsp
- CFI_ADJUST_CFA_OFFSET -(REST_SKIP)
+ .macro RESTORE_C_REGS_EXCEPT_RCX
+ RESTORE_C_REGS_HELPER 1,0,1,1,1
.endm
-
- .macro SAVE_ALL
- SAVE_ARGS
- SAVE_REST
+ .macro RESTORE_C_REGS_EXCEPT_R11
+ RESTORE_C_REGS_HELPER 1,1,0,1,1
+ .endm
+ .macro RESTORE_C_REGS_EXCEPT_RCX_R11
+ RESTORE_C_REGS_HELPER 1,0,0,1,1
+ .endm
+ .macro RESTORE_RSI_RDI
+ RESTORE_C_REGS_HELPER 0,0,0,0,0
+ .endm
+ .macro RESTORE_RSI_RDI_RDX
+ RESTORE_C_REGS_HELPER 0,0,0,0,1
.endm
- .macro RESTORE_ALL addskip=0
- RESTORE_REST
- RESTORE_ARGS 1, \addskip
+ .macro REMOVE_PT_GPREGS_FROM_STACK addskip=0
+ addq $15*8+\addskip, %rsp
+ CFI_ADJUST_CFA_OFFSET -(15*8+\addskip)
.endm
.macro icebp
*/
.macro SAVE_ALL
- pushl_cfi %eax
- CFI_REL_OFFSET eax, 0
- pushl_cfi %ebp
- CFI_REL_OFFSET ebp, 0
- pushl_cfi %edi
- CFI_REL_OFFSET edi, 0
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
- pushl_cfi %edx
- CFI_REL_OFFSET edx, 0
- pushl_cfi %ecx
- CFI_REL_OFFSET ecx, 0
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
+ pushl_cfi_reg eax
+ pushl_cfi_reg ebp
+ pushl_cfi_reg edi
+ pushl_cfi_reg esi
+ pushl_cfi_reg edx
+ pushl_cfi_reg ecx
+ pushl_cfi_reg ebx
.endm
.macro RESTORE_ALL
- popl_cfi %ebx
- CFI_RESTORE ebx
- popl_cfi %ecx
- CFI_RESTORE ecx
- popl_cfi %edx
- CFI_RESTORE edx
- popl_cfi %esi
- CFI_RESTORE esi
- popl_cfi %edi
- CFI_RESTORE edi
- popl_cfi %ebp
- CFI_RESTORE ebp
- popl_cfi %eax
- CFI_RESTORE eax
+ popl_cfi_reg ebx
+ popl_cfi_reg ecx
+ popl_cfi_reg edx
+ popl_cfi_reg esi
+ popl_cfi_reg edi
+ popl_cfi_reg ebp
+ popl_cfi_reg eax
.endm
#endif /* CONFIG_X86_64 */
sp = task_pt_regs(current)->sp;
} else {
/* -128 for the x32 ABI redzone */
- sp = this_cpu_read(old_rsp) - 128;
+ sp = task_pt_regs(current)->sp - 128;
}
return (void __user *)round_down(sp - len, 16);
#define X86_FEATURE_RDSEED ( 9*32+18) /* The RDSEED instruction */
#define X86_FEATURE_ADX ( 9*32+19) /* The ADCX and ADOX instructions */
#define X86_FEATURE_SMAP ( 9*32+20) /* Supervisor Mode Access Prevention */
+#define X86_FEATURE_PCOMMIT ( 9*32+22) /* PCOMMIT instruction */
#define X86_FEATURE_CLFLUSHOPT ( 9*32+23) /* CLFLUSHOPT instruction */
+#define X86_FEATURE_CLWB ( 9*32+24) /* CLWB instruction */
#define X86_FEATURE_AVX512PF ( 9*32+26) /* AVX-512 Prefetch */
#define X86_FEATURE_AVX512ER ( 9*32+27) /* AVX-512 Exponential and Reciprocal */
#define X86_FEATURE_AVX512CD ( 9*32+28) /* AVX-512 Conflict Detection */
" .word %P0\n" /* 1: do replace */
" .byte 2b - 1b\n" /* source len */
" .byte 0\n" /* replacement len */
+ " .byte 0\n" /* pad len */
".previous\n"
/* skipping size check since replacement size = 0 */
: : "i" (X86_FEATURE_ALWAYS) : : t_warn);
" .word %P0\n" /* feature bit */
" .byte 2b - 1b\n" /* source len */
" .byte 0\n" /* replacement len */
+ " .byte 0\n" /* pad len */
".previous\n"
/* skipping size check since replacement size = 0 */
: : "i" (bit) : : t_no);
" .word %P1\n" /* feature bit */
" .byte 2b - 1b\n" /* source len */
" .byte 4f - 3f\n" /* replacement len */
+ " .byte 0\n" /* pad len */
".previous\n"
".section .discard,\"aw\",@progbits\n"
" .byte 0xff + (4f-3f) - (2b-1b)\n" /* size check */
static __always_inline __pure bool _static_cpu_has_safe(u16 bit)
{
#ifdef CC_HAVE_ASM_GOTO
-/*
- * We need to spell the jumps to the compiler because, depending on the offset,
- * the replacement jump can be bigger than the original jump, and this we cannot
- * have. Thus, we force the jump to the widest, 4-byte, signed relative
- * offset even though the last would often fit in less bytes.
- */
- asm_volatile_goto("1: .byte 0xe9\n .long %l[t_dynamic] - 2f\n"
+ asm_volatile_goto("1: jmp %l[t_dynamic]\n"
"2:\n"
+ ".skip -(((5f-4f) - (2b-1b)) > 0) * "
+ "((5f-4f) - (2b-1b)),0x90\n"
+ "3:\n"
".section .altinstructions,\"a\"\n"
" .long 1b - .\n" /* src offset */
- " .long 3f - .\n" /* repl offset */
+ " .long 4f - .\n" /* repl offset */
" .word %P1\n" /* always replace */
- " .byte 2b - 1b\n" /* src len */
- " .byte 4f - 3f\n" /* repl len */
+ " .byte 3b - 1b\n" /* src len */
+ " .byte 5f - 4f\n" /* repl len */
+ " .byte 3b - 2b\n" /* pad len */
".previous\n"
".section .altinstr_replacement,\"ax\"\n"
- "3: .byte 0xe9\n .long %l[t_no] - 2b\n"
- "4:\n"
+ "4: jmp %l[t_no]\n"
+ "5:\n"
".previous\n"
".section .altinstructions,\"a\"\n"
" .long 1b - .\n" /* src offset */
" .long 0\n" /* no replacement */
" .word %P0\n" /* feature bit */
- " .byte 2b - 1b\n" /* src len */
+ " .byte 3b - 1b\n" /* src len */
" .byte 0\n" /* repl len */
+ " .byte 0\n" /* pad len */
".previous\n"
: : "i" (bit), "i" (X86_FEATURE_ALWAYS)
: : t_dynamic, t_no);
" .word %P2\n" /* always replace */
" .byte 2b - 1b\n" /* source len */
" .byte 4f - 3f\n" /* replacement len */
+ " .byte 0\n" /* pad len */
".previous\n"
".section .discard,\"aw\",@progbits\n"
" .byte 0xff + (4f-3f) - (2b-1b)\n" /* size check */
" .word %P1\n" /* feature bit */
" .byte 4b - 3b\n" /* src len */
" .byte 6f - 5f\n" /* repl len */
+ " .byte 0\n" /* pad len */
".previous\n"
".section .discard,\"aw\",@progbits\n"
" .byte 0xff + (6f-5f) - (4b-3b)\n" /* size check */
* Pentium F0 0F bugfix can have resulted in the mapped
* IDT being write-protected.
*/
-#define set_intr_gate(n, addr) \
+#define set_intr_gate_notrace(n, addr) \
do { \
BUG_ON((unsigned)n > 0xFF); \
_set_gate(n, GATE_INTERRUPT, (void *)addr, 0, 0, \
__KERNEL_CS); \
+ } while (0)
+
+#define set_intr_gate(n, addr) \
+ do { \
+ set_intr_gate_notrace(n, addr); \
_trace_set_gate(n, GATE_INTERRUPT, (void *)trace_##addr,\
0, 0, __KERNEL_CS); \
} while (0)
CFI_ADJUST_CFA_OFFSET 8
.endm
+ .macro pushq_cfi_reg reg
+ pushq %\reg
+ CFI_ADJUST_CFA_OFFSET 8
+ CFI_REL_OFFSET \reg, 0
+ .endm
+
.macro popq_cfi reg
popq \reg
CFI_ADJUST_CFA_OFFSET -8
.endm
+ .macro popq_cfi_reg reg
+ popq %\reg
+ CFI_ADJUST_CFA_OFFSET -8
+ CFI_RESTORE \reg
+ .endm
+
.macro pushfq_cfi
pushfq
CFI_ADJUST_CFA_OFFSET 8
CFI_ADJUST_CFA_OFFSET 4
.endm
+ .macro pushl_cfi_reg reg
+ pushl %\reg
+ CFI_ADJUST_CFA_OFFSET 4
+ CFI_REL_OFFSET \reg, 0
+ .endm
+
.macro popl_cfi reg
popl \reg
CFI_ADJUST_CFA_OFFSET -4
.endm
+ .macro popl_cfi_reg reg
+ popl %\reg
+ CFI_ADJUST_CFA_OFFSET -4
+ CFI_RESTORE \reg
+ .endm
+
.macro pushfl_cfi
pushfl
CFI_ADJUST_CFA_OFFSET 4
#define _ASM_X86_EFI_H
#include <asm/i387.h>
+#include <asm/pgtable.h>
+
/*
* We map the EFI regions needed for runtime services non-contiguously,
* with preserved alignment on virtual addresses starting from -4G down
extern struct efi_scratch efi_scratch;
extern void __init efi_set_executable(efi_memory_desc_t *md, bool executable);
extern int __init efi_memblock_x86_reserve_range(void);
-extern void __init efi_call_phys_prolog(void);
-extern void __init efi_call_phys_epilog(void);
+extern pgd_t * __init efi_call_phys_prolog(void);
+extern void __init efi_call_phys_epilog(pgd_t *save_pgd);
extern void __init efi_unmap_memmap(void);
extern void __init efi_memory_uc(u64 addr, unsigned long size);
extern void __init efi_map_region(efi_memory_desc_t *md);
static inline void elf_common_init(struct thread_struct *t,
struct pt_regs *regs, const u16 ds)
{
- regs->ax = regs->bx = regs->cx = regs->dx = 0;
- regs->si = regs->di = regs->bp = 0;
+ /* Commented-out registers are cleared in stub_execve */
+ /*regs->ax = regs->bx =*/ regs->cx = regs->dx = 0;
+ regs->si = regs->di /*= regs->bp*/ = 0;
regs->r8 = regs->r9 = regs->r10 = regs->r11 = 0;
- regs->r12 = regs->r13 = regs->r14 = regs->r15 = 0;
+ /*regs->r12 = regs->r13 = regs->r14 = regs->r15 = 0;*/
t->fs = t->gs = 0;
t->fsindex = t->gsindex = 0;
t->ds = t->es = ds;
struct va_alignment {
int flags;
unsigned long mask;
+ unsigned long bits;
} ____cacheline_aligned;
extern struct va_alignment va_align;
static inline void finit_soft_fpu(struct i387_soft_struct *soft) {}
#endif
+/*
+ * Must be run with preemption disabled: this clears the fpu_owner_task,
+ * on this CPU.
+ *
+ * This will disable any lazy FPU state restore of the current FPU state,
+ * but if the current thread owns the FPU, it will still be saved by.
+ */
+static inline void __cpu_disable_lazy_restore(unsigned int cpu)
+{
+ per_cpu(fpu_owner_task, cpu) = NULL;
+}
+
+/*
+ * Used to indicate that the FPU state in memory is newer than the FPU
+ * state in registers, and the FPU state should be reloaded next time the
+ * task is run. Only safe on the current task, or non-running tasks.
+ */
+static inline void task_disable_lazy_fpu_restore(struct task_struct *tsk)
+{
+ tsk->thread.fpu.last_cpu = ~0;
+}
+
+static inline int fpu_lazy_restore(struct task_struct *new, unsigned int cpu)
+{
+ return new == this_cpu_read_stable(fpu_owner_task) &&
+ cpu == new->thread.fpu.last_cpu;
+}
+
static inline int is_ia32_compat_frame(void)
{
return config_enabled(CONFIG_IA32_EMULATION) &&
static inline void fx_finit(struct i387_fxsave_struct *fx)
{
- memset(fx, 0, xstate_size);
fx->cwd = 0x37f;
fx->mxcsr = MXCSR_DEFAULT;
}
__thread_set_has_fpu(tsk);
}
-static inline void __drop_fpu(struct task_struct *tsk)
+static inline void drop_fpu(struct task_struct *tsk)
{
+ /*
+ * Forget coprocessor state..
+ */
+ preempt_disable();
+ tsk->thread.fpu_counter = 0;
+
if (__thread_has_fpu(tsk)) {
/* Ignore delayed exceptions from user space */
asm volatile("1: fwait\n"
_ASM_EXTABLE(1b, 2b));
__thread_fpu_end(tsk);
}
+
+ clear_stopped_child_used_math(tsk);
+ preempt_enable();
}
-static inline void drop_fpu(struct task_struct *tsk)
+static inline void restore_init_xstate(void)
{
- /*
- * Forget coprocessor state..
- */
- preempt_disable();
- tsk->thread.fpu_counter = 0;
- __drop_fpu(tsk);
- clear_used_math();
- preempt_enable();
+ if (use_xsave())
+ xrstor_state(init_xstate_buf, -1);
+ else
+ fxrstor_checking(&init_xstate_buf->i387);
}
-static inline void drop_init_fpu(struct task_struct *tsk)
+/*
+ * Reset the FPU state in the eager case and drop it in the lazy case (later use
+ * will reinit it).
+ */
+static inline void fpu_reset_state(struct task_struct *tsk)
{
if (!use_eager_fpu())
drop_fpu(tsk);
- else {
- if (use_xsave())
- xrstor_state(init_xstate_buf, -1);
- else
- fxrstor_checking(&init_xstate_buf->i387);
- }
+ else
+ restore_init_xstate();
}
/*
*/
typedef struct { int preload; } fpu_switch_t;
-/*
- * Must be run with preemption disabled: this clears the fpu_owner_task,
- * on this CPU.
- *
- * This will disable any lazy FPU state restore of the current FPU state,
- * but if the current thread owns the FPU, it will still be saved by.
- */
-static inline void __cpu_disable_lazy_restore(unsigned int cpu)
-{
- per_cpu(fpu_owner_task, cpu) = NULL;
-}
-
-static inline int fpu_lazy_restore(struct task_struct *new, unsigned int cpu)
-{
- return new == this_cpu_read_stable(fpu_owner_task) &&
- cpu == new->thread.fpu.last_cpu;
-}
-
static inline fpu_switch_t switch_fpu_prepare(struct task_struct *old, struct task_struct *new, int cpu)
{
fpu_switch_t fpu;
* If the task has used the math, pre-load the FPU on xsave processors
* or if the past 5 consecutive context-switches used math.
*/
- fpu.preload = tsk_used_math(new) && (use_eager_fpu() ||
- new->thread.fpu_counter > 5);
+ fpu.preload = tsk_used_math(new) &&
+ (use_eager_fpu() || new->thread.fpu_counter > 5);
+
if (__thread_has_fpu(old)) {
if (!__save_init_fpu(old))
- cpu = ~0;
- old->thread.fpu.last_cpu = cpu;
- old->thread.fpu.has_fpu = 0; /* But leave fpu_owner_task! */
+ task_disable_lazy_fpu_restore(old);
+ else
+ old->thread.fpu.last_cpu = cpu;
+
+ /* But leave fpu_owner_task! */
+ old->thread.fpu.has_fpu = 0;
/* Don't change CR0.TS if we just switch! */
if (fpu.preload) {
stts();
} else {
old->thread.fpu_counter = 0;
- old->thread.fpu.last_cpu = ~0;
+ task_disable_lazy_fpu_restore(old);
if (fpu.preload) {
new->thread.fpu_counter++;
- if (!use_eager_fpu() && fpu_lazy_restore(new, cpu))
+ if (fpu_lazy_restore(new, cpu))
fpu.preload = 0;
else
prefetch(new->thread.fpu.state);
{
if (fpu.preload) {
if (unlikely(restore_fpu_checking(new)))
- drop_init_fpu(new);
+ fpu_reset_state(new);
}
}
}
/*
- * Need to be preemption-safe.
+ * Needs to be preemption-safe.
*
* NOTE! user_fpu_begin() must be used only immediately before restoring
- * it. This function does not do any save/restore on their own.
+ * the save state. It does not do any saving/restoring on its own. In
+ * lazy FPU mode, it is just an optimization to avoid a #NM exception,
+ * the task can lose the FPU right after preempt_enable().
*/
static inline void user_fpu_begin(void)
{
fpu_fxsave(&tsk->thread.fpu);
}
-/*
- * These disable preemption on their own and are safe
- */
-static inline void save_init_fpu(struct task_struct *tsk)
-{
- WARN_ON_ONCE(!__thread_has_fpu(tsk));
-
- if (use_eager_fpu()) {
- __save_fpu(tsk);
- return;
- }
-
- preempt_disable();
- __save_init_fpu(tsk);
- __thread_fpu_end(tsk);
- preempt_enable();
-}
-
/*
* i387 state interaction
*/
extern __visible void smp_invalidate_interrupt(struct pt_regs *);
#endif
-extern void (*__initconst interrupt[FIRST_SYSTEM_VECTOR
- - FIRST_EXTERNAL_VECTOR])(void);
+extern char irq_entries_start[];
#ifdef CONFIG_TRACING
-#define trace_interrupt interrupt
+#define trace_irq_entries_start irq_entries_start
#endif
#define VECTOR_UNDEFINED (-1)
const insn_byte_t *next_byte;
};
-#define MAX_INSN_SIZE 16
+#define MAX_INSN_SIZE 15
#define X86_MODRM_MOD(modrm) (((modrm) & 0xc0) >> 6)
#define X86_MODRM_REG(modrm) (((modrm) & 0x38) >> 3)
* d). Similar to the 'init', except that this gets called from pci_iommu_init
* where we do have a memory allocator.
*
- * The standard vs the _FINISH differs in that the _FINISH variant will
- * continue detecting other IOMMUs in the call list after the
- * the detection routine returns a positive number. The _FINISH will
- * stop the execution chain. Both will still call the 'init' and
- * 'late_init' functions if they are set.
+ * The standard IOMMU_INIT differs from the IOMMU_INIT_FINISH variant
+ * in that the former will continue detecting other IOMMUs in the call
+ * list after the detection routine returns a positive number, while the
+ * latter will stop the execution chain upon first successful detection.
+ * Both variants will still call the 'init' and 'late_init' functions if
+ * they are set.
*/
#define IOMMU_INIT_FINISH(_detect, _depend, _init, _late_init) \
__IOMMU_INIT(_detect, _depend, _init, _late_init, 1)
#define USERGS_SYSRET32 \
swapgs; \
sysretl
-#define ENABLE_INTERRUPTS_SYSEXIT32 \
- swapgs; \
- sti; \
- sysexit
#else
#define INTERRUPT_RETURN iret
return arch_irqs_disabled_flags(flags);
}
+#endif /* !__ASSEMBLY__ */
+#ifdef __ASSEMBLY__
+#ifdef CONFIG_TRACE_IRQFLAGS
+# define TRACE_IRQS_ON call trace_hardirqs_on_thunk;
+# define TRACE_IRQS_OFF call trace_hardirqs_off_thunk;
#else
-
-#ifdef CONFIG_X86_64
-#define ARCH_LOCKDEP_SYS_EXIT call lockdep_sys_exit_thunk
-#define ARCH_LOCKDEP_SYS_EXIT_IRQ \
+# define TRACE_IRQS_ON
+# define TRACE_IRQS_OFF
+#endif
+#ifdef CONFIG_DEBUG_LOCK_ALLOC
+# ifdef CONFIG_X86_64
+# define LOCKDEP_SYS_EXIT call lockdep_sys_exit_thunk
+# define LOCKDEP_SYS_EXIT_IRQ \
TRACE_IRQS_ON; \
sti; \
- SAVE_REST; \
- LOCKDEP_SYS_EXIT; \
- RESTORE_REST; \
+ call lockdep_sys_exit_thunk; \
cli; \
TRACE_IRQS_OFF;
-
-#else
-#define ARCH_LOCKDEP_SYS_EXIT \
+# else
+# define LOCKDEP_SYS_EXIT \
pushl %eax; \
pushl %ecx; \
pushl %edx; \
popl %edx; \
popl %ecx; \
popl %eax;
-
-#define ARCH_LOCKDEP_SYS_EXIT_IRQ
-#endif
-
-#ifdef CONFIG_TRACE_IRQFLAGS
-# define TRACE_IRQS_ON call trace_hardirqs_on_thunk;
-# define TRACE_IRQS_OFF call trace_hardirqs_off_thunk;
+# define LOCKDEP_SYS_EXIT_IRQ
+# endif
#else
-# define TRACE_IRQS_ON
-# define TRACE_IRQS_OFF
-#endif
-#ifdef CONFIG_DEBUG_LOCK_ALLOC
-# define LOCKDEP_SYS_EXIT ARCH_LOCKDEP_SYS_EXIT
-# define LOCKDEP_SYS_EXIT_IRQ ARCH_LOCKDEP_SYS_EXIT_IRQ
-# else
# define LOCKDEP_SYS_EXIT
# define LOCKDEP_SYS_EXIT_IRQ
-# endif
-
+#endif
#endif /* __ASSEMBLY__ */
+
#endif
#ifndef _ASM_X86_JUMP_LABEL_H
#define _ASM_X86_JUMP_LABEL_H
-#ifdef __KERNEL__
+#ifndef __ASSEMBLY__
#include <linux/stringify.h>
#include <linux/types.h>
return true;
}
-#endif /* __KERNEL__ */
-
#ifdef CONFIG_X86_64
typedef u64 jump_label_t;
#else
jump_label_t key;
};
+#endif /* __ASSEMBLY__ */
#endif
(base_gfn >> KVM_HPAGE_GFN_SHIFT(level));
}
-#define SELECTOR_TI_MASK (1 << 2)
-#define SELECTOR_RPL_MASK 0x03
-
-#define IOPL_SHIFT 12
-
#define KVM_PERMILLE_MMU_PAGES 20
#define KVM_MIN_ALLOC_MMU_PAGES 64
#define KVM_MMU_HASH_SHIFT 10
enum {
KVM_DEBUGREG_BP_ENABLED = 1,
KVM_DEBUGREG_WONT_EXIT = 2,
+ KVM_DEBUGREG_RELOAD = 4,
};
struct kvm_vcpu_arch {
int cpuid_nent;
struct kvm_cpuid_entry2 cpuid_entries[KVM_MAX_CPUID_ENTRIES];
+
+ int maxphyaddr;
+
/* emulate context */
struct x86_emulate_ctxt emulate_ctxt;
struct kvm_lpage_info *lpage_info[KVM_NR_PAGE_SIZES - 1];
};
+/*
+ * We use as the mode the number of bits allocated in the LDR for the
+ * logical processor ID. It happens that these are all powers of two.
+ * This makes it is very easy to detect cases where the APICs are
+ * configured for multiple modes; in that case, we cannot use the map and
+ * hence cannot use kvm_irq_delivery_to_apic_fast either.
+ */
+#define KVM_APIC_MODE_XAPIC_CLUSTER 4
+#define KVM_APIC_MODE_XAPIC_FLAT 8
+#define KVM_APIC_MODE_X2APIC 16
+
struct kvm_apic_map {
struct rcu_head rcu;
- u8 ldr_bits;
- /* fields bellow are used to decode ldr values in different modes */
- u32 cid_shift, cid_mask, lid_mask, broadcast;
+ u8 mode;
struct kvm_lapic *phys_map[256];
/* first index is cluster id second is cpu id in a cluster */
struct kvm_lapic *logical_map[16][16];
void kvm_mmu_reset_context(struct kvm_vcpu *vcpu);
void kvm_mmu_slot_remove_write_access(struct kvm *kvm,
struct kvm_memory_slot *memslot);
+void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
+ struct kvm_memory_slot *memslot);
void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
struct kvm_memory_slot *memslot);
void kvm_mmu_slot_largepage_remove_write_access(struct kvm *kvm,
int kvm_fast_pio_out(struct kvm_vcpu *vcpu, int size, unsigned short port);
void kvm_emulate_cpuid(struct kvm_vcpu *vcpu);
int kvm_emulate_halt(struct kvm_vcpu *vcpu);
+int kvm_vcpu_halt(struct kvm_vcpu *vcpu);
int kvm_emulate_wbinvd(struct kvm_vcpu *vcpu);
void kvm_get_segment(struct kvm_vcpu *vcpu, struct kvm_segment *var, int seg);
int kvm_age_hva(struct kvm *kvm, unsigned long start, unsigned long end);
int kvm_test_age_hva(struct kvm *kvm, unsigned long hva);
void kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);
-int cpuid_maxphyaddr(struct kvm_vcpu *vcpu);
int kvm_cpu_has_injectable_intr(struct kvm_vcpu *v);
int kvm_cpu_has_interrupt(struct kvm_vcpu *vcpu);
int kvm_arch_interrupt_allowed(struct kvm_vcpu *vcpu);
static inline bool kvm_para_available(void)
{
- return 0;
+ return false;
}
static inline unsigned int kvm_arch_para_features(void)
u32 rip_msr;
};
+struct mce_vendor_flags {
+ __u64 overflow_recov : 1, /* cpuid_ebx(80000007) */
+ __reserved_0 : 63;
+};
+extern struct mce_vendor_flags mce_flags;
+
extern struct mca_config mca_cfg;
extern void mce_register_decode_chain(struct notifier_block *nb);
extern void mce_unregister_decode_chain(struct notifier_block *nb);
#ifdef CONFIG_X86_MCE
int mcheck_init(void);
void mcheck_cpu_init(struct cpuinfo_x86 *c);
+void mcheck_vendor_init_severity(void);
#else
static inline int mcheck_init(void) { return 0; }
static inline void mcheck_cpu_init(struct cpuinfo_x86 *c) {}
+static inline void mcheck_vendor_init_severity(void) {}
#endif
#ifdef CONFIG_X86_ANCIENT_MCE
DECLARE_PER_CPU(mce_banks_t, mce_poll_banks);
enum mcp_flags {
- MCP_TIMESTAMP = (1 << 0), /* log time stamp */
- MCP_UC = (1 << 1), /* log uncorrected errors */
- MCP_DONTLOG = (1 << 2), /* only clear, don't log */
+ MCP_TIMESTAMP = BIT(0), /* log time stamp */
+ MCP_UC = BIT(1), /* log uncorrected errors */
+ MCP_DONTLOG = BIT(2), /* only clear, don't log */
};
-void machine_check_poll(enum mcp_flags flags, mce_banks_t *b);
+bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b);
int mce_notify_irq(void);
#ifdef CONFIG_MICROCODE_EARLY
#define MAX_UCODE_COUNT 128
+
+#define QCHAR(a, b, c, d) ((a) + ((b) << 8) + ((c) << 16) + ((d) << 24))
+#define CPUID_INTEL1 QCHAR('G', 'e', 'n', 'u')
+#define CPUID_INTEL2 QCHAR('i', 'n', 'e', 'I')
+#define CPUID_INTEL3 QCHAR('n', 't', 'e', 'l')
+#define CPUID_AMD1 QCHAR('A', 'u', 't', 'h')
+#define CPUID_AMD2 QCHAR('e', 'n', 't', 'i')
+#define CPUID_AMD3 QCHAR('c', 'A', 'M', 'D')
+
+#define CPUID_IS(a, b, c, ebx, ecx, edx) \
+ (!((ebx ^ (a))|(edx ^ (b))|(ecx ^ (c))))
+
+/*
+ * In early loading microcode phase on BSP, boot_cpu_data is not set up yet.
+ * x86_vendor() gets vendor id for BSP.
+ *
+ * In 32 bit AP case, accessing boot_cpu_data needs linear address. To simplify
+ * coding, we still use x86_vendor() to get vendor id for AP.
+ *
+ * x86_vendor() gets vendor information directly from CPUID.
+ */
+static inline int x86_vendor(void)
+{
+ u32 eax = 0x00000000;
+ u32 ebx, ecx = 0, edx;
+
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+
+ if (CPUID_IS(CPUID_INTEL1, CPUID_INTEL2, CPUID_INTEL3, ebx, ecx, edx))
+ return X86_VENDOR_INTEL;
+
+ if (CPUID_IS(CPUID_AMD1, CPUID_AMD2, CPUID_AMD3, ebx, ecx, edx))
+ return X86_VENDOR_AMD;
+
+ return X86_VENDOR_UNKNOWN;
+}
+
+static inline unsigned int __x86_family(unsigned int sig)
+{
+ unsigned int x86;
+
+ x86 = (sig >> 8) & 0xf;
+
+ if (x86 == 0xf)
+ x86 += (sig >> 20) & 0xff;
+
+ return x86;
+}
+
+static inline unsigned int x86_family(void)
+{
+ u32 eax = 0x00000001;
+ u32 ebx, ecx = 0, edx;
+
+ native_cpuid(&eax, &ebx, &ecx, &edx);
+
+ return __x86_family(eax);
+}
+
+static inline unsigned int x86_model(unsigned int sig)
+{
+ unsigned int x86, model;
+
+ x86 = __x86_family(sig);
+
+ model = (sig >> 4) & 0xf;
+
+ if (x86 == 0x6 || x86 == 0xf)
+ model += ((sig >> 16) & 0xf) << 4;
+
+ return model;
+}
+
extern void __init load_ucode_bsp(void);
extern void load_ucode_ap(void);
extern int __init save_microcode_in_initrd(void);
#define exttable_size(et) ((et)->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE)
-extern int
-get_matching_microcode(unsigned int csig, int cpf, void *mc, int rev);
+extern int get_matching_microcode(unsigned int csig, int cpf, int rev, void *mc);
extern int microcode_sanity_check(void *mc, int print_err);
-extern int get_matching_sig(unsigned int csig, int cpf, void *mc, int rev);
-extern int
-update_match_revision(struct microcode_header_intel *mc_header, int rev);
+extern int get_matching_sig(unsigned int csig, int cpf, int rev, void *mc);
+
+static inline int
+revision_is_newer(struct microcode_header_intel *mc_header, int rev)
+{
+ return (mc_header->rev <= rev) ? 0 : 1;
+}
#ifdef CONFIG_MICROCODE_INTEL_EARLY
extern void __init load_ucode_intel_bsp(void);
:: "a" (eax), "c" (ecx));
}
+static inline void __sti_mwait(unsigned long eax, unsigned long ecx)
+{
+ trace_hardirqs_on();
+ /* "mwait %eax, %ecx;" */
+ asm volatile("sti; .byte 0x0f, 0x01, 0xc9;"
+ :: "a" (eax), "c" (ecx));
+}
+
/*
* This uses new MONITOR/MWAIT instructions on P4 processors with PNI,
* which can obviate IPI to trigger checking of need_resched.
extern unsigned long max_low_pfn_mapped;
extern unsigned long max_pfn_mapped;
-extern bool kaslr_enabled;
-
static inline phys_addr_t get_max_mapped(void)
{
return (phys_addr_t)max_pfn_mapped << PAGE_SHIFT;
PARA_SITE(PARA_PATCH(pv_cpu_ops, PV_CPU_usergs_sysret64), \
CLBR_NONE, \
jmp PARA_INDIRECT(pv_cpu_ops+PV_CPU_usergs_sysret64))
-
-#define ENABLE_INTERRUPTS_SYSEXIT32 \
- PARA_SITE(PARA_PATCH(pv_cpu_ops, PV_CPU_irq_enable_sysexit), \
- CLBR_NONE, \
- jmp PARA_INDIRECT(pv_cpu_ops+PV_CPU_irq_enable_sysexit))
#endif /* CONFIG_X86_32 */
#endif /* __ASSEMBLY__ */
extern int (*pcibios_enable_irq)(struct pci_dev *dev);
extern void (*pcibios_disable_irq)(struct pci_dev *dev);
+extern bool mp_should_keep_irq(struct device *dev);
+
struct pci_raw_ops {
int (*read)(unsigned int domain, unsigned int bus, unsigned int devfn,
int reg, int len, u32 *val);
unsigned long sp0;
unsigned short ss0, __ss0h;
unsigned long sp1;
- /* ss1 caches MSR_IA32_SYSENTER_CS: */
- unsigned short ss1, __ss1h;
+
+ /*
+ * We don't use ring 1, so ss1 is a convenient scratch space in
+ * the same cacheline as sp0. We use ss1 to cache the value in
+ * MSR_IA32_SYSENTER_CS. When we context switch
+ * MSR_IA32_SYSENTER_CS, we first check if the new value being
+ * written matches ss1, and, if it's not, then we wrmsr the new
+ * value and update ss1.
+ *
+ * The only reason we context switch MSR_IA32_SYSENTER_CS is
+ * that we set it to zero in vm86 tasks to avoid corrupting the
+ * stack if we were to go through the sysenter path from vm86
+ * mode.
+ */
+ unsigned short ss1; /* MSR_IA32_SYSENTER_CS */
+
+ unsigned short __ss1h;
unsigned long sp2;
unsigned short ss2, __ss2h;
unsigned long __cr3;
unsigned long io_bitmap[IO_BITMAP_LONGS + 1];
/*
- * .. and then another 0x100 bytes for the emergency kernel stack:
+ * Space for the temporary SYSENTER stack:
*/
- unsigned long stack[64];
+ unsigned long SYSENTER_stack[64];
} ____cacheline_aligned;
-DECLARE_PER_CPU_SHARED_ALIGNED(struct tss_struct, init_tss);
+DECLARE_PER_CPU_SHARED_ALIGNED(struct tss_struct, cpu_tss);
+
+#ifdef CONFIG_X86_32
+DECLARE_PER_CPU(unsigned long, cpu_current_top_of_stack);
+#endif
/*
* Save the original ist values for checking stack pointers during debugging
#ifdef CONFIG_X86_32
unsigned long sysenter_cs;
#else
- unsigned long usersp; /* Copy from PDA */
unsigned short es;
unsigned short ds;
unsigned short fsindex;
#endif
}
+static inline unsigned long current_top_of_stack(void)
+{
+#ifdef CONFIG_X86_64
+ return this_cpu_read_stable(cpu_tss.x86_tss.sp0);
+#else
+ /* sp0 on x86_32 is special in and around vm86 mode. */
+ return this_cpu_read_stable(cpu_current_top_of_stack);
+#endif
+}
+
#ifdef CONFIG_PARAVIRT
#include <asm/paravirt.h>
#else
#define ARCH_HAS_SPINLOCK_PREFETCH
#ifdef CONFIG_X86_32
-# define BASE_PREFETCH ASM_NOP4
+# define BASE_PREFETCH ""
# define ARCH_HAS_PREFETCH
#else
-# define BASE_PREFETCH "prefetcht0 (%1)"
+# define BASE_PREFETCH "prefetcht0 %P1"
#endif
/*
*/
static inline void prefetch(const void *x)
{
- alternative_input(BASE_PREFETCH,
- "prefetchnta (%1)",
+ alternative_input(BASE_PREFETCH, "prefetchnta %P1",
X86_FEATURE_XMM,
- "r" (x));
+ "m" (*(const char *)x));
}
/*
*/
static inline void prefetchw(const void *x)
{
- alternative_input(BASE_PREFETCH,
- "prefetchw (%1)",
- X86_FEATURE_3DNOW,
- "r" (x));
+ alternative_input(BASE_PREFETCH, "prefetchw %P1",
+ X86_FEATURE_3DNOWPREFETCH,
+ "m" (*(const char *)x));
}
static inline void spin_lock_prefetch(const void *x)
prefetchw(x);
}
+#define TOP_OF_INIT_STACK ((unsigned long)&init_stack + sizeof(init_stack) - \
+ TOP_OF_KERNEL_STACK_PADDING)
+
#ifdef CONFIG_X86_32
/*
* User space process size: 3GB (default).
#define STACK_TOP_MAX STACK_TOP
#define INIT_THREAD { \
- .sp0 = sizeof(init_stack) + (long)&init_stack, \
+ .sp0 = TOP_OF_INIT_STACK, \
.vm86_info = NULL, \
.sysenter_cs = __KERNEL_CS, \
.io_bitmap_ptr = NULL, \
}
-/*
- * Note that the .io_bitmap member must be extra-big. This is because
- * the CPU will access an additional byte beyond the end of the IO
- * permission bitmap. The extra byte must be all 1 bits, and must
- * be within the limit.
- */
-#define INIT_TSS { \
- .x86_tss = { \
- .sp0 = sizeof(init_stack) + (long)&init_stack, \
- .ss0 = __KERNEL_DS, \
- .ss1 = __KERNEL_CS, \
- .io_bitmap_base = INVALID_IO_BITMAP_OFFSET, \
- }, \
- .io_bitmap = { [0 ... IO_BITMAP_LONGS] = ~0 }, \
-}
-
extern unsigned long thread_saved_pc(struct task_struct *tsk);
-#define THREAD_SIZE_LONGS (THREAD_SIZE/sizeof(unsigned long))
-#define KSTK_TOP(info) \
-({ \
- unsigned long *__ptr = (unsigned long *)(info); \
- (unsigned long)(&__ptr[THREAD_SIZE_LONGS]); \
-})
-
/*
- * The below -8 is to reserve 8 bytes on top of the ring0 stack.
+ * TOP_OF_KERNEL_STACK_PADDING reserves 8 bytes on top of the ring0 stack.
* This is necessary to guarantee that the entire "struct pt_regs"
* is accessible even if the CPU haven't stored the SS/ESP registers
* on the stack (interrupt gate does not save these registers
* "struct pt_regs" is possible, but they may contain the
* completely wrong values.
*/
-#define task_pt_regs(task) \
-({ \
- struct pt_regs *__regs__; \
- __regs__ = (struct pt_regs *)(KSTK_TOP(task_stack_page(task))-8); \
- __regs__ - 1; \
+#define task_pt_regs(task) \
+({ \
+ unsigned long __ptr = (unsigned long)task_stack_page(task); \
+ __ptr += THREAD_SIZE - TOP_OF_KERNEL_STACK_PADDING; \
+ ((struct pt_regs *)__ptr) - 1; \
})
#define KSTK_ESP(task) (task_pt_regs(task)->sp)
#define STACK_TOP_MAX TASK_SIZE_MAX
#define INIT_THREAD { \
- .sp0 = (unsigned long)&init_stack + sizeof(init_stack) \
-}
-
-#define INIT_TSS { \
- .x86_tss.sp0 = (unsigned long)&init_stack + sizeof(init_stack) \
+ .sp0 = TOP_OF_INIT_STACK \
}
/*
#define task_pt_regs(tsk) ((struct pt_regs *)(tsk)->thread.sp0 - 1)
extern unsigned long KSTK_ESP(struct task_struct *task);
-/*
- * User space RSP while inside the SYSCALL fast path
- */
-DECLARE_PER_CPU(unsigned long, old_rsp);
-
#endif /* CONFIG_X86_64 */
extern void start_thread(struct pt_regs *regs, unsigned long new_ip,
#else /* __i386__ */
struct pt_regs {
+/*
+ * C ABI says these regs are callee-preserved. They aren't saved on kernel entry
+ * unless syscall needs a complete, fully filled "struct pt_regs".
+ */
unsigned long r15;
unsigned long r14;
unsigned long r13;
unsigned long r12;
unsigned long bp;
unsigned long bx;
-/* arguments: non interrupts/non tracing syscalls only save up to here*/
+/* These regs are callee-clobbered. Always saved on kernel entry. */
unsigned long r11;
unsigned long r10;
unsigned long r9;
unsigned long dx;
unsigned long si;
unsigned long di;
+/*
+ * On syscall entry, this is syscall#. On CPU exception, this is error code.
+ * On hw interrupt, it's IRQ number:
+ */
unsigned long orig_ax;
-/* end of arguments */
-/* cpu exception frame or undefined */
+/* Return frame for iretq */
unsigned long ip;
unsigned long cs;
unsigned long flags;
}
/*
- * user_mode_vm(regs) determines whether a register set came from user mode.
- * This is true if V8086 mode was enabled OR if the register set was from
- * protected mode with RPL-3 CS value. This tricky test checks that with
- * one comparison. Many places in the kernel can bypass this full check
- * if they have already ruled out V8086 mode, so user_mode(regs) can be used.
+ * user_mode(regs) determines whether a register set came from user
+ * mode. On x86_32, this is true if V8086 mode was enabled OR if the
+ * register set was from protected mode with RPL-3 CS value. This
+ * tricky test checks that with one comparison.
+ *
+ * On x86_64, vm86 mode is mercifully nonexistent, and we don't need
+ * the extra check.
*/
static inline int user_mode(struct pt_regs *regs)
{
#endif
}
-static inline int user_mode_vm(struct pt_regs *regs)
-{
-#ifdef CONFIG_X86_32
- return ((regs->cs & SEGMENT_RPL_MASK) | (regs->flags & X86_VM_MASK)) >=
- USER_RPL;
-#else
- return user_mode(regs);
-#endif
-}
-
static inline int v8086_mode(struct pt_regs *regs)
{
#ifdef CONFIG_X86_32
#endif
}
-#define current_user_stack_pointer() this_cpu_read(old_rsp)
-/* ia32 vs. x32 difference */
-#define compat_user_stack_pointer() \
- (test_thread_flag(TIF_IA32) \
- ? current_pt_regs()->sp \
- : this_cpu_read(old_rsp))
+#define current_user_stack_pointer() current_pt_regs()->sp
+#define compat_user_stack_pointer() current_pt_regs()->sp
#endif
#ifdef CONFIG_X86_32
*/
#define arch_ptrace_stop_needed(code, info) \
({ \
- set_thread_flag(TIF_NOTIFY_RESUME); \
+ force_iret(); \
false; \
})
struct pvclock_vsyscall_time_info {
struct pvclock_vcpu_time_info pvti;
+ u32 migrate_count;
} __attribute__((__aligned__(SMP_CACHE_BYTES)));
#define PVTI_SIZE sizeof(struct pvclock_vsyscall_time_info)
#include <linux/const.h>
-/* Constructor for a conventional segment GDT (or LDT) entry */
-/* This is a macro so it can be used in initializers */
+/*
+ * Constructor for a conventional segment GDT (or LDT) entry.
+ * This is a macro so it can be used in initializers.
+ */
#define GDT_ENTRY(flags, base, limit) \
((((base) & _AC(0xff000000,ULL)) << (56-24)) | \
(((flags) & _AC(0x0000f0ff,ULL)) << 40) | \
(((base) & _AC(0x00ffffff,ULL)) << 16) | \
(((limit) & _AC(0x0000ffff,ULL))))
-/* Simple and small GDT entries for booting only */
+/* Simple and small GDT entries for booting only: */
#define GDT_ENTRY_BOOT_CS 2
-#define __BOOT_CS (GDT_ENTRY_BOOT_CS * 8)
+#define GDT_ENTRY_BOOT_DS 3
+#define GDT_ENTRY_BOOT_TSS 4
+#define __BOOT_CS (GDT_ENTRY_BOOT_CS*8)
+#define __BOOT_DS (GDT_ENTRY_BOOT_DS*8)
+#define __BOOT_TSS (GDT_ENTRY_BOOT_TSS*8)
+
+/*
+ * Bottom two bits of selector give the ring
+ * privilege level
+ */
+#define SEGMENT_RPL_MASK 0x3
-#define GDT_ENTRY_BOOT_DS (GDT_ENTRY_BOOT_CS + 1)
-#define __BOOT_DS (GDT_ENTRY_BOOT_DS * 8)
+/* User mode is privilege level 3: */
+#define USER_RPL 0x3
-#define GDT_ENTRY_BOOT_TSS (GDT_ENTRY_BOOT_CS + 2)
-#define __BOOT_TSS (GDT_ENTRY_BOOT_TSS * 8)
+/* Bit 2 is Table Indicator (TI): selects between LDT or GDT */
+#define SEGMENT_TI_MASK 0x4
+/* LDT segment has TI set ... */
+#define SEGMENT_LDT 0x4
+/* ... GDT has it cleared */
+#define SEGMENT_GDT 0x0
-#define SEGMENT_RPL_MASK 0x3 /*
- * Bottom two bits of selector give the ring
- * privilege level
- */
-#define SEGMENT_TI_MASK 0x4 /* Bit 2 is table indicator (LDT/GDT) */
-#define USER_RPL 0x3 /* User mode is privilege level 3 */
-#define SEGMENT_LDT 0x4 /* LDT segment has TI set... */
-#define SEGMENT_GDT 0x0 /* ... GDT has it cleared */
+#define GDT_ENTRY_INVALID_SEG 0
#ifdef CONFIG_X86_32
/*
* The layout of the per-CPU GDT under Linux:
*
- * 0 - null
+ * 0 - null <=== cacheline #1
* 1 - reserved
* 2 - reserved
* 3 - reserved
*
- * 4 - unused <==== new cacheline
+ * 4 - unused <=== cacheline #2
* 5 - unused
*
* ------- start of TLS (Thread-Local Storage) segments:
*
* 6 - TLS segment #1 [ glibc's TLS segment ]
* 7 - TLS segment #2 [ Wine's %fs Win32 segment ]
- * 8 - TLS segment #3
+ * 8 - TLS segment #3 <=== cacheline #3
* 9 - reserved
* 10 - reserved
* 11 - reserved
*
* ------- start of kernel segments:
*
- * 12 - kernel code segment <==== new cacheline
+ * 12 - kernel code segment <=== cacheline #4
* 13 - kernel data segment
* 14 - default user CS
* 15 - default user DS
- * 16 - TSS
+ * 16 - TSS <=== cacheline #5
* 17 - LDT
* 18 - PNPBIOS support (16->32 gate)
* 19 - PNPBIOS support
- * 20 - PNPBIOS support
+ * 20 - PNPBIOS support <=== cacheline #6
* 21 - PNPBIOS support
* 22 - PNPBIOS support
* 23 - APM BIOS support
- * 24 - APM BIOS support
+ * 24 - APM BIOS support <=== cacheline #7
* 25 - APM BIOS support
*
* 26 - ESPFIX small SS
* 27 - per-cpu [ offset to per-cpu data area ]
- * 28 - stack_canary-20 [ for stack protector ]
+ * 28 - stack_canary-20 [ for stack protector ] <=== cacheline #8
* 29 - unused
* 30 - unused
* 31 - TSS for double fault handler
*/
-#define GDT_ENTRY_TLS_MIN 6
-#define GDT_ENTRY_TLS_MAX (GDT_ENTRY_TLS_MIN + GDT_ENTRY_TLS_ENTRIES - 1)
+#define GDT_ENTRY_TLS_MIN 6
+#define GDT_ENTRY_TLS_MAX (GDT_ENTRY_TLS_MIN + GDT_ENTRY_TLS_ENTRIES - 1)
+#define GDT_ENTRY_KERNEL_CS 12
+#define GDT_ENTRY_KERNEL_DS 13
#define GDT_ENTRY_DEFAULT_USER_CS 14
-
#define GDT_ENTRY_DEFAULT_USER_DS 15
+#define GDT_ENTRY_TSS 16
+#define GDT_ENTRY_LDT 17
+#define GDT_ENTRY_PNPBIOS_CS32 18
+#define GDT_ENTRY_PNPBIOS_CS16 19
+#define GDT_ENTRY_PNPBIOS_DS 20
+#define GDT_ENTRY_PNPBIOS_TS1 21
+#define GDT_ENTRY_PNPBIOS_TS2 22
+#define GDT_ENTRY_APMBIOS_BASE 23
+
+#define GDT_ENTRY_ESPFIX_SS 26
+#define GDT_ENTRY_PERCPU 27
+#define GDT_ENTRY_STACK_CANARY 28
+
+#define GDT_ENTRY_DOUBLEFAULT_TSS 31
-#define GDT_ENTRY_KERNEL_BASE (12)
+/*
+ * Number of entries in the GDT table:
+ */
+#define GDT_ENTRIES 32
-#define GDT_ENTRY_KERNEL_CS (GDT_ENTRY_KERNEL_BASE+0)
+/*
+ * Segment selector values corresponding to the above entries:
+ */
-#define GDT_ENTRY_KERNEL_DS (GDT_ENTRY_KERNEL_BASE+1)
+#define __KERNEL_CS (GDT_ENTRY_KERNEL_CS*8)
+#define __KERNEL_DS (GDT_ENTRY_KERNEL_DS*8)
+#define __USER_DS (GDT_ENTRY_DEFAULT_USER_DS*8 + 3)
+#define __USER_CS (GDT_ENTRY_DEFAULT_USER_CS*8 + 3)
+#define __ESPFIX_SS (GDT_ENTRY_ESPFIX_SS*8)
-#define GDT_ENTRY_TSS (GDT_ENTRY_KERNEL_BASE+4)
-#define GDT_ENTRY_LDT (GDT_ENTRY_KERNEL_BASE+5)
+/* segment for calling fn: */
+#define PNP_CS32 (GDT_ENTRY_PNPBIOS_CS32*8)
+/* code segment for BIOS: */
+#define PNP_CS16 (GDT_ENTRY_PNPBIOS_CS16*8)
-#define GDT_ENTRY_PNPBIOS_BASE (GDT_ENTRY_KERNEL_BASE+6)
-#define GDT_ENTRY_APMBIOS_BASE (GDT_ENTRY_KERNEL_BASE+11)
+/* "Is this PNP code selector (PNP_CS32 or PNP_CS16)?" */
+#define SEGMENT_IS_PNP_CODE(x) (((x) & 0xf4) == PNP_CS32)
-#define GDT_ENTRY_ESPFIX_SS (GDT_ENTRY_KERNEL_BASE+14)
-#define __ESPFIX_SS (GDT_ENTRY_ESPFIX_SS*8)
+/* data segment for BIOS: */
+#define PNP_DS (GDT_ENTRY_PNPBIOS_DS*8)
+/* transfer data segment: */
+#define PNP_TS1 (GDT_ENTRY_PNPBIOS_TS1*8)
+/* another data segment: */
+#define PNP_TS2 (GDT_ENTRY_PNPBIOS_TS2*8)
-#define GDT_ENTRY_PERCPU (GDT_ENTRY_KERNEL_BASE+15)
#ifdef CONFIG_SMP
-#define __KERNEL_PERCPU (GDT_ENTRY_PERCPU * 8)
+# define __KERNEL_PERCPU (GDT_ENTRY_PERCPU*8)
#else
-#define __KERNEL_PERCPU 0
+# define __KERNEL_PERCPU 0
#endif
-#define GDT_ENTRY_STACK_CANARY (GDT_ENTRY_KERNEL_BASE+16)
#ifdef CONFIG_CC_STACKPROTECTOR
-#define __KERNEL_STACK_CANARY (GDT_ENTRY_STACK_CANARY*8)
+# define __KERNEL_STACK_CANARY (GDT_ENTRY_STACK_CANARY*8)
#else
-#define __KERNEL_STACK_CANARY 0
+# define __KERNEL_STACK_CANARY 0
#endif
-#define GDT_ENTRY_DOUBLEFAULT_TSS 31
-
-/*
- * The GDT has 32 entries
- */
-#define GDT_ENTRIES 32
+#else /* 64-bit: */
-/* The PnP BIOS entries in the GDT */
-#define GDT_ENTRY_PNPBIOS_CS32 (GDT_ENTRY_PNPBIOS_BASE + 0)
-#define GDT_ENTRY_PNPBIOS_CS16 (GDT_ENTRY_PNPBIOS_BASE + 1)
-#define GDT_ENTRY_PNPBIOS_DS (GDT_ENTRY_PNPBIOS_BASE + 2)
-#define GDT_ENTRY_PNPBIOS_TS1 (GDT_ENTRY_PNPBIOS_BASE + 3)
-#define GDT_ENTRY_PNPBIOS_TS2 (GDT_ENTRY_PNPBIOS_BASE + 4)
-
-/* The PnP BIOS selectors */
-#define PNP_CS32 (GDT_ENTRY_PNPBIOS_CS32 * 8) /* segment for calling fn */
-#define PNP_CS16 (GDT_ENTRY_PNPBIOS_CS16 * 8) /* code segment for BIOS */
-#define PNP_DS (GDT_ENTRY_PNPBIOS_DS * 8) /* data segment for BIOS */
-#define PNP_TS1 (GDT_ENTRY_PNPBIOS_TS1 * 8) /* transfer data segment */
-#define PNP_TS2 (GDT_ENTRY_PNPBIOS_TS2 * 8) /* another data segment */
+#include <asm/cache.h>
+#define GDT_ENTRY_KERNEL32_CS 1
+#define GDT_ENTRY_KERNEL_CS 2
+#define GDT_ENTRY_KERNEL_DS 3
/*
- * Matching rules for certain types of segments.
+ * We cannot use the same code segment descriptor for user and kernel mode,
+ * not even in long flat mode, because of different DPL.
+ *
+ * GDT layout to get 64-bit SYSCALL/SYSRET support right. SYSRET hardcodes
+ * selectors:
+ *
+ * if returning to 32-bit userspace: cs = STAR.SYSRET_CS,
+ * if returning to 64-bit userspace: cs = STAR.SYSRET_CS+16,
+ *
+ * ss = STAR.SYSRET_CS+8 (in either case)
+ *
+ * thus USER_DS should be between 32-bit and 64-bit code selectors:
*/
+#define GDT_ENTRY_DEFAULT_USER32_CS 4
+#define GDT_ENTRY_DEFAULT_USER_DS 5
+#define GDT_ENTRY_DEFAULT_USER_CS 6
-/* Matches PNP_CS32 and PNP_CS16 (they must be consecutive) */
-#define SEGMENT_IS_PNP_CODE(x) (((x) & 0xf4) == GDT_ENTRY_PNPBIOS_BASE * 8)
-
+/* Needs two entries */
+#define GDT_ENTRY_TSS 8
+/* Needs two entries */
+#define GDT_ENTRY_LDT 10
-#else
-#include <asm/cache.h>
-
-#define GDT_ENTRY_KERNEL32_CS 1
-#define GDT_ENTRY_KERNEL_CS 2
-#define GDT_ENTRY_KERNEL_DS 3
+#define GDT_ENTRY_TLS_MIN 12
+#define GDT_ENTRY_TLS_MAX 14
-#define __KERNEL32_CS (GDT_ENTRY_KERNEL32_CS * 8)
+/* Abused to load per CPU data from limit */
+#define GDT_ENTRY_PER_CPU 15
/*
- * we cannot use the same code segment descriptor for user and kernel
- * -- not even in the long flat mode, because of different DPL /kkeil
- * The segment offset needs to contain a RPL. Grr. -AK
- * GDT layout to get 64bit syscall right (sysret hardcodes gdt offsets)
+ * Number of entries in the GDT table:
*/
-#define GDT_ENTRY_DEFAULT_USER32_CS 4
-#define GDT_ENTRY_DEFAULT_USER_DS 5
-#define GDT_ENTRY_DEFAULT_USER_CS 6
-#define __USER32_CS (GDT_ENTRY_DEFAULT_USER32_CS*8+3)
-#define __USER32_DS __USER_DS
-
-#define GDT_ENTRY_TSS 8 /* needs two entries */
-#define GDT_ENTRY_LDT 10 /* needs two entries */
-#define GDT_ENTRY_TLS_MIN 12
-#define GDT_ENTRY_TLS_MAX 14
-
-#define GDT_ENTRY_PER_CPU 15 /* Abused to load per CPU data from limit */
-#define __PER_CPU_SEG (GDT_ENTRY_PER_CPU * 8 + 3)
+#define GDT_ENTRIES 16
-/* TLS indexes for 64bit - hardcoded in arch_prctl */
-#define FS_TLS 0
-#define GS_TLS 1
-
-#define GS_TLS_SEL ((GDT_ENTRY_TLS_MIN+GS_TLS)*8 + 3)
-#define FS_TLS_SEL ((GDT_ENTRY_TLS_MIN+FS_TLS)*8 + 3)
-
-#define GDT_ENTRIES 16
+/*
+ * Segment selector values corresponding to the above entries:
+ *
+ * Note, selectors also need to have a correct RPL,
+ * expressed with the +3 value for user-space selectors:
+ */
+#define __KERNEL32_CS (GDT_ENTRY_KERNEL32_CS*8)
+#define __KERNEL_CS (GDT_ENTRY_KERNEL_CS*8)
+#define __KERNEL_DS (GDT_ENTRY_KERNEL_DS*8)
+#define __USER32_CS (GDT_ENTRY_DEFAULT_USER32_CS*8 + 3)
+#define __USER_DS (GDT_ENTRY_DEFAULT_USER_DS*8 + 3)
+#define __USER32_DS __USER_DS
+#define __USER_CS (GDT_ENTRY_DEFAULT_USER_CS*8 + 3)
+#define __PER_CPU_SEG (GDT_ENTRY_PER_CPU*8 + 3)
+
+/* TLS indexes for 64-bit - hardcoded in arch_prctl(): */
+#define FS_TLS 0
+#define GS_TLS 1
+
+#define GS_TLS_SEL ((GDT_ENTRY_TLS_MIN+GS_TLS)*8 + 3)
+#define FS_TLS_SEL ((GDT_ENTRY_TLS_MIN+FS_TLS)*8 + 3)
#endif
-#define __KERNEL_CS (GDT_ENTRY_KERNEL_CS*8)
-#define __KERNEL_DS (GDT_ENTRY_KERNEL_DS*8)
-#define __USER_DS (GDT_ENTRY_DEFAULT_USER_DS*8+3)
-#define __USER_CS (GDT_ENTRY_DEFAULT_USER_CS*8+3)
#ifndef CONFIG_PARAVIRT
-#define get_kernel_rpl() 0
+# define get_kernel_rpl() 0
#endif
-#define IDT_ENTRIES 256
-#define NUM_EXCEPTION_VECTORS 32
-/* Bitmask of exception vectors which push an error code on the stack */
-#define EXCEPTION_ERRCODE_MASK 0x00027d00
-#define GDT_SIZE (GDT_ENTRIES * 8)
-#define GDT_ENTRY_TLS_ENTRIES 3
-#define TLS_SIZE (GDT_ENTRY_TLS_ENTRIES * 8)
+#define IDT_ENTRIES 256
+#define NUM_EXCEPTION_VECTORS 32
+
+/* Bitmask of exception vectors which push an error code on the stack: */
+#define EXCEPTION_ERRCODE_MASK 0x00027d00
+
+#define GDT_SIZE (GDT_ENTRIES*8)
+#define GDT_ENTRY_TLS_ENTRIES 3
+#define TLS_SIZE (GDT_ENTRY_TLS_ENTRIES* 8)
#ifdef __KERNEL__
#ifndef __ASSEMBLY__
+
extern const char early_idt_handlers[NUM_EXCEPTION_VECTORS][2+2+5];
#ifdef CONFIG_TRACING
-#define trace_early_idt_handlers early_idt_handlers
+# define trace_early_idt_handlers early_idt_handlers
#endif
/*
} while (0)
/*
- * Save a segment register away
+ * Save a segment register away:
*/
#define savesegment(seg, value) \
asm("mov %%" #seg ",%0":"=r" (value) : : "memory")
/*
- * x86_32 user gs accessors.
+ * x86-32 user GS accessors:
*/
#ifdef CONFIG_X86_32
-#ifdef CONFIG_X86_32_LAZY_GS
-#define get_user_gs(regs) (u16)({unsigned long v; savesegment(gs, v); v;})
-#define set_user_gs(regs, v) loadsegment(gs, (unsigned long)(v))
-#define task_user_gs(tsk) ((tsk)->thread.gs)
-#define lazy_save_gs(v) savesegment(gs, (v))
-#define lazy_load_gs(v) loadsegment(gs, (v))
-#else /* X86_32_LAZY_GS */
-#define get_user_gs(regs) (u16)((regs)->gs)
-#define set_user_gs(regs, v) do { (regs)->gs = (v); } while (0)
-#define task_user_gs(tsk) (task_pt_regs(tsk)->gs)
-#define lazy_save_gs(v) do { } while (0)
-#define lazy_load_gs(v) do { } while (0)
-#endif /* X86_32_LAZY_GS */
+# ifdef CONFIG_X86_32_LAZY_GS
+# define get_user_gs(regs) (u16)({ unsigned long v; savesegment(gs, v); v; })
+# define set_user_gs(regs, v) loadsegment(gs, (unsigned long)(v))
+# define task_user_gs(tsk) ((tsk)->thread.gs)
+# define lazy_save_gs(v) savesegment(gs, (v))
+# define lazy_load_gs(v) loadsegment(gs, (v))
+# else /* X86_32_LAZY_GS */
+# define get_user_gs(regs) (u16)((regs)->gs)
+# define set_user_gs(regs, v) do { (regs)->gs = (v); } while (0)
+# define task_user_gs(tsk) (task_pt_regs(tsk)->gs)
+# define lazy_save_gs(v) do { } while (0)
+# define lazy_load_gs(v) do { } while (0)
+# endif /* X86_32_LAZY_GS */
#endif /* X86_32 */
-static inline unsigned long get_limit(unsigned long segment)
-{
- unsigned long __limit;
- asm("lsll %1,%0" : "=r" (__limit) : "r" (segment));
- return __limit + 1;
-}
-
#endif /* !__ASSEMBLY__ */
#endif /* __KERNEL__ */
*/
extern struct boot_params boot_params;
+static inline bool kaslr_enabled(void)
+{
+ return !!(boot_params.hdr.loadflags & KASLR_FLAG);
+}
+
/*
* Do NOT EVER look at the BIOS memory size location.
* It does not work on many machines.
unsigned long ip;
unsigned long flags;
unsigned short cs;
- unsigned short gs;
- unsigned short fs;
- unsigned short __pad0;
+ unsigned short __pad2; /* Was called gs, but was always zero. */
+ unsigned short __pad1; /* Was called fs, but was always zero. */
+ unsigned short ss;
unsigned long err;
unsigned long trapno;
unsigned long oldmask;
X86_EFLAGS_CF | X86_EFLAGS_RF)
void signal_fault(struct pt_regs *regs, void __user *frame, char *where);
-
-int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc,
- unsigned long *pax);
+int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc);
int setup_sigcontext(struct sigcontext __user *sc, void __user *fpstate,
struct pt_regs *regs, unsigned long mask);
#ifdef CONFIG_X86_SMAP
-#define ASM_CLAC \
- 661: ASM_NOP3 ; \
- .pushsection .altinstr_replacement, "ax" ; \
- 662: __ASM_CLAC ; \
- .popsection ; \
- .pushsection .altinstructions, "a" ; \
- altinstruction_entry 661b, 662b, X86_FEATURE_SMAP, 3, 3 ; \
- .popsection
-
-#define ASM_STAC \
- 661: ASM_NOP3 ; \
- .pushsection .altinstr_replacement, "ax" ; \
- 662: __ASM_STAC ; \
- .popsection ; \
- .pushsection .altinstructions, "a" ; \
- altinstruction_entry 661b, 662b, X86_FEATURE_SMAP, 3, 3 ; \
- .popsection
+#define ASM_CLAC \
+ ALTERNATIVE "", __stringify(__ASM_CLAC), X86_FEATURE_SMAP
+
+#define ASM_STAC \
+ ALTERNATIVE "", __stringify(__ASM_STAC), X86_FEATURE_SMAP
#else /* CONFIG_X86_SMAP */
static __always_inline void clac(void)
{
/* Note: a barrier is implicit in alternative() */
- alternative(ASM_NOP3, __stringify(__ASM_CLAC), X86_FEATURE_SMAP);
+ alternative("", __stringify(__ASM_CLAC), X86_FEATURE_SMAP);
}
static __always_inline void stac(void)
{
/* Note: a barrier is implicit in alternative() */
- alternative(ASM_NOP3, __stringify(__ASM_STAC), X86_FEATURE_SMAP);
+ alternative("", __stringify(__ASM_STAC), X86_FEATURE_SMAP);
}
/* These macros can be used in asm() statements */
#define ASM_CLAC \
- ALTERNATIVE(ASM_NOP3, __stringify(__ASM_CLAC), X86_FEATURE_SMAP)
+ ALTERNATIVE("", __stringify(__ASM_CLAC), X86_FEATURE_SMAP)
#define ASM_STAC \
- ALTERNATIVE(ASM_NOP3, __stringify(__ASM_STAC), X86_FEATURE_SMAP)
+ ALTERNATIVE("", __stringify(__ASM_STAC), X86_FEATURE_SMAP)
#else /* CONFIG_X86_SMAP */
void native_smp_prepare_boot_cpu(void);
void native_smp_prepare_cpus(unsigned int max_cpus);
void native_smp_cpus_done(unsigned int max_cpus);
+void common_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_up(unsigned int cpunum, struct task_struct *tidle);
int native_cpu_disable(void);
void native_cpu_die(unsigned int cpu);
#ifdef __KERNEL__
+#include <asm/nops.h>
+
static inline void native_clts(void)
{
asm volatile("clts");
"+m" (*(volatile char __force *)__p));
}
+static inline void clwb(volatile void *__p)
+{
+ volatile struct { char x[64]; } *p = __p;
+
+ asm volatile(ALTERNATIVE_2(
+ ".byte " __stringify(NOP_DS_PREFIX) "; clflush (%[pax])",
+ ".byte 0x66; clflush (%[pax])", /* clflushopt (%%rax) */
+ X86_FEATURE_CLFLUSHOPT,
+ ".byte 0x66, 0x0f, 0xae, 0x30", /* clwb (%%rax) */
+ X86_FEATURE_CLWB)
+ : [p] "+m" (*p)
+ : [pax] "a" (p));
+}
+
+static inline void pcommit_sfence(void)
+{
+ alternative(ASM_NOP7,
+ ".byte 0x66, 0x0f, 0xae, 0xf8\n\t" /* pcommit */
+ "sfence",
+ X86_FEATURE_PCOMMIT);
+}
+
#define nop() asm volatile ("nop")
#include <asm/percpu.h>
#include <asm/types.h>
+/*
+ * TOP_OF_KERNEL_STACK_PADDING is a number of unused bytes that we
+ * reserve at the top of the kernel stack. We do it because of a nasty
+ * 32-bit corner case. On x86_32, the hardware stack frame is
+ * variable-length. Except for vm86 mode, struct pt_regs assumes a
+ * maximum-length frame. If we enter from CPL 0, the top 8 bytes of
+ * pt_regs don't actually exist. Ordinarily this doesn't matter, but it
+ * does in at least one case:
+ *
+ * If we take an NMI early enough in SYSENTER, then we can end up with
+ * pt_regs that extends above sp0. On the way out, in the espfix code,
+ * we can read the saved SS value, but that value will be above sp0.
+ * Without this offset, that can result in a page fault. (We are
+ * careful that, in this case, the value we read doesn't matter.)
+ *
+ * In vm86 mode, the hardware frame is much longer still, but we neither
+ * access the extra members from NMI context, nor do we write such a
+ * frame at sp0 at all.
+ *
+ * x86_64 has a fixed-length stack frame.
+ */
+#ifdef CONFIG_X86_32
+# define TOP_OF_KERNEL_STACK_PADDING 8
+#else
+# define TOP_OF_KERNEL_STACK_PADDING 0
+#endif
+
/*
* low level task data that entry.S needs immediate access to
* - this struct should fit entirely inside of one cache line
#define _TIF_WORK_CTXSW_NEXT (_TIF_WORK_CTXSW)
#define STACK_WARN (THREAD_SIZE/8)
-#define KERNEL_STACK_OFFSET (5*(BITS_PER_LONG/8))
/*
* macros/functions for gaining access to the thread information structure
static inline struct thread_info *current_thread_info(void)
{
- struct thread_info *ti;
- ti = (void *)(this_cpu_read_stable(kernel_stack) +
- KERNEL_STACK_OFFSET - THREAD_SIZE);
- return ti;
+ return (struct thread_info *)(current_top_of_stack() - THREAD_SIZE);
}
static inline unsigned long current_stack_pointer(void)
#else /* !__ASSEMBLY__ */
-/* how to get the thread information struct from ASM */
+/* Load thread_info address into "reg" */
#define GET_THREAD_INFO(reg) \
_ASM_MOV PER_CPU_VAR(kernel_stack),reg ; \
- _ASM_SUB $(THREAD_SIZE-KERNEL_STACK_OFFSET),reg ;
+ _ASM_SUB $(THREAD_SIZE),reg ;
/*
- * Same if PER_CPU_VAR(kernel_stack) is, perhaps with some offset, already in
- * a certain register (to be used in assembler memory operands).
+ * ASM operand which evaluates to a 'thread_info' address of
+ * the current task, if it is known that "reg" is exactly "off"
+ * bytes below the top of the stack currently.
+ *
+ * ( The kernel stack's size is known at build time, it is usually
+ * 2 or 4 pages, and the bottom of the kernel stack contains
+ * the thread_info structure. So to access the thread_info very
+ * quickly from assembly code we can calculate down from the
+ * top of the kernel stack to the bottom, using constant,
+ * build-time calculations only. )
+ *
+ * For example, to fetch the current thread_info->flags value into %eax
+ * on x86-64 defconfig kernels, in syscall entry code where RSP is
+ * currently at exactly SIZEOF_PTREGS bytes away from the top of the
+ * stack:
+ *
+ * mov ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS), %eax
+ *
+ * will translate to:
+ *
+ * 8b 84 24 b8 c0 ff ff mov -0x3f48(%rsp), %eax
+ *
+ * which is below the current RSP by almost 16K.
*/
-#define THREAD_INFO(reg, off) KERNEL_STACK_OFFSET+(off)-THREAD_SIZE(reg)
+#define ASM_THREAD_INFO(field, reg, off) ((field)+(off)-THREAD_SIZE)(reg)
#endif
#endif
return false;
}
+
+/*
+ * Force syscall return via IRET by making it look as if there was
+ * some work pending. IRET is our most capable (but slowest) syscall
+ * return path, which is able to restore modified SS, CS and certain
+ * EFLAGS values that other (fast) syscall return instructions
+ * are not able to restore properly.
+ */
+#define force_iret() set_thread_flag(TIF_NOTIFY_RESUME)
+
#endif /* !__ASSEMBLY__ */
#ifndef __ASSEMBLY__
}
unsigned long
-copy_user_handle_tail(char *to, char *from, unsigned len, unsigned zerorest);
+copy_user_handle_tail(char *to, char *from, unsigned len);
#endif /* _ASM_X86_UACCESS_64_H */
if (boot_cpu_has(X86_FEATURE_XSAVES))
asm volatile("1:"XSAVES"\n\t"
"2:\n\t"
- : : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
+ xstate_fault
+ : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
: "memory");
else
asm volatile("1:"XSAVE"\n\t"
"2:\n\t"
- : : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
+ xstate_fault
+ : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
: "memory");
-
- asm volatile(xstate_fault
- : "0" (0)
- : "memory");
-
return err;
}
if (boot_cpu_has(X86_FEATURE_XSAVES))
asm volatile("1:"XRSTORS"\n\t"
"2:\n\t"
- : : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
+ xstate_fault
+ : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
: "memory");
else
asm volatile("1:"XRSTOR"\n\t"
"2:\n\t"
- : : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
+ xstate_fault
+ : "D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
: "memory");
-
- asm volatile(xstate_fault
- : "0" (0)
- : "memory");
-
return err;
}
*/
alternative_input_2(
"1:"XSAVE,
- "1:"XSAVEOPT,
+ XSAVEOPT,
X86_FEATURE_XSAVEOPT,
- "1:"XSAVES,
+ XSAVES,
X86_FEATURE_XSAVES,
[fx] "D" (fx), "a" (lmask), "d" (hmask) :
"memory");
*/
alternative_input(
"1: " XRSTOR,
- "1: " XRSTORS,
+ XRSTORS,
X86_FEATURE_XSAVES,
"D" (fx), "m" (*fx), "a" (lmask), "d" (hmask)
: "memory");
#define SETUP_DTB 2
#define SETUP_PCI 3
#define SETUP_EFI 4
-#define SETUP_KASLR 5
/* ram_size flags */
#define RAMDISK_IMAGE_START_MASK 0x07FF
/* loadflags */
#define LOADED_HIGH (1<<0)
+#define KASLR_FLAG (1<<1)
#define QUIET_FLAG (1<<5)
#define KEEP_SEGMENTS (1<<6)
#define CAN_USE_HEAP (1<<7)
#else /* __i386__ */
#if defined(__ASSEMBLY__) || defined(__FRAME_OFFSETS)
+/*
+ * C ABI says these regs are callee-preserved. They aren't saved on kernel entry
+ * unless syscall needs a complete, fully filled "struct pt_regs".
+ */
#define R15 0
#define R14 8
#define R13 16
#define R12 24
#define RBP 32
#define RBX 40
-/* arguments: interrupts/non tracing syscalls only save up to here*/
+/* These regs are callee-clobbered. Always saved on kernel entry. */
#define R11 48
#define R10 56
#define R9 64
#define RDX 96
#define RSI 104
#define RDI 112
-#define ORIG_RAX 120 /* = ERROR */
-/* end of arguments */
-/* cpu exception frame or undefined in case of fast syscall. */
+/*
+ * On syscall entry, this is syscall#. On CPU exception, this is error code.
+ * On hw interrupt, it's IRQ number:
+ */
+#define ORIG_RAX 120
+/* Return frame for iretq */
#define RIP 128
#define CS 136
#define EFLAGS 144
#define RSP 152
#define SS 160
-#define ARGOFFSET R11
#endif /* __ASSEMBLY__ */
/* top of stack page */
#ifndef __KERNEL__
struct pt_regs {
+/*
+ * C ABI says these regs are callee-preserved. They aren't saved on kernel entry
+ * unless syscall needs a complete, fully filled "struct pt_regs".
+ */
unsigned long r15;
unsigned long r14;
unsigned long r13;
unsigned long r12;
unsigned long rbp;
unsigned long rbx;
-/* arguments: non interrupts/non tracing syscalls only save up to here*/
+/* These regs are callee-clobbered. Always saved on kernel entry. */
unsigned long r11;
unsigned long r10;
unsigned long r9;
unsigned long rdx;
unsigned long rsi;
unsigned long rdi;
+/*
+ * On syscall entry, this is syscall#. On CPU exception, this is error code.
+ * On hw interrupt, it's IRQ number:
+ */
unsigned long orig_rax;
-/* end of arguments */
-/* cpu exception frame or undefined */
+/* Return frame for iretq */
unsigned long rip;
unsigned long cs;
unsigned long eflags;
__u64 rip;
__u64 eflags; /* RFLAGS */
__u16 cs;
- __u16 gs;
- __u16 fs;
- __u16 __pad0;
+
+ /*
+ * Prior to 2.5.64 ("[PATCH] x86-64 updates for 2.5.64-bk3"),
+ * Linux saved and restored fs and gs in these slots. This
+ * was counterproductive, as fsbase and gsbase were never
+ * saved, so arch_prctl was presumably unreliable.
+ *
+ * If these slots are ever needed for any other purpose, there
+ * is some risk that very old 64-bit binaries could get
+ * confused. I doubt that many such binaries still work,
+ * though, since the same patch in 2.5.64 also removed the
+ * 64-bit set_thread_area syscall, so it appears that there is
+ * no TLS API that works in both pre- and post-2.5.64 kernels.
+ */
+ __u16 __pad2; /* Was gs. */
+ __u16 __pad1; /* Was fs. */
+
+ __u16 ss;
__u64 err;
__u64 trapno;
__u64 oldmask;
#define EXIT_REASON_EPT_VIOLATION 48
#define EXIT_REASON_EPT_MISCONFIG 49
#define EXIT_REASON_INVEPT 50
+#define EXIT_REASON_RDTSCP 51
#define EXIT_REASON_PREEMPTION_TIMER 52
#define EXIT_REASON_INVVPID 53
#define EXIT_REASON_WBINVD 54
obj-$(CONFIG_X86_64) += sys_x86_64.o x8664_ksyms_64.o
obj-$(CONFIG_X86_64) += mcount_64.o
obj-y += syscall_$(BITS).o vsyscall_gtod.o
+obj-$(CONFIG_IA32_EMULATION) += syscall_32.o
obj-$(CONFIG_X86_VSYSCALL_EMULATION) += vsyscall_64.o vsyscall_emu_64.o
obj-$(CONFIG_X86_ESPFIX64) += espfix_64.o
obj-$(CONFIG_SYSFS) += ksysfs.o
return 0;
}
+/*
+ * ACPI offers an alternative platform interface model that removes
+ * ACPI hardware requirements for platforms that do not implement
+ * the PC Architecture.
+ *
+ * We initialize the Hardware-reduced ACPI model here:
+ */
+static void __init acpi_reduced_hw_init(void)
+{
+ if (acpi_gbl_reduced_hardware) {
+ /*
+ * Override x86_init functions and bypass legacy pic
+ * in Hardware-reduced ACPI mode
+ */
+ x86_init.timers.timer_init = x86_init_noop;
+ x86_init.irqs.pre_vector_init = x86_init_noop;
+ legacy_pic = &null_legacy_pic;
+ }
+}
+
/*
* If your system is blacklisted here, but you find that acpi=force
* works for you, please contact linux-acpi@vger.kernel.org
*/
early_acpi_process_madt();
+ /*
+ * Hardware-reduced ACPI mode initialization:
+ */
+ acpi_reduced_hw_init();
+
return 0;
}
__setup("noreplace-paravirt", setup_noreplace_paravirt);
#endif
-#define DPRINTK(fmt, ...) \
-do { \
- if (debug_alternative) \
- printk(KERN_DEBUG fmt, ##__VA_ARGS__); \
+#define DPRINTK(fmt, args...) \
+do { \
+ if (debug_alternative) \
+ printk(KERN_DEBUG "%s: " fmt "\n", __func__, ##args); \
+} while (0)
+
+#define DUMP_BYTES(buf, len, fmt, args...) \
+do { \
+ if (unlikely(debug_alternative)) { \
+ int j; \
+ \
+ if (!(len)) \
+ break; \
+ \
+ printk(KERN_DEBUG fmt, ##args); \
+ for (j = 0; j < (len) - 1; j++) \
+ printk(KERN_CONT "%02hhx ", buf[j]); \
+ printk(KERN_CONT "%02hhx\n", buf[j]); \
+ } \
} while (0)
/*
extern s32 __smp_locks[], __smp_locks_end[];
void *text_poke_early(void *addr, const void *opcode, size_t len);
-/* Replace instructions with better alternatives for this CPU type.
- This runs before SMP is initialized to avoid SMP problems with
- self modifying code. This implies that asymmetric systems where
- APs have less capabilities than the boot processor are not handled.
- Tough. Make sure you disable such features by hand. */
+/*
+ * Are we looking at a near JMP with a 1 or 4-byte displacement.
+ */
+static inline bool is_jmp(const u8 opcode)
+{
+ return opcode == 0xeb || opcode == 0xe9;
+}
+
+static void __init_or_module
+recompute_jump(struct alt_instr *a, u8 *orig_insn, u8 *repl_insn, u8 *insnbuf)
+{
+ u8 *next_rip, *tgt_rip;
+ s32 n_dspl, o_dspl;
+ int repl_len;
+
+ if (a->replacementlen != 5)
+ return;
+
+ o_dspl = *(s32 *)(insnbuf + 1);
+
+ /* next_rip of the replacement JMP */
+ next_rip = repl_insn + a->replacementlen;
+ /* target rip of the replacement JMP */
+ tgt_rip = next_rip + o_dspl;
+ n_dspl = tgt_rip - orig_insn;
+
+ DPRINTK("target RIP: %p, new_displ: 0x%x", tgt_rip, n_dspl);
+
+ if (tgt_rip - orig_insn >= 0) {
+ if (n_dspl - 2 <= 127)
+ goto two_byte_jmp;
+ else
+ goto five_byte_jmp;
+ /* negative offset */
+ } else {
+ if (((n_dspl - 2) & 0xff) == (n_dspl - 2))
+ goto two_byte_jmp;
+ else
+ goto five_byte_jmp;
+ }
+
+two_byte_jmp:
+ n_dspl -= 2;
+
+ insnbuf[0] = 0xeb;
+ insnbuf[1] = (s8)n_dspl;
+ add_nops(insnbuf + 2, 3);
+
+ repl_len = 2;
+ goto done;
+
+five_byte_jmp:
+ n_dspl -= 5;
+
+ insnbuf[0] = 0xe9;
+ *(s32 *)&insnbuf[1] = n_dspl;
+ repl_len = 5;
+
+done:
+
+ DPRINTK("final displ: 0x%08x, JMP 0x%lx",
+ n_dspl, (unsigned long)orig_insn + n_dspl + repl_len);
+}
+
+static void __init_or_module optimize_nops(struct alt_instr *a, u8 *instr)
+{
+ if (instr[0] != 0x90)
+ return;
+
+ add_nops(instr + (a->instrlen - a->padlen), a->padlen);
+
+ DUMP_BYTES(instr, a->instrlen, "%p: [%d:%d) optimized NOPs: ",
+ instr, a->instrlen - a->padlen, a->padlen);
+}
+
+/*
+ * Replace instructions with better alternatives for this CPU type. This runs
+ * before SMP is initialized to avoid SMP problems with self modifying code.
+ * This implies that asymmetric systems where APs have less capabilities than
+ * the boot processor are not handled. Tough. Make sure you disable such
+ * features by hand.
+ */
void __init_or_module apply_alternatives(struct alt_instr *start,
struct alt_instr *end)
{
u8 *instr, *replacement;
u8 insnbuf[MAX_PATCH_LEN];
- DPRINTK("%s: alt table %p -> %p\n", __func__, start, end);
+ DPRINTK("alt table %p -> %p", start, end);
/*
* The scan order should be from start to end. A later scanned
- * alternative code can overwrite a previous scanned alternative code.
+ * alternative code can overwrite previously scanned alternative code.
* Some kernel functions (e.g. memcpy, memset, etc) use this order to
* patch code.
*
* order.
*/
for (a = start; a < end; a++) {
+ int insnbuf_sz = 0;
+
instr = (u8 *)&a->instr_offset + a->instr_offset;
replacement = (u8 *)&a->repl_offset + a->repl_offset;
- BUG_ON(a->replacementlen > a->instrlen);
BUG_ON(a->instrlen > sizeof(insnbuf));
BUG_ON(a->cpuid >= (NCAPINTS + NBUGINTS) * 32);
- if (!boot_cpu_has(a->cpuid))
+ if (!boot_cpu_has(a->cpuid)) {
+ if (a->padlen > 1)
+ optimize_nops(a, instr);
+
continue;
+ }
+
+ DPRINTK("feat: %d*32+%d, old: (%p, len: %d), repl: (%p, len: %d), pad: %d",
+ a->cpuid >> 5,
+ a->cpuid & 0x1f,
+ instr, a->instrlen,
+ replacement, a->replacementlen, a->padlen);
+
+ DUMP_BYTES(instr, a->instrlen, "%p: old_insn: ", instr);
+ DUMP_BYTES(replacement, a->replacementlen, "%p: rpl_insn: ", replacement);
memcpy(insnbuf, replacement, a->replacementlen);
+ insnbuf_sz = a->replacementlen;
/* 0xe8 is a relative jump; fix the offset. */
- if (*insnbuf == 0xe8 && a->replacementlen == 5)
- *(s32 *)(insnbuf + 1) += replacement - instr;
+ if (*insnbuf == 0xe8 && a->replacementlen == 5) {
+ *(s32 *)(insnbuf + 1) += replacement - instr;
+ DPRINTK("Fix CALL offset: 0x%x, CALL 0x%lx",
+ *(s32 *)(insnbuf + 1),
+ (unsigned long)instr + *(s32 *)(insnbuf + 1) + 5);
+ }
+
+ if (a->replacementlen && is_jmp(replacement[0]))
+ recompute_jump(a, instr, replacement, insnbuf);
- add_nops(insnbuf + a->replacementlen,
- a->instrlen - a->replacementlen);
+ if (a->instrlen > a->replacementlen) {
+ add_nops(insnbuf + a->replacementlen,
+ a->instrlen - a->replacementlen);
+ insnbuf_sz += a->instrlen - a->replacementlen;
+ }
+ DUMP_BYTES(insnbuf, insnbuf_sz, "%p: final_insn: ", instr);
- text_poke_early(instr, insnbuf, a->instrlen);
+ text_poke_early(instr, insnbuf, insnbuf_sz);
}
}
#ifdef CONFIG_SMP
-
static void alternatives_smp_lock(const s32 *start, const s32 *end,
u8 *text, u8 *text_end)
{
smp->locks_end = locks_end;
smp->text = text;
smp->text_end = text_end;
- DPRINTK("%s: locks %p -> %p, text %p -> %p, name %s\n",
- __func__, smp->locks, smp->locks_end,
+ DPRINTK("locks %p -> %p, text %p -> %p, name %s\n",
+ smp->locks, smp->locks_end,
smp->text, smp->text_end, smp->name);
list_add_tail(&smp->next, &smp_alt_modules);
return 0;
}
-#endif
+#endif /* CONFIG_SMP */
#ifdef CONFIG_PARAVIRT
void __init_or_module apply_paravirt(struct paravirt_patch_site *start,
if (likely(!bp_patching_in_progress))
return 0;
- if (user_mode_vm(regs) || regs->ip != (unsigned long)bp_int3_addr)
+ if (user_mode(regs) || regs->ip != (unsigned long)bp_int3_addr)
return 0;
/* set up the specified breakpoint handler */
local_irq_restore(flags);
}
-/*
- * This is to verify that we're looking at a real local APIC.
- * Check these against your board if the CPUs aren't getting
- * started for no apparent reason.
- */
-int __init verify_local_APIC(void)
-{
- unsigned int reg0, reg1;
-
- /*
- * The version register is read-only in a real APIC.
- */
- reg0 = apic_read(APIC_LVR);
- apic_printk(APIC_DEBUG, "Getting VERSION: %x\n", reg0);
- apic_write(APIC_LVR, reg0 ^ APIC_LVR_MASK);
- reg1 = apic_read(APIC_LVR);
- apic_printk(APIC_DEBUG, "Getting VERSION: %x\n", reg1);
-
- /*
- * The two version reads above should print the same
- * numbers. If the second one is different, then we
- * poke at a non-APIC.
- */
- if (reg1 != reg0)
- return 0;
-
- /*
- * Check if the version looks reasonably.
- */
- reg1 = GET_APIC_VERSION(reg0);
- if (reg1 == 0x00 || reg1 == 0xff)
- return 0;
- reg1 = lapic_get_maxlvt();
- if (reg1 < 0x02 || reg1 == 0xff)
- return 0;
-
- /*
- * The ID register is read/write in a real APIC.
- */
- reg0 = apic_read(APIC_ID);
- apic_printk(APIC_DEBUG, "Getting ID: %x\n", reg0);
- apic_write(APIC_ID, reg0 ^ apic->apic_id_mask);
- reg1 = apic_read(APIC_ID);
- apic_printk(APIC_DEBUG, "Getting ID: %x\n", reg1);
- apic_write(APIC_ID, reg0);
- if (reg1 != (reg0 ^ apic->apic_id_mask))
- return 0;
-
- /*
- * The next two are just to see if we have sane values.
- * They're only really relevant if we're in Virtual Wire
- * compatibility mode, but most boxes are anymore.
- */
- reg0 = apic_read(APIC_LVT0);
- apic_printk(APIC_DEBUG, "Getting LVT0: %x\n", reg0);
- reg1 = apic_read(APIC_LVT1);
- apic_printk(APIC_DEBUG, "Getting LVT1: %x\n", reg1);
-
- return 1;
-}
-
/**
* sync_Arb_IDs - synchronize APIC bus arbitration IDs
*/
disable_ioapic_support();
default_setup_apic_routing();
- verify_local_APIC();
apic_bsp_setup(true);
return 0;
}
static unsigned int get_apic_id(unsigned long x)
{
unsigned long value;
- unsigned int id;
+ unsigned int id = (x >> 24) & 0xff;
- rdmsrl(MSR_FAM10H_NODE_ID, value);
- id = ((x >> 24) & 0xffU) | ((value << 2) & 0xff00U);
+ if (static_cpu_has_safe(X86_FEATURE_NODEID_MSR)) {
+ rdmsrl(MSR_FAM10H_NODE_ID, value);
+ id |= (value << 2) & 0xff00;
+ }
return id;
}
static void fixup_cpu_id(struct cpuinfo_x86 *c, int node)
{
- if (c->phys_proc_id != node) {
- c->phys_proc_id = node;
- per_cpu(cpu_llc_id, smp_processor_id()) = node;
+ u64 val;
+ u32 nodes = 1;
+
+ this_cpu_write(cpu_llc_id, node);
+
+ /* Account for nodes per socket in multi-core-module processors */
+ if (static_cpu_has_safe(X86_FEATURE_NODEID_MSR)) {
+ rdmsrl(MSR_FAM10H_NODE_ID, val);
+ nodes = ((val >> 3) & 7) + 1;
}
+
+ c->phys_proc_id = node / nodes;
}
static int __init numachip_system_init(void)
per_cpu(x86_cpu_to_logical_apicid, this_cpu) = apic_read(APIC_LDR);
- __cpu_set(this_cpu, per_cpu(cpus_in_cluster, this_cpu));
+ cpumask_set_cpu(this_cpu, per_cpu(cpus_in_cluster, this_cpu));
for_each_online_cpu(cpu) {
if (x2apic_cluster(this_cpu) != x2apic_cluster(cpu))
continue;
- __cpu_set(this_cpu, per_cpu(cpus_in_cluster, cpu));
- __cpu_set(cpu, per_cpu(cpus_in_cluster, this_cpu));
+ cpumask_set_cpu(this_cpu, per_cpu(cpus_in_cluster, cpu));
+ cpumask_set_cpu(cpu, per_cpu(cpus_in_cluster, this_cpu));
}
}
BUG_ON(!per_cpu(cpus_in_cluster, cpu) || !per_cpu(ipi_mask, cpu));
- __cpu_set(cpu, per_cpu(cpus_in_cluster, cpu));
+ cpumask_set_cpu(cpu, per_cpu(cpus_in_cluster, cpu));
register_hotcpu_notifier(&x2apic_cpu_notifier);
return 1;
}
static int __init uv_acpi_madt_oem_check(char *oem_id, char *oem_table_id)
{
- int pnodeid, is_uv1, is_uv2, is_uv3;
-
- is_uv1 = !strcmp(oem_id, "SGI");
- is_uv2 = !strcmp(oem_id, "SGI2");
- is_uv3 = !strncmp(oem_id, "SGI3", 4); /* there are varieties of UV3 */
- if (is_uv1 || is_uv2 || is_uv3) {
- uv_hub_info->hub_revision =
- (is_uv1 ? UV1_HUB_REVISION_BASE :
- (is_uv2 ? UV2_HUB_REVISION_BASE :
- UV3_HUB_REVISION_BASE));
- pnodeid = early_get_pnodeid();
- early_get_apic_pnode_shift();
- x86_platform.is_untracked_pat_range = uv_is_untracked_pat_range;
- x86_platform.nmi_init = uv_nmi_init;
- if (!strcmp(oem_table_id, "UVL"))
- uv_system_type = UV_LEGACY_APIC;
- else if (!strcmp(oem_table_id, "UVX"))
- uv_system_type = UV_X2APIC;
- else if (!strcmp(oem_table_id, "UVH")) {
- __this_cpu_write(x2apic_extra_bits,
- pnodeid << uvh_apicid.s.pnode_shift);
- uv_system_type = UV_NON_UNIQUE_APIC;
- uv_set_apicid_hibit();
- return 1;
- }
+ int pnodeid;
+ int uv_apic;
+
+ if (strncmp(oem_id, "SGI", 3) != 0)
+ return 0;
+
+ /*
+ * Determine UV arch type.
+ * SGI: UV100/1000
+ * SGI2: UV2000/3000
+ * SGI3: UV300 (truncated to 4 chars because of different varieties)
+ */
+ uv_hub_info->hub_revision =
+ !strncmp(oem_id, "SGI3", 4) ? UV3_HUB_REVISION_BASE :
+ !strcmp(oem_id, "SGI2") ? UV2_HUB_REVISION_BASE :
+ !strcmp(oem_id, "SGI") ? UV1_HUB_REVISION_BASE : 0;
+
+ if (uv_hub_info->hub_revision == 0)
+ goto badbios;
+
+ pnodeid = early_get_pnodeid();
+ early_get_apic_pnode_shift();
+ x86_platform.is_untracked_pat_range = uv_is_untracked_pat_range;
+ x86_platform.nmi_init = uv_nmi_init;
+
+ if (!strcmp(oem_table_id, "UVX")) { /* most common */
+ uv_system_type = UV_X2APIC;
+ uv_apic = 0;
+
+ } else if (!strcmp(oem_table_id, "UVH")) { /* only UV1 systems */
+ uv_system_type = UV_NON_UNIQUE_APIC;
+ __this_cpu_write(x2apic_extra_bits,
+ pnodeid << uvh_apicid.s.pnode_shift);
+ uv_set_apicid_hibit();
+ uv_apic = 1;
+
+ } else if (!strcmp(oem_table_id, "UVL")) { /* only used for */
+ uv_system_type = UV_LEGACY_APIC; /* very small systems */
+ uv_apic = 0;
+
+ } else {
+ goto badbios;
}
- return 0;
+
+ pr_info("UV: OEM IDs %s/%s, System/HUB Types %d/%d, uv_apic %d\n",
+ oem_id, oem_table_id, uv_system_type,
+ uv_min_hub_revision_id, uv_apic);
+
+ return uv_apic;
+
+badbios:
+ pr_err("UV: OEM_ID:%s OEM_TABLE_ID:%s\n", oem_id, oem_table_id);
+ pr_err("Current BIOS not supported, update kernel and/or BIOS\n");
+ BUG();
}
enum uv_system_type get_uv_system_type(void)
unsigned long mmr_base, present, paddr;
unsigned short pnode_mask;
unsigned char n_lshift;
- char *hub = (is_uv1_hub() ? "UV1" :
- (is_uv2_hub() ? "UV2" :
- "UV3"));
+ char *hub = (is_uv1_hub() ? "UV100/1000" :
+ (is_uv2_hub() ? "UV2000/3000" :
+ (is_uv3_hub() ? "UV300" : NULL)));
+ if (!hub) {
+ pr_err("UV: Unknown/unsupported UV hub\n");
+ return;
+ }
pr_info("UV: Found %s hub\n", hub);
map_low_mmrs();
/* Offset from the sysenter stack to tss.sp0 */
DEFINE(TSS_sysenter_sp0, offsetof(struct tss_struct, x86_tss.sp0) -
- sizeof(struct tss_struct));
+ offsetofend(struct tss_struct, SYSENTER_stack));
#if defined(CONFIG_LGUEST) || defined(CONFIG_LGUEST_GUEST) || defined(CONFIG_LGUEST_MODULE)
BLANK();
#undef ENTRY
OFFSET(TSS_ist, tss_struct, x86_tss.ist);
+ OFFSET(TSS_sp0, tss_struct, x86_tss.sp0);
BLANK();
DEFINE(__NR_syscall_max, sizeof(syscalls_64) - 1);
#include <linux/io.h>
#include <linux/sched.h>
+#include <linux/random.h>
#include <asm/processor.h>
#include <asm/apic.h>
#include <asm/cpu.h>
va_align.mask = (upperbit - 1) & PAGE_MASK;
va_align.flags = ALIGN_VA_32 | ALIGN_VA_64;
+
+ /* A random value per boot for bit slice [12:upper_bit) */
+ va_align.bits = get_random_int() & va_align.mask;
}
}
set_cpu_bug(c, X86_BUG_AMD_APIC_C1E);
rdmsr_safe(MSR_AMD64_PATCH_LEVEL, &c->microcode, &dummy);
+
+ /* 3DNow or LM implies PREFETCHW */
+ if (!cpu_has(c, X86_FEATURE_3DNOWPREFETCH))
+ if (cpu_has(c, X86_FEATURE_3DNOW) || cpu_has(c, X86_FEATURE_LM))
+ set_cpu_cap(c, X86_FEATURE_3DNOWPREFETCH);
}
#ifdef CONFIG_X86_32
#endif
}
-#ifdef CONFIG_X86_64
-#ifdef CONFIG_IA32_EMULATION
-/* May not be __init: called during resume */
-static void syscall32_cpu_init(void)
-{
- /* Load these always in case some future AMD CPU supports
- SYSENTER from compat mode too. */
- wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)__KERNEL_CS);
- wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL);
- wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)ia32_sysenter_target);
-
- wrmsrl(MSR_CSTAR, ia32_cstar_target);
-}
-#endif /* CONFIG_IA32_EMULATION */
-#endif /* CONFIG_X86_64 */
-
+/*
+ * Set up the CPU state needed to execute SYSENTER/SYSEXIT instructions
+ * on 32-bit kernels:
+ */
#ifdef CONFIG_X86_32
void enable_sep_cpu(void)
{
- int cpu = get_cpu();
- struct tss_struct *tss = &per_cpu(init_tss, cpu);
+ struct tss_struct *tss;
+ int cpu;
- if (!boot_cpu_has(X86_FEATURE_SEP)) {
- put_cpu();
- return;
- }
+ cpu = get_cpu();
+ tss = &per_cpu(cpu_tss, cpu);
+
+ if (!boot_cpu_has(X86_FEATURE_SEP))
+ goto out;
+
+ /*
+ * We cache MSR_IA32_SYSENTER_CS's value in the TSS's ss1 field --
+ * see the big comment in struct x86_hw_tss's definition.
+ */
tss->x86_tss.ss1 = __KERNEL_CS;
- tss->x86_tss.sp1 = sizeof(struct tss_struct) + (unsigned long) tss;
- wrmsr(MSR_IA32_SYSENTER_CS, __KERNEL_CS, 0);
- wrmsr(MSR_IA32_SYSENTER_ESP, tss->x86_tss.sp1, 0);
- wrmsr(MSR_IA32_SYSENTER_EIP, (unsigned long) ia32_sysenter_target, 0);
+ wrmsr(MSR_IA32_SYSENTER_CS, tss->x86_tss.ss1, 0);
+
+ wrmsr(MSR_IA32_SYSENTER_ESP,
+ (unsigned long)tss + offsetofend(struct tss_struct, SYSENTER_stack),
+ 0);
+
+ wrmsr(MSR_IA32_SYSENTER_EIP, (unsigned long)ia32_sysenter_target, 0);
+
+out:
put_cpu();
}
#endif
__setup("clearcpuid=", setup_disablecpuid);
DEFINE_PER_CPU(unsigned long, kernel_stack) =
- (unsigned long)&init_thread_union - KERNEL_STACK_OFFSET + THREAD_SIZE;
+ (unsigned long)&init_thread_union + THREAD_SIZE;
EXPORT_PER_CPU_SYMBOL(kernel_stack);
#ifdef CONFIG_X86_64
irq_stack_union) __aligned(PAGE_SIZE) __visible;
/*
- * The following four percpu variables are hot. Align current_task to
- * cacheline size such that all four fall in the same cacheline.
+ * The following percpu variables are hot. Align current_task to
+ * cacheline size such that they fall in the same cacheline.
*/
DEFINE_PER_CPU(struct task_struct *, current_task) ____cacheline_aligned =
&init_task;
*/
wrmsrl(MSR_STAR, ((u64)__USER32_CS)<<48 | ((u64)__KERNEL_CS)<<32);
wrmsrl(MSR_LSTAR, system_call);
- wrmsrl(MSR_CSTAR, ignore_sysret);
#ifdef CONFIG_IA32_EMULATION
- syscall32_cpu_init();
+ wrmsrl(MSR_CSTAR, ia32_cstar_target);
+ /*
+ * This only works on Intel CPUs.
+ * On AMD CPUs these MSRs are 32-bit, CPU truncates MSR_IA32_SYSENTER_EIP.
+ * This does not cause SYSENTER to jump to the wrong location, because
+ * AMD doesn't allow SYSENTER in long mode (either 32- or 64-bit).
+ */
+ wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)__KERNEL_CS);
+ wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL);
+ wrmsrl_safe(MSR_IA32_SYSENTER_EIP, (u64)ia32_sysenter_target);
+#else
+ wrmsrl(MSR_CSTAR, ignore_sysret);
+ wrmsrl_safe(MSR_IA32_SYSENTER_CS, (u64)GDT_ENTRY_INVALID_SEG);
+ wrmsrl_safe(MSR_IA32_SYSENTER_ESP, 0ULL);
+ wrmsrl_safe(MSR_IA32_SYSENTER_EIP, 0ULL);
#endif
/* Flags to clear on syscall */
EXPORT_PER_CPU_SYMBOL(__preempt_count);
DEFINE_PER_CPU(struct task_struct *, fpu_owner_task);
+/*
+ * On x86_32, vm86 modifies tss.sp0, so sp0 isn't a reliable way to find
+ * the top of the kernel stack. Use an extra percpu variable to track the
+ * top of the kernel stack directly.
+ */
+DEFINE_PER_CPU(unsigned long, cpu_current_top_of_stack) =
+ (unsigned long)&init_thread_union + THREAD_SIZE;
+EXPORT_PER_CPU_SYMBOL(cpu_current_top_of_stack);
+
#ifdef CONFIG_CC_STACKPROTECTOR
DEFINE_PER_CPU_ALIGNED(struct stack_canary, stack_canary);
#endif
*/
load_ucode_ap();
- t = &per_cpu(init_tss, cpu);
+ t = &per_cpu(cpu_tss, cpu);
oist = &per_cpu(orig_ist, cpu);
#ifdef CONFIG_NUMA
{
int cpu = smp_processor_id();
struct task_struct *curr = current;
- struct tss_struct *t = &per_cpu(init_tss, cpu);
+ struct tss_struct *t = &per_cpu(cpu_tss, cpu);
struct thread_struct *thread = &curr->thread;
wait_for_master_cpu(cpu);
+ /*
+ * Initialize the CR4 shadow before doing anything that could
+ * try to read it.
+ */
+ cr4_init_shadow();
+
show_ucode_info_early();
printk(KERN_INFO "Initializing CPU#%d\n", cpu);
{ 0xb2, TLB_INST_4K, 64, " TLB_INST 4KByte pages, 4-way set associative" },
{ 0xb3, TLB_DATA_4K, 128, " TLB_DATA 4 KByte pages, 4-way set associative" },
{ 0xb4, TLB_DATA_4K, 256, " TLB_DATA 4 KByte pages, 4-way associative" },
- { 0xb5, TLB_INST_4K, 64, " TLB_INST 4 KByte pages, 8-way set ssociative" },
- { 0xb6, TLB_INST_4K, 128, " TLB_INST 4 KByte pages, 8-way set ssociative" },
+ { 0xb5, TLB_INST_4K, 64, " TLB_INST 4 KByte pages, 8-way set associative" },
+ { 0xb6, TLB_INST_4K, 128, " TLB_INST 4 KByte pages, 8-way set associative" },
{ 0xba, TLB_DATA_4K, 64, " TLB_DATA 4 KByte pages, 4-way associative" },
{ 0xc0, TLB_DATA_4K_4M, 8, " TLB_DATA 4 KByte and 4 MByte pages, 4-way associative" },
{ 0xc1, STLB_4K_2M, 1024, " STLB 4 KByte and 2 MByte pages, 8-way associative" },
* Andi Kleen / Andreas Herrmann : CPUID4 emulation on AMD.
*/
-#include <linux/init.h>
#include <linux/slab.h>
-#include <linux/device.h>
-#include <linux/compiler.h>
+#include <linux/cacheinfo.h>
#include <linux/cpu.h>
#include <linux/sched.h>
+#include <linux/sysfs.h>
#include <linux/pci.h>
#include <asm/processor.h>
-#include <linux/smp.h>
#include <asm/amd_nb.h>
#include <asm/smp.h>
enum _cache_type {
- CACHE_TYPE_NULL = 0,
- CACHE_TYPE_DATA = 1,
- CACHE_TYPE_INST = 2,
- CACHE_TYPE_UNIFIED = 3
+ CTYPE_NULL = 0,
+ CTYPE_DATA = 1,
+ CTYPE_INST = 2,
+ CTYPE_UNIFIED = 3
};
union _cpuid4_leaf_eax {
struct amd_northbridge *nb;
};
-struct _cpuid4_info {
- struct _cpuid4_info_regs base;
- DECLARE_BITMAP(shared_cpu_map, NR_CPUS);
-};
-
unsigned short num_cache_leaves;
/* AMD doesn't have CPUID4. Emulate it here to report the same
static const unsigned char levels[] = { 1, 1, 2, 3 };
static const unsigned char types[] = { 1, 2, 3, 3 };
+static const enum cache_type cache_type_map[] = {
+ [CTYPE_NULL] = CACHE_TYPE_NOCACHE,
+ [CTYPE_DATA] = CACHE_TYPE_DATA,
+ [CTYPE_INST] = CACHE_TYPE_INST,
+ [CTYPE_UNIFIED] = CACHE_TYPE_UNIFIED,
+};
+
static void
amd_cpuid4(int leaf, union _cpuid4_leaf_eax *eax,
union _cpuid4_leaf_ebx *ebx,
(ebx->split.ways_of_associativity + 1) - 1;
}
-struct _cache_attr {
- struct attribute attr;
- ssize_t (*show)(struct _cpuid4_info *, char *, unsigned int);
- ssize_t (*store)(struct _cpuid4_info *, const char *, size_t count,
- unsigned int);
-};
-
#if defined(CONFIG_AMD_NB) && defined(CONFIG_SYSFS)
+
/*
* L3 cache descriptors
*/
l3->indices = (max(max3(sc0, sc1, sc2), sc3) << 10) - 1;
}
-static void amd_init_l3_cache(struct _cpuid4_info_regs *this_leaf, int index)
-{
- int node;
-
- /* only for L3, and not in virtualized environments */
- if (index < 3)
- return;
-
- node = amd_get_nb_id(smp_processor_id());
- this_leaf->nb = node_to_amd_nb(node);
- if (this_leaf->nb && !this_leaf->nb->l3_cache.indices)
- amd_calc_l3_indices(this_leaf->nb);
-}
-
/*
* check whether a slot used for disabling an L3 index is occupied.
* @l3: L3 cache descriptor
return -1;
}
-static ssize_t show_cache_disable(struct _cpuid4_info *this_leaf, char *buf,
+static ssize_t show_cache_disable(struct cacheinfo *this_leaf, char *buf,
unsigned int slot)
{
int index;
+ struct amd_northbridge *nb = this_leaf->priv;
- if (!this_leaf->base.nb || !amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE))
- return -EINVAL;
-
- index = amd_get_l3_disable_slot(this_leaf->base.nb, slot);
+ index = amd_get_l3_disable_slot(nb, slot);
if (index >= 0)
return sprintf(buf, "%d\n", index);
#define SHOW_CACHE_DISABLE(slot) \
static ssize_t \
-show_cache_disable_##slot(struct _cpuid4_info *this_leaf, char *buf, \
- unsigned int cpu) \
+cache_disable_##slot##_show(struct device *dev, \
+ struct device_attribute *attr, char *buf) \
{ \
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev); \
return show_cache_disable(this_leaf, buf, slot); \
}
SHOW_CACHE_DISABLE(0)
return 0;
}
-static ssize_t store_cache_disable(struct _cpuid4_info *this_leaf,
- const char *buf, size_t count,
- unsigned int slot)
+static ssize_t store_cache_disable(struct cacheinfo *this_leaf,
+ const char *buf, size_t count,
+ unsigned int slot)
{
unsigned long val = 0;
int cpu, err = 0;
+ struct amd_northbridge *nb = this_leaf->priv;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- if (!this_leaf->base.nb || !amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE))
- return -EINVAL;
-
- cpu = cpumask_first(to_cpumask(this_leaf->shared_cpu_map));
+ cpu = cpumask_first(&this_leaf->shared_cpu_map);
if (kstrtoul(buf, 10, &val) < 0)
return -EINVAL;
- err = amd_set_l3_disable_slot(this_leaf->base.nb, cpu, slot, val);
+ err = amd_set_l3_disable_slot(nb, cpu, slot, val);
if (err) {
if (err == -EEXIST)
pr_warning("L3 slot %d in use/index already disabled!\n",
#define STORE_CACHE_DISABLE(slot) \
static ssize_t \
-store_cache_disable_##slot(struct _cpuid4_info *this_leaf, \
- const char *buf, size_t count, \
- unsigned int cpu) \
+cache_disable_##slot##_store(struct device *dev, \
+ struct device_attribute *attr, \
+ const char *buf, size_t count) \
{ \
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev); \
return store_cache_disable(this_leaf, buf, count, slot); \
}
STORE_CACHE_DISABLE(0)
STORE_CACHE_DISABLE(1)
-static struct _cache_attr cache_disable_0 = __ATTR(cache_disable_0, 0644,
- show_cache_disable_0, store_cache_disable_0);
-static struct _cache_attr cache_disable_1 = __ATTR(cache_disable_1, 0644,
- show_cache_disable_1, store_cache_disable_1);
-
-static ssize_t
-show_subcaches(struct _cpuid4_info *this_leaf, char *buf, unsigned int cpu)
+static ssize_t subcaches_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
{
- if (!this_leaf->base.nb || !amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
- return -EINVAL;
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev);
+ int cpu = cpumask_first(&this_leaf->shared_cpu_map);
return sprintf(buf, "%x\n", amd_get_subcaches(cpu));
}
-static ssize_t
-store_subcaches(struct _cpuid4_info *this_leaf, const char *buf, size_t count,
- unsigned int cpu)
+static ssize_t subcaches_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
{
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev);
+ int cpu = cpumask_first(&this_leaf->shared_cpu_map);
unsigned long val;
if (!capable(CAP_SYS_ADMIN))
return -EPERM;
- if (!this_leaf->base.nb || !amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
- return -EINVAL;
-
if (kstrtoul(buf, 16, &val) < 0)
return -EINVAL;
return count;
}
-static struct _cache_attr subcaches =
- __ATTR(subcaches, 0644, show_subcaches, store_subcaches);
+static DEVICE_ATTR_RW(cache_disable_0);
+static DEVICE_ATTR_RW(cache_disable_1);
+static DEVICE_ATTR_RW(subcaches);
+
+static umode_t
+cache_private_attrs_is_visible(struct kobject *kobj,
+ struct attribute *attr, int unused)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct cacheinfo *this_leaf = dev_get_drvdata(dev);
+ umode_t mode = attr->mode;
+
+ if (!this_leaf->priv)
+ return 0;
+
+ if ((attr == &dev_attr_subcaches.attr) &&
+ amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
+ return mode;
+
+ if ((attr == &dev_attr_cache_disable_0.attr ||
+ attr == &dev_attr_cache_disable_1.attr) &&
+ amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE))
+ return mode;
+
+ return 0;
+}
+
+static struct attribute_group cache_private_group = {
+ .is_visible = cache_private_attrs_is_visible,
+};
+
+static void init_amd_l3_attrs(void)
+{
+ int n = 1;
+ static struct attribute **amd_l3_attrs;
+
+ if (amd_l3_attrs) /* already initialized */
+ return;
+
+ if (amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE))
+ n += 2;
+ if (amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
+ n += 1;
+
+ amd_l3_attrs = kcalloc(n, sizeof(*amd_l3_attrs), GFP_KERNEL);
+ if (!amd_l3_attrs)
+ return;
+
+ n = 0;
+ if (amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE)) {
+ amd_l3_attrs[n++] = &dev_attr_cache_disable_0.attr;
+ amd_l3_attrs[n++] = &dev_attr_cache_disable_1.attr;
+ }
+ if (amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
+ amd_l3_attrs[n++] = &dev_attr_subcaches.attr;
+ cache_private_group.attrs = amd_l3_attrs;
+}
+
+const struct attribute_group *
+cache_get_priv_group(struct cacheinfo *this_leaf)
+{
+ struct amd_northbridge *nb = this_leaf->priv;
+
+ if (this_leaf->level < 3 || !nb)
+ return NULL;
+
+ if (nb && nb->l3_cache.indices)
+ init_amd_l3_attrs();
+
+ return &cache_private_group;
+}
+
+static void amd_init_l3_cache(struct _cpuid4_info_regs *this_leaf, int index)
+{
+ int node;
+
+ /* only for L3, and not in virtualized environments */
+ if (index < 3)
+ return;
+
+ node = amd_get_nb_id(smp_processor_id());
+ this_leaf->nb = node_to_amd_nb(node);
+ if (this_leaf->nb && !this_leaf->nb->l3_cache.indices)
+ amd_calc_l3_indices(this_leaf->nb);
+}
#else
#define amd_init_l3_cache(x, y)
#endif /* CONFIG_AMD_NB && CONFIG_SYSFS */
cpuid_count(4, index, &eax.full, &ebx.full, &ecx.full, &edx);
}
- if (eax.split.type == CACHE_TYPE_NULL)
+ if (eax.split.type == CTYPE_NULL)
return -EIO; /* better error ? */
this_leaf->eax = eax;
/* Do cpuid(op) loop to find out num_cache_leaves */
cpuid_count(op, i, &eax, &ebx, &ecx, &edx);
cache_eax.full = eax;
- } while (cache_eax.split.type != CACHE_TYPE_NULL);
+ } while (cache_eax.split.type != CTYPE_NULL);
return i;
}
switch (this_leaf.eax.split.level) {
case 1:
- if (this_leaf.eax.split.type == CACHE_TYPE_DATA)
+ if (this_leaf.eax.split.type == CTYPE_DATA)
new_l1d = this_leaf.size/1024;
- else if (this_leaf.eax.split.type == CACHE_TYPE_INST)
+ else if (this_leaf.eax.split.type == CTYPE_INST)
new_l1i = this_leaf.size/1024;
break;
case 2:
return l2;
}
-#ifdef CONFIG_SYSFS
-
-/* pointer to _cpuid4_info array (for each cache leaf) */
-static DEFINE_PER_CPU(struct _cpuid4_info *, ici_cpuid4_info);
-#define CPUID4_INFO_IDX(x, y) (&((per_cpu(ici_cpuid4_info, x))[y]))
-
-#ifdef CONFIG_SMP
-
-static int cache_shared_amd_cpu_map_setup(unsigned int cpu, int index)
+static int __cache_amd_cpumap_setup(unsigned int cpu, int index,
+ struct _cpuid4_info_regs *base)
{
- struct _cpuid4_info *this_leaf;
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ struct cacheinfo *this_leaf;
int i, sibling;
if (cpu_has_topoext) {
unsigned int apicid, nshared, first, last;
- if (!per_cpu(ici_cpuid4_info, cpu))
- return 0;
-
- this_leaf = CPUID4_INFO_IDX(cpu, index);
- nshared = this_leaf->base.eax.split.num_threads_sharing + 1;
+ this_leaf = this_cpu_ci->info_list + index;
+ nshared = base->eax.split.num_threads_sharing + 1;
apicid = cpu_data(cpu).apicid;
first = apicid - (apicid % nshared);
last = first + nshared - 1;
for_each_online_cpu(i) {
+ this_cpu_ci = get_cpu_cacheinfo(i);
+ if (!this_cpu_ci->info_list)
+ continue;
+
apicid = cpu_data(i).apicid;
if ((apicid < first) || (apicid > last))
continue;
- if (!per_cpu(ici_cpuid4_info, i))
- continue;
- this_leaf = CPUID4_INFO_IDX(i, index);
+
+ this_leaf = this_cpu_ci->info_list + index;
for_each_online_cpu(sibling) {
apicid = cpu_data(sibling).apicid;
if ((apicid < first) || (apicid > last))
continue;
- set_bit(sibling, this_leaf->shared_cpu_map);
+ cpumask_set_cpu(sibling,
+ &this_leaf->shared_cpu_map);
}
}
} else if (index == 3) {
for_each_cpu(i, cpu_llc_shared_mask(cpu)) {
- if (!per_cpu(ici_cpuid4_info, i))
+ this_cpu_ci = get_cpu_cacheinfo(i);
+ if (!this_cpu_ci->info_list)
continue;
- this_leaf = CPUID4_INFO_IDX(i, index);
+ this_leaf = this_cpu_ci->info_list + index;
for_each_cpu(sibling, cpu_llc_shared_mask(cpu)) {
if (!cpu_online(sibling))
continue;
- set_bit(sibling, this_leaf->shared_cpu_map);
+ cpumask_set_cpu(sibling,
+ &this_leaf->shared_cpu_map);
}
}
} else
return 1;
}
-static void cache_shared_cpu_map_setup(unsigned int cpu, int index)
+static void __cache_cpumap_setup(unsigned int cpu, int index,
+ struct _cpuid4_info_regs *base)
{
- struct _cpuid4_info *this_leaf, *sibling_leaf;
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ struct cacheinfo *this_leaf, *sibling_leaf;
unsigned long num_threads_sharing;
int index_msb, i;
struct cpuinfo_x86 *c = &cpu_data(cpu);
if (c->x86_vendor == X86_VENDOR_AMD) {
- if (cache_shared_amd_cpu_map_setup(cpu, index))
+ if (__cache_amd_cpumap_setup(cpu, index, base))
return;
}
- this_leaf = CPUID4_INFO_IDX(cpu, index);
- num_threads_sharing = 1 + this_leaf->base.eax.split.num_threads_sharing;
+ this_leaf = this_cpu_ci->info_list + index;
+ num_threads_sharing = 1 + base->eax.split.num_threads_sharing;
+ cpumask_set_cpu(cpu, &this_leaf->shared_cpu_map);
if (num_threads_sharing == 1)
- cpumask_set_cpu(cpu, to_cpumask(this_leaf->shared_cpu_map));
- else {
- index_msb = get_count_order(num_threads_sharing);
-
- for_each_online_cpu(i) {
- if (cpu_data(i).apicid >> index_msb ==
- c->apicid >> index_msb) {
- cpumask_set_cpu(i,
- to_cpumask(this_leaf->shared_cpu_map));
- if (i != cpu && per_cpu(ici_cpuid4_info, i)) {
- sibling_leaf =
- CPUID4_INFO_IDX(i, index);
- cpumask_set_cpu(cpu, to_cpumask(
- sibling_leaf->shared_cpu_map));
- }
- }
- }
- }
-}
-static void cache_remove_shared_cpu_map(unsigned int cpu, int index)
-{
- struct _cpuid4_info *this_leaf, *sibling_leaf;
- int sibling;
-
- this_leaf = CPUID4_INFO_IDX(cpu, index);
- for_each_cpu(sibling, to_cpumask(this_leaf->shared_cpu_map)) {
- sibling_leaf = CPUID4_INFO_IDX(sibling, index);
- cpumask_clear_cpu(cpu,
- to_cpumask(sibling_leaf->shared_cpu_map));
- }
-}
-#else
-static void cache_shared_cpu_map_setup(unsigned int cpu, int index)
-{
-}
-
-static void cache_remove_shared_cpu_map(unsigned int cpu, int index)
-{
-}
-#endif
-
-static void free_cache_attributes(unsigned int cpu)
-{
- int i;
-
- for (i = 0; i < num_cache_leaves; i++)
- cache_remove_shared_cpu_map(cpu, i);
-
- kfree(per_cpu(ici_cpuid4_info, cpu));
- per_cpu(ici_cpuid4_info, cpu) = NULL;
-}
-
-static void get_cpu_leaves(void *_retval)
-{
- int j, *retval = _retval, cpu = smp_processor_id();
+ return;
- /* Do cpuid and store the results */
- for (j = 0; j < num_cache_leaves; j++) {
- struct _cpuid4_info *this_leaf = CPUID4_INFO_IDX(cpu, j);
+ index_msb = get_count_order(num_threads_sharing);
- *retval = cpuid4_cache_lookup_regs(j, &this_leaf->base);
- if (unlikely(*retval < 0)) {
- int i;
+ for_each_online_cpu(i)
+ if (cpu_data(i).apicid >> index_msb == c->apicid >> index_msb) {
+ struct cpu_cacheinfo *sib_cpu_ci = get_cpu_cacheinfo(i);
- for (i = 0; i < j; i++)
- cache_remove_shared_cpu_map(cpu, i);
- break;
+ if (i == cpu || !sib_cpu_ci->info_list)
+ continue;/* skip if itself or no cacheinfo */
+ sibling_leaf = sib_cpu_ci->info_list + index;
+ cpumask_set_cpu(i, &this_leaf->shared_cpu_map);
+ cpumask_set_cpu(cpu, &sibling_leaf->shared_cpu_map);
}
- cache_shared_cpu_map_setup(cpu, j);
- }
}
-static int detect_cache_attributes(unsigned int cpu)
+static void ci_leaf_init(struct cacheinfo *this_leaf,
+ struct _cpuid4_info_regs *base)
{
- int retval;
-
- if (num_cache_leaves == 0)
- return -ENOENT;
-
- per_cpu(ici_cpuid4_info, cpu) = kzalloc(
- sizeof(struct _cpuid4_info) * num_cache_leaves, GFP_KERNEL);
- if (per_cpu(ici_cpuid4_info, cpu) == NULL)
- return -ENOMEM;
-
- smp_call_function_single(cpu, get_cpu_leaves, &retval, true);
- if (retval) {
- kfree(per_cpu(ici_cpuid4_info, cpu));
- per_cpu(ici_cpuid4_info, cpu) = NULL;
- }
-
- return retval;
+ this_leaf->level = base->eax.split.level;
+ this_leaf->type = cache_type_map[base->eax.split.type];
+ this_leaf->coherency_line_size =
+ base->ebx.split.coherency_line_size + 1;
+ this_leaf->ways_of_associativity =
+ base->ebx.split.ways_of_associativity + 1;
+ this_leaf->size = base->size;
+ this_leaf->number_of_sets = base->ecx.split.number_of_sets + 1;
+ this_leaf->physical_line_partition =
+ base->ebx.split.physical_line_partition + 1;
+ this_leaf->priv = base->nb;
}
-#include <linux/kobject.h>
-#include <linux/sysfs.h>
-#include <linux/cpu.h>
-
-/* pointer to kobject for cpuX/cache */
-static DEFINE_PER_CPU(struct kobject *, ici_cache_kobject);
-
-struct _index_kobject {
- struct kobject kobj;
- unsigned int cpu;
- unsigned short index;
-};
-
-/* pointer to array of kobjects for cpuX/cache/indexY */
-static DEFINE_PER_CPU(struct _index_kobject *, ici_index_kobject);
-#define INDEX_KOBJECT_PTR(x, y) (&((per_cpu(ici_index_kobject, x))[y]))
-
-#define show_one_plus(file_name, object, val) \
-static ssize_t show_##file_name(struct _cpuid4_info *this_leaf, char *buf, \
- unsigned int cpu) \
-{ \
- return sprintf(buf, "%lu\n", (unsigned long)this_leaf->object + val); \
-}
-
-show_one_plus(level, base.eax.split.level, 0);
-show_one_plus(coherency_line_size, base.ebx.split.coherency_line_size, 1);
-show_one_plus(physical_line_partition, base.ebx.split.physical_line_partition, 1);
-show_one_plus(ways_of_associativity, base.ebx.split.ways_of_associativity, 1);
-show_one_plus(number_of_sets, base.ecx.split.number_of_sets, 1);
-
-static ssize_t show_size(struct _cpuid4_info *this_leaf, char *buf,
- unsigned int cpu)
-{
- return sprintf(buf, "%luK\n", this_leaf->base.size / 1024);
-}
-
-static ssize_t show_shared_cpu_map_func(struct _cpuid4_info *this_leaf,
- int type, char *buf)
-{
- const struct cpumask *mask = to_cpumask(this_leaf->shared_cpu_map);
- int ret;
-
- if (type)
- ret = scnprintf(buf, PAGE_SIZE - 1, "%*pbl",
- cpumask_pr_args(mask));
- else
- ret = scnprintf(buf, PAGE_SIZE - 1, "%*pb",
- cpumask_pr_args(mask));
- buf[ret++] = '\n';
- buf[ret] = '\0';
- return ret;
-}
-
-static inline ssize_t show_shared_cpu_map(struct _cpuid4_info *leaf, char *buf,
- unsigned int cpu)
+static int __init_cache_level(unsigned int cpu)
{
- return show_shared_cpu_map_func(leaf, 0, buf);
-}
-
-static inline ssize_t show_shared_cpu_list(struct _cpuid4_info *leaf, char *buf,
- unsigned int cpu)
-{
- return show_shared_cpu_map_func(leaf, 1, buf);
-}
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
-static ssize_t show_type(struct _cpuid4_info *this_leaf, char *buf,
- unsigned int cpu)
-{
- switch (this_leaf->base.eax.split.type) {
- case CACHE_TYPE_DATA:
- return sprintf(buf, "Data\n");
- case CACHE_TYPE_INST:
- return sprintf(buf, "Instruction\n");
- case CACHE_TYPE_UNIFIED:
- return sprintf(buf, "Unified\n");
- default:
- return sprintf(buf, "Unknown\n");
- }
-}
-
-#define to_object(k) container_of(k, struct _index_kobject, kobj)
-#define to_attr(a) container_of(a, struct _cache_attr, attr)
-
-#define define_one_ro(_name) \
-static struct _cache_attr _name = \
- __ATTR(_name, 0444, show_##_name, NULL)
-
-define_one_ro(level);
-define_one_ro(type);
-define_one_ro(coherency_line_size);
-define_one_ro(physical_line_partition);
-define_one_ro(ways_of_associativity);
-define_one_ro(number_of_sets);
-define_one_ro(size);
-define_one_ro(shared_cpu_map);
-define_one_ro(shared_cpu_list);
-
-static struct attribute *default_attrs[] = {
- &type.attr,
- &level.attr,
- &coherency_line_size.attr,
- &physical_line_partition.attr,
- &ways_of_associativity.attr,
- &number_of_sets.attr,
- &size.attr,
- &shared_cpu_map.attr,
- &shared_cpu_list.attr,
- NULL
-};
-
-#ifdef CONFIG_AMD_NB
-static struct attribute **amd_l3_attrs(void)
-{
- static struct attribute **attrs;
- int n;
-
- if (attrs)
- return attrs;
-
- n = ARRAY_SIZE(default_attrs);
-
- if (amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE))
- n += 2;
-
- if (amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
- n += 1;
-
- attrs = kzalloc(n * sizeof (struct attribute *), GFP_KERNEL);
- if (attrs == NULL)
- return attrs = default_attrs;
-
- for (n = 0; default_attrs[n]; n++)
- attrs[n] = default_attrs[n];
-
- if (amd_nb_has_feature(AMD_NB_L3_INDEX_DISABLE)) {
- attrs[n++] = &cache_disable_0.attr;
- attrs[n++] = &cache_disable_1.attr;
- }
-
- if (amd_nb_has_feature(AMD_NB_L3_PARTITIONING))
- attrs[n++] = &subcaches.attr;
-
- return attrs;
-}
-#endif
-
-static ssize_t show(struct kobject *kobj, struct attribute *attr, char *buf)
-{
- struct _cache_attr *fattr = to_attr(attr);
- struct _index_kobject *this_leaf = to_object(kobj);
- ssize_t ret;
-
- ret = fattr->show ?
- fattr->show(CPUID4_INFO_IDX(this_leaf->cpu, this_leaf->index),
- buf, this_leaf->cpu) :
- 0;
- return ret;
-}
-
-static ssize_t store(struct kobject *kobj, struct attribute *attr,
- const char *buf, size_t count)
-{
- struct _cache_attr *fattr = to_attr(attr);
- struct _index_kobject *this_leaf = to_object(kobj);
- ssize_t ret;
-
- ret = fattr->store ?
- fattr->store(CPUID4_INFO_IDX(this_leaf->cpu, this_leaf->index),
- buf, count, this_leaf->cpu) :
- 0;
- return ret;
-}
-
-static const struct sysfs_ops sysfs_ops = {
- .show = show,
- .store = store,
-};
-
-static struct kobj_type ktype_cache = {
- .sysfs_ops = &sysfs_ops,
- .default_attrs = default_attrs,
-};
-
-static struct kobj_type ktype_percpu_entry = {
- .sysfs_ops = &sysfs_ops,
-};
-
-static void cpuid4_cache_sysfs_exit(unsigned int cpu)
-{
- kfree(per_cpu(ici_cache_kobject, cpu));
- kfree(per_cpu(ici_index_kobject, cpu));
- per_cpu(ici_cache_kobject, cpu) = NULL;
- per_cpu(ici_index_kobject, cpu) = NULL;
- free_cache_attributes(cpu);
-}
-
-static int cpuid4_cache_sysfs_init(unsigned int cpu)
-{
- int err;
-
- if (num_cache_leaves == 0)
+ if (!num_cache_leaves)
return -ENOENT;
-
- err = detect_cache_attributes(cpu);
- if (err)
- return err;
-
- /* Allocate all required memory */
- per_cpu(ici_cache_kobject, cpu) =
- kzalloc(sizeof(struct kobject), GFP_KERNEL);
- if (unlikely(per_cpu(ici_cache_kobject, cpu) == NULL))
- goto err_out;
-
- per_cpu(ici_index_kobject, cpu) = kzalloc(
- sizeof(struct _index_kobject) * num_cache_leaves, GFP_KERNEL);
- if (unlikely(per_cpu(ici_index_kobject, cpu) == NULL))
- goto err_out;
-
+ if (!this_cpu_ci)
+ return -EINVAL;
+ this_cpu_ci->num_levels = 3;
+ this_cpu_ci->num_leaves = num_cache_leaves;
return 0;
-
-err_out:
- cpuid4_cache_sysfs_exit(cpu);
- return -ENOMEM;
}
-static DECLARE_BITMAP(cache_dev_map, NR_CPUS);
-
-/* Add/Remove cache interface for CPU device */
-static int cache_add_dev(struct device *dev)
+static int __populate_cache_leaves(unsigned int cpu)
{
- unsigned int cpu = dev->id;
- unsigned long i, j;
- struct _index_kobject *this_object;
- struct _cpuid4_info *this_leaf;
- int retval;
-
- retval = cpuid4_cache_sysfs_init(cpu);
- if (unlikely(retval < 0))
- return retval;
-
- retval = kobject_init_and_add(per_cpu(ici_cache_kobject, cpu),
- &ktype_percpu_entry,
- &dev->kobj, "%s", "cache");
- if (retval < 0) {
- cpuid4_cache_sysfs_exit(cpu);
- return retval;
- }
+ unsigned int idx, ret;
+ struct cpu_cacheinfo *this_cpu_ci = get_cpu_cacheinfo(cpu);
+ struct cacheinfo *this_leaf = this_cpu_ci->info_list;
+ struct _cpuid4_info_regs id4_regs = {};
- for (i = 0; i < num_cache_leaves; i++) {
- this_object = INDEX_KOBJECT_PTR(cpu, i);
- this_object->cpu = cpu;
- this_object->index = i;
-
- this_leaf = CPUID4_INFO_IDX(cpu, i);
-
- ktype_cache.default_attrs = default_attrs;
-#ifdef CONFIG_AMD_NB
- if (this_leaf->base.nb)
- ktype_cache.default_attrs = amd_l3_attrs();
-#endif
- retval = kobject_init_and_add(&(this_object->kobj),
- &ktype_cache,
- per_cpu(ici_cache_kobject, cpu),
- "index%1lu", i);
- if (unlikely(retval)) {
- for (j = 0; j < i; j++)
- kobject_put(&(INDEX_KOBJECT_PTR(cpu, j)->kobj));
- kobject_put(per_cpu(ici_cache_kobject, cpu));
- cpuid4_cache_sysfs_exit(cpu);
- return retval;
- }
- kobject_uevent(&(this_object->kobj), KOBJ_ADD);
+ for (idx = 0; idx < this_cpu_ci->num_leaves; idx++) {
+ ret = cpuid4_cache_lookup_regs(idx, &id4_regs);
+ if (ret)
+ return ret;
+ ci_leaf_init(this_leaf++, &id4_regs);
+ __cache_cpumap_setup(cpu, idx, &id4_regs);
}
- cpumask_set_cpu(cpu, to_cpumask(cache_dev_map));
-
- kobject_uevent(per_cpu(ici_cache_kobject, cpu), KOBJ_ADD);
return 0;
}
-static void cache_remove_dev(struct device *dev)
-{
- unsigned int cpu = dev->id;
- unsigned long i;
-
- if (per_cpu(ici_cpuid4_info, cpu) == NULL)
- return;
- if (!cpumask_test_cpu(cpu, to_cpumask(cache_dev_map)))
- return;
- cpumask_clear_cpu(cpu, to_cpumask(cache_dev_map));
-
- for (i = 0; i < num_cache_leaves; i++)
- kobject_put(&(INDEX_KOBJECT_PTR(cpu, i)->kobj));
- kobject_put(per_cpu(ici_cache_kobject, cpu));
- cpuid4_cache_sysfs_exit(cpu);
-}
-
-static int cacheinfo_cpu_callback(struct notifier_block *nfb,
- unsigned long action, void *hcpu)
-{
- unsigned int cpu = (unsigned long)hcpu;
- struct device *dev;
-
- dev = get_cpu_device(cpu);
- switch (action) {
- case CPU_ONLINE:
- case CPU_ONLINE_FROZEN:
- cache_add_dev(dev);
- break;
- case CPU_DEAD:
- case CPU_DEAD_FROZEN:
- cache_remove_dev(dev);
- break;
- }
- return NOTIFY_OK;
-}
-
-static struct notifier_block cacheinfo_cpu_notifier = {
- .notifier_call = cacheinfo_cpu_callback,
-};
-
-static int __init cache_sysfs_init(void)
-{
- int i, err = 0;
-
- if (num_cache_leaves == 0)
- return 0;
-
- cpu_notifier_register_begin();
- for_each_online_cpu(i) {
- struct device *dev = get_cpu_device(i);
-
- err = cache_add_dev(dev);
- if (err)
- goto out;
- }
- __register_hotcpu_notifier(&cacheinfo_cpu_notifier);
-
-out:
- cpu_notifier_register_done();
- return err;
-}
-
-device_initcall(cache_sysfs_init);
-
-#endif
+DEFINE_SMP_CALL_CACHE_FUNCTION(init_cache_level)
+DEFINE_SMP_CALL_CACHE_FUNCTION(populate_cache_leaves)
};
#define ATTR_LEN 16
+#define INITIAL_CHECK_INTERVAL 5 * 60 /* 5 minutes */
/* One object for each MCE bank, shared by all CPUs */
struct mce_bank {
char attrname[ATTR_LEN]; /* attribute name */
};
-int mce_severity(struct mce *a, int tolerant, char **msg, bool is_excp);
+extern int (*mce_severity)(struct mce *a, int tolerant, char **msg, bool is_excp);
struct dentry *mce_get_debugfs_dir(void);
extern struct mce_bank *mce_banks;
extern mce_banks_t mce_banks_ce_disabled;
#ifdef CONFIG_X86_MCE_INTEL
-unsigned long mce_intel_adjust_timer(unsigned long interval);
-void mce_intel_cmci_poll(void);
+unsigned long cmci_intel_adjust_timer(unsigned long interval);
+bool mce_intel_cmci_poll(void);
void mce_intel_hcpu_update(unsigned long cpu);
void cmci_disable_bank(int bank);
#else
-# define mce_intel_adjust_timer mce_adjust_timer_default
-static inline void mce_intel_cmci_poll(void) { }
+# define cmci_intel_adjust_timer mce_adjust_timer_default
+static inline bool mce_intel_cmci_poll(void) { return false; }
static inline void mce_intel_hcpu_update(unsigned long cpu) { }
static inline void cmci_disable_bank(int bank) { }
#endif
return ((m->cs & 3) == 3) ? IN_USER : IN_KERNEL;
}
-int mce_severity(struct mce *m, int tolerant, char **msg, bool is_excp)
+/*
+ * See AMD Error Scope Hierarchy table in a newer BKDG. For example
+ * 49125_15h_Models_30h-3Fh_BKDG.pdf, section "RAS Features"
+ */
+static int mce_severity_amd(struct mce *m, int tolerant, char **msg, bool is_excp)
+{
+ enum context ctx = error_context(m);
+
+ /* Processor Context Corrupt, no need to fumble too much, die! */
+ if (m->status & MCI_STATUS_PCC)
+ return MCE_PANIC_SEVERITY;
+
+ if (m->status & MCI_STATUS_UC) {
+
+ /*
+ * On older systems where overflow_recov flag is not present, we
+ * should simply panic if an error overflow occurs. If
+ * overflow_recov flag is present and set, then software can try
+ * to at least kill process to prolong system operation.
+ */
+ if (mce_flags.overflow_recov) {
+ /* software can try to contain */
+ if (!(m->mcgstatus & MCG_STATUS_RIPV) && (ctx == IN_KERNEL))
+ return MCE_PANIC_SEVERITY;
+
+ /* kill current process */
+ return MCE_AR_SEVERITY;
+ } else {
+ /* at least one error was not logged */
+ if (m->status & MCI_STATUS_OVER)
+ return MCE_PANIC_SEVERITY;
+ }
+
+ /*
+ * For any other case, return MCE_UC_SEVERITY so that we log the
+ * error and exit #MC handler.
+ */
+ return MCE_UC_SEVERITY;
+ }
+
+ /*
+ * deferred error: poll handler catches these and adds to mce_ring so
+ * memory-failure can take recovery actions.
+ */
+ if (m->status & MCI_STATUS_DEFERRED)
+ return MCE_DEFERRED_SEVERITY;
+
+ /*
+ * corrected error: poll handler catches these and passes responsibility
+ * of decoding the error to EDAC
+ */
+ return MCE_KEEP_SEVERITY;
+}
+
+static int mce_severity_intel(struct mce *m, int tolerant, char **msg, bool is_excp)
{
enum exception excp = (is_excp ? EXCP_CONTEXT : NO_EXCP);
enum context ctx = error_context(m);
}
}
+/* Default to mce_severity_intel */
+int (*mce_severity)(struct mce *m, int tolerant, char **msg, bool is_excp) =
+ mce_severity_intel;
+
+void __init mcheck_vendor_init_severity(void)
+{
+ if (boot_cpu_data.x86_vendor == X86_VENDOR_AMD)
+ mce_severity = mce_severity_amd;
+}
+
#ifdef CONFIG_DEBUG_FS
static void *s_start(struct seq_file *f, loff_t *pos)
{
#define CREATE_TRACE_POINTS
#include <trace/events/mce.h>
-#define SPINUNIT 100 /* 100ns */
+#define SPINUNIT 100 /* 100ns */
DEFINE_PER_CPU(unsigned, mce_exception_count);
struct mce_bank *mce_banks __read_mostly;
+struct mce_vendor_flags mce_flags __read_mostly;
struct mca_config mca_cfg __read_mostly = {
.bootlog = -1,
static DEFINE_PER_CPU(struct mce, mces_seen);
static int cpu_missing;
-/* CMCI storm detection filter */
-static DEFINE_PER_CPU(unsigned long, mce_polled_error);
-
/*
* MCA banks polled by the period polling timer for corrected events.
* With Intel CMCI, this only has MCA banks which do not support CMCI (if any).
* is already totally * confused. In this case it's likely it will
* not fully execute the machine check handler either.
*/
-void machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
+bool machine_check_poll(enum mcp_flags flags, mce_banks_t *b)
{
+ bool error_logged = false;
struct mce m;
int severity;
int i;
if (!(m.status & MCI_STATUS_VAL))
continue;
- this_cpu_write(mce_polled_error, 1);
+
/*
* Uncorrected or signalled events are handled by the exception
* handler when it is enabled, so don't process those here.
* Don't get the IP here because it's unlikely to
* have anything to do with the actual error location.
*/
- if (!(flags & MCP_DONTLOG) && !mca_cfg.dont_log_ce)
+ if (!(flags & MCP_DONTLOG) && !mca_cfg.dont_log_ce) {
+ error_logged = true;
mce_log(&m);
+ }
/*
* Clear state for this bank.
*/
sync_core();
+
+ return error_logged;
}
EXPORT_SYMBOL_GPL(machine_check_poll);
* other CPUs.
*/
if (m && global_worst >= MCE_PANIC_SEVERITY && mca_cfg.tolerant < 3)
- mce_panic("Fatal Machine check", m, msg);
+ mce_panic("Fatal machine check", m, msg);
/*
* For UC somewhere we let the CPU who detects it handle it.
* source or one CPU is hung. Panic.
*/
if (global_worst <= MCE_KEEP_SEVERITY && mca_cfg.tolerant < 3)
- mce_panic("Machine check from unknown source", NULL, NULL);
+ mce_panic("Fatal machine check from unknown source", NULL, NULL);
/*
* Now clear all the mces_seen so that they don't reappear on
* poller finds an MCE, poll 2x faster. When the poller finds no more
* errors, poll 2x slower (up to check_interval seconds).
*/
-static unsigned long check_interval = 5 * 60; /* 5 minutes */
+static unsigned long check_interval = INITIAL_CHECK_INTERVAL;
static DEFINE_PER_CPU(unsigned long, mce_next_interval); /* in jiffies */
static DEFINE_PER_CPU(struct timer_list, mce_timer);
return interval;
}
-static unsigned long (*mce_adjust_timer)(unsigned long interval) =
- mce_adjust_timer_default;
+static unsigned long (*mce_adjust_timer)(unsigned long interval) = mce_adjust_timer_default;
-static int cmc_error_seen(void)
+static void __restart_timer(struct timer_list *t, unsigned long interval)
{
- unsigned long *v = this_cpu_ptr(&mce_polled_error);
+ unsigned long when = jiffies + interval;
+ unsigned long flags;
+
+ local_irq_save(flags);
- return test_and_clear_bit(0, v);
+ if (timer_pending(t)) {
+ if (time_before(when, t->expires))
+ mod_timer_pinned(t, when);
+ } else {
+ t->expires = round_jiffies(when);
+ add_timer_on(t, smp_processor_id());
+ }
+
+ local_irq_restore(flags);
}
static void mce_timer_fn(unsigned long data)
{
struct timer_list *t = this_cpu_ptr(&mce_timer);
+ int cpu = smp_processor_id();
unsigned long iv;
- int notify;
- WARN_ON(smp_processor_id() != data);
+ WARN_ON(cpu != data);
+
+ iv = __this_cpu_read(mce_next_interval);
if (mce_available(this_cpu_ptr(&cpu_info))) {
- machine_check_poll(MCP_TIMESTAMP,
- this_cpu_ptr(&mce_poll_banks));
- mce_intel_cmci_poll();
+ machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_poll_banks));
+
+ if (mce_intel_cmci_poll()) {
+ iv = mce_adjust_timer(iv);
+ goto done;
+ }
}
/*
- * Alert userspace if needed. If we logged an MCE, reduce the
- * polling interval, otherwise increase the polling interval.
+ * Alert userspace if needed. If we logged an MCE, reduce the polling
+ * interval, otherwise increase the polling interval.
*/
- iv = __this_cpu_read(mce_next_interval);
- notify = mce_notify_irq();
- notify |= cmc_error_seen();
- if (notify) {
+ if (mce_notify_irq())
iv = max(iv / 2, (unsigned long) HZ/100);
- } else {
+ else
iv = min(iv * 2, round_jiffies_relative(check_interval * HZ));
- iv = mce_adjust_timer(iv);
- }
+
+done:
__this_cpu_write(mce_next_interval, iv);
- /* Might have become 0 after CMCI storm subsided */
- if (iv) {
- t->expires = jiffies + iv;
- add_timer_on(t, smp_processor_id());
- }
+ __restart_timer(t, iv);
}
/*
void mce_timer_kick(unsigned long interval)
{
struct timer_list *t = this_cpu_ptr(&mce_timer);
- unsigned long when = jiffies + interval;
unsigned long iv = __this_cpu_read(mce_next_interval);
- if (timer_pending(t)) {
- if (time_before(when, t->expires))
- mod_timer_pinned(t, when);
- } else {
- t->expires = round_jiffies(when);
- add_timer_on(t, smp_processor_id());
- }
+ __restart_timer(t, interval);
+
if (interval < iv)
__this_cpu_write(mce_next_interval, interval);
}
* Various K7s with broken bank 0 around. Always disable
* by default.
*/
- if (c->x86 == 6 && cfg->banks > 0)
+ if (c->x86 == 6 && cfg->banks > 0)
mce_banks[0].ctl = 0;
- /*
- * Turn off MC4_MISC thresholding banks on those models since
- * they're not supported there.
- */
- if (c->x86 == 0x15 &&
- (c->x86_model >= 0x10 && c->x86_model <= 0x1f)) {
- int i;
- u64 val, hwcr;
- bool need_toggle;
- u32 msrs[] = {
+ /*
+ * overflow_recov is supported for F15h Models 00h-0fh
+ * even though we don't have a CPUID bit for it.
+ */
+ if (c->x86 == 0x15 && c->x86_model <= 0xf)
+ mce_flags.overflow_recov = 1;
+
+ /*
+ * Turn off MC4_MISC thresholding banks on those models since
+ * they're not supported there.
+ */
+ if (c->x86 == 0x15 &&
+ (c->x86_model >= 0x10 && c->x86_model <= 0x1f)) {
+ int i;
+ u64 hwcr;
+ bool need_toggle;
+ u32 msrs[] = {
0x00000413, /* MC4_MISC0 */
0xc0000408, /* MC4_MISC1 */
- };
+ };
- rdmsrl(MSR_K7_HWCR, hwcr);
+ rdmsrl(MSR_K7_HWCR, hwcr);
- /* McStatusWrEn has to be set */
- need_toggle = !(hwcr & BIT(18));
+ /* McStatusWrEn has to be set */
+ need_toggle = !(hwcr & BIT(18));
- if (need_toggle)
- wrmsrl(MSR_K7_HWCR, hwcr | BIT(18));
+ if (need_toggle)
+ wrmsrl(MSR_K7_HWCR, hwcr | BIT(18));
- for (i = 0; i < ARRAY_SIZE(msrs); i++) {
- rdmsrl(msrs[i], val);
+ /* Clear CntP bit safely */
+ for (i = 0; i < ARRAY_SIZE(msrs); i++)
+ msr_clear_bit(msrs[i], 62);
- /* CntP bit set? */
- if (val & BIT_64(62)) {
- val &= ~BIT_64(62);
- wrmsrl(msrs[i], val);
- }
- }
-
- /* restore old settings */
- if (need_toggle)
- wrmsrl(MSR_K7_HWCR, hwcr);
- }
+ /* restore old settings */
+ if (need_toggle)
+ wrmsrl(MSR_K7_HWCR, hwcr);
+ }
}
if (c->x86_vendor == X86_VENDOR_INTEL) {
switch (c->x86_vendor) {
case X86_VENDOR_INTEL:
mce_intel_feature_init(c);
- mce_adjust_timer = mce_intel_adjust_timer;
+ mce_adjust_timer = cmci_intel_adjust_timer;
break;
case X86_VENDOR_AMD:
mce_amd_feature_init(c);
+ mce_flags.overflow_recov = cpuid_ebx(0x80000007) & 0x1;
break;
default:
break;
int __init mcheck_init(void)
{
mcheck_intel_therm_init();
+ mcheck_vendor_init_severity();
return 0;
}
return (bank == 4);
}
-static const char * const bank4_names(struct threshold_block *b)
+static const char *bank4_names(const struct threshold_block *b)
{
switch (b->address) {
/* MSR4_MISC0 */
if (!b.interrupt_capable)
goto init;
+ b.interrupt_enable = 1;
new = (high & MASK_LVTOFF_HI) >> 20;
offset = setup_APIC_mce(offset, new);
log:
mce_setup(&m);
rdmsrl(MSR_IA32_MCx_STATUS(bank), m.status);
+ if (!(m.status & MCI_STATUS_VAL))
+ return;
m.misc = ((u64)high << 32) | low;
m.bank = bank;
mce_log(&m);
b->interrupt_capable = lvt_interrupt_supported(bank, high);
b->threshold_limit = THRESHOLD_MAX;
- if (b->interrupt_capable)
+ if (b->interrupt_capable) {
threshold_ktype.default_attrs[2] = &interrupt_enable.attr;
- else
+ b->interrupt_enable = 1;
+ } else {
threshold_ktype.default_attrs[2] = NULL;
+ }
INIT_LIST_HEAD(&b->miscj);
*/
static DEFINE_PER_CPU(mce_banks_t, mce_banks_owned);
+/*
+ * CMCI storm detection backoff counter
+ *
+ * During storm, we reset this counter to INITIAL_CHECK_INTERVAL in case we've
+ * encountered an error. If not, we decrement it by one. We signal the end of
+ * the CMCI storm when it reaches 0.
+ */
+static DEFINE_PER_CPU(int, cmci_backoff_cnt);
+
/*
* cmci_discover_lock protects against parallel discovery attempts
* which could race against each other.
#define CMCI_THRESHOLD 1
#define CMCI_POLL_INTERVAL (30 * HZ)
-#define CMCI_STORM_INTERVAL (1 * HZ)
+#define CMCI_STORM_INTERVAL (HZ)
#define CMCI_STORM_THRESHOLD 15
static DEFINE_PER_CPU(unsigned long, cmci_time_stamp);
return !!(cap & MCG_CMCI_P);
}
-void mce_intel_cmci_poll(void)
+bool mce_intel_cmci_poll(void)
{
if (__this_cpu_read(cmci_storm_state) == CMCI_STORM_NONE)
- return;
- machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned));
+ return false;
+
+ /*
+ * Reset the counter if we've logged an error in the last poll
+ * during the storm.
+ */
+ if (machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned)))
+ this_cpu_write(cmci_backoff_cnt, INITIAL_CHECK_INTERVAL);
+ else
+ this_cpu_dec(cmci_backoff_cnt);
+
+ return true;
}
void mce_intel_hcpu_update(unsigned long cpu)
per_cpu(cmci_storm_state, cpu) = CMCI_STORM_NONE;
}
-unsigned long mce_intel_adjust_timer(unsigned long interval)
+unsigned long cmci_intel_adjust_timer(unsigned long interval)
{
- int r;
-
- if (interval < CMCI_POLL_INTERVAL)
- return interval;
+ if ((this_cpu_read(cmci_backoff_cnt) > 0) &&
+ (__this_cpu_read(cmci_storm_state) == CMCI_STORM_ACTIVE)) {
+ mce_notify_irq();
+ return CMCI_STORM_INTERVAL;
+ }
switch (__this_cpu_read(cmci_storm_state)) {
case CMCI_STORM_ACTIVE:
+
/*
* We switch back to interrupt mode once the poll timer has
- * silenced itself. That means no events recorded and the
- * timer interval is back to our poll interval.
+ * silenced itself. That means no events recorded and the timer
+ * interval is back to our poll interval.
*/
__this_cpu_write(cmci_storm_state, CMCI_STORM_SUBSIDED);
- r = atomic_sub_return(1, &cmci_storm_on_cpus);
- if (r == 0)
+ if (!atomic_sub_return(1, &cmci_storm_on_cpus))
pr_notice("CMCI storm subsided: switching to interrupt mode\n");
+
/* FALLTHROUGH */
case CMCI_STORM_SUBSIDED:
/*
- * We wait for all cpus to go back to SUBSIDED
- * state. When that happens we switch back to
- * interrupt mode.
+ * We wait for all CPUs to go back to SUBSIDED state. When that
+ * happens we switch back to interrupt mode.
*/
if (!atomic_read(&cmci_storm_on_cpus)) {
__this_cpu_write(cmci_storm_state, CMCI_STORM_NONE);
}
return CMCI_POLL_INTERVAL;
default:
- /*
- * We have shiny weather. Let the poll do whatever it
- * thinks.
- */
+
+ /* We have shiny weather. Let the poll do whatever it thinks. */
return interval;
}
}
cmci_storm_disable_banks();
__this_cpu_write(cmci_storm_state, CMCI_STORM_ACTIVE);
r = atomic_add_return(1, &cmci_storm_on_cpus);
- mce_timer_kick(CMCI_POLL_INTERVAL);
+ mce_timer_kick(CMCI_STORM_INTERVAL);
+ this_cpu_write(cmci_backoff_cnt, INITIAL_CHECK_INTERVAL);
if (r == 1)
pr_notice("CMCI storm detected: switching to poll mode\n");
{
if (cmci_storm_detect())
return;
+
machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned));
mce_notify_irq();
}
if (!mce_available(raw_cpu_ptr(&cpu_info)) || !cmci_supported(&banks))
return;
+
local_irq_save(flags);
machine_check_poll(MCP_TIMESTAMP, this_cpu_ptr(&mce_banks_owned));
local_irq_restore(flags);
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/firmware.h>
-#include <linux/pci_ids.h>
#include <linux/uaccess.h>
#include <linux/vmalloc.h>
#include <linux/kernel.h>
#include <asm/processor.h>
#include <asm/cmdline.h>
-#define QCHAR(a, b, c, d) ((a) + ((b) << 8) + ((c) << 16) + ((d) << 24))
-#define CPUID_INTEL1 QCHAR('G', 'e', 'n', 'u')
-#define CPUID_INTEL2 QCHAR('i', 'n', 'e', 'I')
-#define CPUID_INTEL3 QCHAR('n', 't', 'e', 'l')
-#define CPUID_AMD1 QCHAR('A', 'u', 't', 'h')
-#define CPUID_AMD2 QCHAR('e', 'n', 't', 'i')
-#define CPUID_AMD3 QCHAR('c', 'A', 'M', 'D')
-
-#define CPUID_IS(a, b, c, ebx, ecx, edx) \
- (!((ebx ^ (a))|(edx ^ (b))|(ecx ^ (c))))
-
-/*
- * In early loading microcode phase on BSP, boot_cpu_data is not set up yet.
- * x86_vendor() gets vendor id for BSP.
- *
- * In 32 bit AP case, accessing boot_cpu_data needs linear address. To simplify
- * coding, we still use x86_vendor() to get vendor id for AP.
- *
- * x86_vendor() gets vendor information directly through cpuid.
- */
-static int x86_vendor(void)
-{
- u32 eax = 0x00000000;
- u32 ebx, ecx = 0, edx;
-
- native_cpuid(&eax, &ebx, &ecx, &edx);
-
- if (CPUID_IS(CPUID_INTEL1, CPUID_INTEL2, CPUID_INTEL3, ebx, ecx, edx))
- return X86_VENDOR_INTEL;
-
- if (CPUID_IS(CPUID_AMD1, CPUID_AMD2, CPUID_AMD3, ebx, ecx, edx))
- return X86_VENDOR_AMD;
-
- return X86_VENDOR_UNKNOWN;
-}
-
-static int x86_family(void)
-{
- u32 eax = 0x00000001;
- u32 ebx, ecx = 0, edx;
- int x86;
-
- native_cpuid(&eax, &ebx, &ecx, &edx);
-
- x86 = (eax >> 8) & 0xf;
- if (x86 == 15)
- x86 += (eax >> 20) & 0xff;
-
- return x86;
-}
-
static bool __init check_loader_disabled_bsp(void)
{
#ifdef CONFIG_X86_32
void __init load_ucode_bsp(void)
{
- int vendor, x86;
+ int vendor, family;
if (check_loader_disabled_bsp())
return;
return;
vendor = x86_vendor();
- x86 = x86_family();
+ family = x86_family();
switch (vendor) {
case X86_VENDOR_INTEL:
- if (x86 >= 6)
+ if (family >= 6)
load_ucode_intel_bsp();
break;
case X86_VENDOR_AMD:
- if (x86 >= 0x10)
+ if (family >= 0x10)
load_ucode_amd_bsp();
break;
default:
void load_ucode_ap(void)
{
- int vendor, x86;
+ int vendor, family;
if (check_loader_disabled_ap())
return;
return;
vendor = x86_vendor();
- x86 = x86_family();
+ family = x86_family();
switch (vendor) {
case X86_VENDOR_INTEL:
- if (x86 >= 6)
+ if (family >= 6)
load_ucode_intel_ap();
break;
case X86_VENDOR_AMD:
- if (x86 >= 0x10)
+ if (family >= 0x10)
load_ucode_amd_ap();
break;
default:
void reload_early_microcode(void)
{
- int vendor, x86;
+ int vendor, family;
vendor = x86_vendor();
- x86 = x86_family();
+ family = x86_family();
switch (vendor) {
case X86_VENDOR_INTEL:
- if (x86 >= 6)
+ if (family >= 6)
reload_ucode_intel();
break;
case X86_VENDOR_AMD:
- if (x86 >= 0x10)
+ if (family >= 0x10)
reload_ucode_amd();
break;
default:
cpf = cpu_sig.pf;
crev = cpu_sig.rev;
- return get_matching_microcode(csig, cpf, mc_intel, crev);
+ return get_matching_microcode(csig, cpf, crev, mc_intel);
}
static int apply_microcode_intel(int cpu)
csig = uci->cpu_sig.sig;
cpf = uci->cpu_sig.pf;
- if (get_matching_microcode(csig, cpf, mc, new_rev)) {
+ if (get_matching_microcode(csig, cpf, new_rev, mc)) {
vfree(new_mc);
new_rev = mc_header.rev;
new_mc = mc;
* as published by the Free Software Foundation; either version
* 2 of the License, or (at your option) any later version.
*/
+
+/*
+ * This needs to be before all headers so that pr_debug in printk.h doesn't turn
+ * printk calls into no_printk().
+ *
+ *#define DEBUG
+ */
+
#include <linux/module.h>
#include <linux/mm.h>
#include <linux/slab.h>
#include <asm/tlbflush.h>
#include <asm/setup.h>
+#undef pr_fmt
+#define pr_fmt(fmt) "microcode: " fmt
+
static unsigned long mc_saved_in_initrd[MAX_UCODE_COUNT];
static struct mc_saved_data {
unsigned int mc_saved_count;
} mc_saved_data;
static enum ucode_state
-generic_load_microcode_early(struct microcode_intel **mc_saved_p,
- unsigned int mc_saved_count,
- struct ucode_cpu_info *uci)
+load_microcode_early(struct microcode_intel **saved,
+ unsigned int num_saved, struct ucode_cpu_info *uci)
{
struct microcode_intel *ucode_ptr, *new_mc = NULL;
- int new_rev = uci->cpu_sig.rev;
- enum ucode_state state = UCODE_OK;
- unsigned int mc_size;
- struct microcode_header_intel *mc_header;
- unsigned int csig = uci->cpu_sig.sig;
- unsigned int cpf = uci->cpu_sig.pf;
- int i;
+ struct microcode_header_intel *mc_hdr;
+ int new_rev, ret, i;
- for (i = 0; i < mc_saved_count; i++) {
- ucode_ptr = mc_saved_p[i];
+ new_rev = uci->cpu_sig.rev;
- mc_header = (struct microcode_header_intel *)ucode_ptr;
- mc_size = get_totalsize(mc_header);
- if (get_matching_microcode(csig, cpf, ucode_ptr, new_rev)) {
- new_rev = mc_header->rev;
- new_mc = ucode_ptr;
- }
- }
+ for (i = 0; i < num_saved; i++) {
+ ucode_ptr = saved[i];
+ mc_hdr = (struct microcode_header_intel *)ucode_ptr;
- if (!new_mc) {
- state = UCODE_NFOUND;
- goto out;
+ ret = get_matching_microcode(uci->cpu_sig.sig,
+ uci->cpu_sig.pf,
+ new_rev,
+ ucode_ptr);
+ if (!ret)
+ continue;
+
+ new_rev = mc_hdr->rev;
+ new_mc = ucode_ptr;
}
+ if (!new_mc)
+ return UCODE_NFOUND;
+
uci->mc = (struct microcode_intel *)new_mc;
-out:
- return state;
+ return UCODE_OK;
}
-static void
-microcode_pointer(struct microcode_intel **mc_saved,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start, int mc_saved_count)
+static inline void
+copy_initrd_ptrs(struct microcode_intel **mc_saved, unsigned long *initrd,
+ unsigned long off, int num_saved)
{
int i;
- for (i = 0; i < mc_saved_count; i++)
- mc_saved[i] = (struct microcode_intel *)
- (mc_saved_in_initrd[i] + initrd_start);
+ for (i = 0; i < num_saved; i++)
+ mc_saved[i] = (struct microcode_intel *)(initrd[i] + off);
}
#ifdef CONFIG_X86_32
#endif
static enum ucode_state
-load_microcode(struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start,
- struct ucode_cpu_info *uci)
+load_microcode(struct mc_saved_data *mc_saved_data, unsigned long *initrd,
+ unsigned long initrd_start, struct ucode_cpu_info *uci)
{
struct microcode_intel *mc_saved_tmp[MAX_UCODE_COUNT];
unsigned int count = mc_saved_data->mc_saved_count;
if (!mc_saved_data->mc_saved) {
- microcode_pointer(mc_saved_tmp, mc_saved_in_initrd,
- initrd_start, count);
+ copy_initrd_ptrs(mc_saved_tmp, initrd, initrd_start, count);
- return generic_load_microcode_early(mc_saved_tmp, count, uci);
+ return load_microcode_early(mc_saved_tmp, count, uci);
} else {
#ifdef CONFIG_X86_32
microcode_phys(mc_saved_tmp, mc_saved_data);
- return generic_load_microcode_early(mc_saved_tmp, count, uci);
+ return load_microcode_early(mc_saved_tmp, count, uci);
#else
- return generic_load_microcode_early(mc_saved_data->mc_saved,
+ return load_microcode_early(mc_saved_data->mc_saved,
count, uci);
#endif
}
}
-static u8 get_x86_family(unsigned long sig)
-{
- u8 x86;
-
- x86 = (sig >> 8) & 0xf;
-
- if (x86 == 0xf)
- x86 += (sig >> 20) & 0xff;
-
- return x86;
-}
-
-static u8 get_x86_model(unsigned long sig)
-{
- u8 x86, x86_model;
-
- x86 = get_x86_family(sig);
- x86_model = (sig >> 4) & 0xf;
-
- if (x86 == 0x6 || x86 == 0xf)
- x86_model += ((sig >> 16) & 0xf) << 4;
-
- return x86_model;
-}
-
/*
* Given CPU signature and a microcode patch, this function finds if the
* microcode patch has matching family and model with the CPU.
matching_model_microcode(struct microcode_header_intel *mc_header,
unsigned long sig)
{
- u8 x86, x86_model;
- u8 x86_ucode, x86_model_ucode;
+ unsigned int fam, model;
+ unsigned int fam_ucode, model_ucode;
struct extended_sigtable *ext_header;
unsigned long total_size = get_totalsize(mc_header);
unsigned long data_size = get_datasize(mc_header);
int ext_sigcount, i;
struct extended_signature *ext_sig;
- x86 = get_x86_family(sig);
- x86_model = get_x86_model(sig);
+ fam = __x86_family(sig);
+ model = x86_model(sig);
- x86_ucode = get_x86_family(mc_header->sig);
- x86_model_ucode = get_x86_model(mc_header->sig);
+ fam_ucode = __x86_family(mc_header->sig);
+ model_ucode = x86_model(mc_header->sig);
- if (x86 == x86_ucode && x86_model == x86_model_ucode)
+ if (fam == fam_ucode && model == model_ucode)
return UCODE_OK;
/* Look for ext. headers: */
if (total_size <= data_size + MC_HEADER_SIZE)
return UCODE_NFOUND;
- ext_header = (struct extended_sigtable *)
- mc_header + data_size + MC_HEADER_SIZE;
+ ext_header = (void *) mc_header + data_size + MC_HEADER_SIZE;
+ ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
ext_sigcount = ext_header->count;
- ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
for (i = 0; i < ext_sigcount; i++) {
- x86_ucode = get_x86_family(ext_sig->sig);
- x86_model_ucode = get_x86_model(ext_sig->sig);
+ fam_ucode = __x86_family(ext_sig->sig);
+ model_ucode = x86_model(ext_sig->sig);
- if (x86 == x86_ucode && x86_model == x86_model_ucode)
+ if (fam == fam_ucode && model == model_ucode)
return UCODE_OK;
ext_sig++;
}
-
return UCODE_NFOUND;
}
unsigned int mc_saved_count)
{
int i, j;
- struct microcode_intel **mc_saved_p;
+ struct microcode_intel **saved_ptr;
int ret;
if (!mc_saved_count)
/*
* Copy new microcode data.
*/
- mc_saved_p = kmalloc(mc_saved_count*sizeof(struct microcode_intel *),
- GFP_KERNEL);
- if (!mc_saved_p)
+ saved_ptr = kcalloc(mc_saved_count, sizeof(struct microcode_intel *), GFP_KERNEL);
+ if (!saved_ptr)
return -ENOMEM;
for (i = 0; i < mc_saved_count; i++) {
- struct microcode_intel *mc = mc_saved_src[i];
- struct microcode_header_intel *mc_header = &mc->hdr;
- unsigned long mc_size = get_totalsize(mc_header);
- mc_saved_p[i] = kmalloc(mc_size, GFP_KERNEL);
- if (!mc_saved_p[i]) {
- ret = -ENOMEM;
- goto err;
- }
+ struct microcode_header_intel *mc_hdr;
+ struct microcode_intel *mc;
+ unsigned long size;
+
if (!mc_saved_src[i]) {
ret = -EINVAL;
goto err;
}
- memcpy(mc_saved_p[i], mc, mc_size);
+
+ mc = mc_saved_src[i];
+ mc_hdr = &mc->hdr;
+ size = get_totalsize(mc_hdr);
+
+ saved_ptr[i] = kmalloc(size, GFP_KERNEL);
+ if (!saved_ptr[i]) {
+ ret = -ENOMEM;
+ goto err;
+ }
+
+ memcpy(saved_ptr[i], mc, size);
}
/*
* Point to newly saved microcode.
*/
- mc_saved_data->mc_saved = mc_saved_p;
+ mc_saved_data->mc_saved = saved_ptr;
mc_saved_data->mc_saved_count = mc_saved_count;
return 0;
err:
for (j = 0; j <= i; j++)
- kfree(mc_saved_p[j]);
- kfree(mc_saved_p);
+ kfree(saved_ptr[j]);
+ kfree(saved_ptr);
return ret;
}
* - or if it is a newly discovered microcode patch.
*
* The microcode patch should have matching model with CPU.
+ *
+ * Returns: The updated number @num_saved of saved microcode patches.
*/
-static void _save_mc(struct microcode_intel **mc_saved, u8 *ucode_ptr,
- unsigned int *mc_saved_count_p)
+static unsigned int _save_mc(struct microcode_intel **mc_saved,
+ u8 *ucode_ptr, unsigned int num_saved)
{
- int i;
- int found = 0;
- unsigned int mc_saved_count = *mc_saved_count_p;
- struct microcode_header_intel *mc_header;
+ struct microcode_header_intel *mc_hdr, *mc_saved_hdr;
+ unsigned int sig, pf, new_rev;
+ int found = 0, i;
+
+ mc_hdr = (struct microcode_header_intel *)ucode_ptr;
+
+ for (i = 0; i < num_saved; i++) {
+ mc_saved_hdr = (struct microcode_header_intel *)mc_saved[i];
+ sig = mc_saved_hdr->sig;
+ pf = mc_saved_hdr->pf;
+ new_rev = mc_hdr->rev;
+
+ if (!get_matching_sig(sig, pf, new_rev, ucode_ptr))
+ continue;
+
+ found = 1;
+
+ if (!revision_is_newer(mc_hdr, new_rev))
+ continue;
- mc_header = (struct microcode_header_intel *)ucode_ptr;
- for (i = 0; i < mc_saved_count; i++) {
- unsigned int sig, pf;
- unsigned int new_rev;
- struct microcode_header_intel *mc_saved_header =
- (struct microcode_header_intel *)mc_saved[i];
- sig = mc_saved_header->sig;
- pf = mc_saved_header->pf;
- new_rev = mc_header->rev;
-
- if (get_matching_sig(sig, pf, ucode_ptr, new_rev)) {
- found = 1;
- if (update_match_revision(mc_header, new_rev)) {
- /*
- * Found an older ucode saved before.
- * Replace the older one with this newer
- * one.
- */
- mc_saved[i] =
- (struct microcode_intel *)ucode_ptr;
- break;
- }
- }
- }
- if (i >= mc_saved_count && !found)
/*
- * This ucode is first time discovered in ucode file.
- * Save it to memory.
+ * Found an older ucode saved earlier. Replace it with
+ * this newer one.
*/
- mc_saved[mc_saved_count++] =
- (struct microcode_intel *)ucode_ptr;
+ mc_saved[i] = (struct microcode_intel *)ucode_ptr;
+ break;
+ }
+
+ /* Newly detected microcode, save it to memory. */
+ if (i >= num_saved && !found)
+ mc_saved[num_saved++] = (struct microcode_intel *)ucode_ptr;
- *mc_saved_count_p = mc_saved_count;
+ return num_saved;
}
/*
continue;
}
- _save_mc(mc_saved_tmp, ucode_ptr, &mc_saved_count);
+ mc_saved_count = _save_mc(mc_saved_tmp, ucode_ptr, mc_saved_count);
ucode_ptr += mc_size;
}
static int collect_cpu_info_early(struct ucode_cpu_info *uci)
{
unsigned int val[2];
- u8 x86, x86_model;
+ unsigned int family, model;
struct cpu_signature csig;
unsigned int eax, ebx, ecx, edx;
native_cpuid(&eax, &ebx, &ecx, &edx);
csig.sig = eax;
- x86 = get_x86_family(csig.sig);
- x86_model = get_x86_model(csig.sig);
+ family = __x86_family(csig.sig);
+ model = x86_model(csig.sig);
- if ((x86_model >= 5) || (x86 > 6)) {
+ if ((model >= 5) || (family > 6)) {
/* get processor flags from MSR 0x17 */
native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
csig.pf = 1 << ((val[1] >> 18) & 7);
sig = uci.cpu_sig.sig;
pf = uci.cpu_sig.pf;
rev = uci.cpu_sig.rev;
- pr_debug("CPU%d: sig=0x%x, pf=0x%x, rev=0x%x\n",
- smp_processor_id(), sig, pf, rev);
+ pr_debug("CPU: sig=0x%x, pf=0x%x, rev=0x%x\n", sig, pf, rev);
for (i = 0; i < mc_saved_data.mc_saved_count; i++) {
struct microcode_header_intel *mc_saved_header;
if (total_size <= data_size + MC_HEADER_SIZE)
continue;
- ext_header = (struct extended_sigtable *)
- mc_saved_header + data_size + MC_HEADER_SIZE;
+ ext_header = (void *) mc_saved_header + data_size + MC_HEADER_SIZE;
ext_sigcount = ext_header->count;
ext_sig = (void *)ext_header + EXT_HEADER_SIZE;
* Save the microcode patch mc in mc_save_tmp structure if it's a newer
* version.
*/
-
- _save_mc(mc_saved_tmp, mc, &mc_saved_count);
+ mc_saved_count = _save_mc(mc_saved_tmp, mc, mc_saved_count);
/*
* Save the mc_save_tmp in global mc_saved_data.
static __initdata char ucode_name[] = "kernel/x86/microcode/GenuineIntel.bin";
static __init enum ucode_state
-scan_microcode(unsigned long start, unsigned long end,
- struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- struct ucode_cpu_info *uci)
+scan_microcode(struct mc_saved_data *mc_saved_data, unsigned long *initrd,
+ unsigned long start, unsigned long size,
+ struct ucode_cpu_info *uci)
{
- unsigned int size = end - start + 1;
struct cpio_data cd;
long offset = 0;
#ifdef CONFIG_X86_32
if (!cd.data)
return UCODE_ERROR;
-
return get_matching_model_microcode(0, start, cd.data, cd.size,
- mc_saved_data, mc_saved_in_initrd,
- uci);
+ mc_saved_data, initrd, uci);
}
/*
if (count == 0)
return ret;
- microcode_pointer(mc_saved, mc_saved_in_initrd, initrd_start, count);
+ copy_initrd_ptrs(mc_saved, mc_saved_in_initrd, initrd_start, count);
ret = save_microcode(&mc_saved_data, mc_saved, count);
if (ret)
pr_err("Cannot save microcode patches from initrd.\n");
static void __init
_load_ucode_intel_bsp(struct mc_saved_data *mc_saved_data,
- unsigned long *mc_saved_in_initrd,
- unsigned long initrd_start_early,
- unsigned long initrd_end_early,
- struct ucode_cpu_info *uci)
+ unsigned long *initrd,
+ unsigned long start, unsigned long size)
{
+ struct ucode_cpu_info uci;
enum ucode_state ret;
- collect_cpu_info_early(uci);
- scan_microcode(initrd_start_early, initrd_end_early, mc_saved_data,
- mc_saved_in_initrd, uci);
+ collect_cpu_info_early(&uci);
- ret = load_microcode(mc_saved_data, mc_saved_in_initrd,
- initrd_start_early, uci);
+ ret = scan_microcode(mc_saved_data, initrd, start, size, &uci);
+ if (ret != UCODE_OK)
+ return;
- if (ret == UCODE_OK)
- apply_microcode_early(uci, true);
+ ret = load_microcode(mc_saved_data, initrd, start, &uci);
+ if (ret != UCODE_OK)
+ return;
+
+ apply_microcode_early(&uci, true);
}
-void __init
-load_ucode_intel_bsp(void)
+void __init load_ucode_intel_bsp(void)
{
- u64 ramdisk_image, ramdisk_size;
- unsigned long initrd_start_early, initrd_end_early;
- struct ucode_cpu_info uci;
+ u64 start, size;
#ifdef CONFIG_X86_32
- struct boot_params *boot_params_p;
+ struct boot_params *p;
- boot_params_p = (struct boot_params *)__pa_nodebug(&boot_params);
- ramdisk_image = boot_params_p->hdr.ramdisk_image;
- ramdisk_size = boot_params_p->hdr.ramdisk_size;
- initrd_start_early = ramdisk_image;
- initrd_end_early = initrd_start_early + ramdisk_size;
+ p = (struct boot_params *)__pa_nodebug(&boot_params);
+ start = p->hdr.ramdisk_image;
+ size = p->hdr.ramdisk_size;
_load_ucode_intel_bsp(
- (struct mc_saved_data *)__pa_nodebug(&mc_saved_data),
- (unsigned long *)__pa_nodebug(&mc_saved_in_initrd),
- initrd_start_early, initrd_end_early, &uci);
+ (struct mc_saved_data *)__pa_nodebug(&mc_saved_data),
+ (unsigned long *)__pa_nodebug(&mc_saved_in_initrd),
+ start, size);
#else
- ramdisk_image = boot_params.hdr.ramdisk_image;
- ramdisk_size = boot_params.hdr.ramdisk_size;
- initrd_start_early = ramdisk_image + PAGE_OFFSET;
- initrd_end_early = initrd_start_early + ramdisk_size;
-
- _load_ucode_intel_bsp(&mc_saved_data, mc_saved_in_initrd,
- initrd_start_early, initrd_end_early,
- &uci);
+ start = boot_params.hdr.ramdisk_image + PAGE_OFFSET;
+ size = boot_params.hdr.ramdisk_size;
+
+ _load_ucode_intel_bsp(&mc_saved_data, mc_saved_in_initrd, start, size);
#endif
}
struct ucode_cpu_info uci;
unsigned long *mc_saved_in_initrd_p;
unsigned long initrd_start_addr;
+ enum ucode_state ret;
#ifdef CONFIG_X86_32
unsigned long *initrd_start_p;
return;
collect_cpu_info_early(&uci);
- load_microcode(mc_saved_data_p, mc_saved_in_initrd_p,
- initrd_start_addr, &uci);
+ ret = load_microcode(mc_saved_data_p, mc_saved_in_initrd_p,
+ initrd_start_addr, &uci);
+
+ if (ret != UCODE_OK)
+ return;
+
apply_microcode_early(&uci, true);
}
collect_cpu_info_early(&uci);
- ret = generic_load_microcode_early(mc_saved_data.mc_saved,
- mc_saved_data.mc_saved_count, &uci);
+ ret = load_microcode_early(mc_saved_data.mc_saved,
+ mc_saved_data.mc_saved_count, &uci);
if (ret != UCODE_OK)
return;
return (!sigmatch(sig, csig, pf, cpf)) ? 0 : 1;
}
-int
-update_match_revision(struct microcode_header_intel *mc_header, int rev)
-{
- return (mc_header->rev <= rev) ? 0 : 1;
-}
-
int microcode_sanity_check(void *mc, int print_err)
{
unsigned long total_size, data_size, ext_table_size;
EXPORT_SYMBOL_GPL(microcode_sanity_check);
/*
- * return 0 - no update found
- * return 1 - found update
+ * Returns 1 if update has been found, 0 otherwise.
*/
-int get_matching_sig(unsigned int csig, int cpf, void *mc, int rev)
+int get_matching_sig(unsigned int csig, int cpf, int rev, void *mc)
{
struct microcode_header_intel *mc_header = mc;
struct extended_sigtable *ext_header;
}
/*
- * return 0 - no update found
- * return 1 - found update
+ * Returns 1 if update has been found, 0 otherwise.
*/
-int get_matching_microcode(unsigned int csig, int cpf, void *mc, int rev)
+int get_matching_microcode(unsigned int csig, int cpf, int rev, void *mc)
{
- struct microcode_header_intel *mc_header = mc;
+ struct microcode_header_intel *mc_hdr = mc;
- if (!update_match_revision(mc_header, rev))
+ if (!revision_is_newer(mc_hdr, rev))
return 0;
- return get_matching_sig(csig, cpf, mc, rev);
+ return get_matching_sig(csig, cpf, rev, mc);
}
EXPORT_SYMBOL_GPL(get_matching_microcode);
IN=$1
OUT=$2
-function dump_array()
+dump_array()
{
ARRAY=$1
SIZE=$2
*/
static unsigned long code_segment_base(struct pt_regs *regs)
{
+ /*
+ * For IA32 we look at the GDT/LDT segment base to convert the
+ * effective IP to a linear address.
+ */
+
+#ifdef CONFIG_X86_32
/*
* If we are in VM86 mode, add the segment offset to convert to a
* linear address.
if (regs->flags & X86_VM_MASK)
return 0x10 * regs->cs;
- /*
- * For IA32 we look at the GDT/LDT segment base to convert the
- * effective IP to a linear address.
- */
-#ifdef CONFIG_X86_32
if (user_mode(regs) && regs->cs != __USER_CS)
return get_segment_base(regs->cs);
#else
- if (test_thread_flag(TIF_IA32)) {
- if (user_mode(regs) && regs->cs != __USER32_CS)
- return get_segment_base(regs->cs);
- }
+ if (user_mode(regs) && !user_64bit_mode(regs) &&
+ regs->cs != __USER32_CS)
+ return get_segment_base(regs->cs);
#endif
return 0;
}
INTEL_UEVENT_CONSTRAINT(0x01c0, 0x2), /* INST_RETIRED.PREC_DIST */
INTEL_EVENT_CONSTRAINT(0xcd, 0x8), /* MEM_TRANS_RETIRED.LOAD_LATENCY */
/* CYCLE_ACTIVITY.CYCLES_L1D_PENDING */
- INTEL_EVENT_CONSTRAINT(0x08a3, 0x4),
+ INTEL_UEVENT_CONSTRAINT(0x08a3, 0x4),
/* CYCLE_ACTIVITY.STALLS_L1D_PENDING */
- INTEL_EVENT_CONSTRAINT(0x0ca3, 0x4),
+ INTEL_UEVENT_CONSTRAINT(0x0ca3, 0x4),
/* CYCLE_ACTIVITY.CYCLES_NO_EXECUTE */
- INTEL_EVENT_CONSTRAINT(0x04a3, 0xf),
+ INTEL_UEVENT_CONSTRAINT(0x04a3, 0xf),
EVENT_CONSTRAINT_END
};
if (c)
return c;
- c = intel_pebs_constraints(event);
+ c = intel_shared_regs_constraints(cpuc, event);
if (c)
return c;
- c = intel_shared_regs_constraints(cpuc, event);
+ c = intel_pebs_constraints(event);
if (c)
return c;
#ifdef CONFIG_X86_32
struct pt_regs fixed_regs;
- if (!user_mode_vm(regs)) {
+ if (!user_mode(regs)) {
crash_fixup_ss_esp(&fixed_regs, regs);
regs = &fixed_regs;
}
initial_boot_params = dt = early_memremap(initial_dtb, map_len);
size = of_get_flat_dt_size();
if (map_len < size) {
- early_iounmap(dt, map_len);
+ early_memunmap(dt, map_len);
initial_boot_params = dt = early_memremap(initial_dtb, size);
map_len = size;
}
unflatten_and_copy_device_tree();
- early_iounmap(dt, map_len);
+ early_memunmap(dt, map_len);
}
#else
static inline void x86_flattree_get_config(void) { }
int kstack_depth_to_print = 3 * STACKSLOTS_PER_LINE;
static int die_counter;
-static void printk_stack_address(unsigned long address, int reliable)
+static void printk_stack_address(unsigned long address, int reliable,
+ void *data)
{
- pr_cont(" [<%p>] %s%pB\n",
- (void *)address, reliable ? "" : "? ", (void *)address);
+ printk("%s [<%p>] %s%pB\n",
+ (char *)data, (void *)address, reliable ? "" : "? ",
+ (void *)address);
}
void printk_address(unsigned long address)
static void print_trace_address(void *data, unsigned long addr, int reliable)
{
touch_nmi_watchdog();
- printk(data);
- printk_stack_address(addr, reliable);
+ printk_stack_address(addr, reliable, data);
}
static const struct stacktrace_ops print_trace_ops = {
print_modules();
show_regs(regs);
#ifdef CONFIG_X86_32
- if (user_mode_vm(regs)) {
+ if (user_mode(regs)) {
sp = regs->sp;
ss = regs->ss & 0xffff;
} else {
unsigned long flags = oops_begin();
int sig = SIGSEGV;
- if (!user_mode_vm(regs))
+ if (!user_mode(regs))
report_bug(regs->ip, regs);
if (__die(str, regs, err))
for (i = 0; i < kstack_depth_to_print; i++) {
if (kstack_end(stack))
break;
- if (i && ((i % STACKSLOTS_PER_LINE) == 0))
- pr_cont("\n");
- pr_cont(" %08lx", *stack++);
+ if ((i % STACKSLOTS_PER_LINE) == 0) {
+ if (i != 0)
+ pr_cont("\n");
+ printk("%s %08lx", log_lvl, *stack++);
+ } else
+ pr_cont(" %08lx", *stack++);
touch_nmi_watchdog();
}
pr_cont("\n");
int i;
show_regs_print_info(KERN_EMERG);
- __show_regs(regs, !user_mode_vm(regs));
+ __show_regs(regs, !user_mode(regs));
/*
* When in-kernel, we also print out the stack and code at the
* time of the fault..
*/
- if (!user_mode_vm(regs)) {
+ if (!user_mode(regs)) {
unsigned int code_prologue = code_bytes * 43 / 64;
unsigned int code_len = code_bytes;
unsigned char c;
pr_cont(" <EOI> ");
}
} else {
- if (((long) stack & (THREAD_SIZE-1)) == 0)
+ if (kstack_end(stack))
break;
}
- if (i && ((i % STACKSLOTS_PER_LINE) == 0))
- pr_cont("\n");
- pr_cont(" %016lx", *stack++);
+ if ((i % STACKSLOTS_PER_LINE) == 0) {
+ if (i != 0)
+ pr_cont("\n");
+ printk("%s %016lx", log_lvl, *stack++);
+ } else
+ pr_cont(" %016lx", *stack++);
touch_nmi_watchdog();
}
preempt_enable();
extmap = (struct e820entry *)(sdata->data);
__append_e820_map(extmap, entries);
sanitize_e820_map(e820.map, ARRAY_SIZE(e820.map), &e820.nr_map);
- early_iounmap(sdata, data_len);
+ early_memunmap(sdata, data_len);
printk(KERN_INFO "e820: extended physical RAM map:\n");
e820_print_map("extended");
}
#define DLL 0 /* Divisor Latch Low */
#define DLH 1 /* Divisor latch High */
-static void mem32_serial_out(unsigned long addr, int offset, int value)
-{
- uint32_t *vaddr = (uint32_t *)addr;
- /* shift implied by pointer type */
- writel(value, vaddr + offset);
-}
-
-static unsigned int mem32_serial_in(unsigned long addr, int offset)
-{
- uint32_t *vaddr = (uint32_t *)addr;
- /* shift implied by pointer type */
- return readl(vaddr + offset);
-}
-
static unsigned int io_serial_in(unsigned long addr, int offset)
{
return inb(addr + offset);
}
#ifdef CONFIG_PCI
+static void mem32_serial_out(unsigned long addr, int offset, int value)
+{
+ u32 *vaddr = (u32 *)addr;
+ /* shift implied by pointer type */
+ writel(value, vaddr + offset);
+}
+
+static unsigned int mem32_serial_in(unsigned long addr, int offset)
+{
+ u32 *vaddr = (u32 *)addr;
+ /* shift implied by pointer type */
+ return readl(vaddr + offset);
+}
+
/*
* early_pci_serial_init()
*
unsigned divisor;
unsigned long baud = DEFAULT_BAUD;
u8 bus, slot, func;
- uint32_t classcode, bar0;
- uint16_t cmdreg;
+ u32 classcode, bar0;
+ u16 cmdreg;
char *e;
/*CFI_REL_OFFSET cs, 0*/
/*
* Push current_thread_info()->sysenter_return to the stack.
- * A tiny bit of offset fixup is necessary - 4*4 means the 4 words
- * pushed above; +8 corresponds to copy_thread's esp0 setting.
+ * A tiny bit of offset fixup is necessary: TI_sysenter_return
+ * is relative to thread_info, which is at the bottom of the
+ * kernel stack page. 4*4 means the 4 words pushed above;
+ * TOP_OF_KERNEL_STACK_PADDING takes us to the top of the stack;
+ * and THREAD_SIZE takes us to the bottom.
*/
- pushl_cfi ((TI_sysenter_return)-THREAD_SIZE+8+4*4)(%esp)
+ pushl_cfi ((TI_sysenter_return) - THREAD_SIZE + TOP_OF_KERNEL_STACK_PADDING + 4*4)(%esp)
CFI_REL_OFFSET eip, 0
pushl_cfi %eax
TRACE_IRQS_OFF
movl TI_flags(%ebp), %ecx
testl $_TIF_ALLWORK_MASK, %ecx
- jne sysexit_audit
+ jnz sysexit_audit
sysenter_exit:
/* if something modifies registers it must also disable sysexit */
movl PT_EIP(%esp), %edx
sysexit_audit:
testl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), %ecx
- jne syscall_exit_work
+ jnz syscall_exit_work
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_ANY)
movl %eax,%edx /* second arg, syscall return value */
TRACE_IRQS_OFF
movl TI_flags(%ebp), %ecx
testl $(_TIF_ALLWORK_MASK & ~_TIF_SYSCALL_AUDIT), %ecx
- jne syscall_exit_work
+ jnz syscall_exit_work
movl PT_EAX(%esp),%eax /* reload syscall return value */
jmp sysenter_exit
#endif
TRACE_IRQS_OFF
movl TI_flags(%ebp), %ecx
testl $_TIF_ALLWORK_MASK, %ecx # current->work
- jne syscall_exit_work
+ jnz syscall_exit_work
restore_all:
TRACE_IRQS_IRET
#ifdef CONFIG_VM86
testl $X86_EFLAGS_VM, PT_EFLAGS(%esp)
movl %esp, %eax
- jne work_notifysig_v86 # returning to kernel-space or
+ jnz work_notifysig_v86 # returning to kernel-space or
# vm86-space
1:
#else
.endm
/*
- * Build the entry stubs and pointer table with some assembler magic.
- * We pack 7 stubs into a single 32-byte chunk, which will fit in a
- * single cache line on all modern x86 implementations.
+ * Build the entry stubs with some assembler magic.
+ * We pack 1 stub into every 8-byte block.
*/
-.section .init.rodata,"a"
-ENTRY(interrupt)
-.section .entry.text, "ax"
- .p2align 5
- .p2align CONFIG_X86_L1_CACHE_SHIFT
+ .align 8
ENTRY(irq_entries_start)
RING0_INT_FRAME
-vector=FIRST_EXTERNAL_VECTOR
-.rept (FIRST_SYSTEM_VECTOR-FIRST_EXTERNAL_VECTOR+6)/7
- .balign 32
- .rept 7
- .if vector < FIRST_SYSTEM_VECTOR
- .if vector <> FIRST_EXTERNAL_VECTOR
+ vector=FIRST_EXTERNAL_VECTOR
+ .rept (FIRST_SYSTEM_VECTOR - FIRST_EXTERNAL_VECTOR)
+ pushl_cfi $(~vector+0x80) /* Note: always in signed byte range */
+ vector=vector+1
+ jmp common_interrupt
CFI_ADJUST_CFA_OFFSET -4
- .endif
-1: pushl_cfi $(~vector+0x80) /* Note: always in signed byte range */
- .if ((vector-FIRST_EXTERNAL_VECTOR)%7) <> 6
- jmp 2f
- .endif
- .previous
- .long 1b
- .section .entry.text, "ax"
-vector=vector+1
- .endif
- .endr
-2: jmp common_interrupt
-.endr
+ .align 8
+ .endr
END(irq_entries_start)
-.previous
-END(interrupt)
-.previous
-
/*
* the CPU automatically disables interrupts when executing an IRQ vector,
* so IRQ-flags tracing has to follow that:
pushl_cfi $0
#ifdef CONFIG_X86_INVD_BUG
/* AMD 486 bug: invd from userspace calls exception 19 instead of #GP */
-661: pushl_cfi $do_general_protection
-662:
-.section .altinstructions,"a"
- altinstruction_entry 661b, 663f, X86_FEATURE_XMM, 662b-661b, 664f-663f
-.previous
-.section .altinstr_replacement,"ax"
-663: pushl $do_simd_coprocessor_error
-664:
-.previous
+ ALTERNATIVE "pushl_cfi $do_general_protection", \
+ "pushl $do_simd_coprocessor_error", \
+ X86_FEATURE_XMM
#else
pushl_cfi $do_simd_coprocessor_error
#endif
ENTRY(xen_do_upcall)
1: mov %esp, %eax
call xen_evtchn_do_upcall
+#ifndef CONFIG_PREEMPT
+ call xen_maybe_preempt_hcall
+#endif
jmp ret_from_intr
CFI_ENDPROC
ENDPROC(xen_hypervisor_callback)
/*CFI_REL_OFFSET es, 0*/
pushl_cfi %ds
/*CFI_REL_OFFSET ds, 0*/
- pushl_cfi %eax
- CFI_REL_OFFSET eax, 0
- pushl_cfi %ebp
- CFI_REL_OFFSET ebp, 0
- pushl_cfi %edi
- CFI_REL_OFFSET edi, 0
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
- pushl_cfi %edx
- CFI_REL_OFFSET edx, 0
- pushl_cfi %ecx
- CFI_REL_OFFSET ecx, 0
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
+ pushl_cfi_reg eax
+ pushl_cfi_reg ebp
+ pushl_cfi_reg edi
+ pushl_cfi_reg esi
+ pushl_cfi_reg edx
+ pushl_cfi_reg ecx
+ pushl_cfi_reg ebx
cld
movl $(__KERNEL_PERCPU), %ecx
movl %ecx, %fs
* NOTE: This code handles signal-recognition, which happens every time
* after an interrupt and after each system call.
*
- * Normal syscalls and interrupts don't save a full stack frame, this is
- * only done for syscall tracing, signals or fork/exec et.al.
- *
* A note on terminology:
- * - top of stack: Architecture defined interrupt frame from SS to RIP
+ * - iret frame: Architecture defined interrupt frame from SS to RIP
* at the top of the kernel process stack.
- * - partial stack frame: partially saved registers up to R11.
- * - full stack frame: Like partial stack frame, but all register saved.
*
* Some macro usage:
* - CFI macros are used to generate dwarf2 unwind information for better
* backtraces. They don't change any code.
- * - SAVE_ALL/RESTORE_ALL - Save/restore all registers
- * - SAVE_ARGS/RESTORE_ARGS - Save/restore registers that C functions modify.
- * There are unfortunately lots of special cases where some registers
- * not touched. The macro is a big mess that should be cleaned up.
- * - SAVE_REST/RESTORE_REST - Handle the registers not saved by SAVE_ARGS.
- * Gives a full stack frame.
* - ENTRY/END Define functions in the symbol table.
- * - FIXUP_TOP_OF_STACK/RESTORE_TOP_OF_STACK - Fix up the hardware stack
- * frame that is otherwise undefined after a SYSCALL
* - TRACE_IRQ_* - Trace hard interrupt state for lock debugging.
* - idtentry - Define exception entry points.
*/
.section .entry.text, "ax"
-#ifndef CONFIG_PREEMPT
-#define retint_kernel retint_restore_args
-#endif
-
#ifdef CONFIG_PARAVIRT
ENTRY(native_usergs_sysret64)
swapgs
#endif /* CONFIG_PARAVIRT */
-.macro TRACE_IRQS_IRETQ offset=ARGOFFSET
+.macro TRACE_IRQS_IRETQ
#ifdef CONFIG_TRACE_IRQFLAGS
- bt $9,EFLAGS-\offset(%rsp) /* interrupts off? */
+ bt $9,EFLAGS(%rsp) /* interrupts off? */
jnc 1f
TRACE_IRQS_ON
1:
call debug_stack_reset
.endm
-.macro TRACE_IRQS_IRETQ_DEBUG offset=ARGOFFSET
- bt $9,EFLAGS-\offset(%rsp) /* interrupts off? */
+.macro TRACE_IRQS_IRETQ_DEBUG
+ bt $9,EFLAGS(%rsp) /* interrupts off? */
jnc 1f
TRACE_IRQS_ON_DEBUG
1:
#endif
/*
- * C code is not supposed to know about undefined top of stack. Every time
- * a C function with an pt_regs argument is called from the SYSCALL based
- * fast path FIXUP_TOP_OF_STACK is needed.
- * RESTORE_TOP_OF_STACK syncs the syscall state after any possible ptregs
- * manipulation.
- */
-
- /* %rsp:at FRAMEEND */
- .macro FIXUP_TOP_OF_STACK tmp offset=0
- movq PER_CPU_VAR(old_rsp),\tmp
- movq \tmp,RSP+\offset(%rsp)
- movq $__USER_DS,SS+\offset(%rsp)
- movq $__USER_CS,CS+\offset(%rsp)
- movq RIP+\offset(%rsp),\tmp /* get rip */
- movq \tmp,RCX+\offset(%rsp) /* copy it to rcx as sysret would do */
- movq R11+\offset(%rsp),\tmp /* get eflags */
- movq \tmp,EFLAGS+\offset(%rsp)
- .endm
-
- .macro RESTORE_TOP_OF_STACK tmp offset=0
- movq RSP+\offset(%rsp),\tmp
- movq \tmp,PER_CPU_VAR(old_rsp)
- movq EFLAGS+\offset(%rsp),\tmp
- movq \tmp,R11+\offset(%rsp)
- .endm
-
-/*
- * initial frame state for interrupts (and exceptions without error code)
+ * empty frame
*/
.macro EMPTY_FRAME start=1 offset=0
.if \start
* initial frame state for interrupts (and exceptions without error code)
*/
.macro INTR_FRAME start=1 offset=0
- EMPTY_FRAME \start, SS+8+\offset-RIP
- /*CFI_REL_OFFSET ss, SS+\offset-RIP*/
- CFI_REL_OFFSET rsp, RSP+\offset-RIP
- /*CFI_REL_OFFSET rflags, EFLAGS+\offset-RIP*/
- /*CFI_REL_OFFSET cs, CS+\offset-RIP*/
- CFI_REL_OFFSET rip, RIP+\offset-RIP
+ EMPTY_FRAME \start, 5*8+\offset
+ /*CFI_REL_OFFSET ss, 4*8+\offset*/
+ CFI_REL_OFFSET rsp, 3*8+\offset
+ /*CFI_REL_OFFSET rflags, 2*8+\offset*/
+ /*CFI_REL_OFFSET cs, 1*8+\offset*/
+ CFI_REL_OFFSET rip, 0*8+\offset
.endm
/*
* with vector already pushed)
*/
.macro XCPT_FRAME start=1 offset=0
- INTR_FRAME \start, RIP+\offset-ORIG_RAX
- .endm
-
-/*
- * frame that enables calling into C.
- */
- .macro PARTIAL_FRAME start=1 offset=0
- XCPT_FRAME \start, ORIG_RAX+\offset-ARGOFFSET
- CFI_REL_OFFSET rdi, RDI+\offset-ARGOFFSET
- CFI_REL_OFFSET rsi, RSI+\offset-ARGOFFSET
- CFI_REL_OFFSET rdx, RDX+\offset-ARGOFFSET
- CFI_REL_OFFSET rcx, RCX+\offset-ARGOFFSET
- CFI_REL_OFFSET rax, RAX+\offset-ARGOFFSET
- CFI_REL_OFFSET r8, R8+\offset-ARGOFFSET
- CFI_REL_OFFSET r9, R9+\offset-ARGOFFSET
- CFI_REL_OFFSET r10, R10+\offset-ARGOFFSET
- CFI_REL_OFFSET r11, R11+\offset-ARGOFFSET
+ INTR_FRAME \start, 1*8+\offset
.endm
/*
* frame that enables passing a complete pt_regs to a C function.
*/
.macro DEFAULT_FRAME start=1 offset=0
- PARTIAL_FRAME \start, R11+\offset-R15
+ XCPT_FRAME \start, ORIG_RAX+\offset
+ CFI_REL_OFFSET rdi, RDI+\offset
+ CFI_REL_OFFSET rsi, RSI+\offset
+ CFI_REL_OFFSET rdx, RDX+\offset
+ CFI_REL_OFFSET rcx, RCX+\offset
+ CFI_REL_OFFSET rax, RAX+\offset
+ CFI_REL_OFFSET r8, R8+\offset
+ CFI_REL_OFFSET r9, R9+\offset
+ CFI_REL_OFFSET r10, R10+\offset
+ CFI_REL_OFFSET r11, R11+\offset
CFI_REL_OFFSET rbx, RBX+\offset
CFI_REL_OFFSET rbp, RBP+\offset
CFI_REL_OFFSET r12, R12+\offset
CFI_REL_OFFSET r15, R15+\offset
.endm
-ENTRY(save_paranoid)
- XCPT_FRAME 1 RDI+8
- cld
- movq %rdi, RDI+8(%rsp)
- movq %rsi, RSI+8(%rsp)
- movq_cfi rdx, RDX+8
- movq_cfi rcx, RCX+8
- movq_cfi rax, RAX+8
- movq %r8, R8+8(%rsp)
- movq %r9, R9+8(%rsp)
- movq %r10, R10+8(%rsp)
- movq %r11, R11+8(%rsp)
- movq_cfi rbx, RBX+8
- movq %rbp, RBP+8(%rsp)
- movq %r12, R12+8(%rsp)
- movq %r13, R13+8(%rsp)
- movq %r14, R14+8(%rsp)
- movq %r15, R15+8(%rsp)
- movl $1,%ebx
- movl $MSR_GS_BASE,%ecx
- rdmsr
- testl %edx,%edx
- js 1f /* negative -> in kernel */
- SWAPGS
- xorl %ebx,%ebx
-1: ret
- CFI_ENDPROC
-END(save_paranoid)
-
/*
- * A newly forked process directly context switches into this address.
+ * 64bit SYSCALL instruction entry. Up to 6 arguments in registers.
*
- * rdi: prev task we switched from
- */
-ENTRY(ret_from_fork)
- DEFAULT_FRAME
-
- LOCK ; btr $TIF_FORK,TI_flags(%r8)
-
- pushq_cfi $0x0002
- popfq_cfi # reset kernel eflags
-
- call schedule_tail # rdi: 'prev' task parameter
-
- GET_THREAD_INFO(%rcx)
-
- RESTORE_REST
-
- testl $3, CS-ARGOFFSET(%rsp) # from kernel_thread?
- jz 1f
-
- testl $_TIF_IA32, TI_flags(%rcx) # 32-bit compat task needs IRET
- jnz int_ret_from_sys_call
-
- RESTORE_TOP_OF_STACK %rdi, -ARGOFFSET
- jmp ret_from_sys_call # go to the SYSRET fastpath
-
-1:
- subq $REST_SKIP, %rsp # leave space for volatiles
- CFI_ADJUST_CFA_OFFSET REST_SKIP
- movq %rbp, %rdi
- call *%rbx
- movl $0, RAX(%rsp)
- RESTORE_REST
- jmp int_ret_from_sys_call
- CFI_ENDPROC
-END(ret_from_fork)
-
-/*
- * System call entry. Up to 6 arguments in registers are supported.
+ * 64bit SYSCALL saves rip to rcx, clears rflags.RF, then saves rflags to r11,
+ * then loads new ss, cs, and rip from previously programmed MSRs.
+ * rflags gets masked by a value from another MSR (so CLD and CLAC
+ * are not needed). SYSCALL does not save anything on the stack
+ * and does not change rsp.
*
- * SYSCALL does not save anything on the stack and does not change the
- * stack pointer. However, it does mask the flags register for us, so
- * CLD and CLAC are not needed.
- */
-
-/*
- * Register setup:
+ * Registers on entry:
* rax system call number
+ * rcx return address
+ * r11 saved rflags (note: r11 is callee-clobbered register in C ABI)
* rdi arg0
- * rcx return address for syscall/sysret, C arg3
* rsi arg1
* rdx arg2
- * r10 arg3 (--> moved to rcx for C)
+ * r10 arg3 (needs to be moved to rcx to conform to C ABI)
* r8 arg4
* r9 arg5
- * r11 eflags for syscall/sysret, temporary for C
- * r12-r15,rbp,rbx saved by C code, not touched.
+ * (note: r12-r15,rbp,rbx are callee-preserved in C ABI)
*
- * Interrupts are off on entry.
* Only called from user space.
*
- * XXX if we had a free scratch register we could save the RSP into the stack frame
- * and report it properly in ps. Unfortunately we haven't.
- *
- * When user can change the frames always force IRET. That is because
+ * When user can change pt_regs->foo always force IRET. That is because
* it deals with uncanonical addresses better. SYSRET has trouble
* with them due to bugs in both AMD and Intel CPUs.
*/
ENTRY(system_call)
CFI_STARTPROC simple
CFI_SIGNAL_FRAME
- CFI_DEF_CFA rsp,KERNEL_STACK_OFFSET
+ CFI_DEF_CFA rsp,0
CFI_REGISTER rip,rcx
/*CFI_REGISTER rflags,r11*/
+
+ /*
+ * Interrupts are off on entry.
+ * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON,
+ * it is too small to ever cause noticeable irq latency.
+ */
SWAPGS_UNSAFE_STACK
/*
* A hypervisor implementation might want to use a label
*/
GLOBAL(system_call_after_swapgs)
- movq %rsp,PER_CPU_VAR(old_rsp)
+ movq %rsp,PER_CPU_VAR(rsp_scratch)
movq PER_CPU_VAR(kernel_stack),%rsp
+
+ /* Construct struct pt_regs on stack */
+ pushq_cfi $__USER_DS /* pt_regs->ss */
+ pushq_cfi PER_CPU_VAR(rsp_scratch) /* pt_regs->sp */
/*
- * No need to follow this irqs off/on section - it's straight
- * and short:
+ * Re-enable interrupts.
+ * We use 'rsp_scratch' as a scratch space, hence irq-off block above
+ * must execute atomically in the face of possible interrupt-driven
+ * task preemption. We must enable interrupts only after we're done
+ * with using rsp_scratch:
*/
ENABLE_INTERRUPTS(CLBR_NONE)
- SAVE_ARGS 8, 0, rax_enosys=1
- movq_cfi rax,(ORIG_RAX-ARGOFFSET)
- movq %rcx,RIP-ARGOFFSET(%rsp)
- CFI_REL_OFFSET rip,RIP-ARGOFFSET
- testl $_TIF_WORK_SYSCALL_ENTRY,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
+ pushq_cfi %r11 /* pt_regs->flags */
+ pushq_cfi $__USER_CS /* pt_regs->cs */
+ pushq_cfi %rcx /* pt_regs->ip */
+ CFI_REL_OFFSET rip,0
+ pushq_cfi_reg rax /* pt_regs->orig_ax */
+ pushq_cfi_reg rdi /* pt_regs->di */
+ pushq_cfi_reg rsi /* pt_regs->si */
+ pushq_cfi_reg rdx /* pt_regs->dx */
+ pushq_cfi_reg rcx /* pt_regs->cx */
+ pushq_cfi $-ENOSYS /* pt_regs->ax */
+ pushq_cfi_reg r8 /* pt_regs->r8 */
+ pushq_cfi_reg r9 /* pt_regs->r9 */
+ pushq_cfi_reg r10 /* pt_regs->r10 */
+ pushq_cfi_reg r11 /* pt_regs->r11 */
+ sub $(6*8),%rsp /* pt_regs->bp,bx,r12-15 not saved */
+ CFI_ADJUST_CFA_OFFSET 6*8
+
+ testl $_TIF_WORK_SYSCALL_ENTRY, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
jnz tracesys
system_call_fastpath:
#if __SYSCALL_MASK == ~0
andl $__SYSCALL_MASK,%eax
cmpl $__NR_syscall_max,%eax
#endif
- ja ret_from_sys_call /* and return regs->ax */
+ ja 1f /* return -ENOSYS (already in pt_regs->ax) */
movq %r10,%rcx
- call *sys_call_table(,%rax,8) # XXX: rip relative
- movq %rax,RAX-ARGOFFSET(%rsp)
+ call *sys_call_table(,%rax,8)
+ movq %rax,RAX(%rsp)
+1:
/*
- * Syscall return path ending with SYSRET (fast path)
- * Has incomplete stack frame and undefined top of stack.
+ * Syscall return path ending with SYSRET (fast path).
+ * Has incompletely filled pt_regs.
*/
-ret_from_sys_call:
- testl $_TIF_ALLWORK_MASK,TI_flags+THREAD_INFO(%rsp,RIP-ARGOFFSET)
- jnz int_ret_from_sys_call_fixup /* Go the the slow path */
-
LOCKDEP_SYS_EXIT
+ /*
+ * We do not frame this tiny irq-off block with TRACE_IRQS_OFF/ON,
+ * it is too small to ever cause noticeable irq latency.
+ */
DISABLE_INTERRUPTS(CLBR_NONE)
- TRACE_IRQS_OFF
- CFI_REMEMBER_STATE
+
/*
- * sysretq will re-enable interrupts:
+ * We must check ti flags with interrupts (or at least preemption)
+ * off because we must *never* return to userspace without
+ * processing exit work that is enqueued if we're preempted here.
+ * In particular, returning to userspace with any of the one-shot
+ * flags (TIF_NOTIFY_RESUME, TIF_USER_RETURN_NOTIFY, etc) set is
+ * very bad.
*/
- TRACE_IRQS_ON
- movq RIP-ARGOFFSET(%rsp),%rcx
+ testl $_TIF_ALLWORK_MASK, ASM_THREAD_INFO(TI_flags, %rsp, SIZEOF_PTREGS)
+ jnz int_ret_from_sys_call_irqs_off /* Go to the slow path */
+
+ CFI_REMEMBER_STATE
+
+ RESTORE_C_REGS_EXCEPT_RCX_R11
+ movq RIP(%rsp),%rcx
CFI_REGISTER rip,rcx
- RESTORE_ARGS 1,-ARG_SKIP,0
+ movq EFLAGS(%rsp),%r11
/*CFI_REGISTER rflags,r11*/
- movq PER_CPU_VAR(old_rsp), %rsp
+ movq RSP(%rsp),%rsp
+ /*
+ * 64bit SYSRET restores rip from rcx,
+ * rflags from r11 (but RF and VM bits are forced to 0),
+ * cs and ss are loaded from MSRs.
+ * Restoration of rflags re-enables interrupts.
+ */
USERGS_SYSRET64
CFI_RESTORE_STATE
-int_ret_from_sys_call_fixup:
- FIXUP_TOP_OF_STACK %r11, -ARGOFFSET
- jmp int_ret_from_sys_call
-
- /* Do syscall tracing */
+ /* Do syscall entry tracing */
tracesys:
- leaq -REST_SKIP(%rsp), %rdi
- movq $AUDIT_ARCH_X86_64, %rsi
+ movq %rsp, %rdi
+ movl $AUDIT_ARCH_X86_64, %esi
call syscall_trace_enter_phase1
test %rax, %rax
jnz tracesys_phase2 /* if needed, run the slow path */
- LOAD_ARGS 0 /* else restore clobbered regs */
+ RESTORE_C_REGS_EXCEPT_RAX /* else restore clobbered regs */
+ movq ORIG_RAX(%rsp), %rax
jmp system_call_fastpath /* and return to the fast path */
tracesys_phase2:
- SAVE_REST
- FIXUP_TOP_OF_STACK %rdi
+ SAVE_EXTRA_REGS
movq %rsp, %rdi
- movq $AUDIT_ARCH_X86_64, %rsi
+ movl $AUDIT_ARCH_X86_64, %esi
movq %rax,%rdx
call syscall_trace_enter_phase2
/*
- * Reload arg registers from stack in case ptrace changed them.
+ * Reload registers from stack in case ptrace changed them.
* We don't reload %rax because syscall_trace_entry_phase2() returned
* the value it wants us to use in the table lookup.
*/
- LOAD_ARGS ARGOFFSET, 1
- RESTORE_REST
+ RESTORE_C_REGS_EXCEPT_RAX
+ RESTORE_EXTRA_REGS
#if __SYSCALL_MASK == ~0
cmpq $__NR_syscall_max,%rax
#else
andl $__SYSCALL_MASK,%eax
cmpl $__NR_syscall_max,%eax
#endif
- ja int_ret_from_sys_call /* RAX(%rsp) is already set */
+ ja 1f /* return -ENOSYS (already in pt_regs->ax) */
movq %r10,%rcx /* fixup for C */
call *sys_call_table(,%rax,8)
- movq %rax,RAX-ARGOFFSET(%rsp)
- /* Use IRET because user could have changed frame */
+ movq %rax,RAX(%rsp)
+1:
+ /* Use IRET because user could have changed pt_regs->foo */
/*
* Syscall return path ending with IRET.
- * Has correct top of stack, but partial stack frame.
+ * Has correct iret frame.
*/
GLOBAL(int_ret_from_sys_call)
DISABLE_INTERRUPTS(CLBR_NONE)
+int_ret_from_sys_call_irqs_off: /* jumps come here from the irqs-off SYSRET path */
TRACE_IRQS_OFF
movl $_TIF_ALLWORK_MASK,%edi
/* edi: mask to check */
movl TI_flags(%rcx),%edx
andl %edi,%edx
jnz int_careful
- andl $~TS_COMPAT,TI_status(%rcx)
- jmp retint_swapgs
+ andl $~TS_COMPAT,TI_status(%rcx)
+ jmp syscall_return
/* Either reschedule or signal or syscall exit tracking needed. */
/* First do a reschedule test. */
TRACE_IRQS_OFF
jmp int_with_check
- /* handle signals and tracing -- both require a full stack frame */
+ /* handle signals and tracing -- both require a full pt_regs */
int_very_careful:
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
-int_check_syscall_exit_work:
- SAVE_REST
+ SAVE_EXTRA_REGS
/* Check for syscall exit trace */
testl $_TIF_WORK_SYSCALL_EXIT,%edx
jz int_signal
call do_notify_resume
1: movl $_TIF_WORK_MASK,%edi
int_restore_rest:
- RESTORE_REST
+ RESTORE_EXTRA_REGS
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
jmp int_with_check
+
+syscall_return:
+ /* The IRETQ could re-enable interrupts: */
+ DISABLE_INTERRUPTS(CLBR_ANY)
+ TRACE_IRQS_IRETQ
+
+ /*
+ * Try to use SYSRET instead of IRET if we're returning to
+ * a completely clean 64-bit userspace context.
+ */
+ movq RCX(%rsp),%rcx
+ cmpq %rcx,RIP(%rsp) /* RCX == RIP */
+ jne opportunistic_sysret_failed
+
+ /*
+ * On Intel CPUs, SYSRET with non-canonical RCX/RIP will #GP
+ * in kernel space. This essentially lets the user take over
+ * the kernel, since userspace controls RSP. It's not worth
+ * testing for canonicalness exactly -- this check detects any
+ * of the 17 high bits set, which is true for non-canonical
+ * or kernel addresses. (This will pessimize vsyscall=native.
+ * Big deal.)
+ *
+ * If virtual addresses ever become wider, this will need
+ * to be updated to remain correct on both old and new CPUs.
+ */
+ .ifne __VIRTUAL_MASK_SHIFT - 47
+ .error "virtual address width changed -- SYSRET checks need update"
+ .endif
+ shr $__VIRTUAL_MASK_SHIFT, %rcx
+ jnz opportunistic_sysret_failed
+
+ cmpq $__USER_CS,CS(%rsp) /* CS must match SYSRET */
+ jne opportunistic_sysret_failed
+
+ movq R11(%rsp),%r11
+ cmpq %r11,EFLAGS(%rsp) /* R11 == RFLAGS */
+ jne opportunistic_sysret_failed
+
+ /*
+ * SYSRET can't restore RF. SYSRET can restore TF, but unlike IRET,
+ * restoring TF results in a trap from userspace immediately after
+ * SYSRET. This would cause an infinite loop whenever #DB happens
+ * with register state that satisfies the opportunistic SYSRET
+ * conditions. For example, single-stepping this user code:
+ *
+ * movq $stuck_here,%rcx
+ * pushfq
+ * popq %r11
+ * stuck_here:
+ *
+ * would never get past 'stuck_here'.
+ */
+ testq $(X86_EFLAGS_RF|X86_EFLAGS_TF), %r11
+ jnz opportunistic_sysret_failed
+
+ /* nothing to check for RSP */
+
+ cmpq $__USER_DS,SS(%rsp) /* SS must match SYSRET */
+ jne opportunistic_sysret_failed
+
+ /*
+ * We win! This label is here just for ease of understanding
+ * perf profiles. Nothing jumps here.
+ */
+syscall_return_via_sysret:
+ CFI_REMEMBER_STATE
+ /* r11 is already restored (see code above) */
+ RESTORE_C_REGS_EXCEPT_R11
+ movq RSP(%rsp),%rsp
+ USERGS_SYSRET64
+ CFI_RESTORE_STATE
+
+opportunistic_sysret_failed:
+ SWAPGS
+ jmp restore_c_regs_and_iret
CFI_ENDPROC
END(system_call)
+
.macro FORK_LIKE func
ENTRY(stub_\func)
CFI_STARTPROC
- popq %r11 /* save return address */
- PARTIAL_FRAME 0
- SAVE_REST
- pushq %r11 /* put it back on stack */
- FIXUP_TOP_OF_STACK %r11, 8
- DEFAULT_FRAME 0 8 /* offset 8: return address */
- call sys_\func
- RESTORE_TOP_OF_STACK %r11, 8
- ret $REST_SKIP /* pop extended registers */
+ DEFAULT_FRAME 0, 8 /* offset 8: return address */
+ SAVE_EXTRA_REGS 8
+ jmp sys_\func
CFI_ENDPROC
END(stub_\func)
.endm
- .macro FIXED_FRAME label,func
-ENTRY(\label)
- CFI_STARTPROC
- PARTIAL_FRAME 0 8 /* offset 8: return address */
- FIXUP_TOP_OF_STACK %r11, 8-ARGOFFSET
- call \func
- RESTORE_TOP_OF_STACK %r11, 8-ARGOFFSET
- ret
- CFI_ENDPROC
-END(\label)
- .endm
-
FORK_LIKE clone
FORK_LIKE fork
FORK_LIKE vfork
- FIXED_FRAME stub_iopl, sys_iopl
ENTRY(stub_execve)
CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
- call sys_execve
- movq %rax,RAX(%rsp)
- RESTORE_REST
- jmp int_ret_from_sys_call
+ DEFAULT_FRAME 0, 8
+ call sys_execve
+return_from_execve:
+ testl %eax, %eax
+ jz 1f
+ /* exec failed, can use fast SYSRET code path in this case */
+ ret
+1:
+ /* must use IRET code path (pt_regs->cs may have changed) */
+ addq $8, %rsp
+ CFI_ADJUST_CFA_OFFSET -8
+ ZERO_EXTRA_REGS
+ movq %rax,RAX(%rsp)
+ jmp int_ret_from_sys_call
CFI_ENDPROC
END(stub_execve)
-
-ENTRY(stub_execveat)
+/*
+ * Remaining execve stubs are only 7 bytes long.
+ * ENTRY() often aligns to 16 bytes, which in this case has no benefits.
+ */
+ .align 8
+GLOBAL(stub_execveat)
CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
- call sys_execveat
- RESTORE_TOP_OF_STACK %r11
- movq %rax,RAX(%rsp)
- RESTORE_REST
- jmp int_ret_from_sys_call
+ DEFAULT_FRAME 0, 8
+ call sys_execveat
+ jmp return_from_execve
CFI_ENDPROC
END(stub_execveat)
+#ifdef CONFIG_X86_X32_ABI
+ .align 8
+GLOBAL(stub_x32_execve)
+ CFI_STARTPROC
+ DEFAULT_FRAME 0, 8
+ call compat_sys_execve
+ jmp return_from_execve
+ CFI_ENDPROC
+END(stub_x32_execve)
+ .align 8
+GLOBAL(stub_x32_execveat)
+ CFI_STARTPROC
+ DEFAULT_FRAME 0, 8
+ call compat_sys_execveat
+ jmp return_from_execve
+ CFI_ENDPROC
+END(stub_x32_execveat)
+#endif
+
+#ifdef CONFIG_IA32_EMULATION
+ .align 8
+GLOBAL(stub32_execve)
+ CFI_STARTPROC
+ call compat_sys_execve
+ jmp return_from_execve
+ CFI_ENDPROC
+END(stub32_execve)
+ .align 8
+GLOBAL(stub32_execveat)
+ CFI_STARTPROC
+ call compat_sys_execveat
+ jmp return_from_execve
+ CFI_ENDPROC
+END(stub32_execveat)
+#endif
+
/*
* sigreturn is special because it needs to restore all registers on return.
* This cannot be done with SYSRET, so use the IRET return path instead.
*/
ENTRY(stub_rt_sigreturn)
CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
+ DEFAULT_FRAME 0, 8
+ /*
+ * SAVE_EXTRA_REGS result is not normally needed:
+ * sigreturn overwrites all pt_regs->GPREGS.
+ * But sigreturn can fail (!), and there is no easy way to detect that.
+ * To make sure RESTORE_EXTRA_REGS doesn't restore garbage on error,
+ * we SAVE_EXTRA_REGS here.
+ */
+ SAVE_EXTRA_REGS 8
call sys_rt_sigreturn
- movq %rax,RAX(%rsp) # fixme, this could be done at the higher layer
- RESTORE_REST
+return_from_stub:
+ addq $8, %rsp
+ CFI_ADJUST_CFA_OFFSET -8
+ RESTORE_EXTRA_REGS
+ movq %rax,RAX(%rsp)
jmp int_ret_from_sys_call
CFI_ENDPROC
END(stub_rt_sigreturn)
#ifdef CONFIG_X86_X32_ABI
ENTRY(stub_x32_rt_sigreturn)
CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
+ DEFAULT_FRAME 0, 8
+ SAVE_EXTRA_REGS 8
call sys32_x32_rt_sigreturn
- movq %rax,RAX(%rsp) # fixme, this could be done at the higher layer
- RESTORE_REST
- jmp int_ret_from_sys_call
+ jmp return_from_stub
CFI_ENDPROC
END(stub_x32_rt_sigreturn)
+#endif
-ENTRY(stub_x32_execve)
- CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
- call compat_sys_execve
- RESTORE_TOP_OF_STACK %r11
- movq %rax,RAX(%rsp)
- RESTORE_REST
- jmp int_ret_from_sys_call
- CFI_ENDPROC
-END(stub_x32_execve)
+/*
+ * A newly forked process directly context switches into this address.
+ *
+ * rdi: prev task we switched from
+ */
+ENTRY(ret_from_fork)
+ DEFAULT_FRAME
-ENTRY(stub_x32_execveat)
- CFI_STARTPROC
- addq $8, %rsp
- PARTIAL_FRAME 0
- SAVE_REST
- FIXUP_TOP_OF_STACK %r11
- call compat_sys_execveat
- RESTORE_TOP_OF_STACK %r11
- movq %rax,RAX(%rsp)
- RESTORE_REST
+ LOCK ; btr $TIF_FORK,TI_flags(%r8)
+
+ pushq_cfi $0x0002
+ popfq_cfi # reset kernel eflags
+
+ call schedule_tail # rdi: 'prev' task parameter
+
+ RESTORE_EXTRA_REGS
+
+ testl $3,CS(%rsp) # from kernel_thread?
+
+ /*
+ * By the time we get here, we have no idea whether our pt_regs,
+ * ti flags, and ti status came from the 64-bit SYSCALL fast path,
+ * the slow path, or one of the ia32entry paths.
+ * Use IRET code path to return, since it can safely handle
+ * all of the above.
+ */
+ jnz int_ret_from_sys_call
+
+ /* We came from kernel_thread */
+ /* nb: we depend on RESTORE_EXTRA_REGS above */
+ movq %rbp, %rdi
+ call *%rbx
+ movl $0, RAX(%rsp)
+ RESTORE_EXTRA_REGS
jmp int_ret_from_sys_call
CFI_ENDPROC
-END(stub_x32_execveat)
-
-#endif
+END(ret_from_fork)
/*
- * Build the entry stubs and pointer table with some assembler magic.
- * We pack 7 stubs into a single 32-byte chunk, which will fit in a
- * single cache line on all modern x86 implementations.
+ * Build the entry stubs with some assembler magic.
+ * We pack 1 stub into every 8-byte block.
*/
- .section .init.rodata,"a"
-ENTRY(interrupt)
- .section .entry.text
- .p2align 5
- .p2align CONFIG_X86_L1_CACHE_SHIFT
+ .align 8
ENTRY(irq_entries_start)
INTR_FRAME
-vector=FIRST_EXTERNAL_VECTOR
-.rept (FIRST_SYSTEM_VECTOR-FIRST_EXTERNAL_VECTOR+6)/7
- .balign 32
- .rept 7
- .if vector < FIRST_SYSTEM_VECTOR
- .if vector <> FIRST_EXTERNAL_VECTOR
+ vector=FIRST_EXTERNAL_VECTOR
+ .rept (FIRST_SYSTEM_VECTOR - FIRST_EXTERNAL_VECTOR)
+ pushq_cfi $(~vector+0x80) /* Note: always in signed byte range */
+ vector=vector+1
+ jmp common_interrupt
CFI_ADJUST_CFA_OFFSET -8
- .endif
-1: pushq_cfi $(~vector+0x80) /* Note: always in signed byte range */
- .if ((vector-FIRST_EXTERNAL_VECTOR)%7) <> 6
- jmp 2f
- .endif
- .previous
- .quad 1b
- .section .entry.text
-vector=vector+1
- .endif
- .endr
-2: jmp common_interrupt
-.endr
+ .align 8
+ .endr
CFI_ENDPROC
END(irq_entries_start)
-.previous
-END(interrupt)
-.previous
-
/*
* Interrupt entry/exit.
*
/* 0(%rsp): ~(interrupt number) */
.macro interrupt func
- /* reserve pt_regs for scratch regs and rbp */
- subq $ORIG_RAX-RBP, %rsp
- CFI_ADJUST_CFA_OFFSET ORIG_RAX-RBP
cld
- /* start from rbp in pt_regs and jump over */
- movq_cfi rdi, (RDI-RBP)
- movq_cfi rsi, (RSI-RBP)
- movq_cfi rdx, (RDX-RBP)
- movq_cfi rcx, (RCX-RBP)
- movq_cfi rax, (RAX-RBP)
- movq_cfi r8, (R8-RBP)
- movq_cfi r9, (R9-RBP)
- movq_cfi r10, (R10-RBP)
- movq_cfi r11, (R11-RBP)
-
- /* Save rbp so that we can unwind from get_irq_regs() */
- movq_cfi rbp, 0
-
- /* Save previous stack value */
- movq %rsp, %rsi
+ /*
+ * Since nothing in interrupt handling code touches r12...r15 members
+ * of "struct pt_regs", and since interrupts can nest, we can save
+ * four stack slots and simultaneously provide
+ * an unwind-friendly stack layout by saving "truncated" pt_regs
+ * exactly up to rbp slot, without these members.
+ */
+ ALLOC_PT_GPREGS_ON_STACK -RBP
+ SAVE_C_REGS -RBP
+ /* this goes to 0(%rsp) for unwinder, not for saving the value: */
+ SAVE_EXTRA_REGS_RBP -RBP
+
+ leaq -RBP(%rsp),%rdi /* arg1 for \func (pointer to pt_regs) */
- leaq -RBP(%rsp),%rdi /* arg1 for handler */
- testl $3, CS-RBP(%rsi)
+ testl $3, CS-RBP(%rsp)
je 1f
SWAPGS
+1:
/*
+ * Save previous stack pointer, optionally switch to interrupt stack.
* irq_count is used to check if a CPU is already on an interrupt stack
* or not. While this is essentially redundant with preempt_count it is
* a little cheaper to use a separate counter in the PDA (short of
* moving irq_enter into assembly, which would be too much work)
*/
-1: incl PER_CPU_VAR(irq_count)
+ movq %rsp, %rsi
+ incl PER_CPU_VAR(irq_count)
cmovzq PER_CPU_VAR(irq_stack_ptr),%rsp
CFI_DEF_CFA_REGISTER rsi
-
- /* Store previous stack value */
pushq %rsi
+ /*
+ * For debugger:
+ * "CFA (Current Frame Address) is the value on stack + offset"
+ */
CFI_ESCAPE 0x0f /* DW_CFA_def_cfa_expression */, 6, \
- 0x77 /* DW_OP_breg7 */, 0, \
+ 0x77 /* DW_OP_breg7 (rsp) */, 0, \
0x06 /* DW_OP_deref */, \
- 0x08 /* DW_OP_const1u */, SS+8-RBP, \
+ 0x08 /* DW_OP_const1u */, SIZEOF_PTREGS-RBP, \
0x22 /* DW_OP_plus */
/* We entered an interrupt context - irqs are off: */
TRACE_IRQS_OFF
ASM_CLAC
addq $-0x80,(%rsp) /* Adjust vector to [-256,-1] range */
interrupt do_IRQ
- /* 0(%rsp): old_rsp-ARGOFFSET */
+ /* 0(%rsp): old RSP */
ret_from_intr:
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
/* Restore saved previous stack */
popq %rsi
- CFI_DEF_CFA rsi,SS+8-RBP /* reg/off reset after def_cfa_expr */
- leaq ARGOFFSET-RBP(%rsi), %rsp
+ CFI_DEF_CFA rsi,SIZEOF_PTREGS-RBP /* reg/off reset after def_cfa_expr */
+ /* return code expects complete pt_regs - adjust rsp accordingly: */
+ leaq -RBP(%rsi),%rsp
CFI_DEF_CFA_REGISTER rsp
- CFI_ADJUST_CFA_OFFSET RBP-ARGOFFSET
+ CFI_ADJUST_CFA_OFFSET RBP
-exit_intr:
- GET_THREAD_INFO(%rcx)
- testl $3,CS-ARGOFFSET(%rsp)
+ testl $3,CS(%rsp)
je retint_kernel
-
/* Interrupt came from user space */
+
+ GET_THREAD_INFO(%rcx)
/*
- * Has a correct top of stack, but a partial stack frame
* %rcx: thread info. Interrupts off.
*/
retint_with_reschedule:
DISABLE_INTERRUPTS(CLBR_ANY)
TRACE_IRQS_IRETQ
- /*
- * Try to use SYSRET instead of IRET if we're returning to
- * a completely clean 64-bit userspace context.
- */
- movq (RCX-R11)(%rsp), %rcx
- cmpq %rcx,(RIP-R11)(%rsp) /* RCX == RIP */
- jne opportunistic_sysret_failed
-
- /*
- * On Intel CPUs, sysret with non-canonical RCX/RIP will #GP
- * in kernel space. This essentially lets the user take over
- * the kernel, since userspace controls RSP. It's not worth
- * testing for canonicalness exactly -- this check detects any
- * of the 17 high bits set, which is true for non-canonical
- * or kernel addresses. (This will pessimize vsyscall=native.
- * Big deal.)
- *
- * If virtual addresses ever become wider, this will need
- * to be updated to remain correct on both old and new CPUs.
- */
- .ifne __VIRTUAL_MASK_SHIFT - 47
- .error "virtual address width changed -- sysret checks need update"
- .endif
- shr $__VIRTUAL_MASK_SHIFT, %rcx
- jnz opportunistic_sysret_failed
-
- cmpq $__USER_CS,(CS-R11)(%rsp) /* CS must match SYSRET */
- jne opportunistic_sysret_failed
-
- movq (R11-ARGOFFSET)(%rsp), %r11
- cmpq %r11,(EFLAGS-ARGOFFSET)(%rsp) /* R11 == RFLAGS */
- jne opportunistic_sysret_failed
-
- testq $X86_EFLAGS_RF,%r11 /* sysret can't restore RF */
- jnz opportunistic_sysret_failed
-
- /* nothing to check for RSP */
-
- cmpq $__USER_DS,(SS-ARGOFFSET)(%rsp) /* SS must match SYSRET */
- jne opportunistic_sysret_failed
-
- /*
- * We win! This label is here just for ease of understanding
- * perf profiles. Nothing jumps here.
- */
-irq_return_via_sysret:
- CFI_REMEMBER_STATE
- RESTORE_ARGS 1,8,1
- movq (RSP-RIP)(%rsp),%rsp
- USERGS_SYSRET64
- CFI_RESTORE_STATE
-
-opportunistic_sysret_failed:
SWAPGS
- jmp restore_args
+ jmp restore_c_regs_and_iret
-retint_restore_args: /* return to kernel space */
- DISABLE_INTERRUPTS(CLBR_ANY)
+/* Returning to kernel space */
+retint_kernel:
+#ifdef CONFIG_PREEMPT
+ /* Interrupts are off */
+ /* Check if we need preemption */
+ bt $9,EFLAGS(%rsp) /* interrupts were off? */
+ jnc 1f
+0: cmpl $0,PER_CPU_VAR(__preempt_count)
+ jnz 1f
+ call preempt_schedule_irq
+ jmp 0b
+1:
+#endif
/*
* The iretq could re-enable interrupts:
*/
TRACE_IRQS_IRETQ
-restore_args:
- RESTORE_ARGS 1,8,1
+
+/*
+ * At this label, code paths which return to kernel and to user,
+ * which come from interrupts/exception and from syscalls, merge.
+ */
+restore_c_regs_and_iret:
+ RESTORE_C_REGS
+ REMOVE_PT_GPREGS_FROM_STACK 8
irq_return:
INTERRUPT_RETURN
jz retint_swapgs
TRACE_IRQS_ON
ENABLE_INTERRUPTS(CLBR_NONE)
- SAVE_REST
+ SAVE_EXTRA_REGS
movq $-1,ORIG_RAX(%rsp)
xorl %esi,%esi # oldset
movq %rsp,%rdi # &pt_regs
call do_notify_resume
- RESTORE_REST
+ RESTORE_EXTRA_REGS
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
GET_THREAD_INFO(%rcx)
jmp retint_with_reschedule
-#ifdef CONFIG_PREEMPT
- /* Returning to kernel space. Check if we need preemption */
- /* rcx: threadinfo. interrupts off. */
-ENTRY(retint_kernel)
- cmpl $0,PER_CPU_VAR(__preempt_count)
- jnz retint_restore_args
- bt $9,EFLAGS-ARGOFFSET(%rsp) /* interrupts off? */
- jnc retint_restore_args
- call preempt_schedule_irq
- jmp exit_intr
-#endif
CFI_ENDPROC
END(common_interrupt)
/*
* Exception entry points.
*/
-#define INIT_TSS_IST(x) PER_CPU_VAR(init_tss) + (TSS_ist + ((x) - 1) * 8)
+#define CPU_TSS_IST(x) PER_CPU_VAR(cpu_tss) + (TSS_ist + ((x) - 1) * 8)
.macro idtentry sym do_sym has_error_code:req paranoid=0 shift_ist=-1
ENTRY(\sym)
pushq_cfi $-1 /* ORIG_RAX: no syscall to restart */
.endif
- subq $ORIG_RAX-R15, %rsp
- CFI_ADJUST_CFA_OFFSET ORIG_RAX-R15
+ ALLOC_PT_GPREGS_ON_STACK
.if \paranoid
.if \paranoid == 1
testl $3, CS(%rsp) /* If coming from userspace, switch */
jnz 1f /* stacks. */
.endif
- call save_paranoid
+ call paranoid_entry
.else
call error_entry
.endif
+ /* returned flag: ebx=0: need swapgs on exit, ebx=1: don't need it */
DEFAULT_FRAME 0
.endif
.if \shift_ist != -1
- subq $EXCEPTION_STKSZ, INIT_TSS_IST(\shift_ist)
+ subq $EXCEPTION_STKSZ, CPU_TSS_IST(\shift_ist)
.endif
call \do_sym
.if \shift_ist != -1
- addq $EXCEPTION_STKSZ, INIT_TSS_IST(\shift_ist)
+ addq $EXCEPTION_STKSZ, CPU_TSS_IST(\shift_ist)
.endif
+ /* these procedures expect "no swapgs" flag in ebx */
.if \paranoid
- jmp paranoid_exit /* %ebx: no swapgs flag */
+ jmp paranoid_exit
.else
- jmp error_exit /* %ebx: no swapgs flag */
+ jmp error_exit
.endif
.if \paranoid == 1
popq %rsp
CFI_DEF_CFA_REGISTER rsp
decl PER_CPU_VAR(irq_count)
+#ifndef CONFIG_PREEMPT
+ call xen_maybe_preempt_hcall
+#endif
jmp error_exit
CFI_ENDPROC
END(xen_do_hypervisor_callback)
addq $0x30,%rsp
CFI_ADJUST_CFA_OFFSET -0x30
pushq_cfi $-1 /* orig_ax = -1 => not a system call */
- SAVE_ALL
+ ALLOC_PT_GPREGS_ON_STACK
+ SAVE_C_REGS
+ SAVE_EXTRA_REGS
jmp error_exit
CFI_ENDPROC
END(xen_failsafe_callback)
idtentry machine_check has_error_code=0 paranoid=1 do_sym=*machine_check_vector(%rip)
#endif
- /*
- * "Paranoid" exit path from exception stack. This is invoked
- * only on return from non-NMI IST interrupts that came
- * from kernel space.
- *
- * We may be returning to very strange contexts (e.g. very early
- * in syscall entry), so checking for preemption here would
- * be complicated. Fortunately, we there's no good reason
- * to try to handle preemption here.
- */
+/*
+ * Save all registers in pt_regs, and switch gs if needed.
+ * Use slow, but surefire "are we in kernel?" check.
+ * Return: ebx=0: need swapgs on exit, ebx=1: otherwise
+ */
+ENTRY(paranoid_entry)
+ XCPT_FRAME 1 15*8
+ cld
+ SAVE_C_REGS 8
+ SAVE_EXTRA_REGS 8
+ movl $1,%ebx
+ movl $MSR_GS_BASE,%ecx
+ rdmsr
+ testl %edx,%edx
+ js 1f /* negative -> in kernel */
+ SWAPGS
+ xorl %ebx,%ebx
+1: ret
+ CFI_ENDPROC
+END(paranoid_entry)
- /* ebx: no swapgs flag */
+/*
+ * "Paranoid" exit path from exception stack. This is invoked
+ * only on return from non-NMI IST interrupts that came
+ * from kernel space.
+ *
+ * We may be returning to very strange contexts (e.g. very early
+ * in syscall entry), so checking for preemption here would
+ * be complicated. Fortunately, we there's no good reason
+ * to try to handle preemption here.
+ */
+/* On entry, ebx is "no swapgs" flag (1: don't need swapgs, 0: need it) */
ENTRY(paranoid_exit)
DEFAULT_FRAME
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF_DEBUG
testl %ebx,%ebx /* swapgs needed? */
- jnz paranoid_restore
- TRACE_IRQS_IRETQ 0
+ jnz paranoid_exit_no_swapgs
+ TRACE_IRQS_IRETQ
SWAPGS_UNSAFE_STACK
- RESTORE_ALL 8
- INTERRUPT_RETURN
-paranoid_restore:
- TRACE_IRQS_IRETQ_DEBUG 0
- RESTORE_ALL 8
+ jmp paranoid_exit_restore
+paranoid_exit_no_swapgs:
+ TRACE_IRQS_IRETQ_DEBUG
+paranoid_exit_restore:
+ RESTORE_EXTRA_REGS
+ RESTORE_C_REGS
+ REMOVE_PT_GPREGS_FROM_STACK 8
INTERRUPT_RETURN
CFI_ENDPROC
END(paranoid_exit)
/*
- * Exception entry point. This expects an error code/orig_rax on the stack.
- * returns in "no swapgs flag" in %ebx.
+ * Save all registers in pt_regs, and switch gs if needed.
+ * Return: ebx=0: need swapgs on exit, ebx=1: otherwise
*/
ENTRY(error_entry)
- XCPT_FRAME
- CFI_ADJUST_CFA_OFFSET 15*8
- /* oldrax contains error code */
+ XCPT_FRAME 1 15*8
cld
- movq %rdi, RDI+8(%rsp)
- movq %rsi, RSI+8(%rsp)
- movq %rdx, RDX+8(%rsp)
- movq %rcx, RCX+8(%rsp)
- movq %rax, RAX+8(%rsp)
- movq %r8, R8+8(%rsp)
- movq %r9, R9+8(%rsp)
- movq %r10, R10+8(%rsp)
- movq %r11, R11+8(%rsp)
- movq_cfi rbx, RBX+8
- movq %rbp, RBP+8(%rsp)
- movq %r12, R12+8(%rsp)
- movq %r13, R13+8(%rsp)
- movq %r14, R14+8(%rsp)
- movq %r15, R15+8(%rsp)
+ SAVE_C_REGS 8
+ SAVE_EXTRA_REGS 8
xorl %ebx,%ebx
testl $3,CS+8(%rsp)
je error_kernelspace
TRACE_IRQS_OFF
ret
-/*
- * There are two places in the kernel that can potentially fault with
- * usergs. Handle them here. B stepping K8s sometimes report a
- * truncated RIP for IRET exceptions returning to compat mode. Check
- * for these here too.
- */
+ /*
+ * There are two places in the kernel that can potentially fault with
+ * usergs. Handle them here. B stepping K8s sometimes report a
+ * truncated RIP for IRET exceptions returning to compat mode. Check
+ * for these here too.
+ */
error_kernelspace:
CFI_REL_OFFSET rcx, RCX+8
incl %ebx
END(error_entry)
-/* ebx: no swapgs flag (1: don't need swapgs, 0: need it) */
+/* On entry, ebx is "no swapgs" flag (1: don't need swapgs, 0: need it) */
ENTRY(error_exit)
DEFAULT_FRAME
movl %ebx,%eax
- RESTORE_REST
+ RESTORE_EXTRA_REGS
DISABLE_INTERRUPTS(CLBR_NONE)
TRACE_IRQS_OFF
GET_THREAD_INFO(%rcx)
CFI_ENDPROC
END(error_exit)
-/*
- * Test if a given stack is an NMI stack or not.
- */
- .macro test_in_nmi reg stack nmi_ret normal_ret
- cmpq %\reg, \stack
- ja \normal_ret
- subq $EXCEPTION_STKSZ, %\reg
- cmpq %\reg, \stack
- jb \normal_ret
- jmp \nmi_ret
- .endm
-
- /* runs on exception stack */
+/* Runs on exception stack */
ENTRY(nmi)
INTR_FRAME
PARAVIRT_ADJUST_EXCEPTION_FRAME
* NMI.
*/
- /* Use %rdx as out temp variable throughout */
+ /* Use %rdx as our temp variable throughout */
pushq_cfi %rdx
CFI_REL_OFFSET rdx, 0
* We check the variable because the first NMI could be in a
* breakpoint routine using a breakpoint stack.
*/
- lea 6*8(%rsp), %rdx
- test_in_nmi rdx, 4*8(%rsp), nested_nmi, first_nmi
+ lea 6*8(%rsp), %rdx
+ /* Compare the NMI stack (rdx) with the stack we came from (4*8(%rsp)) */
+ cmpq %rdx, 4*8(%rsp)
+ /* If the stack pointer is above the NMI stack, this is a normal NMI */
+ ja first_nmi
+ subq $EXCEPTION_STKSZ, %rdx
+ cmpq %rdx, 4*8(%rsp)
+ /* If it is below the NMI stack, it is a normal NMI */
+ jb first_nmi
+ /* Ah, it is within the NMI stack, treat it as nested */
+
CFI_REMEMBER_STATE
nested_nmi:
.rept 5
pushq_cfi 11*8(%rsp)
.endr
- CFI_DEF_CFA_OFFSET SS+8-RIP
+ CFI_DEF_CFA_OFFSET 5*8
/* Everything up to here is safe from nested NMIs */
pushq_cfi -6*8(%rsp)
.endr
subq $(5*8), %rsp
- CFI_DEF_CFA_OFFSET SS+8-RIP
+ CFI_DEF_CFA_OFFSET 5*8
end_repeat_nmi:
/*
* so that we repeat another NMI.
*/
pushq_cfi $-1 /* ORIG_RAX: no syscall to restart */
- subq $ORIG_RAX-R15, %rsp
- CFI_ADJUST_CFA_OFFSET ORIG_RAX-R15
+ ALLOC_PT_GPREGS_ON_STACK
+
/*
- * Use save_paranoid to handle SWAPGS, but no need to use paranoid_exit
+ * Use paranoid_entry to handle SWAPGS, but no need to use paranoid_exit
* as we should not be calling schedule in NMI context.
* Even with normal interrupts enabled. An NMI should not be
* setting NEED_RESCHED or anything that normal interrupts and
* exceptions might do.
*/
- call save_paranoid
+ call paranoid_entry
DEFAULT_FRAME 0
/*
nmi_swapgs:
SWAPGS_UNSAFE_STACK
nmi_restore:
+ RESTORE_EXTRA_REGS
+ RESTORE_C_REGS
/* Pop the extra iret frame at once */
- RESTORE_ALL 6*8
+ REMOVE_PT_GPREGS_FROM_STACK 6*8
/* Clear the NMI executing stack variable */
movq $0, 5*8(%rsp)
*/
load_ucode_bsp();
- if (console_loglevel >= CONSOLE_LOGLEVEL_DEBUG)
- early_printk("Kernel alive\n");
-
clear_page(init_level4_pgt);
/* set init_level4_pgt kernel high mapping*/
init_level4_pgt[511] = early_level4_pgt[511];
#include <asm/cpufeature.h>
#include <asm/percpu.h>
#include <asm/nops.h>
+#include <asm/bootparam.h>
/* Physical address */
#define pa(X) ((X) - __PAGE_OFFSET)
/* test KEEP_SEGMENTS flag to see if the bootloader is asking
us to not reload segments */
- testb $(1<<6), BP_loadflags(%esi)
+ testb $KEEP_SEGMENTS, BP_loadflags(%esi)
jnz 2f
/*
/*
- * linux/arch/x86_64/kernel/head.S -- start in 32bit and switch to 64bit
+ * linux/arch/x86/kernel/head_64.S -- start in 32bit and switch to 64bit
*
* Copyright (C) 2000 Andrea Arcangeli <andrea@suse.de> SuSE
* Copyright (C) 2000 Pavel Machek <pavel@suse.cz>
* %rsi holds a physical pointer to real_mode_data.
*
* We come here either directly from a 64bit bootloader, or from
- * arch/x86_64/boot/compressed/head.S.
+ * arch/x86/boot/compressed/head_64.S.
*
* We only come here initially at boot nothing else comes here.
*
leaq level2_kernel_pgt(%rip), %rdi
leaq 4096(%rdi), %r8
/* See if it is a valid page table entry */
-1: testq $1, 0(%rdi)
+1: testb $1, 0(%rdi)
jz 2f
addq %rbp, 0(%rdi)
/* Go to the next page */
* be set (so that the clts/stts pair does nothing that is
* visible in the interrupted kernel thread).
*
- * Except for the eagerfpu case when we return 1 unless we've already
- * been eager and saved the state in kernel_fpu_begin().
+ * Except for the eagerfpu case when we return true; in the likely case
+ * the thread has FPU but we are not going to set/clear TS.
*/
static inline bool interrupted_kernel_fpu_idle(void)
{
return false;
if (use_eager_fpu())
- return __thread_has_fpu(current);
+ return true;
return !__thread_has_fpu(current) &&
(read_cr0() & X86_CR0_TS);
static inline bool interrupted_user_mode(void)
{
struct pt_regs *regs = get_irq_regs();
- return regs && user_mode_vm(regs);
+ return regs && user_mode(regs);
}
/*
if (__thread_has_fpu(me)) {
__save_init_fpu(me);
- } else if (!use_eager_fpu()) {
+ } else {
this_cpu_write(fpu_owner_task, NULL);
- clts();
+ if (!use_eager_fpu())
+ clts();
}
}
EXPORT_SYMBOL(__kernel_fpu_begin);
if (__thread_has_fpu(me)) {
if (WARN_ON(restore_fpu_checking(me)))
- drop_init_fpu(me);
+ fpu_reset_state(me);
} else if (!use_eager_fpu()) {
stts();
}
{
preempt_disable();
if (__thread_has_fpu(tsk)) {
- __save_init_fpu(tsk);
- __thread_fpu_end(tsk);
- } else
- tsk->thread.fpu_counter = 0;
+ if (use_eager_fpu()) {
+ __save_fpu(tsk);
+ } else {
+ __save_init_fpu(tsk);
+ __thread_fpu_end(tsk);
+ }
+ }
preempt_enable();
}
EXPORT_SYMBOL(unlazy_fpu);
return;
}
+ memset(fpu->state, 0, xstate_size);
+
if (cpu_has_fxsr) {
fx_finit(&fpu->state->fxsave);
} else {
struct i387_fsave_struct *fp = &fpu->state->fsave;
- memset(fp, 0, xstate_size);
fp->cwd = 0xffff037fu;
fp->swd = 0xffff0000u;
fp->twd = 0xffffffffu;
if (tsk_used_math(tsk)) {
if (cpu_has_fpu && tsk == current)
unlazy_fpu(tsk);
- tsk->thread.fpu.last_cpu = ~0;
+ task_disable_lazy_fpu_restore(tsk);
return 0;
}
unsigned int pos, unsigned int count,
void *kbuf, void __user *ubuf)
{
+ struct xsave_struct *xsave = &target->thread.fpu.state->xsave;
int ret;
if (!cpu_has_xsave)
* memory layout in the thread struct, so that we can copy the entire
* xstateregs to the user using one user_regset_copyout().
*/
- memcpy(&target->thread.fpu.state->fxsave.sw_reserved,
- xstate_fx_sw_bytes, sizeof(xstate_fx_sw_bytes));
-
+ memcpy(&xsave->i387.sw_reserved,
+ xstate_fx_sw_bytes, sizeof(xstate_fx_sw_bytes));
/*
* Copy the xstate memory layout.
*/
- ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf,
- &target->thread.fpu.state->xsave, 0, -1);
+ ret = user_regset_copyout(&pos, &count, &kbuf, &ubuf, xsave, 0, -1);
return ret;
}
unsigned int pos, unsigned int count,
const void *kbuf, const void __user *ubuf)
{
+ struct xsave_struct *xsave = &target->thread.fpu.state->xsave;
int ret;
- struct xsave_hdr_struct *xsave_hdr;
if (!cpu_has_xsave)
return -ENODEV;
if (ret)
return ret;
- ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf,
- &target->thread.fpu.state->xsave, 0, -1);
-
+ ret = user_regset_copyin(&pos, &count, &kbuf, &ubuf, xsave, 0, -1);
/*
* mxcsr reserved bits must be masked to zero for security reasons.
*/
- target->thread.fpu.state->fxsave.mxcsr &= mxcsr_feature_mask;
-
- xsave_hdr = &target->thread.fpu.state->xsave.xsave_hdr;
-
- xsave_hdr->xstate_bv &= pcntxt_mask;
+ xsave->i387.mxcsr &= mxcsr_feature_mask;
+ xsave->xsave_hdr.xstate_bv &= pcntxt_mask;
/*
* These bits must be zero.
*/
- memset(xsave_hdr->reserved, 0, 48);
-
+ memset(&xsave->xsave_hdr.reserved, 0, 48);
return ret;
}
* because the ->io_bitmap_max value must match the bitmap
* contents:
*/
- tss = &per_cpu(init_tss, get_cpu());
+ tss = &per_cpu(cpu_tss, get_cpu());
if (turn_on)
bitmap_clear(t->io_bitmap_ptr, from, num);
this_cpu = smp_processor_id();
cpumask_copy(&online_new, cpu_online_mask);
- cpu_clear(this_cpu, online_new);
+ cpumask_clear_cpu(this_cpu, &online_new);
this_count = 0;
for (vector = FIRST_EXTERNAL_VECTOR; vector < NR_VECTORS; vector++) {
data = irq_desc_get_irq_data(desc);
cpumask_copy(&affinity_new, data->affinity);
- cpu_clear(this_cpu, affinity_new);
+ cpumask_clear_cpu(this_cpu, &affinity_new);
/* Do not count inactive or per-cpu irqs. */
if (!irq_has_action(irq) || irqd_is_per_cpu(data))
if (unlikely(!desc))
return false;
- if (user_mode_vm(regs) || !execute_on_irq_stack(overflow, desc, irq)) {
+ if (user_mode(regs) || !execute_on_irq_stack(overflow, desc, irq)) {
if (unlikely(overflow))
print_stack_overflow();
desc->handle_irq(irq, desc);
u64 estack_top, estack_bottom;
u64 curbase = (u64)task_stack_page(current);
- if (user_mode_vm(regs))
+ if (user_mode(regs))
return;
if (regs->sp >= curbase + sizeof(struct thread_info) +
#endif
for_each_clear_bit_from(i, used_vectors, first_system_vector) {
/* IA32_SYSCALL_VECTOR could be used in trap_init already. */
- set_intr_gate(i, interrupt[i - FIRST_EXTERNAL_VECTOR]);
+ set_intr_gate(i, irq_entries_start +
+ 8 * (i - FIRST_EXTERNAL_VECTOR));
}
#ifdef CONFIG_X86_LOCAL_APIC
for_each_clear_bit_from(i, used_vectors, NR_VECTORS)
{ "bx", 8, offsetof(struct pt_regs, bx) },
{ "cx", 8, offsetof(struct pt_regs, cx) },
{ "dx", 8, offsetof(struct pt_regs, dx) },
- { "si", 8, offsetof(struct pt_regs, dx) },
+ { "si", 8, offsetof(struct pt_regs, si) },
{ "di", 8, offsetof(struct pt_regs, di) },
{ "bp", 8, offsetof(struct pt_regs, bp) },
{ "sp", 8, offsetof(struct pt_regs, sp) },
#ifdef CONFIG_X86_32
switch (regno) {
case GDB_SS:
- if (!user_mode_vm(regs))
+ if (!user_mode(regs))
*(unsigned long *)mem = __KERNEL_DS;
break;
case GDB_SP:
- if (!user_mode_vm(regs))
+ if (!user_mode(regs))
*(unsigned long *)mem = kernel_stack_pointer(regs);
break;
case GDB_GS:
__recover_probed_insn(kprobe_opcode_t *buf, unsigned long addr)
{
struct kprobe *kp;
+ unsigned long faddr;
kp = get_kprobe((void *)addr);
- /* There is no probe, return original address */
- if (!kp)
+ faddr = ftrace_location(addr);
+ /*
+ * Addresses inside the ftrace location are refused by
+ * arch_check_ftrace_location(). Something went terribly wrong
+ * if such an address is checked here.
+ */
+ if (WARN_ON(faddr && faddr != addr))
+ return 0UL;
+ /*
+ * Use the current code if it is not modified by Kprobe
+ * and it cannot be modified by ftrace.
+ */
+ if (!kp && !faddr)
return addr;
/*
- * Basically, kp->ainsn.insn has an original instruction.
- * However, RIP-relative instruction can not do single-stepping
- * at different place, __copy_instruction() tweaks the displacement of
- * that instruction. In that case, we can't recover the instruction
- * from the kp->ainsn.insn.
+ * Basically, kp->ainsn.insn has an original instruction.
+ * However, RIP-relative instruction can not do single-stepping
+ * at different place, __copy_instruction() tweaks the displacement of
+ * that instruction. In that case, we can't recover the instruction
+ * from the kp->ainsn.insn.
*
- * On the other hand, kp->opcode has a copy of the first byte of
- * the probed instruction, which is overwritten by int3. And
- * the instruction at kp->addr is not modified by kprobes except
- * for the first byte, we can recover the original instruction
- * from it and kp->opcode.
+ * On the other hand, in case on normal Kprobe, kp->opcode has a copy
+ * of the first byte of the probed instruction, which is overwritten
+ * by int3. And the instruction at kp->addr is not modified by kprobes
+ * except for the first byte, we can recover the original instruction
+ * from it and kp->opcode.
+ *
+ * In case of Kprobes using ftrace, we do not have a copy of
+ * the original instruction. In fact, the ftrace location might
+ * be modified at anytime and even could be in an inconsistent state.
+ * Fortunately, we know that the original code is the ideal 5-byte
+ * long NOP.
*/
- memcpy(buf, kp->addr, MAX_INSN_SIZE * sizeof(kprobe_opcode_t));
- buf[0] = kp->opcode;
+ memcpy(buf, (void *)addr, MAX_INSN_SIZE * sizeof(kprobe_opcode_t));
+ if (faddr)
+ memcpy(buf, ideal_nops[NOP_ATOMIC5], 5);
+ else
+ buf[0] = kp->opcode;
return (unsigned long)buf;
}
* Recover the probed instruction at addr for further analysis.
* Caller must lock kprobes by kprobe_mutex, or disable preemption
* for preventing to release referencing kprobes.
+ * Returns zero if the instruction can not get recovered.
*/
unsigned long recover_probed_instruction(kprobe_opcode_t *buf, unsigned long addr)
{
* normally used, we just go through if there is no kprobe.
*/
__addr = recover_probed_instruction(buf, addr);
+ if (!__addr)
+ return 0;
kernel_insn_init(&insn, (void *)__addr, MAX_INSN_SIZE);
insn_get_length(&insn);
unsigned long recovered_insn =
recover_probed_instruction(buf, (unsigned long)src);
+ if (!recovered_insn)
+ return 0;
kernel_insn_init(&insn, (void *)recovered_insn, MAX_INSN_SIZE);
insn_get_length(&insn);
/* Another subsystem puts a breakpoint, failed to recover */
struct kprobe *p;
struct kprobe_ctlblk *kcb;
- if (user_mode_vm(regs))
+ if (user_mode(regs))
return 0;
addr = (kprobe_opcode_t *)(regs->ip - sizeof(kprobe_opcode_t));
struct die_args *args = data;
int ret = NOTIFY_DONE;
- if (args->regs && user_mode_vm(args->regs))
+ if (args->regs && user_mode(args->regs))
return ret;
if (val == DIE_GPF) {
*/
return 0;
recovered_insn = recover_probed_instruction(buf, addr);
+ if (!recovered_insn)
+ return 0;
kernel_insn_init(&insn, (void *)recovered_insn, MAX_INSN_SIZE);
insn_get_length(&insn);
/* Another subsystem puts a breakpoint */
#include <asm/page.h>
#include <asm/pgtable.h>
+#include <asm/setup.h>
#if 0
#define DEBUGP(fmt, ...) \
static unsigned long int get_module_load_offset(void)
{
- if (kaslr_enabled) {
+ if (kaslr_enabled()) {
mutex_lock(&module_kaslr_mutex);
/*
* Calculate the module_load_offset the first time this
}
/*
- * RIP, flags, and the argument registers are usually saved.
- * orig_ax is probably okay, too.
+ * These registers are always saved on 64-bit syscall entry.
+ * On 32-bit entry points, they are saved too except r8..r11.
*/
regs_user_copy->ip = user_regs->ip;
+ regs_user_copy->ax = user_regs->ax;
regs_user_copy->cx = user_regs->cx;
regs_user_copy->dx = user_regs->dx;
regs_user_copy->si = user_regs->si;
regs_user_copy->r11 = user_regs->r11;
regs_user_copy->orig_ax = user_regs->orig_ax;
regs_user_copy->flags = user_regs->flags;
+ regs_user_copy->sp = user_regs->sp;
+ regs_user_copy->cs = user_regs->cs;
+ regs_user_copy->ss = user_regs->ss;
/*
- * Don't even try to report the "rest" regs.
+ * Most system calls don't save these registers, don't report them.
*/
regs_user_copy->bx = -1;
regs_user_copy->bp = -1;
/*
* For this to be at all useful, we need a reasonable guess for
- * sp and the ABI. Be careful: we're in NMI context, and we're
+ * the ABI. Be careful: we're in NMI context, and we're
* considering current to be the current task, so we should
* be careful not to look at any other percpu variables that might
* change during context switches.
*/
- if (IS_ENABLED(CONFIG_IA32_EMULATION) &&
- task_thread_info(current)->status & TS_COMPAT) {
- /* Easy case: we're in a compat syscall. */
- regs_user->abi = PERF_SAMPLE_REGS_ABI_32;
- regs_user_copy->sp = user_regs->sp;
- regs_user_copy->cs = user_regs->cs;
- regs_user_copy->ss = user_regs->ss;
- } else if (user_regs->orig_ax != -1) {
- /*
- * We're probably in a 64-bit syscall.
- * Warning: this code is severely racy. At least it's better
- * than just blindly copying user_regs.
- */
- regs_user->abi = PERF_SAMPLE_REGS_ABI_64;
- regs_user_copy->sp = this_cpu_read(old_rsp);
- regs_user_copy->cs = __USER_CS;
- regs_user_copy->ss = __USER_DS;
- regs_user_copy->cx = -1; /* usually contains garbage */
- } else {
- /* We're probably in an interrupt or exception. */
- regs_user->abi = user_64bit_mode(user_regs) ?
- PERF_SAMPLE_REGS_ABI_64 : PERF_SAMPLE_REGS_ABI_32;
- regs_user_copy->sp = user_regs->sp;
- regs_user_copy->cs = user_regs->cs;
- regs_user_copy->ss = user_regs->ss;
- }
+ regs_user->abi = user_64bit_mode(user_regs) ?
+ PERF_SAMPLE_REGS_ABI_64 : PERF_SAMPLE_REGS_ABI_32;
regs_user->regs = regs_user_copy;
}
#include <linux/sched.h>
#include <linux/module.h>
#include <linux/pm.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/random.h>
#include <linux/user-return-notifier.h>
#include <linux/dmi.h>
#include <asm/syscalls.h>
#include <asm/idle.h>
#include <asm/uaccess.h>
+#include <asm/mwait.h>
#include <asm/i387.h>
#include <asm/fpu-internal.h>
#include <asm/debugreg.h>
* section. Since TSS's are completely CPU-local, we want them
* on exact cacheline boundaries, to eliminate cacheline ping-pong.
*/
-__visible DEFINE_PER_CPU_SHARED_ALIGNED(struct tss_struct, init_tss) = INIT_TSS;
+__visible DEFINE_PER_CPU_SHARED_ALIGNED(struct tss_struct, cpu_tss) = {
+ .x86_tss = {
+ .sp0 = TOP_OF_INIT_STACK,
+#ifdef CONFIG_X86_32
+ .ss0 = __KERNEL_DS,
+ .ss1 = __KERNEL_CS,
+ .io_bitmap_base = INVALID_IO_BITMAP_OFFSET,
+#endif
+ },
+#ifdef CONFIG_X86_32
+ /*
+ * Note that the .io_bitmap member must be extra-big. This is because
+ * the CPU will access an additional byte beyond the end of the IO
+ * permission bitmap. The extra byte must be all 1 bits, and must
+ * be within the limit.
+ */
+ .io_bitmap = { [0 ... IO_BITMAP_LONGS] = ~0 },
+#endif
+};
+EXPORT_PER_CPU_SYMBOL_GPL(cpu_tss);
#ifdef CONFIG_X86_64
static DEFINE_PER_CPU(unsigned char, is_idle);
dst->thread.fpu_counter = 0;
dst->thread.fpu.has_fpu = 0;
- dst->thread.fpu.last_cpu = ~0;
dst->thread.fpu.state = NULL;
+ task_disable_lazy_fpu_restore(dst);
if (tsk_used_math(src)) {
int err = fpu_alloc(&dst->thread.fpu);
if (err)
unsigned long *bp = t->io_bitmap_ptr;
if (bp) {
- struct tss_struct *tss = &per_cpu(init_tss, get_cpu());
+ struct tss_struct *tss = &per_cpu(cpu_tss, get_cpu());
t->io_bitmap_ptr = NULL;
clear_thread_flag(TIF_IO_BITMAP);
flush_ptrace_hw_breakpoint(tsk);
memset(tsk->thread.tls_array, 0, sizeof(tsk->thread.tls_array));
- drop_init_fpu(tsk);
- /*
- * Free the FPU state for non xsave platforms. They get reallocated
- * lazily at the first use.
- */
- if (!use_eager_fpu())
+
+ if (!use_eager_fpu()) {
+ /* FPU state will be reallocated lazily at the first use. */
+ drop_fpu(tsk);
free_thread_xstate(tsk);
+ } else if (!used_math()) {
+ /* kthread execs. TODO: cleanup this horror. */
+ if (WARN_ON(init_fpu(tsk)))
+ force_sig(SIGKILL, tsk);
+ user_fpu_begin();
+ restore_init_xstate();
+ }
}
static void hard_disable_TSC(void)
if (!cpumask_test_cpu(cpu, amd_e400_c1e_mask)) {
cpumask_set_cpu(cpu, amd_e400_c1e_mask);
- /*
- * Force broadcast so ACPI can not interfere.
- */
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_FORCE,
- &cpu);
+ /* Force broadcast so ACPI can not interfere. */
+ tick_broadcast_force();
pr_info("Switch to broadcast mode on CPU%d\n", cpu);
}
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
+ tick_broadcast_enter();
default_idle();
* called with interrupts disabled.
*/
local_irq_disable();
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
+ tick_broadcast_exit();
local_irq_enable();
} else
default_idle();
}
+/*
+ * Intel Core2 and older machines prefer MWAIT over HALT for C1.
+ * We can't rely on cpuidle installing MWAIT, because it will not load
+ * on systems that support only C1 -- so the boot default must be MWAIT.
+ *
+ * Some AMD machines are the opposite, they depend on using HALT.
+ *
+ * So for default C1, which is used during boot until cpuidle loads,
+ * use MWAIT-C1 on Intel HW that has it, else use HALT.
+ */
+static int prefer_mwait_c1_over_halt(const struct cpuinfo_x86 *c)
+{
+ if (c->x86_vendor != X86_VENDOR_INTEL)
+ return 0;
+
+ if (!cpu_has(c, X86_FEATURE_MWAIT))
+ return 0;
+
+ return 1;
+}
+
+/*
+ * MONITOR/MWAIT with no hints, used for default default C1 state.
+ * This invokes MWAIT with interrutps enabled and no flags,
+ * which is backwards compatible with the original MWAIT implementation.
+ */
+
+static void mwait_idle(void)
+{
+ if (!current_set_polling_and_test()) {
+ if (this_cpu_has(X86_BUG_CLFLUSH_MONITOR)) {
+ smp_mb(); /* quirk */
+ clflush((void *)¤t_thread_info()->flags);
+ smp_mb(); /* quirk */
+ }
+
+ __monitor((void *)¤t_thread_info()->flags, 0, 0);
+ if (!need_resched())
+ __sti_mwait(0, 0);
+ else
+ local_irq_enable();
+ } else {
+ local_irq_enable();
+ }
+ __current_clr_polling();
+}
+
void select_idle_routine(const struct cpuinfo_x86 *c)
{
#ifdef CONFIG_SMP
/* E400: APIC timer interrupt does not wake up CPU from C1e */
pr_info("using AMD E400 aware idle routine\n");
x86_idle = amd_e400_idle;
+ } else if (prefer_mwait_c1_over_halt(c)) {
+ pr_info("using mwait in idle threads\n");
+ x86_idle = mwait_idle;
} else
x86_idle = default_idle;
}
unsigned long sp;
unsigned short ss, gs;
- if (user_mode_vm(regs)) {
+ if (user_mode(regs)) {
sp = regs->sp;
ss = regs->ss & 0xffff;
gs = get_user_gs(regs);
regs->ip = new_ip;
regs->sp = new_sp;
regs->flags = X86_EFLAGS_IF;
- /*
- * force it to the iret return path by making it look as if there was
- * some work pending.
- */
- set_thread_flag(TIF_NOTIFY_RESUME);
+ force_iret();
}
EXPORT_SYMBOL_GPL(start_thread);
struct thread_struct *prev = &prev_p->thread,
*next = &next_p->thread;
int cpu = smp_processor_id();
- struct tss_struct *tss = &per_cpu(init_tss, cpu);
+ struct tss_struct *tss = &per_cpu(cpu_tss, cpu);
fpu_switch_t fpu;
/* never put a printk in __switch_to... printk() calls wake_up*() indirectly */
fpu = switch_fpu_prepare(prev_p, next_p, cpu);
- /*
- * Reload esp0.
- */
- load_sp0(tss, next);
-
/*
* Save away %gs. No need to save %fs, as it was saved on the
* stack on entry. No need to save %es and %ds, as those are
*/
arch_end_context_switch(next_p);
+ /*
+ * Reload esp0, kernel_stack, and current_top_of_stack. This changes
+ * current_thread_info().
+ */
+ load_sp0(tss, next);
this_cpu_write(kernel_stack,
- (unsigned long)task_stack_page(next_p) +
- THREAD_SIZE - KERNEL_STACK_OFFSET);
+ (unsigned long)task_stack_page(next_p) +
+ THREAD_SIZE);
+ this_cpu_write(cpu_current_top_of_stack,
+ (unsigned long)task_stack_page(next_p) +
+ THREAD_SIZE);
/*
* Restore %gs if needed (which is common)
asmlinkage extern void ret_from_fork(void);
-__visible DEFINE_PER_CPU(unsigned long, old_rsp);
+__visible DEFINE_PER_CPU(unsigned long, rsp_scratch);
/* Prints also some state that isn't saved in the pt_regs */
void __show_regs(struct pt_regs *regs, int all)
p->thread.sp0 = (unsigned long)task_stack_page(p) + THREAD_SIZE;
childregs = task_pt_regs(p);
p->thread.sp = (unsigned long) childregs;
- p->thread.usersp = me->thread.usersp;
set_tsk_thread_flag(p, TIF_FORK);
p->thread.io_bitmap_ptr = NULL;
*/
if (clone_flags & CLONE_SETTLS) {
#ifdef CONFIG_IA32_EMULATION
- if (test_thread_flag(TIF_IA32))
+ if (is_ia32_task())
err = do_set_thread_area(p, -1,
(struct user_desc __user *)childregs->si, 0);
else
loadsegment(es, _ds);
loadsegment(ds, _ds);
load_gs_index(0);
- current->thread.usersp = new_sp;
regs->ip = new_ip;
regs->sp = new_sp;
- this_cpu_write(old_rsp, new_sp);
regs->cs = _cs;
regs->ss = _ss;
regs->flags = X86_EFLAGS_IF;
+ force_iret();
}
void
struct thread_struct *prev = &prev_p->thread;
struct thread_struct *next = &next_p->thread;
int cpu = smp_processor_id();
- struct tss_struct *tss = &per_cpu(init_tss, cpu);
+ struct tss_struct *tss = &per_cpu(cpu_tss, cpu);
unsigned fsindex, gsindex;
fpu_switch_t fpu;
fpu = switch_fpu_prepare(prev_p, next_p, cpu);
- /* Reload esp0 and ss1. */
- load_sp0(tss, next);
-
/* We must save %fs and %gs before load_TLS() because
* %fs and %gs may be cleared by load_TLS().
*
/*
* Switch the PDA and FPU contexts.
*/
- prev->usersp = this_cpu_read(old_rsp);
- this_cpu_write(old_rsp, next->usersp);
this_cpu_write(current_task, next_p);
/*
task_thread_info(prev_p)->saved_preempt_count = this_cpu_read(__preempt_count);
this_cpu_write(__preempt_count, task_thread_info(next_p)->saved_preempt_count);
+ /* Reload esp0 and ss1. This changes current_thread_info(). */
+ load_sp0(tss, next);
+
this_cpu_write(kernel_stack,
- (unsigned long)task_stack_page(next_p) +
- THREAD_SIZE - KERNEL_STACK_OFFSET);
+ (unsigned long)task_stack_page(next_p) + THREAD_SIZE);
/*
* Now maybe reload the debug registers and handle I/O bitmaps
unsigned long KSTK_ESP(struct task_struct *task)
{
- return (test_tsk_thread_flag(task, TIF_IA32)) ?
- (task_pt_regs(task)->sp) : ((task)->thread.usersp);
+ return task_pt_regs(task)->sp;
}
case offsetof(struct user_regs_struct,cs):
if (unlikely(value == 0))
return -EIO;
-#ifdef CONFIG_IA32_EMULATION
- if (test_tsk_thread_flag(task, TIF_IA32))
- task_pt_regs(task)->cs = value;
-#endif
+ task_pt_regs(task)->cs = value;
break;
case offsetof(struct user_regs_struct,ss):
if (unlikely(value == 0))
return -EIO;
-#ifdef CONFIG_IA32_EMULATION
- if (test_tsk_thread_flag(task, TIF_IA32))
- task_pt_regs(task)->ss = value;
-#endif
+ task_pt_regs(task)->ss = value;
break;
}
memset(info, 0, sizeof(*info));
info->si_signo = SIGTRAP;
info->si_code = si_code;
- info->si_addr = user_mode_vm(regs) ? (void __user *)regs->ip : NULL;
+ info->si_addr = user_mode(regs) ? (void __user *)regs->ip : NULL;
}
void user_single_step_siginfo(struct task_struct *tsk,
set_normalized_timespec(ts, now.tv_sec, now.tv_nsec);
}
+static struct pvclock_vsyscall_time_info *pvclock_vdso_info;
+
+static struct pvclock_vsyscall_time_info *
+pvclock_get_vsyscall_user_time_info(int cpu)
+{
+ if (!pvclock_vdso_info) {
+ BUG();
+ return NULL;
+ }
+
+ return &pvclock_vdso_info[cpu];
+}
+
+struct pvclock_vcpu_time_info *pvclock_get_vsyscall_time_info(int cpu)
+{
+ return &pvclock_get_vsyscall_user_time_info(cpu)->pvti;
+}
+
#ifdef CONFIG_X86_64
+static int pvclock_task_migrate(struct notifier_block *nb, unsigned long l,
+ void *v)
+{
+ struct task_migration_notifier *mn = v;
+ struct pvclock_vsyscall_time_info *pvti;
+
+ pvti = pvclock_get_vsyscall_user_time_info(mn->from_cpu);
+
+ /* this is NULL when pvclock vsyscall is not initialized */
+ if (unlikely(pvti == NULL))
+ return NOTIFY_DONE;
+
+ pvti->migrate_count++;
+
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block pvclock_migrate = {
+ .notifier_call = pvclock_task_migrate,
+};
+
/*
* Initialize the generic pvclock vsyscall state. This will allocate
* a/some page(s) for the per-vcpu pvclock information, set up a
WARN_ON (size != PVCLOCK_VSYSCALL_NR_PAGES*PAGE_SIZE);
+ pvclock_vdso_info = i;
+
for (idx = 0; idx <= (PVCLOCK_FIXMAP_END-PVCLOCK_FIXMAP_BEGIN); idx++) {
__set_fixmap(PVCLOCK_FIXMAP_BEGIN + idx,
__pa(i) + (idx*PAGE_SIZE),
PAGE_KERNEL_VVAR);
}
+
+ register_task_migration_notifier(&pvclock_migrate);
+
return 0;
}
#endif
},
},
+ /* ASRock */
+ { /* Handle problems with rebooting on ASRock Q1900DC-ITX */
+ .callback = set_pci_reboot,
+ .ident = "ASRock Q1900DC-ITX",
+ .matches = {
+ DMI_MATCH(DMI_BOARD_VENDOR, "ASRock"),
+ DMI_MATCH(DMI_BOARD_NAME, "Q1900DC-ITX"),
+ },
+ },
+
/* ASUS */
{ /* Handle problems with rebooting on ASUS P4S800 */
.callback = set_bios_reboot,
movl (%ebx), %ecx
addl $4, %ebx
1:
- testl $0x1, %ecx /* is it a destination page */
+ testb $0x1, %cl /* is it a destination page */
jz 2f
movl %ecx, %edi
andl $0xfffff000, %edi
jmp 0b
2:
- testl $0x2, %ecx /* is it an indirection page */
+ testb $0x2, %cl /* is it an indirection page */
jz 2f
movl %ecx, %ebx
andl $0xfffff000, %ebx
jmp 0b
2:
- testl $0x4, %ecx /* is it the done indicator */
+ testb $0x4, %cl /* is it the done indicator */
jz 2f
jmp 3f
2:
- testl $0x8, %ecx /* is it the source indicator */
+ testb $0x8, %cl /* is it the source indicator */
jz 0b /* Ignore it otherwise */
movl %ecx, %esi /* For every source page do a copy */
andl $0xfffff000, %esi
* Set cr4 to a known state:
* - physical address extension enabled
*/
- movq $X86_CR4_PAE, %rax
+ movl $X86_CR4_PAE, %eax
movq %rax, %cr4
jmp 1f
movq (%rbx), %rcx
addq $8, %rbx
1:
- testq $0x1, %rcx /* is it a destination page? */
+ testb $0x1, %cl /* is it a destination page? */
jz 2f
movq %rcx, %rdi
andq $0xfffffffffffff000, %rdi
jmp 0b
2:
- testq $0x2, %rcx /* is it an indirection page? */
+ testb $0x2, %cl /* is it an indirection page? */
jz 2f
movq %rcx, %rbx
andq $0xfffffffffffff000, %rbx
jmp 0b
2:
- testq $0x4, %rcx /* is it the done indicator? */
+ testb $0x4, %cl /* is it the done indicator? */
jz 2f
jmp 3f
2:
- testq $0x8, %rcx /* is it the source indicator? */
+ testb $0x8, %cl /* is it the source indicator? */
jz 0b /* Ignore it otherwise */
movq %rcx, %rsi /* For ever source page do a copy */
andq $0xfffffffffffff000, %rsi
movq %rsi, %rax
movq %r10, %rdi
- movq $512, %rcx
+ movl $512, %ecx
rep ; movsq
movq %rax, %rdi
movq %rdx, %rsi
- movq $512, %rcx
+ movl $512, %ecx
rep ; movsq
movq %rdx, %rdi
movq %r10, %rsi
- movq $512, %rcx
+ movl $512, %ecx
rep ; movsq
lea PAGE_SIZE(%rax), %rsi
unsigned long max_low_pfn_mapped;
unsigned long max_pfn_mapped;
-bool __read_mostly kaslr_enabled = false;
-
#ifdef CONFIG_DMI
RESERVE_BRK(dmi_alloc, 65536);
#endif
mapaddr = ramdisk_image & PAGE_MASK;
p = early_memremap(mapaddr, clen+slop);
memcpy(q, p+slop, clen);
- early_iounmap(p, clen+slop);
+ early_memunmap(p, clen+slop);
q += clen;
ramdisk_image += clen;
ramdisk_size -= clen;
}
#endif /* CONFIG_BLK_DEV_INITRD */
-static void __init parse_kaslr_setup(u64 pa_data, u32 data_len)
-{
- kaslr_enabled = (bool)(pa_data + sizeof(struct setup_data));
-}
-
static void __init parse_setup_data(void)
{
struct setup_data *data;
data_len = data->len + sizeof(struct setup_data);
data_type = data->type;
pa_next = data->next;
- early_iounmap(data, sizeof(*data));
+ early_memunmap(data, sizeof(*data));
switch (data_type) {
case SETUP_E820_EXT:
case SETUP_EFI:
parse_efi_setup(pa_data, data_len);
break;
- case SETUP_KASLR:
- parse_kaslr_setup(pa_data, data_len);
- break;
default:
break;
}
E820_RAM, E820_RESERVED_KERN);
found = 1;
pa_data = data->next;
- early_iounmap(data, sizeof(*data));
+ early_memunmap(data, sizeof(*data));
}
if (!found)
return;
data = early_memremap(pa_data, sizeof(*data));
memblock_reserve(pa_data, sizeof(*data) + data->len);
pa_data = data->next;
- early_iounmap(data, sizeof(*data));
+ early_memunmap(data, sizeof(*data));
}
}
static int
dump_kernel_offset(struct notifier_block *self, unsigned long v, void *p)
{
- if (kaslr_enabled)
+ if (kaslr_enabled()) {
pr_emerg("Kernel Offset: 0x%lx from 0x%lx (relocation range: 0x%lx-0x%lx)\n",
(unsigned long)&_text - __START_KERNEL,
__START_KERNEL,
__START_KERNEL_map,
MODULES_VADDR-1);
- else
+ } else {
pr_emerg("Kernel Offset: disabled\n");
+ }
return 0;
}
regs->seg = GET_SEG(seg) | 3; \
} while (0)
-int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc,
- unsigned long *pax)
+int restore_sigcontext(struct pt_regs *regs, struct sigcontext __user *sc)
{
void __user *buf;
unsigned int tmpflags;
#endif /* CONFIG_X86_32 */
COPY(di); COPY(si); COPY(bp); COPY(sp); COPY(bx);
- COPY(dx); COPY(cx); COPY(ip);
+ COPY(dx); COPY(cx); COPY(ip); COPY(ax);
#ifdef CONFIG_X86_64
COPY(r8);
COPY(r15);
#endif /* CONFIG_X86_64 */
-#ifdef CONFIG_X86_32
COPY_SEG_CPL3(cs);
COPY_SEG_CPL3(ss);
-#else /* !CONFIG_X86_32 */
- /* Kernel saves and restores only the CS segment register on signals,
- * which is the bare minimum needed to allow mixed 32/64-bit code.
- * App's signal handler can save/restore other segments if needed. */
- COPY_SEG_CPL3(cs);
-#endif /* CONFIG_X86_32 */
get_user_ex(tmpflags, &sc->flags);
regs->flags = (regs->flags & ~FIX_EFLAGS) | (tmpflags & FIX_EFLAGS);
regs->orig_ax = -1; /* disable syscall checks */
get_user_ex(buf, &sc->fpstate);
-
- get_user_ex(*pax, &sc->ax);
} get_user_catch(err);
err |= restore_xstate_sig(buf, config_enabled(CONFIG_X86_32));
+ force_iret();
+
return err;
}
#else /* !CONFIG_X86_32 */
put_user_ex(regs->flags, &sc->flags);
put_user_ex(regs->cs, &sc->cs);
- put_user_ex(0, &sc->gs);
- put_user_ex(0, &sc->fs);
+ put_user_ex(0, &sc->__pad2);
+ put_user_ex(0, &sc->__pad1);
+ put_user_ex(regs->ss, &sc->ss);
#endif /* CONFIG_X86_32 */
put_user_ex(fpstate, &sc->fpstate);
regs->sp = (unsigned long)frame;
- /* Set up the CS register to run signal handlers in 64-bit mode,
- even if the handler happens to be interrupting 32-bit code. */
+ /*
+ * Set up the CS and SS registers to run signal handlers in
+ * 64-bit mode, even if the handler happens to be interrupting
+ * 32-bit or 16-bit code.
+ *
+ * SS is subtle. In 64-bit mode, we don't need any particular
+ * SS descriptor, but we do need SS to be valid. It's possible
+ * that the old SS is entirely bogus -- this can happen if the
+ * signal we're trying to deliver is #GP or #SS caused by a bad
+ * SS value.
+ */
regs->cs = __USER_CS;
+ regs->ss = __USER_DS;
return 0;
}
{
struct pt_regs *regs = current_pt_regs();
struct sigframe __user *frame;
- unsigned long ax;
sigset_t set;
frame = (struct sigframe __user *)(regs->sp - 8);
set_current_blocked(&set);
- if (restore_sigcontext(regs, &frame->sc, &ax))
+ if (restore_sigcontext(regs, &frame->sc))
goto badframe;
- return ax;
+ return regs->ax;
badframe:
signal_fault(regs, frame, "sigreturn");
{
struct pt_regs *regs = current_pt_regs();
struct rt_sigframe __user *frame;
- unsigned long ax;
sigset_t set;
frame = (struct rt_sigframe __user *)(regs->sp - sizeof(long));
set_current_blocked(&set);
- if (restore_sigcontext(regs, &frame->uc.uc_mcontext, &ax))
+ if (restore_sigcontext(regs, &frame->uc.uc_mcontext))
goto badframe;
if (restore_altstack(&frame->uc.uc_stack))
goto badframe;
- return ax;
+ return regs->ax;
badframe:
signal_fault(regs, frame, "rt_sigreturn");
* Ensure the signal handler starts with the new fpu state.
*/
if (used_math())
- drop_init_fpu(current);
+ fpu_reset_state(current);
}
signal_setup_done(failed, ksig, test_thread_flag(TIF_SINGLESTEP));
}
struct pt_regs *regs = current_pt_regs();
struct rt_sigframe_x32 __user *frame;
sigset_t set;
- unsigned long ax;
frame = (struct rt_sigframe_x32 __user *)(regs->sp - 8);
set_current_blocked(&set);
- if (restore_sigcontext(regs, &frame->uc.uc_mcontext, &ax))
+ if (restore_sigcontext(regs, &frame->uc.uc_mcontext))
goto badframe;
if (compat_restore_altstack(&frame->uc.uc_stack))
goto badframe;
- return ax;
+ return regs->ax;
badframe:
signal_fault(regs, frame, "x32 rt_sigreturn");
return boot_error;
}
+void common_cpu_up(unsigned int cpu, struct task_struct *idle)
+{
+ /* Just in case we booted with a single CPU. */
+ alternatives_enable_smp();
+
+ per_cpu(current_task, cpu) = idle;
+
+#ifdef CONFIG_X86_32
+ /* Stack for startup_32 can be just as for start_secondary onwards */
+ irq_ctx_init(cpu);
+ per_cpu(cpu_current_top_of_stack, cpu) =
+ (unsigned long)task_stack_page(idle) + THREAD_SIZE;
+#else
+ clear_tsk_thread_flag(idle, TIF_FORK);
+ initial_gs = per_cpu_offset(cpu);
+#endif
+ per_cpu(kernel_stack, cpu) =
+ (unsigned long)task_stack_page(idle) + THREAD_SIZE;
+}
+
/*
* NOTE - on most systems this is a PHYSICAL apic ID, but on multiquad
* (ie clustered apic addressing mode), this is a LOGICAL apic ID.
int cpu0_nmi_registered = 0;
unsigned long timeout;
- /* Just in case we booted with a single CPU. */
- alternatives_enable_smp();
-
idle->thread.sp = (unsigned long) (((struct pt_regs *)
(THREAD_SIZE + task_stack_page(idle))) - 1);
- per_cpu(current_task, cpu) = idle;
-#ifdef CONFIG_X86_32
- /* Stack for startup_32 can be just as for start_secondary onwards */
- irq_ctx_init(cpu);
-#else
- clear_tsk_thread_flag(idle, TIF_FORK);
- initial_gs = per_cpu_offset(cpu);
-#endif
- per_cpu(kernel_stack, cpu) =
- (unsigned long)task_stack_page(idle) -
- KERNEL_STACK_OFFSET + THREAD_SIZE;
early_gdt_descr.address = (unsigned long)get_cpu_gdt_table(cpu);
initial_code = (unsigned long)start_secondary;
stack_start = idle->thread.sp;
/* the FPU context is blank, nobody can own it */
__cpu_disable_lazy_restore(cpu);
+ common_cpu_up(cpu, tidle);
+
err = do_boot_cpu(apicid, cpu, tidle);
if (err) {
pr_err("do_boot_cpu failed(%d) to wakeup CPU#%u\n", err, cpu);
return SMP_NO_APIC;
}
- verify_local_APIC();
-
/*
* If SMP should be disabled, then really disable it!
*/
return va_align.mask;
}
+/*
+ * To avoid aliasing in the I$ on AMD F15h, the bits defined by the
+ * va_align.bits, [12:upper_bit), are set to a random value instead of
+ * zeroing them. This random value is computed once per boot. This form
+ * of ASLR is known as "per-boot ASLR".
+ *
+ * To achieve this, the random value is added to the info.align_offset
+ * value before calling vm_unmapped_area() or ORed directly to the
+ * address.
+ */
+static unsigned long get_align_bits(void)
+{
+ return va_align.bits & get_align_mask();
+}
+
unsigned long align_vdso_addr(unsigned long addr)
{
unsigned long align_mask = get_align_mask();
- return (addr + align_mask) & ~align_mask;
+ addr = (addr + align_mask) & ~align_mask;
+ return addr | get_align_bits();
}
static int __init control_va_addr_alignment(char *str)
info.length = len;
info.low_limit = begin;
info.high_limit = end;
- info.align_mask = filp ? get_align_mask() : 0;
+ info.align_mask = 0;
info.align_offset = pgoff << PAGE_SHIFT;
+ if (filp) {
+ info.align_mask = get_align_mask();
+ info.align_offset += get_align_bits();
+ }
return vm_unmapped_area(&info);
}
info.length = len;
info.low_limit = PAGE_SIZE;
info.high_limit = mm->mmap_base;
- info.align_mask = filp ? get_align_mask() : 0;
+ info.align_mask = 0;
info.align_offset = pgoff << PAGE_SHIFT;
+ if (filp) {
+ info.align_mask = get_align_mask();
+ info.align_offset += get_align_bits();
+ }
addr = vm_unmapped_area(&info);
if (!(addr & ~PAGE_MASK))
return addr;
#include <linux/cache.h>
#include <asm/asm-offsets.h>
-#define __SYSCALL_I386(nr, sym, compat) extern asmlinkage void sym(void) ;
+#ifdef CONFIG_IA32_EMULATION
+#define SYM(sym, compat) compat
+#else
+#define SYM(sym, compat) sym
+#define ia32_sys_call_table sys_call_table
+#define __NR_ia32_syscall_max __NR_syscall_max
+#endif
+
+#define __SYSCALL_I386(nr, sym, compat) extern asmlinkage void SYM(sym, compat)(void) ;
#include <asm/syscalls_32.h>
#undef __SYSCALL_I386
-#define __SYSCALL_I386(nr, sym, compat) [nr] = sym,
+#define __SYSCALL_I386(nr, sym, compat) [nr] = SYM(sym, compat),
typedef asmlinkage void (*sys_call_ptr_t)(void);
extern asmlinkage void sys_ni_syscall(void);
-__visible const sys_call_ptr_t sys_call_table[__NR_syscall_max+1] = {
+__visible const sys_call_ptr_t ia32_sys_call_table[__NR_ia32_syscall_max+1] = {
/*
* Smells like a compiler bug -- it doesn't work
* when the & below is removed.
*/
- [0 ... __NR_syscall_max] = &sys_ni_syscall,
+ [0 ... __NR_ia32_syscall_max] = &sys_ni_syscall,
#include <asm/syscalls_32.h>
};
{
unsigned long pc = instruction_pointer(regs);
- if (!user_mode_vm(regs) && in_lock_functions(pc)) {
+ if (!user_mode(regs) && in_lock_functions(pc)) {
#ifdef CONFIG_FRAME_POINTER
return *(unsigned long *)(regs->bp + sizeof(long));
#else
{
enum ctx_state prev_state;
- if (user_mode_vm(regs)) {
+ if (user_mode(regs)) {
/* Other than that, we're just an exception. */
prev_state = exception_enter();
} else {
/* Must be before exception_exit. */
preempt_count_sub(HARDIRQ_OFFSET);
- if (user_mode_vm(regs))
+ if (user_mode(regs))
return exception_exit(prev_state);
else
rcu_nmi_exit();
*
* IST exception handlers normally cannot schedule. As a special
* exception, if the exception interrupted userspace code (i.e.
- * user_mode_vm(regs) would return true) and the exception was not
+ * user_mode(regs) would return true) and the exception was not
* a double fault, it can be safe to schedule. ist_begin_non_atomic()
* begins a non-atomic section within an ist_enter()/ist_exit() region.
* Callers are responsible for enabling interrupts themselves inside
*/
void ist_begin_non_atomic(struct pt_regs *regs)
{
- BUG_ON(!user_mode_vm(regs));
+ BUG_ON(!user_mode(regs));
/*
* Sanity check: we need to be on the normal thread stack. This
* will catch asm bugs and any attempt to use ist_preempt_enable
* from double_fault.
*/
- BUG_ON(((current_stack_pointer() ^ this_cpu_read_stable(kernel_stack))
- & ~(THREAD_SIZE - 1)) != 0);
+ BUG_ON((unsigned long)(current_top_of_stack() -
+ current_stack_pointer()) >= THREAD_SIZE);
preempt_count_sub(HARDIRQ_OFFSET);
}
do_trap_no_signal(struct task_struct *tsk, int trapnr, char *str,
struct pt_regs *regs, long error_code)
{
-#ifdef CONFIG_X86_32
- if (regs->flags & X86_VM_MASK) {
+ if (v8086_mode(regs)) {
/*
* Traps 0, 1, 3, 4, and 5 should be forwarded to vm86.
* On nmi (interrupt 2), do_trap should not be called.
}
return -1;
}
-#endif
+
if (!user_mode(regs)) {
if (!fixup_exception(regs)) {
tsk->thread.error_code = error_code;
prev_state = exception_enter();
conditional_sti(regs);
-#ifdef CONFIG_X86_32
- if (regs->flags & X86_VM_MASK) {
+ if (v8086_mode(regs)) {
local_irq_enable();
handle_vm86_fault((struct kernel_vm86_regs *) regs, error_code);
goto exit;
}
-#endif
tsk = current;
if (!user_mode(regs)) {
/* Copy the remainder of the stack from the current stack. */
memmove(new_stack, s, offsetof(struct bad_iret_stack, regs.ip));
- BUG_ON(!user_mode_vm(&new_stack->regs));
+ BUG_ON(!user_mode(&new_stack->regs));
return new_stack;
}
NOKPROBE_SYMBOL(fixup_bad_iret);
/* It's safe to allow irq's after DR6 has been saved */
preempt_conditional_sti(regs);
- if (regs->flags & X86_VM_MASK) {
+ if (v8086_mode(regs)) {
handle_vm86_trap((struct kernel_vm86_regs *) regs, error_code,
X86_TRAP_DB);
preempt_conditional_cli(regs);
return;
conditional_sti(regs);
- if (!user_mode_vm(regs))
+ if (!user_mode(regs))
{
if (!fixup_exception(regs)) {
task->thread.error_code = error_code;
/*
* Save the info for the exception handler and clear the error.
*/
- save_init_fpu(task);
+ unlazy_fpu(task);
task->thread.trap_nr = trapnr;
task->thread.error_code = error_code;
info.si_signo = SIGFPE;
kernel_fpu_disable();
__thread_fpu_begin(tsk);
if (unlikely(restore_fpu_checking(tsk))) {
- drop_init_fpu(tsk);
+ fpu_reset_state(tsk);
force_sig_info(SIGSEGV, SEND_SIG_PRIV, tsk);
} else {
tsk->thread.fpu_counter++;
/* Set of traps needed for early debugging. */
void __init early_trap_init(void)
{
- set_intr_gate_ist(X86_TRAP_DB, &debug, DEBUG_STACK);
+ /*
+ * Don't use IST to set DEBUG_STACK as it doesn't work until TSS
+ * is ready in cpu_init() <-- trap_init(). Before trap_init(),
+ * CPU runs at ring 0 so it is impossible to hit an invalid
+ * stack. Using the original stack works well enough at this
+ * early stage. DEBUG_STACK will be equipped after cpu_init() in
+ * trap_init().
+ *
+ * We don't need to set trace_idt_table like set_intr_gate(),
+ * since we don't have trace_debug and it will be reset to
+ * 'debug' in trap_init() by set_intr_gate_ist().
+ */
+ set_intr_gate_notrace(X86_TRAP_DB, debug);
/* int3 can be called from all */
- set_system_intr_gate_ist(X86_TRAP_BP, &int3, DEBUG_STACK);
+ set_system_intr_gate(X86_TRAP_BP, &int3);
#ifdef CONFIG_X86_32
set_intr_gate(X86_TRAP_PF, page_fault);
#endif
*/
cpu_init();
+ /*
+ * X86_TRAP_DB and X86_TRAP_BP have been set
+ * in early_trap_init(). However, ITS works only after
+ * cpu_init() loads TSS. See comments in early_trap_init().
+ */
+ set_intr_gate_ist(X86_TRAP_DB, &debug, DEBUG_STACK);
+ /* int3 can be called from all */
+ set_system_intr_gate_ist(X86_TRAP_BP, &int3, DEBUG_STACK);
+
x86_init.irqs.trap_init();
#ifdef CONFIG_X86_64
int ret = NOTIFY_DONE;
/* We are only interested in userspace traps */
- if (regs && !user_mode_vm(regs))
+ if (regs && !user_mode(regs))
return NOTIFY_DONE;
switch (val) {
do_exit(SIGSEGV);
}
- tss = &per_cpu(init_tss, get_cpu());
+ tss = &per_cpu(cpu_tss, get_cpu());
current->thread.sp0 = current->thread.saved_sp0;
current->thread.sysenter_cs = __KERNEL_CS;
load_sp0(tss, ¤t->thread);
tsk->thread.saved_fs = info->regs32->fs;
tsk->thread.saved_gs = get_user_gs(info->regs32);
- tss = &per_cpu(init_tss, get_cpu());
+ tss = &per_cpu(cpu_tss, get_cpu());
tsk->thread.sp0 = (unsigned long) &info->VM86_TSS_ESP0;
if (cpu_has_sep)
tsk->thread.sysenter_cs = 0;
gtod_write_begin(vdata);
/* copy vsyscall data */
- vdata->vclock_mode = tk->tkr.clock->archdata.vclock_mode;
- vdata->cycle_last = tk->tkr.cycle_last;
- vdata->mask = tk->tkr.mask;
- vdata->mult = tk->tkr.mult;
- vdata->shift = tk->tkr.shift;
+ vdata->vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
+ vdata->cycle_last = tk->tkr_mono.cycle_last;
+ vdata->mask = tk->tkr_mono.mask;
+ vdata->mult = tk->tkr_mono.mult;
+ vdata->shift = tk->tkr_mono.shift;
vdata->wall_time_sec = tk->xtime_sec;
- vdata->wall_time_snsec = tk->tkr.xtime_nsec;
+ vdata->wall_time_snsec = tk->tkr_mono.xtime_nsec;
vdata->monotonic_time_sec = tk->xtime_sec
+ tk->wall_to_monotonic.tv_sec;
- vdata->monotonic_time_snsec = tk->tkr.xtime_nsec
+ vdata->monotonic_time_snsec = tk->tkr_mono.xtime_nsec
+ ((u64)tk->wall_to_monotonic.tv_nsec
- << tk->tkr.shift);
+ << tk->tkr_mono.shift);
while (vdata->monotonic_time_snsec >=
- (((u64)NSEC_PER_SEC) << tk->tkr.shift)) {
+ (((u64)NSEC_PER_SEC) << tk->tkr_mono.shift)) {
vdata->monotonic_time_snsec -=
- ((u64)NSEC_PER_SEC) << tk->tkr.shift;
+ ((u64)NSEC_PER_SEC) << tk->tkr_mono.shift;
vdata->monotonic_time_sec++;
}
vdata->wall_time_coarse_sec = tk->xtime_sec;
- vdata->wall_time_coarse_nsec = (long)(tk->tkr.xtime_nsec >>
- tk->tkr.shift);
+ vdata->wall_time_coarse_nsec = (long)(tk->tkr_mono.xtime_nsec >>
+ tk->tkr_mono.shift);
vdata->monotonic_time_coarse_sec =
vdata->wall_time_coarse_sec + tk->wall_to_monotonic.tv_sec;
config_enabled(CONFIG_IA32_EMULATION));
if (!buf) {
- drop_init_fpu(tsk);
+ fpu_reset_state(tsk);
return 0;
}
* thread's fpu state, reconstruct fxstate from the fsave
* header. Sanitize the copied state etc.
*/
- struct xsave_struct *xsave = &tsk->thread.fpu.state->xsave;
+ struct fpu *fpu = &tsk->thread.fpu;
struct user_i387_ia32_struct env;
int err = 0;
*/
drop_fpu(tsk);
- if (__copy_from_user(xsave, buf_fx, state_size) ||
+ if (__copy_from_user(&fpu->state->xsave, buf_fx, state_size) ||
__copy_from_user(&env, buf, sizeof(env))) {
+ fpu_finit(fpu);
err = -1;
} else {
sanitize_restored_xstate(tsk, &env, xstate_bv, fx_only);
- set_used_math();
}
+ set_used_math();
if (use_eager_fpu()) {
preempt_disable();
math_state_restore();
*/
user_fpu_begin();
if (restore_user_xstate(buf_fx, xstate_bv, fx_only)) {
- drop_init_fpu(tsk);
+ fpu_reset_state(tsk);
return -1;
}
}
this_func();
}
-static inline void __init eager_fpu_init_bp(void)
-{
- current->thread.fpu.state =
- alloc_bootmem_align(xstate_size, __alignof__(struct xsave_struct));
- if (!init_xstate_buf)
- setup_init_fpu_buf();
-}
-
-void eager_fpu_init(void)
+/*
+ * setup_init_fpu_buf() is __init and it is OK to call it here because
+ * init_xstate_buf will be unset only once during boot.
+ */
+void __init_refok eager_fpu_init(void)
{
- static __refdata void (*boot_func)(void) = eager_fpu_init_bp;
-
- clear_used_math();
+ WARN_ON(used_math());
current_thread_info()->status = 0;
if (eagerfpu == ENABLE)
return;
}
- if (boot_func) {
- boot_func();
- boot_func = NULL;
- }
-
- /*
- * This is same as math_state_restore(). But use_xsave() is
- * not yet patched to use math_state_restore().
- */
- init_fpu(current);
- __thread_fpu_begin(current);
- if (cpu_has_xsave)
- xrstor_state(init_xstate_buf, -1);
- else
- fxrstor_checking(&init_xstate_buf->i387);
+ if (!init_xstate_buf)
+ setup_init_fpu_buf();
}
/*
-ccflags-y += -Ivirt/kvm -Iarch/x86/kvm
+ccflags-y += -Iarch/x86/kvm
CFLAGS_x86.o := -I.
CFLAGS_svm.o := -I.
((best->eax & 0xff00) >> 8) != 0)
return -EINVAL;
+ /* Update physical-address width */
+ vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
+
kvm_pmu_cpuid_update(vcpu);
return 0;
}
}
}
+int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu)
+{
+ struct kvm_cpuid_entry2 *best;
+
+ best = kvm_find_cpuid_entry(vcpu, 0x80000000, 0);
+ if (!best || best->eax < 0x80000008)
+ goto not_found;
+ best = kvm_find_cpuid_entry(vcpu, 0x80000008, 0);
+ if (best)
+ return best->eax & 0xff;
+not_found:
+ return 36;
+}
+EXPORT_SYMBOL_GPL(cpuid_query_maxphyaddr);
+
/* when an old userspace process fills a new kernel module */
int kvm_vcpu_ioctl_set_cpuid(struct kvm_vcpu *vcpu,
struct kvm_cpuid *cpuid,
}
EXPORT_SYMBOL_GPL(kvm_find_cpuid_entry);
-int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
-{
- struct kvm_cpuid_entry2 *best;
-
- best = kvm_find_cpuid_entry(vcpu, 0x80000000, 0);
- if (!best || best->eax < 0x80000008)
- goto not_found;
- best = kvm_find_cpuid_entry(vcpu, 0x80000008, 0);
- if (best)
- return best->eax & 0xff;
-not_found:
- return 36;
-}
-EXPORT_SYMBOL_GPL(cpuid_maxphyaddr);
-
/*
* If no match is found, check whether we exceed the vCPU's limit
* and return the content of the highest valid _standard_ leaf instead.
struct kvm_cpuid_entry2 __user *entries);
void kvm_cpuid(struct kvm_vcpu *vcpu, u32 *eax, u32 *ebx, u32 *ecx, u32 *edx);
+int cpuid_query_maxphyaddr(struct kvm_vcpu *vcpu);
+
+static inline int cpuid_maxphyaddr(struct kvm_vcpu *vcpu)
+{
+ return vcpu->arch.maxphyaddr;
+}
static inline bool guest_cpuid_has_xsave(struct kvm_vcpu *vcpu)
{
struct kvm_cpuid_entry2 *best;
if (!static_cpu_has(X86_FEATURE_XSAVE))
- return 0;
+ return false;
best = kvm_find_cpuid_entry(vcpu, 1, 0);
return best && (best->ecx & bit(X86_FEATURE_XSAVE));
struct opcode mode64;
};
-/* EFLAGS bit definitions. */
-#define EFLG_ID (1<<21)
-#define EFLG_VIP (1<<20)
-#define EFLG_VIF (1<<19)
-#define EFLG_AC (1<<18)
-#define EFLG_VM (1<<17)
-#define EFLG_RF (1<<16)
-#define EFLG_IOPL (3<<12)
-#define EFLG_NT (1<<14)
-#define EFLG_OF (1<<11)
-#define EFLG_DF (1<<10)
-#define EFLG_IF (1<<9)
-#define EFLG_TF (1<<8)
-#define EFLG_SF (1<<7)
-#define EFLG_ZF (1<<6)
-#define EFLG_AF (1<<4)
-#define EFLG_PF (1<<2)
-#define EFLG_CF (1<<0)
-
#define EFLG_RESERVED_ZEROS_MASK 0xffc0802a
-#define EFLG_RESERVED_ONE_MASK 2
enum x86_transfer_type {
X86_TRANSFER_NONE,
* These EFLAGS bits are restored from saved value during emulation, and
* any changes are written back to the saved value after emulation.
*/
-#define EFLAGS_MASK (EFLG_OF|EFLG_SF|EFLG_ZF|EFLG_AF|EFLG_PF|EFLG_CF)
+#define EFLAGS_MASK (X86_EFLAGS_OF|X86_EFLAGS_SF|X86_EFLAGS_ZF|X86_EFLAGS_AF|\
+ X86_EFLAGS_PF|X86_EFLAGS_CF)
#ifdef CONFIG_X86_64
#define ON64(x) x
*dest = (*dest & ~mask) | (src & mask);
}
+static void assign_register(unsigned long *reg, u64 val, int bytes)
+{
+ /* The 4-byte case *is* correct: in 64-bit mode we zero-extend. */
+ switch (bytes) {
+ case 1:
+ *(u8 *)reg = (u8)val;
+ break;
+ case 2:
+ *(u16 *)reg = (u16)val;
+ break;
+ case 4:
+ *reg = (u32)val;
+ break; /* 64b: zero-extend */
+ case 8:
+ *reg = val;
+ break;
+ }
+}
+
static inline unsigned long ad_mask(struct x86_emulate_ctxt *ctxt)
{
return (1UL << (ctxt->ad_bytes << 3)) - 1;
FASTOP2R(cmp, cmp_r);
+static int em_bsf_c(struct x86_emulate_ctxt *ctxt)
+{
+ /* If src is zero, do not writeback, but update flags */
+ if (ctxt->src.val == 0)
+ ctxt->dst.type = OP_NONE;
+ return fastop(ctxt, em_bsf);
+}
+
+static int em_bsr_c(struct x86_emulate_ctxt *ctxt)
+{
+ /* If src is zero, do not writeback, but update flags */
+ if (ctxt->src.val == 0)
+ ctxt->dst.type = OP_NONE;
+ return fastop(ctxt, em_bsr);
+}
+
static u8 test_cc(unsigned int condition, unsigned long flags)
{
u8 rc;
unsigned int in_page, n;
unsigned int count = ctxt->rep_prefix ?
address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) : 1;
- in_page = (ctxt->eflags & EFLG_DF) ?
+ in_page = (ctxt->eflags & X86_EFLAGS_DF) ?
offset_in_page(reg_read(ctxt, VCPU_REGS_RDI)) :
PAGE_SIZE - offset_in_page(reg_read(ctxt, VCPU_REGS_RDI));
n = min3(in_page, (unsigned int)sizeof(rc->data) / size, count);
}
if (ctxt->rep_prefix && (ctxt->d & String) &&
- !(ctxt->eflags & EFLG_DF)) {
+ !(ctxt->eflags & X86_EFLAGS_DF)) {
ctxt->dst.data = rc->data + rc->pos;
ctxt->dst.type = OP_MEM_STR;
ctxt->dst.count = (rc->end - rc->pos) / size;
static void write_register_operand(struct operand *op)
{
- /* The 4-byte case *is* correct: in 64-bit mode we zero-extend. */
- switch (op->bytes) {
- case 1:
- *(u8 *)op->addr.reg = (u8)op->val;
- break;
- case 2:
- *(u16 *)op->addr.reg = (u16)op->val;
- break;
- case 4:
- *op->addr.reg = (u32)op->val;
- break; /* 64b: zero-extend */
- case 8:
- *op->addr.reg = op->val;
- break;
- }
+ return assign_register(op->addr.reg, op->val, op->bytes);
}
static int writeback(struct x86_emulate_ctxt *ctxt, struct operand *op)
{
int rc;
unsigned long val, change_mask;
- int iopl = (ctxt->eflags & X86_EFLAGS_IOPL) >> IOPL_SHIFT;
+ int iopl = (ctxt->eflags & X86_EFLAGS_IOPL) >> X86_EFLAGS_IOPL_BIT;
int cpl = ctxt->ops->cpl(ctxt);
rc = emulate_pop(ctxt, &val, len);
if (rc != X86EMUL_CONTINUE)
return rc;
- change_mask = EFLG_CF | EFLG_PF | EFLG_AF | EFLG_ZF | EFLG_SF | EFLG_OF
- | EFLG_TF | EFLG_DF | EFLG_NT | EFLG_AC | EFLG_ID;
+ change_mask = X86_EFLAGS_CF | X86_EFLAGS_PF | X86_EFLAGS_AF |
+ X86_EFLAGS_ZF | X86_EFLAGS_SF | X86_EFLAGS_OF |
+ X86_EFLAGS_TF | X86_EFLAGS_DF | X86_EFLAGS_NT |
+ X86_EFLAGS_AC | X86_EFLAGS_ID;
switch(ctxt->mode) {
case X86EMUL_MODE_PROT64:
case X86EMUL_MODE_PROT32:
case X86EMUL_MODE_PROT16:
if (cpl == 0)
- change_mask |= EFLG_IOPL;
+ change_mask |= X86_EFLAGS_IOPL;
if (cpl <= iopl)
- change_mask |= EFLG_IF;
+ change_mask |= X86_EFLAGS_IF;
break;
case X86EMUL_MODE_VM86:
if (iopl < 3)
return emulate_gp(ctxt, 0);
- change_mask |= EFLG_IF;
+ change_mask |= X86_EFLAGS_IF;
break;
default: /* real mode */
- change_mask |= (EFLG_IOPL | EFLG_IF);
+ change_mask |= (X86_EFLAGS_IOPL | X86_EFLAGS_IF);
break;
}
static int em_pushf(struct x86_emulate_ctxt *ctxt)
{
- ctxt->src.val = (unsigned long)ctxt->eflags & ~EFLG_VM;
+ ctxt->src.val = (unsigned long)ctxt->eflags & ~X86_EFLAGS_VM;
return em_push(ctxt);
}
{
int rc = X86EMUL_CONTINUE;
int reg = VCPU_REGS_RDI;
+ u32 val;
while (reg >= VCPU_REGS_RAX) {
if (reg == VCPU_REGS_RSP) {
--reg;
}
- rc = emulate_pop(ctxt, reg_rmw(ctxt, reg), ctxt->op_bytes);
+ rc = emulate_pop(ctxt, &val, ctxt->op_bytes);
if (rc != X86EMUL_CONTINUE)
break;
+ assign_register(reg_rmw(ctxt, reg), val, ctxt->op_bytes);
--reg;
}
return rc;
if (rc != X86EMUL_CONTINUE)
return rc;
- ctxt->eflags &= ~(EFLG_IF | EFLG_TF | EFLG_AC);
+ ctxt->eflags &= ~(X86_EFLAGS_IF | X86_EFLAGS_TF | X86_EFLAGS_AC);
ctxt->src.val = get_segment_selector(ctxt, VCPU_SREG_CS);
rc = em_push(ctxt);
unsigned long temp_eip = 0;
unsigned long temp_eflags = 0;
unsigned long cs = 0;
- unsigned long mask = EFLG_CF | EFLG_PF | EFLG_AF | EFLG_ZF | EFLG_SF | EFLG_TF |
- EFLG_IF | EFLG_DF | EFLG_OF | EFLG_IOPL | EFLG_NT | EFLG_RF |
- EFLG_AC | EFLG_ID | (1 << 1); /* Last one is the reserved bit */
- unsigned long vm86_mask = EFLG_VM | EFLG_VIF | EFLG_VIP;
+ unsigned long mask = X86_EFLAGS_CF | X86_EFLAGS_PF | X86_EFLAGS_AF |
+ X86_EFLAGS_ZF | X86_EFLAGS_SF | X86_EFLAGS_TF |
+ X86_EFLAGS_IF | X86_EFLAGS_DF | X86_EFLAGS_OF |
+ X86_EFLAGS_IOPL | X86_EFLAGS_NT | X86_EFLAGS_RF |
+ X86_EFLAGS_AC | X86_EFLAGS_ID |
+ X86_EFLAGS_FIXED;
+ unsigned long vm86_mask = X86_EFLAGS_VM | X86_EFLAGS_VIF |
+ X86_EFLAGS_VIP;
/* TODO: Add stack limit check */
ctxt->_eip = temp_eip;
-
if (ctxt->op_bytes == 4)
ctxt->eflags = ((temp_eflags & mask) | (ctxt->eflags & vm86_mask));
else if (ctxt->op_bytes == 2) {
}
ctxt->eflags &= ~EFLG_RESERVED_ZEROS_MASK; /* Clear reserved zeros */
- ctxt->eflags |= EFLG_RESERVED_ONE_MASK;
+ ctxt->eflags |= X86_EFLAGS_FIXED;
ctxt->ops->set_nmi_mask(ctxt, false);
return rc;
((u32) (old >> 32) != (u32) reg_read(ctxt, VCPU_REGS_RDX))) {
*reg_write(ctxt, VCPU_REGS_RAX) = (u32) (old >> 0);
*reg_write(ctxt, VCPU_REGS_RDX) = (u32) (old >> 32);
- ctxt->eflags &= ~EFLG_ZF;
+ ctxt->eflags &= ~X86_EFLAGS_ZF;
} else {
ctxt->dst.val64 = ((u64)reg_read(ctxt, VCPU_REGS_RCX) << 32) |
(u32) reg_read(ctxt, VCPU_REGS_RBX);
- ctxt->eflags |= EFLG_ZF;
+ ctxt->eflags |= X86_EFLAGS_ZF;
}
return X86EMUL_CONTINUE;
}
ctxt->src.val = ctxt->dst.orig_val;
fastop(ctxt, em_cmp);
- if (ctxt->eflags & EFLG_ZF) {
+ if (ctxt->eflags & X86_EFLAGS_ZF) {
/* Success: write back to memory; no update of EAX */
ctxt->src.type = OP_NONE;
ctxt->dst.val = ctxt->src.orig_val;
ops->get_msr(ctxt, MSR_SYSCALL_MASK, &msr_data);
ctxt->eflags &= ~msr_data;
- ctxt->eflags |= EFLG_RESERVED_ONE_MASK;
+ ctxt->eflags |= X86_EFLAGS_FIXED;
#endif
} else {
/* legacy mode */
ops->get_msr(ctxt, MSR_STAR, &msr_data);
ctxt->_eip = (u32)msr_data;
- ctxt->eflags &= ~(EFLG_VM | EFLG_IF);
+ ctxt->eflags &= ~(X86_EFLAGS_VM | X86_EFLAGS_IF);
}
return X86EMUL_CONTINUE;
if ((msr_data & 0xfffc) == 0x0)
return emulate_gp(ctxt, 0);
- ctxt->eflags &= ~(EFLG_VM | EFLG_IF);
- cs_sel = (u16)msr_data & ~SELECTOR_RPL_MASK;
+ ctxt->eflags &= ~(X86_EFLAGS_VM | X86_EFLAGS_IF);
+ cs_sel = (u16)msr_data & ~SEGMENT_RPL_MASK;
ss_sel = cs_sel + 8;
if (efer & EFER_LMA) {
cs.d = 0;
return emulate_gp(ctxt, 0);
break;
}
- cs_sel |= SELECTOR_RPL_MASK;
- ss_sel |= SELECTOR_RPL_MASK;
+ cs_sel |= SEGMENT_RPL_MASK;
+ ss_sel |= SEGMENT_RPL_MASK;
ops->set_segment(ctxt, cs_sel, &cs, 0, VCPU_SREG_CS);
ops->set_segment(ctxt, ss_sel, &ss, 0, VCPU_SREG_SS);
return false;
if (ctxt->mode == X86EMUL_MODE_VM86)
return true;
- iopl = (ctxt->eflags & X86_EFLAGS_IOPL) >> IOPL_SHIFT;
+ iopl = (ctxt->eflags & X86_EFLAGS_IOPL) >> X86_EFLAGS_IOPL_BIT;
return ctxt->ops->cpl(ctxt) > iopl;
}
return ret;
ret = __load_segment_descriptor(ctxt, tss->gs, VCPU_SREG_GS, cpl,
X86_TRANSFER_TASK_SWITCH, NULL);
- if (ret != X86EMUL_CONTINUE)
- return ret;
- return X86EMUL_CONTINUE;
+ return ret;
}
static int task_switch_32(struct x86_emulate_ctxt *ctxt,
static void string_addr_inc(struct x86_emulate_ctxt *ctxt, int reg,
struct operand *op)
{
- int df = (ctxt->eflags & EFLG_DF) ? -op->count : op->count;
+ int df = (ctxt->eflags & X86_EFLAGS_DF) ? -op->count : op->count;
register_address_increment(ctxt, reg, df * op->bytes);
op->addr.mem.ea = register_address(ctxt, reg);
return X86EMUL_CONTINUE;
}
-static int em_vmcall(struct x86_emulate_ctxt *ctxt)
+static int em_hypercall(struct x86_emulate_ctxt *ctxt)
{
int rc = ctxt->ops->fix_hypercall(ctxt);
return em_lgdt_lidt(ctxt, true);
}
-static int em_vmmcall(struct x86_emulate_ctxt *ctxt)
-{
- int rc;
-
- rc = ctxt->ops->fix_hypercall(ctxt);
-
- /* Disable writeback. */
- ctxt->dst.type = OP_NONE;
- return rc;
-}
-
static int em_lidt(struct x86_emulate_ctxt *ctxt)
{
return em_lgdt_lidt(ctxt, false);
{
u32 flags;
- flags = EFLG_CF | EFLG_PF | EFLG_AF | EFLG_ZF | EFLG_SF;
+ flags = X86_EFLAGS_CF | X86_EFLAGS_PF | X86_EFLAGS_AF | X86_EFLAGS_ZF |
+ X86_EFLAGS_SF;
flags &= *reg_rmw(ctxt, VCPU_REGS_RAX) >> 8;
ctxt->eflags &= ~0xffUL;
static const struct opcode group7_rm0[] = {
N,
- I(SrcNone | Priv | EmulateOnUD, em_vmcall),
+ I(SrcNone | Priv | EmulateOnUD, em_hypercall),
N, N, N, N, N, N,
};
static const struct opcode group7_rm3[] = {
DIP(SrcNone | Prot | Priv, vmrun, check_svme_pa),
- II(SrcNone | Prot | EmulateOnUD, em_vmmcall, vmmcall),
+ II(SrcNone | Prot | EmulateOnUD, em_hypercall, vmmcall),
DIP(SrcNone | Prot | Priv, vmload, check_svme_pa),
DIP(SrcNone | Prot | Priv, vmsave, check_svme_pa),
DIP(SrcNone | Prot | Priv, stgi, check_svme),
N, N,
G(BitOp, group8),
F(DstMem | SrcReg | ModRM | BitOp | Lock | PageTable, em_btc),
- F(DstReg | SrcMem | ModRM, em_bsf), F(DstReg | SrcMem | ModRM, em_bsr),
+ I(DstReg | SrcMem | ModRM, em_bsf_c),
+ I(DstReg | SrcMem | ModRM, em_bsr_c),
D(DstReg | SrcMem8 | ModRM | Mov), D(DstReg | SrcMem16 | ModRM | Mov),
/* 0xC0 - 0xC7 */
F2bv(DstMem | SrcReg | ModRM | SrcWrite | Lock, em_xadd),
if (((ctxt->b == 0xa6) || (ctxt->b == 0xa7) ||
(ctxt->b == 0xae) || (ctxt->b == 0xaf))
&& (((ctxt->rep_prefix == REPE_PREFIX) &&
- ((ctxt->eflags & EFLG_ZF) == 0))
+ ((ctxt->eflags & X86_EFLAGS_ZF) == 0))
|| ((ctxt->rep_prefix == REPNE_PREFIX) &&
- ((ctxt->eflags & EFLG_ZF) == EFLG_ZF))))
+ ((ctxt->eflags & X86_EFLAGS_ZF) == X86_EFLAGS_ZF))))
return true;
return false;
/* All REP prefixes have the same first termination condition */
if (address_mask(ctxt, reg_read(ctxt, VCPU_REGS_RCX)) == 0) {
ctxt->eip = ctxt->_eip;
- ctxt->eflags &= ~EFLG_RF;
+ ctxt->eflags &= ~X86_EFLAGS_RF;
goto done;
}
}
goto done;
}
}
- ctxt->dst.orig_val = ctxt->dst.val;
+ /* Copy full 64-bit value for CMPXCHG8B. */
+ ctxt->dst.orig_val64 = ctxt->dst.val64;
special_insn:
}
if (ctxt->rep_prefix && (ctxt->d & String))
- ctxt->eflags |= EFLG_RF;
+ ctxt->eflags |= X86_EFLAGS_RF;
else
- ctxt->eflags &= ~EFLG_RF;
+ ctxt->eflags &= ~X86_EFLAGS_RF;
if (ctxt->execute) {
if (ctxt->d & Fastop) {
rc = emulate_int(ctxt, ctxt->src.val);
break;
case 0xce: /* into */
- if (ctxt->eflags & EFLG_OF)
+ if (ctxt->eflags & X86_EFLAGS_OF)
rc = emulate_int(ctxt, 4);
break;
case 0xe9: /* jmp rel */
break;
case 0xf5: /* cmc */
/* complement carry flag from eflags reg */
- ctxt->eflags ^= EFLG_CF;
+ ctxt->eflags ^= X86_EFLAGS_CF;
break;
case 0xf8: /* clc */
- ctxt->eflags &= ~EFLG_CF;
+ ctxt->eflags &= ~X86_EFLAGS_CF;
break;
case 0xf9: /* stc */
- ctxt->eflags |= EFLG_CF;
+ ctxt->eflags |= X86_EFLAGS_CF;
break;
case 0xfc: /* cld */
- ctxt->eflags &= ~EFLG_DF;
+ ctxt->eflags &= ~X86_EFLAGS_DF;
break;
case 0xfd: /* std */
- ctxt->eflags |= EFLG_DF;
+ ctxt->eflags |= X86_EFLAGS_DF;
break;
default:
goto cannot_emulate;
}
goto done; /* skip rip writeback */
}
- ctxt->eflags &= ~EFLG_RF;
+ ctxt->eflags &= ~X86_EFLAGS_RF;
}
ctxt->eip = ctxt->_eip;
case 0x40 ... 0x4f: /* cmov */
if (test_cc(ctxt->b, ctxt->eflags))
ctxt->dst.val = ctxt->src.val;
- else if (ctxt->mode != X86EMUL_MODE_PROT64 ||
- ctxt->op_bytes != 4)
+ else if (ctxt->op_bytes != 4)
ctxt->dst.type = OP_NONE; /* no writeback */
break;
case 0x80 ... 0x8f: /* jnz rel, etc*/
(addr < KVM_PIT_BASE_ADDRESS + KVM_PIT_MEM_LENGTH));
}
-static int pit_ioport_write(struct kvm_io_device *this,
+static int pit_ioport_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr, int len, const void *data)
{
struct kvm_pit *pit = dev_to_pit(this);
return 0;
}
-static int pit_ioport_read(struct kvm_io_device *this,
+static int pit_ioport_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr, int len, void *data)
{
struct kvm_pit *pit = dev_to_pit(this);
return 0;
}
-static int speaker_ioport_write(struct kvm_io_device *this,
+static int speaker_ioport_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr, int len, const void *data)
{
struct kvm_pit *pit = speaker_to_pit(this);
return 0;
}
-static int speaker_ioport_read(struct kvm_io_device *this,
- gpa_t addr, int len, void *data)
+static int speaker_ioport_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, void *data)
{
struct kvm_pit *pit = speaker_to_pit(this);
struct kvm_kpit_state *pit_state = &pit->pit_state;
#include <linux/kthread.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
struct kvm_kpit_channel_state {
u32 count; /* can be 65536 */
return -EOPNOTSUPP;
if (len != 1) {
+ memset(val, 0, len);
pr_pic_unimpl("non byte read\n");
return 0;
}
return 0;
}
-static int picdev_master_write(struct kvm_io_device *dev,
+static int picdev_master_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, const void *val)
{
return picdev_write(container_of(dev, struct kvm_pic, dev_master),
addr, len, val);
}
-static int picdev_master_read(struct kvm_io_device *dev,
+static int picdev_master_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
return picdev_read(container_of(dev, struct kvm_pic, dev_master),
addr, len, val);
}
-static int picdev_slave_write(struct kvm_io_device *dev,
+static int picdev_slave_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, const void *val)
{
return picdev_write(container_of(dev, struct kvm_pic, dev_slave),
addr, len, val);
}
-static int picdev_slave_read(struct kvm_io_device *dev,
+static int picdev_slave_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
return picdev_read(container_of(dev, struct kvm_pic, dev_slave),
addr, len, val);
}
-static int picdev_eclr_write(struct kvm_io_device *dev,
+static int picdev_eclr_write(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, const void *val)
{
return picdev_write(container_of(dev, struct kvm_pic, dev_eclr),
addr, len, val);
}
-static int picdev_eclr_read(struct kvm_io_device *dev,
+static int picdev_eclr_read(struct kvm_vcpu *vcpu, struct kvm_io_device *dev,
gpa_t addr, int len, void *val)
{
return picdev_read(container_of(dev, struct kvm_pic, dev_eclr),
old_irr = ioapic->irr;
ioapic->irr |= mask;
+ if (edge)
+ ioapic->irr_delivered &= ~mask;
if ((edge && old_irr == ioapic->irr) ||
(!edge && entry.fields.remote_irr)) {
ret = 0;
irqe.shorthand = 0;
if (irqe.trig_mode == IOAPIC_EDGE_TRIG)
- ioapic->irr &= ~(1 << irq);
+ ioapic->irr_delivered |= 1 << irq;
if (irq == RTC_GSI && line_status) {
/*
struct kvm_ioapic *ioapic, int vector, int trigger_mode)
{
int i;
+ struct kvm_lapic *apic = vcpu->arch.apic;
for (i = 0; i < IOAPIC_NUM_PINS; i++) {
union kvm_ioapic_redirect_entry *ent = &ioapic->redirtbl[i];
kvm_notify_acked_irq(ioapic->kvm, KVM_IRQCHIP_IOAPIC, i);
spin_lock(&ioapic->lock);
- if (trigger_mode != IOAPIC_LEVEL_TRIG)
+ if (trigger_mode != IOAPIC_LEVEL_TRIG ||
+ kvm_apic_get_reg(apic, APIC_SPIV) & APIC_SPIV_DIRECTED_EOI)
continue;
ASSERT(ent->fields.trig_mode == IOAPIC_LEVEL_TRIG);
}
}
-bool kvm_ioapic_handles_vector(struct kvm *kvm, int vector)
-{
- struct kvm_ioapic *ioapic = kvm->arch.vioapic;
- smp_rmb();
- return test_bit(vector, ioapic->handled_vectors);
-}
-
void kvm_ioapic_update_eoi(struct kvm_vcpu *vcpu, int vector, int trigger_mode)
{
struct kvm_ioapic *ioapic = vcpu->kvm->arch.vioapic;
(addr < ioapic->base_address + IOAPIC_MEM_LENGTH)));
}
-static int ioapic_mmio_read(struct kvm_io_device *this, gpa_t addr, int len,
- void *val)
+static int ioapic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, void *val)
{
struct kvm_ioapic *ioapic = to_ioapic(this);
u32 result;
return 0;
}
-static int ioapic_mmio_write(struct kvm_io_device *this, gpa_t addr, int len,
- const void *val)
+static int ioapic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
+ gpa_t addr, int len, const void *val)
{
struct kvm_ioapic *ioapic = to_ioapic(this);
u32 data;
ioapic->base_address = IOAPIC_DEFAULT_BASE_ADDRESS;
ioapic->ioregsel = 0;
ioapic->irr = 0;
+ ioapic->irr_delivered = 0;
ioapic->id = 0;
memset(ioapic->irq_eoi, 0x00, IOAPIC_NUM_PINS);
rtc_irq_eoi_tracking_reset(ioapic);
spin_lock(&ioapic->lock);
memcpy(state, ioapic, sizeof(struct kvm_ioapic_state));
+ state->irr &= ~ioapic->irr_delivered;
spin_unlock(&ioapic->lock);
return 0;
}
spin_lock(&ioapic->lock);
memcpy(ioapic, state, sizeof(struct kvm_ioapic_state));
ioapic->irr = 0;
+ ioapic->irr_delivered = 0;
update_handled_vectors(ioapic);
kvm_vcpu_request_scan_ioapic(kvm);
kvm_ioapic_inject_all(ioapic, state->irr);
#include <linux/kvm_host.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
struct kvm;
struct kvm_vcpu;
struct rtc_status rtc_status;
struct delayed_work eoi_inject;
u32 irq_eoi[IOAPIC_NUM_PINS];
+ u32 irr_delivered;
};
#ifdef DEBUG
return kvm->arch.vioapic;
}
+static inline bool kvm_ioapic_handles_vector(struct kvm *kvm, int vector)
+{
+ struct kvm_ioapic *ioapic = kvm->arch.vioapic;
+ smp_rmb();
+ return test_bit(vector, ioapic->handled_vectors);
+}
+
void kvm_rtc_eoi_tracking_restore_one(struct kvm_vcpu *vcpu);
bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
int short_hand, unsigned int dest, int dest_mode);
int kvm_apic_compare_prio(struct kvm_vcpu *vcpu1, struct kvm_vcpu *vcpu2);
void kvm_ioapic_update_eoi(struct kvm_vcpu *vcpu, int vector,
int trigger_mode);
-bool kvm_ioapic_handles_vector(struct kvm *kvm, int vector);
int kvm_ioapic_init(struct kvm *kvm);
void kvm_ioapic_destroy(struct kvm *kvm);
int kvm_ioapic_set_irq(struct kvm_ioapic *ioapic, int irq, int irq_source_id,
#include <linux/kvm_host.h>
#include <linux/spinlock.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
#include "ioapic.h"
#include "lapic.h"
return (kvm_apic_get_reg(apic, APIC_ID) >> 24) & 0xff;
}
+/* The logical map is definitely wrong if we have multiple
+ * modes at the same time. (Physical map is always right.)
+ */
+static inline bool kvm_apic_logical_map_valid(struct kvm_apic_map *map)
+{
+ return !(map->mode & (map->mode - 1));
+}
+
+static inline void
+apic_logical_id(struct kvm_apic_map *map, u32 dest_id, u16 *cid, u16 *lid)
+{
+ unsigned lid_bits;
+
+ BUILD_BUG_ON(KVM_APIC_MODE_XAPIC_CLUSTER != 4);
+ BUILD_BUG_ON(KVM_APIC_MODE_XAPIC_FLAT != 8);
+ BUILD_BUG_ON(KVM_APIC_MODE_X2APIC != 16);
+ lid_bits = map->mode;
+
+ *cid = dest_id >> lid_bits;
+ *lid = dest_id & ((1 << lid_bits) - 1);
+}
+
static void recalculate_apic_map(struct kvm *kvm)
{
struct kvm_apic_map *new, *old = NULL;
if (!new)
goto out;
- new->ldr_bits = 8;
- /* flat mode is default */
- new->cid_shift = 8;
- new->cid_mask = 0;
- new->lid_mask = 0xff;
- new->broadcast = APIC_BROADCAST;
-
- kvm_for_each_vcpu(i, vcpu, kvm) {
- struct kvm_lapic *apic = vcpu->arch.apic;
-
- if (!kvm_apic_present(vcpu))
- continue;
-
- if (apic_x2apic_mode(apic)) {
- new->ldr_bits = 32;
- new->cid_shift = 16;
- new->cid_mask = new->lid_mask = 0xffff;
- new->broadcast = X2APIC_BROADCAST;
- } else if (kvm_apic_get_reg(apic, APIC_LDR)) {
- if (kvm_apic_get_reg(apic, APIC_DFR) ==
- APIC_DFR_CLUSTER) {
- new->cid_shift = 4;
- new->cid_mask = 0xf;
- new->lid_mask = 0xf;
- } else {
- new->cid_shift = 8;
- new->cid_mask = 0;
- new->lid_mask = 0xff;
- }
- }
-
- /*
- * All APICs have to be configured in the same mode by an OS.
- * We take advatage of this while building logical id loockup
- * table. After reset APICs are in software disabled mode, so if
- * we find apic with different setting we assume this is the mode
- * OS wants all apics to be in; build lookup table accordingly.
- */
- if (kvm_apic_sw_enabled(apic))
- break;
- }
-
kvm_for_each_vcpu(i, vcpu, kvm) {
struct kvm_lapic *apic = vcpu->arch.apic;
u16 cid, lid;
aid = kvm_apic_id(apic);
ldr = kvm_apic_get_reg(apic, APIC_LDR);
- cid = apic_cluster_id(new, ldr);
- lid = apic_logical_id(new, ldr);
if (aid < ARRAY_SIZE(new->phys_map))
new->phys_map[aid] = apic;
+
+ if (apic_x2apic_mode(apic)) {
+ new->mode |= KVM_APIC_MODE_X2APIC;
+ } else if (ldr) {
+ ldr = GET_APIC_LOGICAL_ID(ldr);
+ if (kvm_apic_get_reg(apic, APIC_DFR) == APIC_DFR_FLAT)
+ new->mode |= KVM_APIC_MODE_XAPIC_FLAT;
+ else
+ new->mode |= KVM_APIC_MODE_XAPIC_CLUSTER;
+ }
+
+ if (!kvm_apic_logical_map_valid(new))
+ continue;
+
+ apic_logical_id(new, ldr, &cid, &lid);
+
if (lid && cid < ARRAY_SIZE(new->logical_map))
new->logical_map[cid][ffs(lid) - 1] = apic;
}
apic_update_ppr(apic);
}
-static bool kvm_apic_broadcast(struct kvm_lapic *apic, u32 dest)
+static bool kvm_apic_broadcast(struct kvm_lapic *apic, u32 mda)
{
- return dest == (apic_x2apic_mode(apic) ?
- X2APIC_BROADCAST : APIC_BROADCAST);
+ if (apic_x2apic_mode(apic))
+ return mda == X2APIC_BROADCAST;
+
+ return GET_APIC_DEST_FIELD(mda) == APIC_BROADCAST;
}
-static bool kvm_apic_match_physical_addr(struct kvm_lapic *apic, u32 dest)
+static bool kvm_apic_match_physical_addr(struct kvm_lapic *apic, u32 mda)
{
- return kvm_apic_id(apic) == dest || kvm_apic_broadcast(apic, dest);
+ if (kvm_apic_broadcast(apic, mda))
+ return true;
+
+ if (apic_x2apic_mode(apic))
+ return mda == kvm_apic_id(apic);
+
+ return mda == SET_APIC_DEST_FIELD(kvm_apic_id(apic));
}
static bool kvm_apic_match_logical_addr(struct kvm_lapic *apic, u32 mda)
&& (logical_id & mda & 0xffff) != 0;
logical_id = GET_APIC_LOGICAL_ID(logical_id);
+ mda = GET_APIC_DEST_FIELD(mda);
switch (kvm_apic_get_reg(apic, APIC_DFR)) {
case APIC_DFR_FLAT:
}
}
+/* KVM APIC implementation has two quirks
+ * - dest always begins at 0 while xAPIC MDA has offset 24,
+ * - IOxAPIC messages have to be delivered (directly) to x2APIC.
+ */
+static u32 kvm_apic_mda(unsigned int dest_id, struct kvm_lapic *source,
+ struct kvm_lapic *target)
+{
+ bool ipi = source != NULL;
+ bool x2apic_mda = apic_x2apic_mode(ipi ? source : target);
+
+ if (!ipi && dest_id == APIC_BROADCAST && x2apic_mda)
+ return X2APIC_BROADCAST;
+
+ return x2apic_mda ? dest_id : SET_APIC_DEST_FIELD(dest_id);
+}
+
bool kvm_apic_match_dest(struct kvm_vcpu *vcpu, struct kvm_lapic *source,
int short_hand, unsigned int dest, int dest_mode)
{
struct kvm_lapic *target = vcpu->arch.apic;
+ u32 mda = kvm_apic_mda(dest, source, target);
apic_debug("target %p, source %p, dest 0x%x, "
"dest_mode 0x%x, short_hand 0x%x\n",
switch (short_hand) {
case APIC_DEST_NOSHORT:
if (dest_mode == APIC_DEST_PHYSICAL)
- return kvm_apic_match_physical_addr(target, dest);
+ return kvm_apic_match_physical_addr(target, mda);
else
- return kvm_apic_match_logical_addr(target, dest);
+ return kvm_apic_match_logical_addr(target, mda);
case APIC_DEST_SELF:
return target == source;
case APIC_DEST_ALLINC:
struct kvm_lapic **dst;
int i;
bool ret = false;
+ bool x2apic_ipi = src && apic_x2apic_mode(src);
*r = -1;
if (irq->shorthand)
return false;
+ if (irq->dest_id == (x2apic_ipi ? X2APIC_BROADCAST : APIC_BROADCAST))
+ return false;
+
rcu_read_lock();
map = rcu_dereference(kvm->arch.apic_map);
if (!map)
goto out;
- if (irq->dest_id == map->broadcast)
- goto out;
-
ret = true;
if (irq->dest_mode == APIC_DEST_PHYSICAL) {
dst = &map->phys_map[irq->dest_id];
} else {
- u32 mda = irq->dest_id << (32 - map->ldr_bits);
- u16 cid = apic_cluster_id(map, mda);
+ u16 cid;
+
+ if (!kvm_apic_logical_map_valid(map)) {
+ ret = false;
+ goto out;
+ }
+
+ apic_logical_id(map, irq->dest_id, &cid, (u16 *)&bitmap);
if (cid >= ARRAY_SIZE(map->logical_map))
goto out;
dst = map->logical_map[cid];
- bitmap = apic_logical_id(map, mda);
-
if (irq->delivery_mode == APIC_DM_LOWEST) {
int l = -1;
for_each_set_bit(i, &bitmap, 16) {
static void kvm_ioapic_send_eoi(struct kvm_lapic *apic, int vector)
{
- if (!(kvm_apic_get_reg(apic, APIC_SPIV) & APIC_SPIV_DIRECTED_EOI) &&
- kvm_ioapic_handles_vector(apic->vcpu->kvm, vector)) {
+ if (kvm_ioapic_handles_vector(apic->vcpu->kvm, vector)) {
int trigger_mode;
if (apic_test_vector(vector, apic->regs + APIC_TMR))
trigger_mode = IOAPIC_LEVEL_TRIG;
addr < apic->base_address + LAPIC_MMIO_LENGTH;
}
-static int apic_mmio_read(struct kvm_io_device *this,
+static int apic_mmio_read(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
gpa_t address, int len, void *data)
{
struct kvm_lapic *apic = to_lapic(this);
return ret;
}
-static int apic_mmio_write(struct kvm_io_device *this,
+static int apic_mmio_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this,
gpa_t address, int len, const void *data)
{
struct kvm_lapic *apic = to_lapic(this);
return;
}
- if (!kvm_vcpu_is_bsp(apic->vcpu))
- value &= ~MSR_IA32_APICBASE_BSP;
vcpu->arch.apic_base = value;
/* update jump label if enable bit changes */
apic_set_reg(apic, APIC_TMR + 0x10 * i, 0);
}
apic->irr_pending = kvm_apic_vid_enabled(vcpu->kvm);
- apic->isr_count = kvm_apic_vid_enabled(vcpu->kvm);
+ apic->isr_count = kvm_x86_ops->hwapic_isr_update ? 1 : 0;
apic->highest_isr_cache = -1;
update_divide_count(apic);
atomic_set(&apic->lapic_timer.pending, 0);
update_divide_count(apic);
start_apic_timer(apic);
apic->irr_pending = true;
- apic->isr_count = kvm_apic_vid_enabled(vcpu->kvm) ?
+ apic->isr_count = kvm_x86_ops->hwapic_isr_update ?
1 : count_vectors(apic->regs + APIC_ISR);
apic->highest_isr_cache = -1;
if (kvm_x86_ops->hwapic_irr_update)
#ifndef __KVM_X86_LAPIC_H
#define __KVM_X86_LAPIC_H
-#include "iodev.h"
+#include <kvm/iodev.h>
#include <linux/kvm_host.h>
return kvm_x86_ops->vm_has_apicv(kvm);
}
-static inline u16 apic_cluster_id(struct kvm_apic_map *map, u32 ldr)
-{
- u16 cid;
- ldr >>= 32 - map->ldr_bits;
- cid = (ldr >> map->cid_shift) & map->cid_mask;
-
- return cid;
-}
-
-static inline u16 apic_logical_id(struct kvm_apic_map *map, u32 ldr)
-{
- ldr >>= (32 - map->ldr_bits);
- return ldr & map->lid_mask;
-}
-
static inline bool kvm_apic_has_events(struct kvm_vcpu *vcpu)
{
return vcpu->arch.apic->pending_events;
kvm_flush_remote_tlbs(kvm);
}
+static bool kvm_mmu_zap_collapsible_spte(struct kvm *kvm,
+ unsigned long *rmapp)
+{
+ u64 *sptep;
+ struct rmap_iterator iter;
+ int need_tlb_flush = 0;
+ pfn_t pfn;
+ struct kvm_mmu_page *sp;
+
+ for (sptep = rmap_get_first(*rmapp, &iter); sptep;) {
+ BUG_ON(!(*sptep & PT_PRESENT_MASK));
+
+ sp = page_header(__pa(sptep));
+ pfn = spte_to_pfn(*sptep);
+
+ /*
+ * Only EPT supported for now; otherwise, one would need to
+ * find out efficiently whether the guest page tables are
+ * also using huge pages.
+ */
+ if (sp->role.direct &&
+ !kvm_is_reserved_pfn(pfn) &&
+ PageTransCompound(pfn_to_page(pfn))) {
+ drop_spte(kvm, sptep);
+ sptep = rmap_get_first(*rmapp, &iter);
+ need_tlb_flush = 1;
+ } else
+ sptep = rmap_get_next(&iter);
+ }
+
+ return need_tlb_flush;
+}
+
+void kvm_mmu_zap_collapsible_sptes(struct kvm *kvm,
+ struct kvm_memory_slot *memslot)
+{
+ bool flush = false;
+ unsigned long *rmapp;
+ unsigned long last_index, index;
+ gfn_t gfn_start, gfn_end;
+
+ spin_lock(&kvm->mmu_lock);
+
+ gfn_start = memslot->base_gfn;
+ gfn_end = memslot->base_gfn + memslot->npages - 1;
+
+ if (gfn_start >= gfn_end)
+ goto out;
+
+ rmapp = memslot->arch.rmap[0];
+ last_index = gfn_to_index(gfn_end, memslot->base_gfn,
+ PT_PAGE_TABLE_LEVEL);
+
+ for (index = 0; index <= last_index; ++index, ++rmapp) {
+ if (*rmapp)
+ flush |= kvm_mmu_zap_collapsible_spte(kvm, rmapp);
+
+ if (need_resched() || spin_needbreak(&kvm->mmu_lock)) {
+ if (flush) {
+ kvm_flush_remote_tlbs(kvm);
+ flush = false;
+ }
+ cond_resched_lock(&kvm->mmu_lock);
+ }
+ }
+
+ if (flush)
+ kvm_flush_remote_tlbs(kvm);
+
+out:
+ spin_unlock(&kvm->mmu_lock);
+}
+
void kvm_mmu_slot_leaf_clear_dirty(struct kvm *kvm,
struct kvm_memory_slot *memslot)
{
};
/* mapping between fixed pmc index and arch_events array */
-int fixed_pmc_events[] = {1, 0, 7};
+static int fixed_pmc_events[] = {1, 0, 7};
static bool pmc_is_gp(struct kvm_pmc *pmc)
{
svm->vcpu.arch.apic_base = APIC_DEFAULT_PHYS_BASE |
MSR_IA32_APICBASE_ENABLE;
- if (kvm_vcpu_is_bsp(&svm->vcpu))
+ if (kvm_vcpu_is_reset_bsp(&svm->vcpu))
svm->vcpu.arch.apic_base |= MSR_IA32_APICBASE_BSP;
svm_init_osvw(&svm->vcpu);
static int halt_interception(struct vcpu_svm *svm)
{
svm->next_rip = kvm_rip_read(&svm->vcpu) + 1;
- skip_emulated_instruction(&svm->vcpu);
return kvm_emulate_halt(&svm->vcpu);
}
static int vmmcall_interception(struct vcpu_svm *svm)
{
svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
- skip_emulated_instruction(&svm->vcpu);
kvm_emulate_hypercall(&svm->vcpu);
return 1;
}
{
struct kvm_vcpu *vcpu = &svm->vcpu;
- trace_kvm_invlpga(svm->vmcb->save.rip, vcpu->arch.regs[VCPU_REGS_RCX],
- vcpu->arch.regs[VCPU_REGS_RAX]);
+ trace_kvm_invlpga(svm->vmcb->save.rip, kvm_register_read(&svm->vcpu, VCPU_REGS_RCX),
+ kvm_register_read(&svm->vcpu, VCPU_REGS_RAX));
/* Let's treat INVLPGA the same as INVLPG (can be optimized!) */
- kvm_mmu_invlpg(vcpu, vcpu->arch.regs[VCPU_REGS_RAX]);
+ kvm_mmu_invlpg(vcpu, kvm_register_read(&svm->vcpu, VCPU_REGS_RAX));
svm->next_rip = kvm_rip_read(&svm->vcpu) + 3;
skip_emulated_instruction(&svm->vcpu);
static int skinit_interception(struct vcpu_svm *svm)
{
- trace_kvm_skinit(svm->vmcb->save.rip, svm->vcpu.arch.regs[VCPU_REGS_RAX]);
+ trace_kvm_skinit(svm->vmcb->save.rip, kvm_register_read(&svm->vcpu, VCPU_REGS_RAX));
kvm_queue_exception(&svm->vcpu, UD_VECTOR);
return 1;
}
+static int wbinvd_interception(struct vcpu_svm *svm)
+{
+ kvm_emulate_wbinvd(&svm->vcpu);
+ return 1;
+}
+
static int xsetbv_interception(struct vcpu_svm *svm)
{
u64 new_bv = kvm_read_edx_eax(&svm->vcpu);
return 1;
}
-bool check_selective_cr0_intercepted(struct vcpu_svm *svm, unsigned long val)
+static bool check_selective_cr0_intercepted(struct vcpu_svm *svm,
+ unsigned long val)
{
unsigned long cr0 = svm->vcpu.arch.cr0;
bool ret = false;
return emulate_on_interception(svm);
reg = svm->vmcb->control.exit_info_1 & SVM_EXITINFO_REG_MASK;
- cr = svm->vmcb->control.exit_code - SVM_EXIT_READ_CR0;
+ if (svm->vmcb->control.exit_code == SVM_EXIT_CR0_SEL_WRITE)
+ cr = SVM_EXIT_WRITE_CR0 - SVM_EXIT_READ_CR0;
+ else
+ cr = svm->vmcb->control.exit_code - SVM_EXIT_READ_CR0;
err = 0;
if (cr >= 16) { /* mov to cr */
static int rdmsr_interception(struct vcpu_svm *svm)
{
- u32 ecx = svm->vcpu.arch.regs[VCPU_REGS_RCX];
+ u32 ecx = kvm_register_read(&svm->vcpu, VCPU_REGS_RCX);
u64 data;
if (svm_get_msr(&svm->vcpu, ecx, &data)) {
} else {
trace_kvm_msr_read(ecx, data);
- svm->vcpu.arch.regs[VCPU_REGS_RAX] = data & 0xffffffff;
- svm->vcpu.arch.regs[VCPU_REGS_RDX] = data >> 32;
+ kvm_register_write(&svm->vcpu, VCPU_REGS_RAX, data & 0xffffffff);
+ kvm_register_write(&svm->vcpu, VCPU_REGS_RDX, data >> 32);
svm->next_rip = kvm_rip_read(&svm->vcpu) + 2;
skip_emulated_instruction(&svm->vcpu);
}
static int wrmsr_interception(struct vcpu_svm *svm)
{
struct msr_data msr;
- u32 ecx = svm->vcpu.arch.regs[VCPU_REGS_RCX];
- u64 data = (svm->vcpu.arch.regs[VCPU_REGS_RAX] & -1u)
- | ((u64)(svm->vcpu.arch.regs[VCPU_REGS_RDX] & -1u) << 32);
+ u32 ecx = kvm_register_read(&svm->vcpu, VCPU_REGS_RCX);
+ u64 data = kvm_read_edx_eax(&svm->vcpu);
msr.data = data;
msr.index = ecx;
[SVM_EXIT_READ_CR3] = cr_interception,
[SVM_EXIT_READ_CR4] = cr_interception,
[SVM_EXIT_READ_CR8] = cr_interception,
- [SVM_EXIT_CR0_SEL_WRITE] = emulate_on_interception,
+ [SVM_EXIT_CR0_SEL_WRITE] = cr_interception,
[SVM_EXIT_WRITE_CR0] = cr_interception,
[SVM_EXIT_WRITE_CR3] = cr_interception,
[SVM_EXIT_WRITE_CR4] = cr_interception,
[SVM_EXIT_STGI] = stgi_interception,
[SVM_EXIT_CLGI] = clgi_interception,
[SVM_EXIT_SKINIT] = skinit_interception,
- [SVM_EXIT_WBINVD] = emulate_on_interception,
+ [SVM_EXIT_WBINVD] = wbinvd_interception,
[SVM_EXIT_MONITOR] = monitor_interception,
[SVM_EXIT_MWAIT] = mwait_interception,
[SVM_EXIT_XSETBV] = xsetbv_interception,
if (exit_code >= ARRAY_SIZE(svm_exit_handlers)
|| !svm_exit_handlers[exit_code]) {
- WARN_ONCE(1, "vmx: unexpected exit reason 0x%x\n", exit_code);
+ WARN_ONCE(1, "svm: unexpected exit reason 0x%x\n", exit_code);
kvm_queue_exception(vcpu, UD_VECTOR);
return 1;
}
return;
}
-static void svm_hwapic_isr_update(struct kvm *kvm, int isr)
-{
- return;
-}
-
static void svm_sync_pir_to_irr(struct kvm_vcpu *vcpu)
{
return;
.set_virtual_x2apic_mode = svm_set_virtual_x2apic_mode,
.vm_has_apicv = svm_vm_has_apicv,
.load_eoi_exitmap = svm_load_eoi_exitmap,
- .hwapic_isr_update = svm_hwapic_isr_update,
.sync_pir_to_irr = svm_sync_pir_to_irr,
.set_tss_addr = svm_set_tss_addr,
{
unsigned long *msr_bitmap;
- if (irqchip_in_kernel(vcpu->kvm) && apic_x2apic_mode(vcpu->arch.apic)) {
+ if (is_guest_mode(vcpu))
+ msr_bitmap = vmx_msr_bitmap_nested;
+ else if (irqchip_in_kernel(vcpu->kvm) &&
+ apic_x2apic_mode(vcpu->arch.apic)) {
if (is_long_mode(vcpu))
msr_bitmap = vmx_msr_bitmap_longmode_x2apic;
else
vmx->nested.nested_vmx_secondary_ctls_low = 0;
vmx->nested.nested_vmx_secondary_ctls_high &=
SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
+ SECONDARY_EXEC_RDTSCP |
SECONDARY_EXEC_VIRTUALIZE_X2APIC_MODE |
SECONDARY_EXEC_APIC_REGISTER_VIRT |
SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
if (enable_ept) {
/* nested EPT: emulate EPT also to L1 */
vmx->nested.nested_vmx_secondary_ctls_high |=
- SECONDARY_EXEC_ENABLE_EPT |
- SECONDARY_EXEC_UNRESTRICTED_GUEST;
+ SECONDARY_EXEC_ENABLE_EPT;
vmx->nested.nested_vmx_ept_caps = VMX_EPT_PAGE_WALK_4_BIT |
VMX_EPTP_WB_BIT | VMX_EPT_2MB_PAGE_BIT |
VMX_EPT_INVEPT_BIT;
} else
vmx->nested.nested_vmx_ept_caps = 0;
+ if (enable_unrestricted_guest)
+ vmx->nested.nested_vmx_secondary_ctls_high |=
+ SECONDARY_EXEC_UNRESTRICTED_GUEST;
+
/* miscellaneous data */
rdmsr(MSR_IA32_VMX_MISC,
vmx->nested.nested_vmx_misc_low,
* default value.
*/
if (seg == VCPU_SREG_CS || seg == VCPU_SREG_SS)
- save->selector &= ~SELECTOR_RPL_MASK;
- save->dpl = save->selector & SELECTOR_RPL_MASK;
+ save->selector &= ~SEGMENT_RPL_MASK;
+ save->dpl = save->selector & SEGMENT_RPL_MASK;
save->s = 1;
}
vmx_set_segment(vcpu, save, seg);
unsigned int cs_rpl;
vmx_get_segment(vcpu, &cs, VCPU_SREG_CS);
- cs_rpl = cs.selector & SELECTOR_RPL_MASK;
+ cs_rpl = cs.selector & SEGMENT_RPL_MASK;
if (cs.unusable)
return false;
unsigned int ss_rpl;
vmx_get_segment(vcpu, &ss, VCPU_SREG_SS);
- ss_rpl = ss.selector & SELECTOR_RPL_MASK;
+ ss_rpl = ss.selector & SEGMENT_RPL_MASK;
if (ss.unusable)
return true;
unsigned int rpl;
vmx_get_segment(vcpu, &var, seg);
- rpl = var.selector & SELECTOR_RPL_MASK;
+ rpl = var.selector & SEGMENT_RPL_MASK;
if (var.unusable)
return true;
if (tr.unusable)
return false;
- if (tr.selector & SELECTOR_TI_MASK) /* TI = 1 */
+ if (tr.selector & SEGMENT_TI_MASK) /* TI = 1 */
return false;
if (tr.type != 3 && tr.type != 11) /* TODO: Check if guest is in IA32e mode */
return false;
if (ldtr.unusable)
return true;
- if (ldtr.selector & SELECTOR_TI_MASK) /* TI = 1 */
+ if (ldtr.selector & SEGMENT_TI_MASK) /* TI = 1 */
return false;
if (ldtr.type != 2)
return false;
vmx_get_segment(vcpu, &cs, VCPU_SREG_CS);
vmx_get_segment(vcpu, &ss, VCPU_SREG_SS);
- return ((cs.selector & SELECTOR_RPL_MASK) ==
- (ss.selector & SELECTOR_RPL_MASK));
+ return ((cs.selector & SEGMENT_RPL_MASK) ==
+ (ss.selector & SEGMENT_RPL_MASK));
}
/*
return 0;
}
+static inline bool kvm_vcpu_trigger_posted_interrupt(struct kvm_vcpu *vcpu)
+{
+#ifdef CONFIG_SMP
+ if (vcpu->mode == IN_GUEST_MODE) {
+ apic->send_IPI_mask(get_cpu_mask(vcpu->cpu),
+ POSTED_INTR_VECTOR);
+ return true;
+ }
+#endif
+ return false;
+}
+
static int vmx_deliver_nested_posted_interrupt(struct kvm_vcpu *vcpu,
int vector)
{
if (is_guest_mode(vcpu) &&
vector == vmx->nested.posted_intr_nv) {
/* the PIR and ON have been set by L1. */
- if (vcpu->mode == IN_GUEST_MODE)
- apic->send_IPI_mask(get_cpu_mask(vcpu->cpu),
- POSTED_INTR_VECTOR);
+ kvm_vcpu_trigger_posted_interrupt(vcpu);
/*
* If a posted intr is not recognized by hardware,
* we will accomplish it in the next vmentry.
r = pi_test_and_set_on(&vmx->pi_desc);
kvm_make_request(KVM_REQ_EVENT, vcpu);
-#ifdef CONFIG_SMP
- if (!r && (vcpu->mode == IN_GUEST_MODE))
- apic->send_IPI_mask(get_cpu_mask(vcpu->cpu),
- POSTED_INTR_VECTOR);
- else
-#endif
+ if (r || !kvm_vcpu_trigger_posted_interrupt(vcpu))
kvm_vcpu_kick(vcpu);
}
vmx->vcpu.arch.regs[VCPU_REGS_RDX] = get_rdx_init_val();
kvm_set_cr8(&vmx->vcpu, 0);
apic_base_msr.data = APIC_DEFAULT_PHYS_BASE | MSR_IA32_APICBASE_ENABLE;
- if (kvm_vcpu_is_bsp(&vmx->vcpu))
+ if (kvm_vcpu_is_reset_bsp(&vmx->vcpu))
apic_base_msr.data |= MSR_IA32_APICBASE_BSP;
apic_base_msr.host_initiated = true;
kvm_set_apic_base(&vmx->vcpu, &apic_base_msr);
if (emulate_instruction(vcpu, 0) == EMULATE_DONE) {
if (vcpu->arch.halt_request) {
vcpu->arch.halt_request = 0;
- return kvm_emulate_halt(vcpu);
+ return kvm_vcpu_halt(vcpu);
}
return 1;
}
}
if (is_invalid_opcode(intr_info)) {
+ if (is_guest_mode(vcpu)) {
+ kvm_queue_exception(vcpu, UD_VECTOR);
+ return 1;
+ }
er = emulate_instruction(vcpu, EMULTYPE_TRAP_UD);
if (er != EMULATE_DONE)
kvm_queue_exception(vcpu, UD_VECTOR);
!(is_page_fault(intr_info) && !(error_code & PFERR_RSVD_MASK))) {
vcpu->run->exit_reason = KVM_EXIT_INTERNAL_ERROR;
vcpu->run->internal.suberror = KVM_INTERNAL_ERROR_SIMUL_EX;
- vcpu->run->internal.ndata = 2;
+ vcpu->run->internal.ndata = 3;
vcpu->run->internal.data[0] = vect_info;
vcpu->run->internal.data[1] = intr_info;
+ vcpu->run->internal.data[2] = error_code;
return 0;
}
static int handle_halt(struct kvm_vcpu *vcpu)
{
- skip_emulated_instruction(vcpu);
return kvm_emulate_halt(vcpu);
}
static int handle_vmcall(struct kvm_vcpu *vcpu)
{
- skip_emulated_instruction(vcpu);
kvm_emulate_hypercall(vcpu);
return 1;
}
static int handle_wbinvd(struct kvm_vcpu *vcpu)
{
- skip_emulated_instruction(vcpu);
kvm_emulate_wbinvd(vcpu);
return 1;
}
gpa_t gpa;
gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS);
- if (!kvm_io_bus_write(vcpu->kvm, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
+ if (!kvm_io_bus_write(vcpu, KVM_FAST_MMIO_BUS, gpa, 0, NULL)) {
skip_emulated_instruction(vcpu);
return 1;
}
if (vcpu->arch.halt_request) {
vcpu->arch.halt_request = 0;
- ret = kvm_emulate_halt(vcpu);
+ ret = kvm_vcpu_halt(vcpu);
goto out;
}
else if (port < 0x10000)
bitmap = vmcs12->io_bitmap_b;
else
- return 1;
+ return true;
bitmap += (port & 0x7fff) / 8;
if (last_bitmap != bitmap)
if (kvm_read_guest(vcpu->kvm, bitmap, &b, 1))
- return 1;
+ return true;
if (b & (1 << (port & 7)))
- return 1;
+ return true;
port++;
size--;
last_bitmap = bitmap;
}
- return 0;
+ return false;
}
/*
gpa_t bitmap;
if (!nested_cpu_has(vmcs12, CPU_BASED_USE_MSR_BITMAPS))
- return 1;
+ return true;
/*
* The MSR_BITMAP page is divided into four 1024-byte bitmaps,
if (msr_index < 1024*8) {
unsigned char b;
if (kvm_read_guest(vcpu->kvm, bitmap + msr_index/8, &b, 1))
- return 1;
+ return true;
return 1 & (b >> (msr_index & 7));
} else
- return 1; /* let L1 handle the wrong parameter */
+ return true; /* let L1 handle the wrong parameter */
}
/*
case 0:
if (vmcs12->cr0_guest_host_mask &
(val ^ vmcs12->cr0_read_shadow))
- return 1;
+ return true;
break;
case 3:
if ((vmcs12->cr3_target_count >= 1 &&
vmcs12->cr3_target_value2 == val) ||
(vmcs12->cr3_target_count >= 4 &&
vmcs12->cr3_target_value3 == val))
- return 0;
+ return false;
if (nested_cpu_has(vmcs12, CPU_BASED_CR3_LOAD_EXITING))
- return 1;
+ return true;
break;
case 4:
if (vmcs12->cr4_guest_host_mask &
(vmcs12->cr4_read_shadow ^ val))
- return 1;
+ return true;
break;
case 8:
if (nested_cpu_has(vmcs12, CPU_BASED_CR8_LOAD_EXITING))
- return 1;
+ return true;
break;
}
break;
case 2: /* clts */
if ((vmcs12->cr0_guest_host_mask & X86_CR0_TS) &&
(vmcs12->cr0_read_shadow & X86_CR0_TS))
- return 1;
+ return true;
break;
case 1: /* mov from cr */
switch (cr) {
case 3:
if (vmcs12->cpu_based_vm_exec_control &
CPU_BASED_CR3_STORE_EXITING)
- return 1;
+ return true;
break;
case 8:
if (vmcs12->cpu_based_vm_exec_control &
CPU_BASED_CR8_STORE_EXITING)
- return 1;
+ return true;
break;
}
break;
*/
if (vmcs12->cr0_guest_host_mask & 0xe &
(val ^ vmcs12->cr0_read_shadow))
- return 1;
+ return true;
if ((vmcs12->cr0_guest_host_mask & 0x1) &&
!(vmcs12->cr0_read_shadow & 0x1) &&
(val & 0x1))
- return 1;
+ return true;
break;
}
- return 0;
+ return false;
}
/*
KVM_ISA_VMX);
if (vmx->nested.nested_run_pending)
- return 0;
+ return false;
if (unlikely(vmx->fail)) {
pr_info_ratelimited("%s failed vm entry %x\n", __func__,
vmcs_read32(VM_INSTRUCTION_ERROR));
- return 1;
+ return true;
}
switch (exit_reason) {
case EXIT_REASON_EXCEPTION_NMI:
if (!is_exception(intr_info))
- return 0;
+ return false;
else if (is_page_fault(intr_info))
return enable_ept;
else if (is_no_device(intr_info) &&
!(vmcs12->guest_cr0 & X86_CR0_TS))
- return 0;
+ return false;
return vmcs12->exception_bitmap &
(1u << (intr_info & INTR_INFO_VECTOR_MASK));
case EXIT_REASON_EXTERNAL_INTERRUPT:
- return 0;
+ return false;
case EXIT_REASON_TRIPLE_FAULT:
- return 1;
+ return true;
case EXIT_REASON_PENDING_INTERRUPT:
return nested_cpu_has(vmcs12, CPU_BASED_VIRTUAL_INTR_PENDING);
case EXIT_REASON_NMI_WINDOW:
return nested_cpu_has(vmcs12, CPU_BASED_VIRTUAL_NMI_PENDING);
case EXIT_REASON_TASK_SWITCH:
- return 1;
+ return true;
case EXIT_REASON_CPUID:
if (kvm_register_read(vcpu, VCPU_REGS_RAX) == 0xa)
- return 0;
- return 1;
+ return false;
+ return true;
case EXIT_REASON_HLT:
return nested_cpu_has(vmcs12, CPU_BASED_HLT_EXITING);
case EXIT_REASON_INVD:
- return 1;
+ return true;
case EXIT_REASON_INVLPG:
return nested_cpu_has(vmcs12, CPU_BASED_INVLPG_EXITING);
case EXIT_REASON_RDPMC:
return nested_cpu_has(vmcs12, CPU_BASED_RDPMC_EXITING);
- case EXIT_REASON_RDTSC:
+ case EXIT_REASON_RDTSC: case EXIT_REASON_RDTSCP:
return nested_cpu_has(vmcs12, CPU_BASED_RDTSC_EXITING);
case EXIT_REASON_VMCALL: case EXIT_REASON_VMCLEAR:
case EXIT_REASON_VMLAUNCH: case EXIT_REASON_VMPTRLD:
* VMX instructions trap unconditionally. This allows L1 to
* emulate them for its L2 guest, i.e., allows 3-level nesting!
*/
- return 1;
+ return true;
case EXIT_REASON_CR_ACCESS:
return nested_vmx_exit_handled_cr(vcpu, vmcs12);
case EXIT_REASON_DR_ACCESS:
case EXIT_REASON_MSR_WRITE:
return nested_vmx_exit_handled_msr(vcpu, vmcs12, exit_reason);
case EXIT_REASON_INVALID_STATE:
- return 1;
+ return true;
case EXIT_REASON_MWAIT_INSTRUCTION:
return nested_cpu_has(vmcs12, CPU_BASED_MWAIT_EXITING);
case EXIT_REASON_MONITOR_INSTRUCTION:
nested_cpu_has2(vmcs12,
SECONDARY_EXEC_PAUSE_LOOP_EXITING);
case EXIT_REASON_MCE_DURING_VMENTRY:
- return 0;
+ return false;
case EXIT_REASON_TPR_BELOW_THRESHOLD:
return nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW);
case EXIT_REASON_APIC_ACCESS:
case EXIT_REASON_APIC_WRITE:
case EXIT_REASON_EOI_INDUCED:
/* apic_write and eoi_induced should exit unconditionally. */
- return 1;
+ return true;
case EXIT_REASON_EPT_VIOLATION:
/*
* L0 always deals with the EPT violation. If nested EPT is
* missing in the guest EPT table (EPT12), the EPT violation
* will be injected with nested_ept_inject_page_fault()
*/
- return 0;
+ return false;
case EXIT_REASON_EPT_MISCONFIG:
/*
* L2 never uses directly L1's EPT, but rather L0's own EPT
* (EPT on EPT). So any problems with the structure of the
* table is L0's fault.
*/
- return 0;
+ return false;
case EXIT_REASON_WBINVD:
return nested_cpu_has2(vmcs12, SECONDARY_EXEC_WBINVD_EXITING);
case EXIT_REASON_XSETBV:
- return 1;
+ return true;
case EXIT_REASON_XSAVES: case EXIT_REASON_XRSTORS:
/*
* This should never happen, since it is not possible to
*/
return nested_cpu_has2(vmcs12, SECONDARY_EXEC_XSAVES);
default:
- return 1;
+ return true;
}
}
exec_control);
}
}
+ if (nested && !vmx->rdtscp_enabled)
+ vmx->nested.nested_vmx_secondary_ctls_high &=
+ ~SECONDARY_EXEC_RDTSCP;
}
/* Exposing INVPCID only when PCID is exposed */
struct vmcs12 *vmcs12)
{
struct vcpu_vmx *vmx = to_vmx(vcpu);
+ int maxphyaddr = cpuid_maxphyaddr(vcpu);
if (nested_cpu_has2(vmcs12, SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES)) {
- /* TODO: Also verify bits beyond physical address width are 0 */
- if (!PAGE_ALIGNED(vmcs12->apic_access_addr))
+ if (!PAGE_ALIGNED(vmcs12->apic_access_addr) ||
+ vmcs12->apic_access_addr >> maxphyaddr)
return false;
/*
}
if (nested_cpu_has(vmcs12, CPU_BASED_TPR_SHADOW)) {
- /* TODO: Also verify bits beyond physical address width are 0 */
- if (!PAGE_ALIGNED(vmcs12->virtual_apic_page_addr))
+ if (!PAGE_ALIGNED(vmcs12->virtual_apic_page_addr) ||
+ vmcs12->virtual_apic_page_addr >> maxphyaddr)
return false;
if (vmx->nested.virtual_apic_page) /* shouldn't happen */
}
if (nested_cpu_has_posted_intr(vmcs12)) {
- if (!IS_ALIGNED(vmcs12->posted_intr_desc_addr, 64))
+ if (!IS_ALIGNED(vmcs12->posted_intr_desc_addr, 64) ||
+ vmcs12->posted_intr_desc_addr >> maxphyaddr)
return false;
if (vmx->nested.pi_desc_page) { /* shouldn't happen */
static int nested_vmx_check_msr_switch(struct kvm_vcpu *vcpu,
unsigned long count_field,
- unsigned long addr_field,
- int maxphyaddr)
+ unsigned long addr_field)
{
+ int maxphyaddr;
u64 count, addr;
if (vmcs12_read_any(vcpu, count_field, &count) ||
}
if (count == 0)
return 0;
+ maxphyaddr = cpuid_maxphyaddr(vcpu);
if (!IS_ALIGNED(addr, 16) || addr >> maxphyaddr ||
(addr + count * sizeof(struct vmx_msr_entry) - 1) >> maxphyaddr) {
pr_warn_ratelimited(
static int nested_vmx_check_msr_switch_controls(struct kvm_vcpu *vcpu,
struct vmcs12 *vmcs12)
{
- int maxphyaddr;
-
if (vmcs12->vm_exit_msr_load_count == 0 &&
vmcs12->vm_exit_msr_store_count == 0 &&
vmcs12->vm_entry_msr_load_count == 0)
return 0; /* Fast path */
- maxphyaddr = cpuid_maxphyaddr(vcpu);
if (nested_vmx_check_msr_switch(vcpu, VM_EXIT_MSR_LOAD_COUNT,
- VM_EXIT_MSR_LOAD_ADDR, maxphyaddr) ||
+ VM_EXIT_MSR_LOAD_ADDR) ||
nested_vmx_check_msr_switch(vcpu, VM_EXIT_MSR_STORE_COUNT,
- VM_EXIT_MSR_STORE_ADDR, maxphyaddr) ||
+ VM_EXIT_MSR_STORE_ADDR) ||
nested_vmx_check_msr_switch(vcpu, VM_ENTRY_MSR_LOAD_COUNT,
- VM_ENTRY_MSR_LOAD_ADDR, maxphyaddr))
+ VM_ENTRY_MSR_LOAD_ADDR))
return -EINVAL;
return 0;
}
exec_control &= ~SECONDARY_EXEC_RDTSCP;
/* Take the following fields only from vmcs12 */
exec_control &= ~(SECONDARY_EXEC_VIRTUALIZE_APIC_ACCESSES |
+ SECONDARY_EXEC_RDTSCP |
SECONDARY_EXEC_VIRTUAL_INTR_DELIVERY |
- SECONDARY_EXEC_APIC_REGISTER_VIRT);
+ SECONDARY_EXEC_APIC_REGISTER_VIRT);
if (nested_cpu_has(vmcs12,
CPU_BASED_ACTIVATE_SECONDARY_CONTROLS))
exec_control |= vmcs12->secondary_vm_exec_control;
}
if (cpu_has_vmx_msr_bitmap() &&
- exec_control & CPU_BASED_USE_MSR_BITMAPS &&
- nested_vmx_merge_msr_bitmap(vcpu, vmcs12)) {
- vmcs_write64(MSR_BITMAP, __pa(vmx_msr_bitmap_nested));
+ exec_control & CPU_BASED_USE_MSR_BITMAPS) {
+ nested_vmx_merge_msr_bitmap(vcpu, vmcs12);
+ /* MSR_BITMAP will be set by following vmx_set_efer. */
} else
exec_control &= ~CPU_BASED_USE_MSR_BITMAPS;
}
if (!nested_get_vmcs12_pages(vcpu, vmcs12)) {
- /*TODO: Also verify bits beyond physical address width are 0*/
nested_vmx_failValid(vcpu, VMXERR_ENTRY_INVALID_CONTROL_FIELD);
return 1;
}
vmcs12->launch_state = 1;
if (vmcs12->guest_activity_state == GUEST_ACTIVITY_HLT)
- return kvm_emulate_halt(vcpu);
+ return kvm_vcpu_halt(vcpu);
vmx->nested.nested_run_pending = 1;
}
EXPORT_SYMBOL_GPL(kvm_get_cr8);
+static void kvm_update_dr0123(struct kvm_vcpu *vcpu)
+{
+ int i;
+
+ if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP)) {
+ for (i = 0; i < KVM_NR_DB_REGS; i++)
+ vcpu->arch.eff_db[i] = vcpu->arch.db[i];
+ vcpu->arch.switch_db_regs |= KVM_DEBUGREG_RELOAD;
+ }
+}
+
static void kvm_update_dr6(struct kvm_vcpu *vcpu)
{
if (!(vcpu->guest_debug & KVM_GUESTDBG_USE_HW_BP))
struct pvclock_gtod_data *vdata = &pvclock_gtod_data;
u64 boot_ns;
- boot_ns = ktime_to_ns(ktime_add(tk->tkr.base_mono, tk->offs_boot));
+ boot_ns = ktime_to_ns(ktime_add(tk->tkr_mono.base, tk->offs_boot));
write_seqcount_begin(&vdata->seq);
/* copy pvclock gtod data */
- vdata->clock.vclock_mode = tk->tkr.clock->archdata.vclock_mode;
- vdata->clock.cycle_last = tk->tkr.cycle_last;
- vdata->clock.mask = tk->tkr.mask;
- vdata->clock.mult = tk->tkr.mult;
- vdata->clock.shift = tk->tkr.shift;
+ vdata->clock.vclock_mode = tk->tkr_mono.clock->archdata.vclock_mode;
+ vdata->clock.cycle_last = tk->tkr_mono.cycle_last;
+ vdata->clock.mask = tk->tkr_mono.mask;
+ vdata->clock.mult = tk->tkr_mono.mult;
+ vdata->clock.shift = tk->tkr_mono.shift;
vdata->boot_ns = boot_ns;
- vdata->nsec_base = tk->tkr.xtime_nsec;
+ vdata->nsec_base = tk->tkr_mono.xtime_nsec;
write_seqcount_end(&vdata->seq);
}
case KVM_CAP_USER_NMI:
case KVM_CAP_REINJECT_CONTROL:
case KVM_CAP_IRQ_INJECT_STATUS:
- case KVM_CAP_IRQFD:
case KVM_CAP_IOEVENTFD:
case KVM_CAP_IOEVENTFD_NO_LENGTH:
case KVM_CAP_PIT2:
return -EINVAL;
memcpy(vcpu->arch.db, dbgregs->db, sizeof(vcpu->arch.db));
+ kvm_update_dr0123(vcpu);
vcpu->arch.dr6 = dbgregs->dr6;
kvm_update_dr6(vcpu);
vcpu->arch.dr7 = dbgregs->dr7;
do {
n = min(len, 8);
if (!(vcpu->arch.apic &&
- !kvm_iodevice_write(&vcpu->arch.apic->dev, addr, n, v))
- && kvm_io_bus_write(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+ !kvm_iodevice_write(vcpu, &vcpu->arch.apic->dev, addr, n, v))
+ && kvm_io_bus_write(vcpu, KVM_MMIO_BUS, addr, n, v))
break;
handled += n;
addr += n;
do {
n = min(len, 8);
if (!(vcpu->arch.apic &&
- !kvm_iodevice_read(&vcpu->arch.apic->dev, addr, n, v))
- && kvm_io_bus_read(vcpu->kvm, KVM_MMIO_BUS, addr, n, v))
+ !kvm_iodevice_read(vcpu, &vcpu->arch.apic->dev,
+ addr, n, v))
+ && kvm_io_bus_read(vcpu, KVM_MMIO_BUS, addr, n, v))
break;
trace_kvm_mmio(KVM_TRACE_MMIO_READ, n, addr, *(u64 *)v);
handled += n;
return X86EMUL_CONTINUE;
}
-int emulator_read_write(struct x86_emulate_ctxt *ctxt, unsigned long addr,
+static int emulator_read_write(struct x86_emulate_ctxt *ctxt,
+ unsigned long addr,
void *val, unsigned int bytes,
struct x86_exception *exception,
const struct read_write_emulator_ops *ops)
exception, &read_emultor);
}
-int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
+static int emulator_write_emulated(struct x86_emulate_ctxt *ctxt,
unsigned long addr,
const void *val,
unsigned int bytes,
int r;
if (vcpu->arch.pio.in)
- r = kvm_io_bus_read(vcpu->kvm, KVM_PIO_BUS, vcpu->arch.pio.port,
+ r = kvm_io_bus_read(vcpu, KVM_PIO_BUS, vcpu->arch.pio.port,
vcpu->arch.pio.size, pd);
else
- r = kvm_io_bus_write(vcpu->kvm, KVM_PIO_BUS,
+ r = kvm_io_bus_write(vcpu, KVM_PIO_BUS,
vcpu->arch.pio.port, vcpu->arch.pio.size,
pd);
return r;
kvm_mmu_invlpg(emul_to_vcpu(ctxt), address);
}
-int kvm_emulate_wbinvd(struct kvm_vcpu *vcpu)
+int kvm_emulate_wbinvd_noskip(struct kvm_vcpu *vcpu)
{
if (!need_emulate_wbinvd(vcpu))
return X86EMUL_CONTINUE;
wbinvd();
return X86EMUL_CONTINUE;
}
+
+int kvm_emulate_wbinvd(struct kvm_vcpu *vcpu)
+{
+ kvm_x86_ops->skip_emulated_instruction(vcpu);
+ return kvm_emulate_wbinvd_noskip(vcpu);
+}
EXPORT_SYMBOL_GPL(kvm_emulate_wbinvd);
+
+
static void emulator_wbinvd(struct x86_emulate_ctxt *ctxt)
{
- kvm_emulate_wbinvd(emul_to_vcpu(ctxt));
+ kvm_emulate_wbinvd_noskip(emul_to_vcpu(ctxt));
}
-int emulator_get_dr(struct x86_emulate_ctxt *ctxt, int dr, unsigned long *dest)
+static int emulator_get_dr(struct x86_emulate_ctxt *ctxt, int dr,
+ unsigned long *dest)
{
return kvm_get_dr(emul_to_vcpu(ctxt), dr, dest);
}
-int emulator_set_dr(struct x86_emulate_ctxt *ctxt, int dr, unsigned long value)
+static int emulator_set_dr(struct x86_emulate_ctxt *ctxt, int dr,
+ unsigned long value)
{
return __kvm_set_dr(emul_to_vcpu(ctxt), dr, value);
free_percpu(shared_msrs);
}
-int kvm_emulate_halt(struct kvm_vcpu *vcpu)
+int kvm_vcpu_halt(struct kvm_vcpu *vcpu)
{
++vcpu->stat.halt_exits;
if (irqchip_in_kernel(vcpu->kvm)) {
return 0;
}
}
+EXPORT_SYMBOL_GPL(kvm_vcpu_halt);
+
+int kvm_emulate_halt(struct kvm_vcpu *vcpu)
+{
+ kvm_x86_ops->skip_emulated_instruction(vcpu);
+ return kvm_vcpu_halt(vcpu);
+}
EXPORT_SYMBOL_GPL(kvm_emulate_halt);
int kvm_hv_hypercall(struct kvm_vcpu *vcpu)
lapic_irq.dest_id = apicid;
lapic_irq.delivery_mode = APIC_DM_REMRD;
- kvm_irq_delivery_to_apic(kvm, 0, &lapic_irq, NULL);
+ kvm_irq_delivery_to_apic(kvm, NULL, &lapic_irq, NULL);
}
int kvm_emulate_hypercall(struct kvm_vcpu *vcpu)
unsigned long nr, a0, a1, a2, a3, ret;
int op_64_bit, r = 1;
+ kvm_x86_ops->skip_emulated_instruction(vcpu);
+
if (kvm_hv_hypercall_enabled(vcpu->kvm))
return kvm_hv_hypercall(vcpu);
}
/*
- * Returns 1 to let __vcpu_run() continue the guest execution loop without
+ * Returns 1 to let vcpu_run() continue the guest execution loop without
* exiting to the userspace. Otherwise, the value will be returned to the
* userspace.
*/
set_debugreg(vcpu->arch.eff_db[2], 2);
set_debugreg(vcpu->arch.eff_db[3], 3);
set_debugreg(vcpu->arch.dr6, 6);
+ vcpu->arch.switch_db_regs &= ~KVM_DEBUGREG_RELOAD;
}
trace_kvm_entry(vcpu->vcpu_id);
return r;
}
+static inline int vcpu_block(struct kvm *kvm, struct kvm_vcpu *vcpu)
+{
+ if (!kvm_arch_vcpu_runnable(vcpu)) {
+ srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx);
+ kvm_vcpu_block(vcpu);
+ vcpu->srcu_idx = srcu_read_lock(&kvm->srcu);
+ if (!kvm_check_request(KVM_REQ_UNHALT, vcpu))
+ return 1;
+ }
+
+ kvm_apic_accept_events(vcpu);
+ switch(vcpu->arch.mp_state) {
+ case KVM_MP_STATE_HALTED:
+ vcpu->arch.pv.pv_unhalted = false;
+ vcpu->arch.mp_state =
+ KVM_MP_STATE_RUNNABLE;
+ case KVM_MP_STATE_RUNNABLE:
+ vcpu->arch.apf.halted = false;
+ break;
+ case KVM_MP_STATE_INIT_RECEIVED:
+ break;
+ default:
+ return -EINTR;
+ break;
+ }
+ return 1;
+}
-static int __vcpu_run(struct kvm_vcpu *vcpu)
+static int vcpu_run(struct kvm_vcpu *vcpu)
{
int r;
struct kvm *kvm = vcpu->kvm;
vcpu->srcu_idx = srcu_read_lock(&kvm->srcu);
- r = 1;
- while (r > 0) {
+ for (;;) {
if (vcpu->arch.mp_state == KVM_MP_STATE_RUNNABLE &&
!vcpu->arch.apf.halted)
r = vcpu_enter_guest(vcpu);
- else {
- srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx);
- kvm_vcpu_block(vcpu);
- vcpu->srcu_idx = srcu_read_lock(&kvm->srcu);
- if (kvm_check_request(KVM_REQ_UNHALT, vcpu)) {
- kvm_apic_accept_events(vcpu);
- switch(vcpu->arch.mp_state) {
- case KVM_MP_STATE_HALTED:
- vcpu->arch.pv.pv_unhalted = false;
- vcpu->arch.mp_state =
- KVM_MP_STATE_RUNNABLE;
- case KVM_MP_STATE_RUNNABLE:
- vcpu->arch.apf.halted = false;
- break;
- case KVM_MP_STATE_INIT_RECEIVED:
- break;
- default:
- r = -EINTR;
- break;
- }
- }
- }
-
+ else
+ r = vcpu_block(kvm, vcpu);
if (r <= 0)
break;
r = -EINTR;
vcpu->run->exit_reason = KVM_EXIT_INTR;
++vcpu->stat.request_irq_exits;
+ break;
}
kvm_check_async_pf_completion(vcpu);
r = -EINTR;
vcpu->run->exit_reason = KVM_EXIT_INTR;
++vcpu->stat.signal_exits;
+ break;
}
if (need_resched()) {
srcu_read_unlock(&kvm->srcu, vcpu->srcu_idx);
} else
WARN_ON(vcpu->arch.pio.count || vcpu->mmio_needed);
- r = __vcpu_run(vcpu);
+ r = vcpu_run(vcpu);
out:
post_kvm_run_save(vcpu);
kvm_clear_exception_queue(vcpu);
memset(vcpu->arch.db, 0, sizeof(vcpu->arch.db));
+ kvm_update_dr0123(vcpu);
vcpu->arch.dr6 = DR6_INIT;
kvm_update_dr6(vcpu);
vcpu->arch.dr7 = DR7_FIXED_1;
kvm_update_dr7(vcpu);
+ vcpu->arch.cr2 = 0;
+
kvm_make_request(KVM_REQ_EVENT, vcpu);
vcpu->arch.apf.msr_val = 0;
vcpu->arch.st.msr_val = 0;
vcpu->arch.pv.pv_unhalted = false;
vcpu->arch.emulate_ctxt.ops = &emulate_ops;
- if (!irqchip_in_kernel(kvm) || kvm_vcpu_is_bsp(vcpu))
+ if (!irqchip_in_kernel(kvm) || kvm_vcpu_is_reset_bsp(vcpu))
vcpu->arch.mp_state = KVM_MP_STATE_RUNNABLE;
else
vcpu->arch.mp_state = KVM_MP_STATE_UNINITIALIZED;
vcpu->arch.guest_supported_xcr0 = 0;
vcpu->arch.guest_xstate_size = XSAVE_HDR_SIZE + XSAVE_HDR_OFFSET;
+ vcpu->arch.maxphyaddr = cpuid_query_maxphyaddr(vcpu);
+
kvm_async_pf_hash_reset(vcpu);
kvm_pmu_init(vcpu);
for (i = 0; i < KVM_NR_PAGE_SIZES; ++i) {
if (!dont || free->arch.rmap[i] != dont->arch.rmap[i]) {
- kvm_kvfree(free->arch.rmap[i]);
+ kvfree(free->arch.rmap[i]);
free->arch.rmap[i] = NULL;
}
if (i == 0)
if (!dont || free->arch.lpage_info[i - 1] !=
dont->arch.lpage_info[i - 1]) {
- kvm_kvfree(free->arch.lpage_info[i - 1]);
+ kvfree(free->arch.lpage_info[i - 1]);
free->arch.lpage_info[i - 1] = NULL;
}
}
out_free:
for (i = 0; i < KVM_NR_PAGE_SIZES; ++i) {
- kvm_kvfree(slot->arch.rmap[i]);
+ kvfree(slot->arch.rmap[i]);
slot->arch.rmap[i] = NULL;
if (i == 0)
continue;
- kvm_kvfree(slot->arch.lpage_info[i - 1]);
+ kvfree(slot->arch.lpage_info[i - 1]);
slot->arch.lpage_info[i - 1] = NULL;
}
return -ENOMEM;
/* It's OK to get 'new' slot here as it has already been installed */
new = id_to_memslot(kvm->memslots, mem->slot);
+ /*
+ * Dirty logging tracks sptes in 4k granularity, meaning that large
+ * sptes have to be split. If live migration is successful, the guest
+ * in the source machine will be destroyed and large sptes will be
+ * created in the destination. However, if the guest continues to run
+ * in the source machine (for example if live migration fails), small
+ * sptes will remain around and cause bad performance.
+ *
+ * Scan sptes if dirty logging has been stopped, dropping those
+ * which can be collapsed into a single large-page spte. Later
+ * page faults will create the large-page sptes.
+ */
+ if ((change != KVM_MR_DELETE) &&
+ (old->flags & KVM_MEM_LOG_DIRTY_PAGES) &&
+ !(new->flags & KVM_MEM_LOG_DIRTY_PAGES))
+ kvm_mmu_zap_collapsible_sptes(kvm, new);
+
/*
* Set up write protection and/or dirty logging for the new slot.
*
config LGUEST_GUEST
bool "Lguest guest support"
- depends on X86_32 && PARAVIRT
+ depends on X86_32 && PARAVIRT && PCI
select TTY
select VIRTUALIZATION
select VIRTIO
help
Lguest is a tiny in-kernel hypervisor. Selecting this will
allow your kernel to boot under lguest. This option will increase
- your kernel size by about 6k. If in doubt, say N.
+ your kernel size by about 10k. If in doubt, say N.
If you say Y here, make sure you say Y (or M) to the virtio block
and net drivers which lguest needs.
/* Some systems map "vectors" to interrupts weirdly. Not us! */
__this_cpu_write(vector_irq[i], i - FIRST_EXTERNAL_VECTOR);
if (i != SYSCALL_VECTOR)
- set_intr_gate(i, interrupt[i - FIRST_EXTERNAL_VECTOR]);
+ set_intr_gate(i, irq_entries_start +
+ 8 * (i - FIRST_EXTERNAL_VECTOR));
}
/*
{
lazy_hcall3(LHCALL_SET_STACK, __KERNEL_DS | 0x1, thread->sp0,
THREAD_SIZE / PAGE_SIZE);
+ tss->x86_tss.sp0 = thread->sp0;
}
/* Let's just say, I wouldn't do debugging under a Guest. */
#include <asm/alternative-asm.h>
#include <asm/dwarf2.h>
-.macro SAVE reg
- pushl_cfi %\reg
- CFI_REL_OFFSET \reg, 0
-.endm
-
-.macro RESTORE reg
- popl_cfi %\reg
- CFI_RESTORE \reg
-.endm
-
.macro read64 reg
movl %ebx, %eax
movl %ecx, %edx
.macro addsub_return func ins insc
ENTRY(atomic64_\func\()_return_cx8)
CFI_STARTPROC
- SAVE ebp
- SAVE ebx
- SAVE esi
- SAVE edi
+ pushl_cfi_reg ebp
+ pushl_cfi_reg ebx
+ pushl_cfi_reg esi
+ pushl_cfi_reg edi
movl %eax, %esi
movl %edx, %edi
10:
movl %ebx, %eax
movl %ecx, %edx
- RESTORE edi
- RESTORE esi
- RESTORE ebx
- RESTORE ebp
+ popl_cfi_reg edi
+ popl_cfi_reg esi
+ popl_cfi_reg ebx
+ popl_cfi_reg ebp
ret
CFI_ENDPROC
ENDPROC(atomic64_\func\()_return_cx8)
.macro incdec_return func ins insc
ENTRY(atomic64_\func\()_return_cx8)
CFI_STARTPROC
- SAVE ebx
+ pushl_cfi_reg ebx
read64 %esi
1:
10:
movl %ebx, %eax
movl %ecx, %edx
- RESTORE ebx
+ popl_cfi_reg ebx
ret
CFI_ENDPROC
ENDPROC(atomic64_\func\()_return_cx8)
ENTRY(atomic64_dec_if_positive_cx8)
CFI_STARTPROC
- SAVE ebx
+ pushl_cfi_reg ebx
read64 %esi
1:
2:
movl %ebx, %eax
movl %ecx, %edx
- RESTORE ebx
+ popl_cfi_reg ebx
ret
CFI_ENDPROC
ENDPROC(atomic64_dec_if_positive_cx8)
ENTRY(atomic64_add_unless_cx8)
CFI_STARTPROC
- SAVE ebp
- SAVE ebx
+ pushl_cfi_reg ebp
+ pushl_cfi_reg ebx
/* these just push these two parameters on the stack */
- SAVE edi
- SAVE ecx
+ pushl_cfi_reg edi
+ pushl_cfi_reg ecx
movl %eax, %ebp
movl %edx, %edi
3:
addl $8, %esp
CFI_ADJUST_CFA_OFFSET -8
- RESTORE ebx
- RESTORE ebp
+ popl_cfi_reg ebx
+ popl_cfi_reg ebp
ret
4:
cmpl %edx, 4(%esp)
ENTRY(atomic64_inc_not_zero_cx8)
CFI_STARTPROC
- SAVE ebx
+ pushl_cfi_reg ebx
read64 %esi
1:
movl $1, %eax
3:
- RESTORE ebx
+ popl_cfi_reg ebx
ret
CFI_ENDPROC
ENDPROC(atomic64_inc_not_zero_cx8)
*/
ENTRY(csum_partial)
CFI_STARTPROC
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
+ pushl_cfi_reg esi
+ pushl_cfi_reg ebx
movl 20(%esp),%eax # Function arg: unsigned int sum
movl 16(%esp),%ecx # Function arg: int len
movl 12(%esp),%esi # Function arg: unsigned char *buff
6: addl %ecx,%eax
adcl $0, %eax
7:
- testl $1, 12(%esp)
+ testb $1, 12(%esp)
jz 8f
roll $8, %eax
8:
- popl_cfi %ebx
- CFI_RESTORE ebx
- popl_cfi %esi
- CFI_RESTORE esi
+ popl_cfi_reg ebx
+ popl_cfi_reg esi
ret
CFI_ENDPROC
ENDPROC(csum_partial)
ENTRY(csum_partial)
CFI_STARTPROC
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
+ pushl_cfi_reg esi
+ pushl_cfi_reg ebx
movl 20(%esp),%eax # Function arg: unsigned int sum
movl 16(%esp),%ecx # Function arg: int len
movl 12(%esp),%esi # Function arg: const unsigned char *buf
addl %ebx,%eax
adcl $0,%eax
80:
- testl $1, 12(%esp)
+ testb $1, 12(%esp)
jz 90f
roll $8, %eax
90:
- popl_cfi %ebx
- CFI_RESTORE ebx
- popl_cfi %esi
- CFI_RESTORE esi
+ popl_cfi_reg ebx
+ popl_cfi_reg esi
ret
CFI_ENDPROC
ENDPROC(csum_partial)
CFI_STARTPROC
subl $4,%esp
CFI_ADJUST_CFA_OFFSET 4
- pushl_cfi %edi
- CFI_REL_OFFSET edi, 0
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
+ pushl_cfi_reg edi
+ pushl_cfi_reg esi
+ pushl_cfi_reg ebx
movl ARGBASE+16(%esp),%eax # sum
movl ARGBASE+12(%esp),%ecx # len
movl ARGBASE+4(%esp),%esi # src
.previous
- popl_cfi %ebx
- CFI_RESTORE ebx
- popl_cfi %esi
- CFI_RESTORE esi
- popl_cfi %edi
- CFI_RESTORE edi
+ popl_cfi_reg ebx
+ popl_cfi_reg esi
+ popl_cfi_reg edi
popl_cfi %ecx # equivalent to addl $4,%esp
ret
CFI_ENDPROC
ENTRY(csum_partial_copy_generic)
CFI_STARTPROC
- pushl_cfi %ebx
- CFI_REL_OFFSET ebx, 0
- pushl_cfi %edi
- CFI_REL_OFFSET edi, 0
- pushl_cfi %esi
- CFI_REL_OFFSET esi, 0
+ pushl_cfi_reg ebx
+ pushl_cfi_reg edi
+ pushl_cfi_reg esi
movl ARGBASE+4(%esp),%esi #src
movl ARGBASE+8(%esp),%edi #dst
movl ARGBASE+12(%esp),%ecx #len
jmp 7b
.previous
- popl_cfi %esi
- CFI_RESTORE esi
- popl_cfi %edi
- CFI_RESTORE edi
- popl_cfi %ebx
- CFI_RESTORE ebx
+ popl_cfi_reg esi
+ popl_cfi_reg edi
+ popl_cfi_reg ebx
ret
CFI_ENDPROC
ENDPROC(csum_partial_copy_generic)
#include <linux/linkage.h>
#include <asm/dwarf2.h>
+#include <asm/cpufeature.h>
#include <asm/alternative-asm.h>
/*
- * Zero a page.
- * rdi page
- */
-ENTRY(clear_page_c)
+ * Most CPUs support enhanced REP MOVSB/STOSB instructions. It is
+ * recommended to use this when possible and we do use them by default.
+ * If enhanced REP MOVSB/STOSB is not available, try to use fast string.
+ * Otherwise, use original.
+ */
+
+/*
+ * Zero a page.
+ * %rdi - page
+ */
+ENTRY(clear_page)
CFI_STARTPROC
+
+ ALTERNATIVE_2 "jmp clear_page_orig", "", X86_FEATURE_REP_GOOD, \
+ "jmp clear_page_c_e", X86_FEATURE_ERMS
+
movl $4096/8,%ecx
xorl %eax,%eax
rep stosq
ret
CFI_ENDPROC
-ENDPROC(clear_page_c)
+ENDPROC(clear_page)
-ENTRY(clear_page_c_e)
+ENTRY(clear_page_orig)
CFI_STARTPROC
- movl $4096,%ecx
- xorl %eax,%eax
- rep stosb
- ret
- CFI_ENDPROC
-ENDPROC(clear_page_c_e)
-ENTRY(clear_page)
- CFI_STARTPROC
xorl %eax,%eax
movl $4096/64,%ecx
.p2align 4
nop
ret
CFI_ENDPROC
-.Lclear_page_end:
-ENDPROC(clear_page)
-
- /*
- * Some CPUs support enhanced REP MOVSB/STOSB instructions.
- * It is recommended to use this when possible.
- * If enhanced REP MOVSB/STOSB is not available, try to use fast string.
- * Otherwise, use original function.
- *
- */
+ENDPROC(clear_page_orig)
-#include <asm/cpufeature.h>
-
- .section .altinstr_replacement,"ax"
-1: .byte 0xeb /* jmp <disp8> */
- .byte (clear_page_c - clear_page) - (2f - 1b) /* offset */
-2: .byte 0xeb /* jmp <disp8> */
- .byte (clear_page_c_e - clear_page) - (3f - 2b) /* offset */
-3:
- .previous
- .section .altinstructions,"a"
- altinstruction_entry clear_page,1b,X86_FEATURE_REP_GOOD,\
- .Lclear_page_end-clear_page, 2b-1b
- altinstruction_entry clear_page,2b,X86_FEATURE_ERMS, \
- .Lclear_page_end-clear_page,3b-2b
- .previous
+ENTRY(clear_page_c_e)
+ CFI_STARTPROC
+ movl $4096,%ecx
+ xorl %eax,%eax
+ rep stosb
+ ret
+ CFI_ENDPROC
+ENDPROC(clear_page_c_e)
#include <linux/linkage.h>
#include <asm/dwarf2.h>
+#include <asm/cpufeature.h>
#include <asm/alternative-asm.h>
+/*
+ * Some CPUs run faster using the string copy instructions (sane microcode).
+ * It is also a lot simpler. Use this when possible. But, don't use streaming
+ * copy unless the CPU indicates X86_FEATURE_REP_GOOD. Could vary the
+ * prefetch distance based on SMP/UP.
+ */
ALIGN
-copy_page_rep:
+ENTRY(copy_page)
CFI_STARTPROC
+ ALTERNATIVE "jmp copy_page_regs", "", X86_FEATURE_REP_GOOD
movl $4096/8, %ecx
rep movsq
ret
CFI_ENDPROC
-ENDPROC(copy_page_rep)
-
-/*
- * Don't use streaming copy unless the CPU indicates X86_FEATURE_REP_GOOD.
- * Could vary the prefetch distance based on SMP/UP.
-*/
+ENDPROC(copy_page)
-ENTRY(copy_page)
+ENTRY(copy_page_regs)
CFI_STARTPROC
subq $2*8, %rsp
CFI_ADJUST_CFA_OFFSET 2*8
addq $2*8, %rsp
CFI_ADJUST_CFA_OFFSET -2*8
ret
-.Lcopy_page_end:
CFI_ENDPROC
-ENDPROC(copy_page)
-
- /* Some CPUs run faster using the string copy instructions.
- It is also a lot simpler. Use this when possible */
-
-#include <asm/cpufeature.h>
-
- .section .altinstr_replacement,"ax"
-1: .byte 0xeb /* jmp <disp8> */
- .byte (copy_page_rep - copy_page) - (2f - 1b) /* offset */
-2:
- .previous
- .section .altinstructions,"a"
- altinstruction_entry copy_page, 1b, X86_FEATURE_REP_GOOD, \
- .Lcopy_page_end-copy_page, 2b-1b
- .previous
+ENDPROC(copy_page_regs)
#include <linux/linkage.h>
#include <asm/dwarf2.h>
-
-#define FIX_ALIGNMENT 1
-
#include <asm/current.h>
#include <asm/asm-offsets.h>
#include <asm/thread_info.h>
#include <asm/asm.h>
#include <asm/smap.h>
-/*
- * By placing feature2 after feature1 in altinstructions section, we logically
- * implement:
- * If CPU has feature2, jmp to alt2 is used
- * else if CPU has feature1, jmp to alt1 is used
- * else jmp to orig is used.
- */
- .macro ALTERNATIVE_JUMP feature1,feature2,orig,alt1,alt2
-0:
- .byte 0xe9 /* 32bit jump */
- .long \orig-1f /* by default jump to orig */
-1:
- .section .altinstr_replacement,"ax"
-2: .byte 0xe9 /* near jump with 32bit immediate */
- .long \alt1-1b /* offset */ /* or alternatively to alt1 */
-3: .byte 0xe9 /* near jump with 32bit immediate */
- .long \alt2-1b /* offset */ /* or alternatively to alt2 */
- .previous
-
- .section .altinstructions,"a"
- altinstruction_entry 0b,2b,\feature1,5,5
- altinstruction_entry 0b,3b,\feature2,5,5
- .previous
- .endm
-
.macro ALIGN_DESTINATION
-#ifdef FIX_ALIGNMENT
/* check for bad alignment of destination */
movl %edi,%ecx
andl $7,%ecx
_ASM_EXTABLE(100b,103b)
_ASM_EXTABLE(101b,103b)
-#endif
.endm
/* Standard copy_to_user with segment limit checking */
jc bad_to_user
cmpq TI_addr_limit(%rax),%rcx
ja bad_to_user
- ALTERNATIVE_JUMP X86_FEATURE_REP_GOOD,X86_FEATURE_ERMS, \
- copy_user_generic_unrolled,copy_user_generic_string, \
- copy_user_enhanced_fast_string
+ ALTERNATIVE_2 "jmp copy_user_generic_unrolled", \
+ "jmp copy_user_generic_string", \
+ X86_FEATURE_REP_GOOD, \
+ "jmp copy_user_enhanced_fast_string", \
+ X86_FEATURE_ERMS
CFI_ENDPROC
ENDPROC(_copy_to_user)
jc bad_from_user
cmpq TI_addr_limit(%rax),%rcx
ja bad_from_user
- ALTERNATIVE_JUMP X86_FEATURE_REP_GOOD,X86_FEATURE_ERMS, \
- copy_user_generic_unrolled,copy_user_generic_string, \
- copy_user_enhanced_fast_string
+ ALTERNATIVE_2 "jmp copy_user_generic_unrolled", \
+ "jmp copy_user_generic_string", \
+ X86_FEATURE_REP_GOOD, \
+ "jmp copy_user_enhanced_fast_string", \
+ X86_FEATURE_ERMS
CFI_ENDPROC
ENDPROC(_copy_from_user)
/* handle last odd byte */
.Lhandle_1:
- testl $1, %r10d
+ testb $1, %r10b
jz .Lende
xorl %ebx, %ebx
source
*/
void insn_init(struct insn *insn, const void *kaddr, int buf_len, int x86_64)
{
+ /*
+ * Instructions longer than MAX_INSN_SIZE (15 bytes) are invalid
+ * even if the input buffer is long enough to hold them.
+ */
+ if (buf_len > MAX_INSN_SIZE)
+ buf_len = MAX_INSN_SIZE;
+
memset(insn, 0, sizeof(*insn));
insn->kaddr = kaddr;
insn->end_kaddr = kaddr + buf_len;
/* VEX.W overrides opnd_size */
insn->opnd_bytes = 8;
} else {
+ /*
+ * For VEX2, fake VEX3-like byte#2.
+ * Makes it easier to decode vex.W, vex.vvvv,
+ * vex.L and vex.pp. Masking with 0x7f sets vex.W == 0.
+ */
+ insn->vex_prefix.bytes[2] = b2 & 0x7f;
insn->vex_prefix.nbytes = 2;
insn->next_byte += 2;
}
/* Copyright 2002 Andi Kleen */
#include <linux/linkage.h>
-
#include <asm/cpufeature.h>
#include <asm/dwarf2.h>
#include <asm/alternative-asm.h>
+/*
+ * We build a jump to memcpy_orig by default which gets NOPped out on
+ * the majority of x86 CPUs which set REP_GOOD. In addition, CPUs which
+ * have the enhanced REP MOVSB/STOSB feature (ERMS), change those NOPs
+ * to a jmp to memcpy_erms which does the REP; MOVSB mem copy.
+ */
+
+.weak memcpy
+
/*
* memcpy - Copy a memory block.
*
* Output:
* rax original destination
*/
+ENTRY(__memcpy)
+ENTRY(memcpy)
+ ALTERNATIVE_2 "jmp memcpy_orig", "", X86_FEATURE_REP_GOOD, \
+ "jmp memcpy_erms", X86_FEATURE_ERMS
-/*
- * memcpy_c() - fast string ops (REP MOVSQ) based variant.
- *
- * This gets patched over the unrolled variant (below) via the
- * alternative instructions framework:
- */
- .section .altinstr_replacement, "ax", @progbits
-.Lmemcpy_c:
movq %rdi, %rax
movq %rdx, %rcx
shrq $3, %rcx
movl %edx, %ecx
rep movsb
ret
-.Lmemcpy_e:
- .previous
+ENDPROC(memcpy)
+ENDPROC(__memcpy)
/*
- * memcpy_c_e() - enhanced fast string memcpy. This is faster and simpler than
- * memcpy_c. Use memcpy_c_e when possible.
- *
- * This gets patched over the unrolled variant (below) via the
- * alternative instructions framework:
+ * memcpy_erms() - enhanced fast string memcpy. This is faster and
+ * simpler than memcpy. Use memcpy_erms when possible.
*/
- .section .altinstr_replacement, "ax", @progbits
-.Lmemcpy_c_e:
+ENTRY(memcpy_erms)
movq %rdi, %rax
movq %rdx, %rcx
rep movsb
ret
-.Lmemcpy_e_e:
- .previous
-
-.weak memcpy
+ENDPROC(memcpy_erms)
-ENTRY(__memcpy)
-ENTRY(memcpy)
+ENTRY(memcpy_orig)
CFI_STARTPROC
movq %rdi, %rax
.Lend:
retq
CFI_ENDPROC
-ENDPROC(memcpy)
-ENDPROC(__memcpy)
-
- /*
- * Some CPUs are adding enhanced REP MOVSB/STOSB feature
- * If the feature is supported, memcpy_c_e() is the first choice.
- * If enhanced rep movsb copy is not available, use fast string copy
- * memcpy_c() when possible. This is faster and code is simpler than
- * original memcpy().
- * Otherwise, original memcpy() is used.
- * In .altinstructions section, ERMS feature is placed after REG_GOOD
- * feature to implement the right patch order.
- *
- * Replace only beginning, memcpy is used to apply alternatives,
- * so it is silly to overwrite itself with nops - reboot is the
- * only outcome...
- */
- .section .altinstructions, "a"
- altinstruction_entry __memcpy,.Lmemcpy_c,X86_FEATURE_REP_GOOD,\
- .Lmemcpy_e-.Lmemcpy_c,.Lmemcpy_e-.Lmemcpy_c
- altinstruction_entry __memcpy,.Lmemcpy_c_e,X86_FEATURE_ERMS, \
- .Lmemcpy_e_e-.Lmemcpy_c_e,.Lmemcpy_e_e-.Lmemcpy_c_e
- .previous
+ENDPROC(memcpy_orig)
* This assembly file is re-written from memmove_64.c file.
* - Copyright 2011 Fenghua Yu <fenghua.yu@intel.com>
*/
-#define _STRING_C
#include <linux/linkage.h>
#include <asm/dwarf2.h>
#include <asm/cpufeature.h>
jg 2f
.Lmemmove_begin_forward:
+ ALTERNATIVE "", "movq %rdx, %rcx; rep movsb; retq", X86_FEATURE_ERMS
+
/*
* movsq instruction have many startup latency
* so we handle small size by general register.
13:
retq
CFI_ENDPROC
-
- .section .altinstr_replacement,"ax"
-.Lmemmove_begin_forward_efs:
- /* Forward moving data. */
- movq %rdx, %rcx
- rep movsb
- retq
-.Lmemmove_end_forward_efs:
- .previous
-
- .section .altinstructions,"a"
- altinstruction_entry .Lmemmove_begin_forward, \
- .Lmemmove_begin_forward_efs,X86_FEATURE_ERMS, \
- .Lmemmove_end_forward-.Lmemmove_begin_forward, \
- .Lmemmove_end_forward_efs-.Lmemmove_begin_forward_efs
- .previous
ENDPROC(__memmove)
ENDPROC(memmove)
#include <asm/cpufeature.h>
#include <asm/alternative-asm.h>
+.weak memset
+
/*
* ISO C memset - set a memory block to a byte value. This function uses fast
* string to get better performance than the original function. The code is
* simpler and shorter than the orignal function as well.
- *
+ *
* rdi destination
- * rsi value (char)
- * rdx count (bytes)
- *
+ * rsi value (char)
+ * rdx count (bytes)
+ *
* rax original destination
- */
- .section .altinstr_replacement, "ax", @progbits
-.Lmemset_c:
+ */
+ENTRY(memset)
+ENTRY(__memset)
+ /*
+ * Some CPUs support enhanced REP MOVSB/STOSB feature. It is recommended
+ * to use it when possible. If not available, use fast string instructions.
+ *
+ * Otherwise, use original memset function.
+ */
+ ALTERNATIVE_2 "jmp memset_orig", "", X86_FEATURE_REP_GOOD, \
+ "jmp memset_erms", X86_FEATURE_ERMS
+
movq %rdi,%r9
movq %rdx,%rcx
andl $7,%edx
rep stosb
movq %r9,%rax
ret
-.Lmemset_e:
- .previous
+ENDPROC(memset)
+ENDPROC(__memset)
/*
* ISO C memset - set a memory block to a byte value. This function uses
*
* rax original destination
*/
- .section .altinstr_replacement, "ax", @progbits
-.Lmemset_c_e:
+ENTRY(memset_erms)
movq %rdi,%r9
movb %sil,%al
movq %rdx,%rcx
rep stosb
movq %r9,%rax
ret
-.Lmemset_e_e:
- .previous
-
-.weak memset
+ENDPROC(memset_erms)
-ENTRY(memset)
-ENTRY(__memset)
+ENTRY(memset_orig)
CFI_STARTPROC
movq %rdi,%r10
jmp .Lafter_bad_alignment
.Lfinal:
CFI_ENDPROC
-ENDPROC(memset)
-ENDPROC(__memset)
-
- /* Some CPUs support enhanced REP MOVSB/STOSB feature.
- * It is recommended to use this when possible.
- *
- * If enhanced REP MOVSB/STOSB feature is not available, use fast string
- * instructions.
- *
- * Otherwise, use original memset function.
- *
- * In .altinstructions section, ERMS feature is placed after REG_GOOD
- * feature to implement the right patch order.
- */
- .section .altinstructions,"a"
- altinstruction_entry __memset,.Lmemset_c,X86_FEATURE_REP_GOOD,\
- .Lfinal-__memset,.Lmemset_e-.Lmemset_c
- altinstruction_entry __memset,.Lmemset_c_e,X86_FEATURE_ERMS, \
- .Lfinal-__memset,.Lmemset_e_e-.Lmemset_c_e
- .previous
+ENDPROC(memset_orig)
.macro op_safe_regs op
ENTRY(\op\()_safe_regs)
CFI_STARTPROC
- pushq_cfi %rbx
- pushq_cfi %rbp
+ pushq_cfi_reg rbx
+ pushq_cfi_reg rbp
movq %rdi, %r10 /* Save pointer */
xorl %r11d, %r11d /* Return value */
movl (%rdi), %eax
movl %ebp, 20(%r10)
movl %esi, 24(%r10)
movl %edi, 28(%r10)
- popq_cfi %rbp
- popq_cfi %rbx
+ popq_cfi_reg rbp
+ popq_cfi_reg rbx
ret
3:
CFI_RESTORE_STATE
.macro op_safe_regs op
ENTRY(\op\()_safe_regs)
CFI_STARTPROC
- pushl_cfi %ebx
- pushl_cfi %ebp
- pushl_cfi %esi
- pushl_cfi %edi
+ pushl_cfi_reg ebx
+ pushl_cfi_reg ebp
+ pushl_cfi_reg esi
+ pushl_cfi_reg edi
pushl_cfi $0 /* Return value */
pushl_cfi %eax
movl 4(%eax), %ecx
movl %esi, 24(%eax)
movl %edi, 28(%eax)
popl_cfi %eax
- popl_cfi %edi
- popl_cfi %esi
- popl_cfi %ebp
- popl_cfi %ebx
+ popl_cfi_reg edi
+ popl_cfi_reg esi
+ popl_cfi_reg ebp
+ popl_cfi_reg ebx
ret
3:
CFI_RESTORE_STATE
*/
#define save_common_regs \
- pushl_cfi %ecx; CFI_REL_OFFSET ecx, 0
+ pushl_cfi_reg ecx
#define restore_common_regs \
- popl_cfi %ecx; CFI_RESTORE ecx
+ popl_cfi_reg ecx
/* Avoid uglifying the argument copying x86-64 needs to do. */
.macro movq src, dst
*/
#define save_common_regs \
- pushq_cfi %rdi; CFI_REL_OFFSET rdi, 0; \
- pushq_cfi %rsi; CFI_REL_OFFSET rsi, 0; \
- pushq_cfi %rcx; CFI_REL_OFFSET rcx, 0; \
- pushq_cfi %r8; CFI_REL_OFFSET r8, 0; \
- pushq_cfi %r9; CFI_REL_OFFSET r9, 0; \
- pushq_cfi %r10; CFI_REL_OFFSET r10, 0; \
- pushq_cfi %r11; CFI_REL_OFFSET r11, 0
+ pushq_cfi_reg rdi; \
+ pushq_cfi_reg rsi; \
+ pushq_cfi_reg rcx; \
+ pushq_cfi_reg r8; \
+ pushq_cfi_reg r9; \
+ pushq_cfi_reg r10; \
+ pushq_cfi_reg r11
#define restore_common_regs \
- popq_cfi %r11; CFI_RESTORE r11; \
- popq_cfi %r10; CFI_RESTORE r10; \
- popq_cfi %r9; CFI_RESTORE r9; \
- popq_cfi %r8; CFI_RESTORE r8; \
- popq_cfi %rcx; CFI_RESTORE rcx; \
- popq_cfi %rsi; CFI_RESTORE rsi; \
- popq_cfi %rdi; CFI_RESTORE rdi
+ popq_cfi_reg r11; \
+ popq_cfi_reg r10; \
+ popq_cfi_reg r9; \
+ popq_cfi_reg r8; \
+ popq_cfi_reg rcx; \
+ popq_cfi_reg rsi; \
+ popq_cfi_reg rdi
#endif
ENTRY(call_rwsem_down_read_failed)
CFI_STARTPROC
save_common_regs
- __ASM_SIZE(push,_cfi) %__ASM_REG(dx)
- CFI_REL_OFFSET __ASM_REG(dx), 0
+ __ASM_SIZE(push,_cfi_reg) __ASM_REG(dx)
movq %rax,%rdi
call rwsem_down_read_failed
- __ASM_SIZE(pop,_cfi) %__ASM_REG(dx)
- CFI_RESTORE __ASM_REG(dx)
+ __ASM_SIZE(pop,_cfi_reg) __ASM_REG(dx)
restore_common_regs
ret
CFI_ENDPROC
ENTRY(call_rwsem_downgrade_wake)
CFI_STARTPROC
save_common_regs
- __ASM_SIZE(push,_cfi) %__ASM_REG(dx)
- CFI_REL_OFFSET __ASM_REG(dx), 0
+ __ASM_SIZE(push,_cfi_reg) __ASM_REG(dx)
movq %rax,%rdi
call rwsem_downgrade_wake
- __ASM_SIZE(pop,_cfi) %__ASM_REG(dx)
- CFI_RESTORE __ASM_REG(dx)
+ __ASM_SIZE(pop,_cfi_reg) __ASM_REG(dx)
restore_common_regs
ret
CFI_ENDPROC
.globl \name
\name:
CFI_STARTPROC
- pushl_cfi %eax
- CFI_REL_OFFSET eax, 0
- pushl_cfi %ecx
- CFI_REL_OFFSET ecx, 0
- pushl_cfi %edx
- CFI_REL_OFFSET edx, 0
+ pushl_cfi_reg eax
+ pushl_cfi_reg ecx
+ pushl_cfi_reg edx
.if \put_ret_addr_in_eax
/* Place EIP in the arg1 */
.endif
call \func
- popl_cfi %edx
- CFI_RESTORE edx
- popl_cfi %ecx
- CFI_RESTORE ecx
- popl_cfi %eax
- CFI_RESTORE eax
+ popl_cfi_reg edx
+ popl_cfi_reg ecx
+ popl_cfi_reg eax
ret
CFI_ENDPROC
_ASM_NOKPROBE(\name)
CFI_STARTPROC
/* this one pushes 9 elems, the next one would be %rIP */
- SAVE_ARGS
+ pushq_cfi_reg rdi
+ pushq_cfi_reg rsi
+ pushq_cfi_reg rdx
+ pushq_cfi_reg rcx
+ pushq_cfi_reg rax
+ pushq_cfi_reg r8
+ pushq_cfi_reg r9
+ pushq_cfi_reg r10
+ pushq_cfi_reg r11
.if \put_ret_addr_in_rdi
+ /* 9*8(%rsp) is return addr on stack */
movq_cfi_restore 9*8, rdi
.endif
#endif
#endif
- /* SAVE_ARGS below is used only for the .cfi directives it contains. */
+#if defined(CONFIG_TRACE_IRQFLAGS) \
+ || defined(CONFIG_DEBUG_LOCK_ALLOC) \
+ || defined(CONFIG_PREEMPT)
CFI_STARTPROC
- SAVE_ARGS
+ CFI_ADJUST_CFA_OFFSET 9*8
restore:
- RESTORE_ARGS
+ popq_cfi_reg r11
+ popq_cfi_reg r10
+ popq_cfi_reg r9
+ popq_cfi_reg r8
+ popq_cfi_reg rax
+ popq_cfi_reg rcx
+ popq_cfi_reg rdx
+ popq_cfi_reg rsi
+ popq_cfi_reg rdi
ret
CFI_ENDPROC
_ASM_NOKPROBE(restore)
+#endif
* it is not necessary to optimize tail handling.
*/
__visible unsigned long
-copy_user_handle_tail(char *to, char *from, unsigned len, unsigned zerorest)
+copy_user_handle_tail(char *to, char *from, unsigned len)
{
- char c;
- unsigned zero_len;
-
for (; len; --len, to++) {
+ char c;
+
if (__get_user_nocheck(c, from++, sizeof(char)))
break;
if (__put_user_nocheck(c, to, sizeof(char)))
break;
}
-
- for (c = 0, zero_len = len; zerorest && zero_len; --zero_len)
- if (__put_user_nocheck(c, to++, sizeof(char)))
- break;
clac();
+
+ /* If the destination is a kernel buffer, we always clear the end */
+ if ((unsigned long)to >= TASK_SIZE_MAX)
+ memset(to, 0, len);
return len;
}
de: ESC
df: ESC
# 0xe0 - 0xef
+# Note: "forced64" is Intel CPU behavior: they ignore 0x66 prefix
+# in 64-bit mode. AMD CPUs accept 0x66 prefix, it causes RIP truncation
+# to 16 bits. In 32-bit mode, 0x66 is accepted by both Intel and AMD.
e0: LOOPNE/LOOPNZ Jb (f64)
e1: LOOPE/LOOPZ Jb (f64)
e2: LOOP Jb (f64)
e5: IN eAX,Ib
e6: OUT Ib,AL
e7: OUT Ib,eAX
+# With 0x66 prefix in 64-bit mode, for AMD CPUs immediate offset
+# in "near" jumps and calls is 16-bit. For CALL,
+# push of return address is 16-bit wide, RSP is decremented by 2
+# but is not truncated to 16 bits, unlike RIP.
e8: CALL Jz (f64)
e9: JMP-near Jz (f64)
ea: JMP-far Ap (i64)
7e: movd/q Ey,Pd | vmovd/q Ey,Vy (66),(v1) | vmovq Vq,Wq (F3),(v1)
7f: movq Qq,Pq | vmovdqa Wx,Vx (66) | vmovdqu Wx,Vx (F3)
# 0x0f 0x80-0x8f
+# Note: "forced64" is Intel CPU behavior (see comment about CALL insn).
80: JO Jz (f64)
81: JNO Jz (f64)
82: JB/JC/JNAE Jz (f64)
GrpTable: Grp5
0: INC Ev
1: DEC Ev
+# Note: "forced64" is Intel CPU behavior (see comment about CALL insn).
2: CALLN Ev (f64)
3: CALLF Ep
4: JMPN Ev (f64)
int ret = 0;
/* kprobe_running() needs smp_processor_id() */
- if (kprobes_built_in() && !user_mode_vm(regs)) {
+ if (kprobes_built_in() && !user_mode(regs)) {
preempt_disable();
if (kprobe_running() && kprobe_fault_handler(regs, 14))
ret = 1;
instr = (void *)convert_ip_to_linear(current, regs);
max_instr = instr + 15;
- if (user_mode(regs) && instr >= (unsigned char *)TASK_SIZE)
+ if (user_mode(regs) && instr >= (unsigned char *)TASK_SIZE_MAX)
return 0;
while (instr < max_instr) {
if (error_code & PF_USER)
return false;
- if (!user_mode_vm(regs) && (regs->flags & X86_EFLAGS_AC))
+ if (!user_mode(regs) && (regs->flags & X86_EFLAGS_AC))
return false;
return true;
* User-mode registers count as a user access even for any
* potential system fault or CPU buglet:
*/
- if (user_mode_vm(regs)) {
+ if (user_mode(regs)) {
local_irq_enable();
error_code |= PF_USER;
flags |= FAULT_FLAG_USER;
/*
* Tables translating between page_cache_type_t and pte encoding.
- * Minimal supported modes are defined statically, modified if more supported
- * cache modes are available.
- * Index into __cachemode2pte_tbl is the cachemode.
- * Index into __pte2cachemode_tbl are the caching attribute bits of the pte
- * (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2.
+ *
+ * Minimal supported modes are defined statically, they are modified
+ * during bootup if more supported cache modes are available.
+ *
+ * Index into __cachemode2pte_tbl[] is the cachemode.
+ *
+ * Index into __pte2cachemode_tbl[] are the caching attribute bits of the pte
+ * (_PAGE_PWT, _PAGE_PCD, _PAGE_PAT) at index bit positions 0, 1, 2.
*/
uint16_t __cachemode2pte_tbl[_PAGE_CACHE_MODE_NUM] = {
- [_PAGE_CACHE_MODE_WB] = 0,
- [_PAGE_CACHE_MODE_WC] = _PAGE_PWT,
- [_PAGE_CACHE_MODE_UC_MINUS] = _PAGE_PCD,
- [_PAGE_CACHE_MODE_UC] = _PAGE_PCD | _PAGE_PWT,
- [_PAGE_CACHE_MODE_WT] = _PAGE_PCD,
- [_PAGE_CACHE_MODE_WP] = _PAGE_PCD,
+ [_PAGE_CACHE_MODE_WB ] = 0 | 0 ,
+ [_PAGE_CACHE_MODE_WC ] = _PAGE_PWT | 0 ,
+ [_PAGE_CACHE_MODE_UC_MINUS] = 0 | _PAGE_PCD,
+ [_PAGE_CACHE_MODE_UC ] = _PAGE_PWT | _PAGE_PCD,
+ [_PAGE_CACHE_MODE_WT ] = 0 | _PAGE_PCD,
+ [_PAGE_CACHE_MODE_WP ] = 0 | _PAGE_PCD,
};
EXPORT_SYMBOL(__cachemode2pte_tbl);
+
uint8_t __pte2cachemode_tbl[8] = {
- [__pte2cm_idx(0)] = _PAGE_CACHE_MODE_WB,
- [__pte2cm_idx(_PAGE_PWT)] = _PAGE_CACHE_MODE_WC,
- [__pte2cm_idx(_PAGE_PCD)] = _PAGE_CACHE_MODE_UC_MINUS,
- [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD)] = _PAGE_CACHE_MODE_UC,
- [__pte2cm_idx(_PAGE_PAT)] = _PAGE_CACHE_MODE_WB,
- [__pte2cm_idx(_PAGE_PWT | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC,
- [__pte2cm_idx(_PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS,
+ [__pte2cm_idx( 0 | 0 | 0 )] = _PAGE_CACHE_MODE_WB,
+ [__pte2cm_idx(_PAGE_PWT | 0 | 0 )] = _PAGE_CACHE_MODE_WC,
+ [__pte2cm_idx( 0 | _PAGE_PCD | 0 )] = _PAGE_CACHE_MODE_UC_MINUS,
+ [__pte2cm_idx(_PAGE_PWT | _PAGE_PCD | 0 )] = _PAGE_CACHE_MODE_UC,
+ [__pte2cm_idx( 0 | 0 | _PAGE_PAT)] = _PAGE_CACHE_MODE_WB,
+ [__pte2cm_idx(_PAGE_PWT | 0 | _PAGE_PAT)] = _PAGE_CACHE_MODE_WC,
+ [__pte2cm_idx(0 | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC_MINUS,
[__pte2cm_idx(_PAGE_PWT | _PAGE_PCD | _PAGE_PAT)] = _PAGE_CACHE_MODE_UC,
};
EXPORT_SYMBOL(__pte2cachemode_tbl);
int after_bootmem;
-int direct_gbpages
-#ifdef CONFIG_DIRECT_GBPAGES
- = 1
-#endif
-;
-
-static void __init init_gbpages(void)
-{
-#ifdef CONFIG_X86_64
- if (direct_gbpages && cpu_has_gbpages)
- printk(KERN_INFO "Using GB pages for direct mapping\n");
- else
- direct_gbpages = 0;
-#endif
-}
+early_param_on_off("gbpages", "nogbpages", direct_gbpages, CONFIG_X86_DIRECT_GBPAGES);
struct map_range {
unsigned long start;
static void __init probe_page_size_mask(void)
{
- init_gbpages();
-
#if !defined(CONFIG_DEBUG_PAGEALLOC) && !defined(CONFIG_KMEMCHECK)
/*
* For CONFIG_DEBUG_PAGEALLOC, identity mapping will use small pages.
* This will simplify cpa(), which otherwise needs to support splitting
* large pages into small in interrupt context, etc.
*/
- if (direct_gbpages)
- page_size_mask |= 1 << PG_LEVEL_1G;
if (cpu_has_pse)
page_size_mask |= 1 << PG_LEVEL_2M;
#endif
if (cpu_has_pge) {
cr4_set_bits_and_update_boot(X86_CR4_PGE);
__supported_pte_mask |= _PAGE_GLOBAL;
+ } else
+ __supported_pte_mask &= ~_PAGE_GLOBAL;
+
+ /* Enable 1 GB linear kernel mappings if available: */
+ if (direct_gbpages && cpu_has_gbpages) {
+ printk(KERN_INFO "Using GB pages for direct mapping\n");
+ page_size_mask |= 1 << PG_LEVEL_1G;
+ } else {
+ direct_gbpages = 0;
}
}
return 0;
}
-static int __init parse_direct_gbpages_off(char *arg)
-{
- direct_gbpages = 0;
- return 0;
-}
-early_param("nogbpages", parse_direct_gbpages_off);
-
-static int __init parse_direct_gbpages_on(char *arg)
-{
- direct_gbpages = 1;
- return 0;
-}
-early_param("gbpages", parse_direct_gbpages_on);
-
/*
* NOTE: pagetable_init alloc all the fixmap pagetables contiguous on the
* physical space so we can cache the place of the first one and move
&memblock.reserved, mb->nid);
}
- /* Mark all kernel nodes. */
+ /*
+ * Mark all kernel nodes.
+ *
+ * When booting with mem=nn[kMG] or in a kdump kernel, numa_meminfo
+ * may not include all the memblock.reserved memory ranges because
+ * trim_snb_memory() reserves specific pages for Sandy Bridge graphics.
+ */
for_each_memblock(reserved, r)
- node_set(r->nid, numa_kernel_nodes);
+ if (r->nid != MAX_NUMNODES)
+ node_set(r->nid, numa_kernel_nodes);
/* Clear MEMBLOCK_HOTPLUG flag for memory in kernel nodes. */
for (i = 0; i < numa_meminfo.nr_blks; i++) {
seq_printf(m, "DirectMap4M: %8lu kB\n",
direct_pages_count[PG_LEVEL_2M] << 12);
#endif
-#ifdef CONFIG_X86_64
if (direct_gbpages)
seq_printf(m, "DirectMap1G: %8lu kB\n",
direct_pages_count[PG_LEVEL_1G] << 20);
-#endif
}
#else
static inline void split_page_count(int level) { }
{
return change_page_attr_clear(&addr, numpages, __pgprot(_PAGE_RW), 0);
}
-EXPORT_SYMBOL_GPL(set_memory_ro);
int set_memory_rw(unsigned long addr, int numpages)
{
return change_page_attr_set(&addr, numpages, __pgprot(_PAGE_RW), 0);
}
-EXPORT_SYMBOL_GPL(set_memory_rw);
int set_memory_np(unsigned long addr, int numpages)
{
}
#ifdef CONFIG_STRICT_DEVMEM
-/* This check is done in drivers/char/mem.c in case of STRICT_DEVMEM*/
+/* This check is done in drivers/char/mem.c in case of STRICT_DEVMEM */
static inline int range_is_allowed(unsigned long pfn, unsigned long size)
{
return 1;
while (cursor < to) {
if (!devmem_is_allowed(pfn)) {
- printk(KERN_INFO "Program %s tried to access /dev/mem between [mem %#010Lx-%#010Lx]\n",
- current->comm, from, to - 1);
+ printk(KERN_INFO "Program %s tried to access /dev/mem between [mem %#010Lx-%#010Lx], PAT prevents it\n",
+ current->comm, from, to - 1);
return 0;
}
cursor += PAGE_SIZE;
}
}
+/*
+ * Xen paravirt assumes pgd table should be in one page. 64 bit kernel also
+ * assumes that pgd should be in one page.
+ *
+ * But kernel with PAE paging that is not running as a Xen domain
+ * only needs to allocate 32 bytes for pgd instead of one page.
+ */
+#ifdef CONFIG_X86_PAE
+
+#include <linux/slab.h>
+
+#define PGD_SIZE (PTRS_PER_PGD * sizeof(pgd_t))
+#define PGD_ALIGN 32
+
+static struct kmem_cache *pgd_cache;
+
+static int __init pgd_cache_init(void)
+{
+ /*
+ * When PAE kernel is running as a Xen domain, it does not use
+ * shared kernel pmd. And this requires a whole page for pgd.
+ */
+ if (!SHARED_KERNEL_PMD)
+ return 0;
+
+ /*
+ * when PAE kernel is not running as a Xen domain, it uses
+ * shared kernel pmd. Shared kernel pmd does not require a whole
+ * page for pgd. We are able to just allocate a 32-byte for pgd.
+ * During boot time, we create a 32-byte slab for pgd table allocation.
+ */
+ pgd_cache = kmem_cache_create("pgd_cache", PGD_SIZE, PGD_ALIGN,
+ SLAB_PANIC, NULL);
+ if (!pgd_cache)
+ return -ENOMEM;
+
+ return 0;
+}
+core_initcall(pgd_cache_init);
+
+static inline pgd_t *_pgd_alloc(void)
+{
+ /*
+ * If no SHARED_KERNEL_PMD, PAE kernel is running as a Xen domain.
+ * We allocate one page for pgd.
+ */
+ if (!SHARED_KERNEL_PMD)
+ return (pgd_t *)__get_free_page(PGALLOC_GFP);
+
+ /*
+ * Now PAE kernel is not running as a Xen domain. We can allocate
+ * a 32-byte slab for pgd to save memory space.
+ */
+ return kmem_cache_alloc(pgd_cache, PGALLOC_GFP);
+}
+
+static inline void _pgd_free(pgd_t *pgd)
+{
+ if (!SHARED_KERNEL_PMD)
+ free_page((unsigned long)pgd);
+ else
+ kmem_cache_free(pgd_cache, pgd);
+}
+#else
+static inline pgd_t *_pgd_alloc(void)
+{
+ return (pgd_t *)__get_free_page(PGALLOC_GFP);
+}
+
+static inline void _pgd_free(pgd_t *pgd)
+{
+ free_page((unsigned long)pgd);
+}
+#endif /* CONFIG_X86_PAE */
+
pgd_t *pgd_alloc(struct mm_struct *mm)
{
pgd_t *pgd;
pmd_t *pmds[PREALLOCATED_PMDS];
- pgd = (pgd_t *)__get_free_page(PGALLOC_GFP);
+ pgd = _pgd_alloc();
if (pgd == NULL)
goto out;
out_free_pmds:
free_pmds(mm, pmds);
out_free_pgd:
- free_page((unsigned long)pgd);
+ _pgd_free(pgd);
out:
return NULL;
}
pgd_mop_up_pmds(mm, pgd);
pgd_dtor(pgd);
paravirt_pgd_free(mm, pgd);
- free_page((unsigned long)pgd);
+ _pgd_free(pgd);
}
/*
{
struct stack_frame *head = (struct stack_frame *)frame_pointer(regs);
- if (!user_mode_vm(regs)) {
+ if (!user_mode(regs)) {
unsigned long stack = kernel_stack_pointer(regs);
if (depth)
dump_trace(NULL, regs, (unsigned long *)stack, 0,
struct list_head *list)
{
int ret;
- struct resource_entry *entry;
+ struct resource_entry *entry, *tmp;
sprintf(info->name, "PCI Bus %04x:%02x", domain, busnum);
info->bridge = device;
dev_dbg(&device->dev,
"no IO and memory resources present in _CRS\n");
else
- resource_list_for_each_entry(entry, list)
- entry->res->name = info->name;
+ resource_list_for_each_entry_safe(entry, tmp, list) {
+ if ((entry->res->flags & IORESOURCE_WINDOW) == 0 ||
+ (entry->res->flags & IORESOURCE_DISABLED))
+ resource_list_destroy_entry(entry);
+ else
+ entry->res->name = info->name;
+ }
}
struct pci_bus *pci_acpi_scan_root(struct acpi_pci_root *root)
if (!bus) {
pci_free_resource_list(&resources);
kfree(sd);
+ return;
}
+ pci_bus_add_devices(bus);
}
void __init pcibios_set_cache_line_size(void)
}
}
-/*
- * Some device drivers assume dev->irq won't change after calling
- * pci_disable_device(). So delay releasing of IRQ resource to driver
- * unbinding time. Otherwise it will break PM subsystem and drivers
- * like xen-pciback etc.
- */
-static int pci_irq_notifier(struct notifier_block *nb, unsigned long action,
- void *data)
-{
- struct pci_dev *dev = to_pci_dev(data);
-
- if (action != BUS_NOTIFY_UNBOUND_DRIVER)
- return NOTIFY_DONE;
-
- if (pcibios_disable_irq)
- pcibios_disable_irq(dev);
-
- return NOTIFY_OK;
-}
-
-static struct notifier_block pci_irq_nb = {
- .notifier_call = pci_irq_notifier,
- .priority = INT_MIN,
-};
-
int __init pcibios_init(void)
{
if (!raw_pci_ops) {
if (pci_bf_sort >= pci_force_bf)
pci_sort_breadthfirst();
-
- bus_register_notifier(&pci_bus_type, &pci_irq_nb);
-
return 0;
}
return 0;
}
+void pcibios_disable_device (struct pci_dev *dev)
+{
+ if (!pci_dev_msi_enabled(dev) && pcibios_disable_irq)
+ pcibios_disable_irq(dev);
+}
+
int pci_ext_cfg_avail(void)
{
if (raw_pci_ext_ops)
static void intel_mid_pci_irq_disable(struct pci_dev *dev)
{
- if (dev->irq_managed && dev->irq > 0) {
+ if (!mp_should_keep_irq(&dev->dev) && dev->irq_managed &&
+ dev->irq > 0) {
mp_unmap_irq(dev->irq);
dev->irq_managed = 0;
- dev->irq = 0;
}
}
return 0;
}
+bool mp_should_keep_irq(struct device *dev)
+{
+ if (dev->power.is_prepared)
+ return true;
+#ifdef CONFIG_PM
+ if (dev->power.runtime_status == RPM_SUSPENDING)
+ return true;
+#endif
+
+ return false;
+}
+
static void pirq_disable_irq(struct pci_dev *dev)
{
- if (io_apic_assign_pci_irqs && dev->irq_managed && dev->irq) {
+ if (io_apic_assign_pci_irqs && !mp_should_keep_irq(&dev->dev) &&
+ dev->irq_managed && dev->irq) {
mp_unmap_irq(dev->irq);
dev->irq = 0;
dev->irq_managed = 0;
image = efi_lookup_mapped_addr(bgrt_tab->image_address);
if (!image) {
- image = early_memremap(bgrt_tab->image_address,
+ image = early_ioremap(bgrt_tab->image_address,
sizeof(bmp_header));
ioremapped = true;
if (!image) {
}
if (ioremapped) {
- image = early_memremap(bgrt_tab->image_address,
+ image = early_ioremap(bgrt_tab->image_address,
bmp_header.size);
if (!image) {
pr_err("Ignoring BGRT: failed to map image memory\n");
efi_memory_desc_t *virtual_map)
{
efi_status_t status;
+ unsigned long flags;
+ pgd_t *save_pgd;
- efi_call_phys_prolog();
+ save_pgd = efi_call_phys_prolog();
+
+ /* Disable interrupts around EFI calls: */
+ local_irq_save(flags);
status = efi_call_phys(efi_phys.set_virtual_address_map,
memory_map_size, descriptor_size,
descriptor_version, virtual_map);
- efi_call_phys_epilog();
+ local_irq_restore(flags);
+
+ efi_call_phys_epilog(save_pgd);
+
return status;
}
if (efi_memmap_init())
return;
- print_efi_memmap();
+ if (efi_enabled(EFI_DBG))
+ print_efi_memmap();
}
void __init efi_late_init(void)
{
if (parse_option_str(str, "old_map"))
set_bit(EFI_OLD_MEMMAP, &efi.flags);
+ if (parse_option_str(str, "debug"))
+ set_bit(EFI_DBG, &efi.flags);
return 0;
}
/*
* To make EFI call EFI runtime service in physical addressing mode we need
- * prolog/epilog before/after the invocation to disable interrupt, to
- * claim EFI runtime service handler exclusively and to duplicate a memory in
- * low memory space say 0 - 3G.
+ * prolog/epilog before/after the invocation to claim the EFI runtime service
+ * handler exclusively and to duplicate a memory mapping in low memory space,
+ * say 0 - 3G.
*/
-static unsigned long efi_rt_eflags;
void efi_sync_low_kernel_mappings(void) {}
void __init efi_dump_pagetable(void) {}
void __init efi_map_region_fixed(efi_memory_desc_t *md) {}
void __init parse_efi_setup(u64 phys_addr, u32 data_len) {}
-void __init efi_call_phys_prolog(void)
+pgd_t * __init efi_call_phys_prolog(void)
{
struct desc_ptr gdt_descr;
+ pgd_t *save_pgd;
- local_irq_save(efi_rt_eflags);
-
+ /* Current pgd is swapper_pg_dir, we'll restore it later: */
+ save_pgd = swapper_pg_dir;
load_cr3(initial_page_table);
__flush_tlb_all();
gdt_descr.address = __pa(get_cpu_gdt_table(0));
gdt_descr.size = GDT_SIZE - 1;
load_gdt(&gdt_descr);
+
+ return save_pgd;
}
-void __init efi_call_phys_epilog(void)
+void __init efi_call_phys_epilog(pgd_t *save_pgd)
{
struct desc_ptr gdt_descr;
gdt_descr.size = GDT_SIZE - 1;
load_gdt(&gdt_descr);
- load_cr3(swapper_pg_dir);
+ load_cr3(save_pgd);
__flush_tlb_all();
-
- local_irq_restore(efi_rt_eflags);
}
void __init efi_runtime_mkexec(void)
#include <asm/realmode.h>
#include <asm/time.h>
-static pgd_t *save_pgd __initdata;
-static unsigned long efi_flags __initdata;
-
/*
* We allocate runtime services regions bottom-up, starting from -4G, i.e.
* 0xffff_ffff_0000_0000 and limit EFI VA mapping space to 64G.
}
}
-void __init efi_call_phys_prolog(void)
+pgd_t * __init efi_call_phys_prolog(void)
{
unsigned long vaddress;
+ pgd_t *save_pgd;
+
int pgd;
int n_pgds;
if (!efi_enabled(EFI_OLD_MEMMAP))
- return;
+ return NULL;
early_code_mapping_set_exec(1);
- local_irq_save(efi_flags);
n_pgds = DIV_ROUND_UP((max_pfn << PAGE_SHIFT), PGDIR_SIZE);
save_pgd = kmalloc(n_pgds * sizeof(pgd_t), GFP_KERNEL);
set_pgd(pgd_offset_k(pgd * PGDIR_SIZE), *pgd_offset_k(vaddress));
}
__flush_tlb_all();
+
+ return save_pgd;
}
-void __init efi_call_phys_epilog(void)
+void __init efi_call_phys_epilog(pgd_t *save_pgd)
{
/*
* After the lock is released, the original page table is restored.
*/
- int pgd;
- int n_pgds = DIV_ROUND_UP((max_pfn << PAGE_SHIFT) , PGDIR_SIZE);
+ int pgd_idx;
+ int nr_pgds;
- if (!efi_enabled(EFI_OLD_MEMMAP))
+ if (!save_pgd)
return;
- for (pgd = 0; pgd < n_pgds; pgd++)
- set_pgd(pgd_offset_k(pgd * PGDIR_SIZE), save_pgd[pgd]);
+ nr_pgds = DIV_ROUND_UP((max_pfn << PAGE_SHIFT) , PGDIR_SIZE);
+
+ for (pgd_idx = 0; pgd_idx < nr_pgds; pgd_idx++)
+ set_pgd(pgd_offset_k(pgd_idx * PGDIR_SIZE), save_pgd[pgd_idx]);
+
kfree(save_pgd);
+
__flush_tlb_all();
- local_irq_restore(efi_flags);
early_code_mapping_set_exec(0);
}
intel_mid_ops = get_intel_mid_ops[__intel_mid_cpu_chip]();
else {
intel_mid_ops = get_intel_mid_ops[INTEL_MID_CPU_CHIP_PENWELL]();
- pr_info("ARCH: Uknown SoC, assuming PENWELL!\n");
+ pr_info("ARCH: Unknown SoC, assuming PENWELL!\n");
}
out:
*/
#include <asm-generic/sections.h>
+#include <asm/cpu_device_id.h>
#include <asm/imr.h>
#include <linux/init.h>
#include <linux/mm.h>
}
}
+static const struct x86_cpu_id imr_ids[] __initconst = {
+ { X86_VENDOR_INTEL, 5, 9 }, /* Intel Quark SoC X1000. */
+ {}
+};
+MODULE_DEVICE_TABLE(x86cpu, imr_ids);
+
/**
* imr_self_test_init - entry point for IMR driver.
*
*/
static int __init imr_self_test_init(void)
{
- imr_self_test();
+ if (x86_match_cpu(imr_ids))
+ imr_self_test();
return 0;
}
if (psy) {
power_supply_changed(psy);
- put_device(psy->dev);
+ power_supply_put(psy);
}
}
if (psy) {
power_supply_changed(psy);
- put_device(psy->dev);
+ power_supply_put(psy);
}
}
if (psy) {
power_supply_changed(psy);
- put_device(psy->dev);
+ power_supply_put(psy);
}
}
if (psy) {
power_supply_changed(psy);
- put_device(psy->dev);
+ power_supply_put(psy);
}
}
struct reset_args reset_args;
reset_args.sender = sender;
- cpus_clear(*mask);
+ cpumask_clear(mask);
/* find a single cpu for each uvhub in this distribution mask */
maskbits = sizeof(struct pnmask) * BITSPERBYTE;
/* each bit is a pnode relative to the partition base pnode */
continue;
apnode = pnode + bcp->partition_base_pnode;
cpu = pnode_to_first_cpu(apnode, smaster);
- cpu_set(cpu, *mask);
+ cpumask_set_cpu(cpu, mask);
}
/* IPI all cpus; preemption is already disabled */
/* don't actually do a shootdown of the local cpu */
cpumask_andnot(flush_mask, cpumask, cpumask_of(cpu));
- if (cpu_isset(cpu, *cpumask))
+ if (cpumask_test_cpu(cpu, cpumask))
stat->s_ntargself++;
bau_desc = bcp->descriptor_base;
static void fix_processor_context(void)
{
int cpu = smp_processor_id();
- struct tss_struct *t = &per_cpu(init_tss, cpu);
+ struct tss_struct *t = &per_cpu(cpu_tss, cpu);
#ifdef CONFIG_X86_64
struct desc_struct *desc = get_cpu_gdt_table(cpu);
tss_desc tss;
110 i386 iopl sys_iopl
111 i386 vhangup sys_vhangup
112 i386 idle
-113 i386 vm86old sys_vm86old sys32_vm86_warning
+113 i386 vm86old sys_vm86old sys_ni_syscall
114 i386 wait4 sys_wait4 compat_sys_wait4
115 i386 swapoff sys_swapoff
116 i386 sysinfo sys_sysinfo compat_sys_sysinfo
163 i386 mremap sys_mremap
164 i386 setresuid sys_setresuid16
165 i386 getresuid sys_getresuid16
-166 i386 vm86 sys_vm86 sys32_vm86_warning
+166 i386 vm86 sys_vm86 sys_ni_syscall
167 i386 query_module
168 i386 poll sys_poll
169 i386 nfsservctl
169 common reboot sys_reboot
170 common sethostname sys_sethostname
171 common setdomainname sys_setdomainname
-172 common iopl stub_iopl
+172 common iopl sys_iopl
173 common ioperm sys_ioperm
174 64 create_module
175 common init_module sys_init_module
*/
static inline void rdtsc_barrier(void)
{
- alternative(ASM_NOP3, "mfence", X86_FEATURE_MFENCE_RDTSC);
- alternative(ASM_NOP3, "lfence", X86_FEATURE_LFENCE_RDTSC);
+ alternative_2("", "mfence", X86_FEATURE_MFENCE_RDTSC,
+ "lfence", X86_FEATURE_LFENCE_RDTSC);
}
#endif
*/
/* Not going to be implemented by UML, since we have no hardware. */
-#define stub_iopl sys_ni_syscall
+#define sys_iopl sys_ni_syscall
#define sys_ioperm sys_ni_syscall
/*
$(obj)/vdso64.so.dbg: $(src)/vdso.lds $(vobjs) FORCE
$(call if_changed,vdso)
-HOST_EXTRACFLAGS += -I$(srctree)/tools/include
+HOST_EXTRACFLAGS += -I$(srctree)/tools/include -I$(srctree)/include/uapi
hostprogs-y += vdso2c
quiet_cmd_vdso2c = VDSO2C $@
PHONY += vdso_install $(vdso_img_insttargets)
vdso_install: $(vdso_img_insttargets) FORCE
-clean-files := vdso32-syscall* vdso32-sysenter* vdso32-int80* vdso64*
+clean-files := vdso32-syscall* vdso32-sysenter* vdso32-int80* vdso64* vdso-image-*.c vdsox32.so*
cycle_t ret;
u64 last;
u32 version;
+ u32 migrate_count;
u8 flags;
unsigned cpu, cpu1;
/*
- * Note: hypervisor must guarantee that:
- * 1. cpu ID number maps 1:1 to per-CPU pvclock time info.
- * 2. that per-CPU pvclock time info is updated if the
- * underlying CPU changes.
- * 3. that version is increased whenever underlying CPU
- * changes.
- *
+ * When looping to get a consistent (time-info, tsc) pair, we
+ * also need to deal with the possibility we can switch vcpus,
+ * so make sure we always re-fetch time-info for the current vcpu.
*/
do {
cpu = __getcpu() & VGETCPU_CPU_MASK;
* __getcpu() calls (Gleb).
*/
- pvti = get_pvti(cpu);
+ /* Make sure migrate_count will change if we leave the VCPU. */
+ do {
+ pvti = get_pvti(cpu);
+ migrate_count = pvti->migrate_count;
+
+ cpu1 = cpu;
+ cpu = __getcpu() & VGETCPU_CPU_MASK;
+ } while (unlikely(cpu != cpu1));
version = __pvclock_read_cycles(&pvti->pvti, &ret, &flags);
/*
* Test we're still on the cpu as well as the version.
- * We could have been migrated just after the first
- * vgetcpu but before fetching the version, so we
- * wouldn't notice a version change.
+ * - We must read TSC of pvti's VCPU.
+ * - KVM doesn't follow the versioning protocol, so data could
+ * change before version if we left the VCPU.
*/
- cpu1 = __getcpu() & VGETCPU_CPU_MASK;
- } while (unlikely(cpu != cpu1 ||
- (pvti->pvti.version & 1) ||
- pvti->pvti.version != version));
+ smp_rmb();
+ } while (unlikely((pvti->pvti.version & 1) ||
+ pvti->pvti.version != version ||
+ pvti->migrate_count != migrate_count));
if (unlikely(!(flags & PVCLOCK_TSC_STABLE_BIT)))
*mode = VCLOCK_NONE;
.text
.globl __kernel_sigreturn
.type __kernel_sigreturn,@function
+ nop /* this guy is needed for .LSTARTFDEDLSI1 below (watch for HACK) */
ALIGN
__kernel_sigreturn:
.LSTART_sigreturn:
.Lpush_ebp:
movl %ecx, %ebp
syscall
- movl $__USER32_DS, %ecx
- movl %ecx, %ss
movl %ebp, %ecx
popl %ebp
.Lpop_ebp:
mcs = xen_mc_entry(0);
MULTI_stack_switch(mcs.mc, __KERNEL_DS, thread->sp0);
xen_mc_issue(PARAVIRT_LAZY_CPU);
+ tss->x86_tss.sp0 = thread->sp0;
}
static void xen_set_iopl_mask(unsigned mask)
BUG_ON(val);
}
#endif
+
+static u64 xen_read_msr_safe(unsigned int msr, int *err)
+{
+ u64 val;
+
+ val = native_read_msr_safe(msr, err);
+ switch (msr) {
+ case MSR_IA32_APICBASE:
+#ifdef CONFIG_X86_X2APIC
+ if (!(cpuid_ecx(1) & (1 << (X86_FEATURE_X2APIC & 31))))
+#endif
+ val &= ~X2APIC_ENABLE;
+ break;
+ }
+ return val;
+}
+
static int xen_write_msr_safe(unsigned int msr, unsigned low, unsigned high)
{
int ret;
.wbinvd = native_wbinvd,
- .read_msr = native_read_msr_safe,
+ .read_msr = xen_read_msr_safe,
.write_msr = xen_write_msr_safe,
.read_tsc = native_read_tsc,
#ifdef CONFIG_X86_32
i386_start_kernel();
#else
+ cr4_init_shadow(); /* 32b kernel does this in i386_start_kernel() */
x86_64_start_reservations((char *)__pa_symbol(&boot_params));
#endif
}
unsigned long xen_max_p2m_pfn __read_mostly;
EXPORT_SYMBOL_GPL(xen_max_p2m_pfn);
+#ifdef CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
+#define P2M_LIMIT CONFIG_XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
+#else
+#define P2M_LIMIT 0
+#endif
+
static DEFINE_SPINLOCK(p2m_update_lock);
static unsigned long *p2m_mid_missing_mfn;
void __init xen_vmalloc_p2m_tree(void)
{
static struct vm_struct vm;
+ unsigned long p2m_limit;
+ p2m_limit = (phys_addr_t)P2M_LIMIT * 1024 * 1024 * 1024 / PAGE_SIZE;
vm.flags = VM_ALLOC;
- vm.size = ALIGN(sizeof(unsigned long) * xen_max_p2m_pfn,
+ vm.size = ALIGN(sizeof(unsigned long) * max(xen_max_p2m_pfn, p2m_limit),
PMD_SIZE * PMDS_PER_MID_PAGE);
vm_area_register_early(&vm, PMD_SIZE * PMDS_PER_MID_PAGE);
pr_notice("p2m virtual area at %p, size is %lx\n", vm.addr, vm.size);
if (p2m_pfn == PFN_DOWN(__pa(p2m_missing)))
p2m_init(p2m);
else
- p2m_init_identity(p2m, pfn);
+ p2m_init_identity(p2m, pfn & ~(P2M_PER_PAGE - 1));
spin_lock_irqsave(&p2m_update_lock, flags);
{
int rc;
- per_cpu(current_task, cpu) = idle;
-#ifdef CONFIG_X86_32
- irq_ctx_init(cpu);
-#else
- clear_tsk_thread_flag(idle, TIF_FORK);
-#endif
- per_cpu(kernel_stack, cpu) =
- (unsigned long)task_stack_page(idle) -
- KERNEL_STACK_OFFSET + THREAD_SIZE;
+ common_cpu_up(cpu, idle);
xen_setup_runstate_info(cpu);
xen_setup_timer(cpu);
if (rc)
return rc;
- if (num_online_cpus() == 1)
- /* Just in case we booted with a single CPU. */
- alternatives_enable_smp();
-
rc = xen_smp_intr_init(cpu);
if (rc)
return rc;
#include <linux/types.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <xen/interface/xen.h>
#include <xen/grant_table.h>
static void xen_vcpu_notify_restore(void *data)
{
- unsigned long reason = (unsigned long)data;
-
/* Boot processor notified via generic timekeeping_resume() */
- if ( smp_processor_id() == 0)
+ if (smp_processor_id() == 0)
return;
- clockevents_notify(reason, NULL);
+ tick_resume_local();
}
void xen_arch_resume(void)
{
- on_each_cpu(xen_vcpu_notify_restore,
- (void *)CLOCK_EVT_NOTIFY_RESUME, 1);
+ on_each_cpu(xen_vcpu_notify_restore, NULL, 1);
}
* We're already on the usermode stack at this point, but
* still with the kernel gs, so we can easily switch back
*/
- movq %rsp, PER_CPU_VAR(old_rsp)
+ movq %rsp, PER_CPU_VAR(rsp_scratch)
movq PER_CPU_VAR(kernel_stack), %rsp
pushq $__USER_DS
- pushq PER_CPU_VAR(old_rsp)
+ pushq PER_CPU_VAR(rsp_scratch)
pushq %r11
pushq $__USER_CS
pushq %rcx
* We're already on the usermode stack at this point, but
* still with the kernel gs, so we can easily switch back
*/
- movq %rsp, PER_CPU_VAR(old_rsp)
+ movq %rsp, PER_CPU_VAR(rsp_scratch)
movq PER_CPU_VAR(kernel_stack), %rsp
pushq $__USER32_DS
- pushq PER_CPU_VAR(old_rsp)
+ pushq PER_CPU_VAR(rsp_scratch)
pushq %r11
pushq $__USER32_CS
pushq %rcx
struct pci_controller *pci_ctrl;
struct list_head resources;
struct pci_bus *bus;
- int next_busno = 0;
+ int next_busno = 0, ret;
printk("PCI: Probing PCI hardware\n");
pci_controller_apertures(pci_ctrl, &resources);
bus = pci_scan_root_bus(NULL, pci_ctrl->first_busno,
pci_ctrl->ops, pci_ctrl, &resources);
+ if (!bus)
+ continue;
+
pci_ctrl->bus = bus;
pci_ctrl->last_busno = bus->busn_res.end;
if (next_busno <= pci_ctrl->last_busno)
next_busno = pci_ctrl->last_busno+1;
}
pci_bus_count = next_busno;
+ ret = platform_pcibios_fixup();
+ if (ret)
+ return ret;
- return platform_pcibios_fixup();
+ for (pci_ctrl = pci_ctrl_head; pci_ctrl; pci_ctrl = pci_ctrl->next) {
+ if (pci_ctrl->bus)
+ pci_bus_add_devices(pci_ctrl->bus);
+ }
+
+ return 0;
}
subsys_initcall(pcibios_init);
if (q->queue_flags & (1 << QUEUE_FLAG_SG_GAPS)) {
struct bio_vec *bprev;
- bprev = &rq->biotail->bi_io_vec[bio->bi_vcnt - 1];
+ bprev = &rq->biotail->bi_io_vec[rq->biotail->bi_vcnt - 1];
if (bvec_gap_to_prev(bprev, bio->bi_io_vec[0].bv_offset))
return false;
}
/*
* We're out of tags on this hardware queue, kick any
* pending IO submits before going to sleep waiting for
- * some to complete.
+ * some to complete. Note that hctx can be NULL here for
+ * reserved tag allocation.
*/
- blk_mq_run_hw_queue(hctx, false);
+ if (hctx)
+ blk_mq_run_hw_queue(hctx, false);
/*
* Retry tag allocation after running the hardware queue,
do {
page = alloc_pages_node(set->numa_node,
- GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY,
+ GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY | __GFP_ZERO,
this_order);
if (page)
break;
left -= to_do * rq_size;
for (j = 0; j < to_do; j++) {
tags->rqs[i] = p;
- tags->rqs[i]->atomic_flags = 0;
- tags->rqs[i]->cmd_flags = 0;
if (set->ops->init_request) {
if (set->ops->init_request(set->driver_data,
tags->rqs[i], hctx_idx, i,
*/
if (percpu_ref_init(&q->mq_usage_counter, blk_mq_usage_counter_release,
PERCPU_REF_INIT_ATOMIC, GFP_KERNEL))
- goto err_map;
+ goto err_mq_usage;
setup_timer(&q->timeout, blk_mq_rq_timer, (unsigned long) q);
blk_queue_rq_timeout(q, 30000);
blk_mq_init_cpu_queues(q, set->nr_hw_queues);
if (blk_mq_init_hw_queues(q, set))
- goto err_hw;
+ goto err_mq_usage;
mutex_lock(&all_q_mutex);
list_add_tail(&q->all_q_node, &all_q_list);
return q;
-err_hw:
+err_mq_usage:
blk_cleanup_queue(q);
err_hctxs:
kfree(map);
b->physical_block_size);
t->io_min = max(t->io_min, b->io_min);
- t->io_opt = lcm(t->io_opt, b->io_opt);
+ t->io_opt = lcm_not_zero(t->io_opt, b->io_opt);
t->cluster &= b->cluster;
t->discard_zeroes_data &= b->discard_zeroes_data;
b->raid_partial_stripes_expensive);
/* Find lowest common alignment_offset */
- t->alignment_offset = lcm(t->alignment_offset, alignment)
+ t->alignment_offset = lcm_not_zero(t->alignment_offset, alignment)
% max(t->physical_block_size, t->io_min);
/* Verify that new alignment_offset is on a logical block boundary */
b->max_discard_sectors);
t->discard_granularity = max(t->discard_granularity,
b->discard_granularity);
- t->discard_alignment = lcm(t->discard_alignment, alignment) %
+ t->discard_alignment = lcm_not_zero(t->discard_alignment, alignment) %
t->discard_granularity;
}
};
struct acpi_ac {
- struct power_supply charger;
+ struct power_supply *charger;
+ struct power_supply_desc charger_desc;
struct acpi_device * device;
unsigned long long state;
struct notifier_block battery_nb;
};
-#define to_acpi_ac(x) container_of(x, struct acpi_ac, charger)
+#define to_acpi_ac(x) power_supply_get_drvdata(x)
#ifdef CONFIG_ACPI_PROCFS_POWER
static const struct file_operations acpi_ac_fops = {
dev_name(&device->dev), event,
(u32) ac->state);
acpi_notifier_call_chain(device, event, (u32) ac->state);
- kobject_uevent(&ac->charger.dev->kobj, KOBJ_CHANGE);
+ kobject_uevent(&ac->charger->dev.kobj, KOBJ_CHANGE);
}
return;
static int acpi_ac_add(struct acpi_device *device)
{
+ struct power_supply_config psy_cfg = {};
int result = 0;
struct acpi_ac *ac = NULL;
if (result)
goto end;
- ac->charger.name = acpi_device_bid(device);
+ psy_cfg.drv_data = ac;
+
+ ac->charger_desc.name = acpi_device_bid(device);
#ifdef CONFIG_ACPI_PROCFS_POWER
result = acpi_ac_add_fs(ac);
if (result)
goto end;
#endif
- ac->charger.type = POWER_SUPPLY_TYPE_MAINS;
- ac->charger.properties = ac_props;
- ac->charger.num_properties = ARRAY_SIZE(ac_props);
- ac->charger.get_property = get_ac_property;
- result = power_supply_register(&ac->device->dev, &ac->charger);
- if (result)
+ ac->charger_desc.type = POWER_SUPPLY_TYPE_MAINS;
+ ac->charger_desc.properties = ac_props;
+ ac->charger_desc.num_properties = ARRAY_SIZE(ac_props);
+ ac->charger_desc.get_property = get_ac_property;
+ ac->charger = power_supply_register(&ac->device->dev,
+ &ac->charger_desc, &psy_cfg);
+ if (IS_ERR(ac->charger)) {
+ result = PTR_ERR(ac->charger);
goto end;
+ }
printk(KERN_INFO PREFIX "%s [%s] (%s)\n",
acpi_device_name(device), acpi_device_bid(device),
if (acpi_ac_get_state(ac))
return 0;
if (old_state != ac->state)
- kobject_uevent(&ac->charger.dev->kobj, KOBJ_CHANGE);
+ kobject_uevent(&ac->charger->dev.kobj, KOBJ_CHANGE);
return 0;
}
#else
ac = acpi_driver_data(device);
- if (ac->charger.dev)
- power_supply_unregister(&ac->charger);
+ power_supply_unregister(ac->charger);
unregister_acpi_notifier(&ac->battery_nb);
#ifdef CONFIG_ACPI_PROCFS_POWER
struct lpss_device_desc {
unsigned int flags;
+ const char *clk_con_id;
unsigned int prv_offset;
size_t prv_size_override;
void (*setup)(struct lpss_private_data *pdata);
static struct lpss_device_desc lpt_uart_dev_desc = {
.flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_LTR,
+ .clk_con_id = "baudclk",
.prv_offset = 0x800,
.setup = lpss_uart_setup,
};
static struct lpss_device_desc byt_uart_dev_desc = {
.flags = LPSS_CLK | LPSS_CLK_GATE | LPSS_CLK_DIVIDER | LPSS_SAVE_CTX,
+ .clk_con_id = "baudclk",
.prv_offset = 0x800,
.setup = lpss_uart_setup,
};
return PTR_ERR(clk);
pdata->clk = clk;
- clk_register_clkdev(clk, NULL, devname);
+ clk_register_clkdev(clk, dev_desc->clk_con_id, devname);
return 0;
}
#include <linux/kthread.h>
#include <linux/freezer.h>
#include <linux/cpu.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/slab.h>
#include <linux/acpi.h>
#include <asm/mwait.h>
static unsigned char tsc_detected_unstable;
static unsigned char tsc_marked_unstable;
-static unsigned char lapic_detected_unstable;
-static unsigned char lapic_marked_unstable;
static void power_saving_mwait_init(void)
{
*/
if (!boot_cpu_has(X86_FEATURE_NONSTOP_TSC))
tsc_detected_unstable = 1;
- if (!boot_cpu_has(X86_FEATURE_ARAT))
- lapic_detected_unstable = 1;
break;
default:
- /* TSC & LAPIC could halt in idle */
+ /* TSC could halt in idle */
tsc_detected_unstable = 1;
- lapic_detected_unstable = 1;
}
#endif
}
sched_setscheduler(current, SCHED_RR, ¶m);
while (!kthread_should_stop()) {
- int cpu;
unsigned long expire_time;
try_to_freeze();
mark_tsc_unstable("TSC halts in idle");
tsc_marked_unstable = 1;
}
- if (lapic_detected_unstable && !lapic_marked_unstable) {
- int i;
- /* LAPIC could halt in idle, so notify users */
- for_each_online_cpu(i)
- clockevents_notify(
- CLOCK_EVT_NOTIFY_BROADCAST_ON,
- &i);
- lapic_marked_unstable = 1;
- }
local_irq_disable();
- cpu = smp_processor_id();
- if (lapic_marked_unstable)
- clockevents_notify(
- CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
+ tick_broadcast_enable();
+ tick_broadcast_enter();
stop_critical_timings();
mwait_idle_with_hints(power_saving_mwait_eax, 1);
start_critical_timings();
- if (lapic_marked_unstable)
- clockevents_notify(
- CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
+ tick_broadcast_exit();
local_irq_enable();
if (time_before(expire_time, jiffies)) {
struct acpi_battery {
struct mutex lock;
struct mutex sysfs_lock;
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct acpi_device *device;
struct notifier_block pm_nb;
unsigned long update_time;
unsigned long flags;
};
-#define to_acpi_battery(x) container_of(x, struct acpi_battery, bat)
+#define to_acpi_battery(x) power_supply_get_drvdata(x)
static inline int acpi_battery_present(struct acpi_battery *battery)
{
static int sysfs_add_battery(struct acpi_battery *battery)
{
- int result;
+ struct power_supply_config psy_cfg = { .drv_data = battery, };
if (battery->power_unit == ACPI_BATTERY_POWER_UNIT_MA) {
- battery->bat.properties = charge_battery_props;
- battery->bat.num_properties =
+ battery->bat_desc.properties = charge_battery_props;
+ battery->bat_desc.num_properties =
ARRAY_SIZE(charge_battery_props);
} else {
- battery->bat.properties = energy_battery_props;
- battery->bat.num_properties =
+ battery->bat_desc.properties = energy_battery_props;
+ battery->bat_desc.num_properties =
ARRAY_SIZE(energy_battery_props);
}
- battery->bat.name = acpi_device_bid(battery->device);
- battery->bat.type = POWER_SUPPLY_TYPE_BATTERY;
- battery->bat.get_property = acpi_battery_get_property;
+ battery->bat_desc.name = acpi_device_bid(battery->device);
+ battery->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ battery->bat_desc.get_property = acpi_battery_get_property;
- result = power_supply_register_no_ws(&battery->device->dev, &battery->bat);
+ battery->bat = power_supply_register_no_ws(&battery->device->dev,
+ &battery->bat_desc, &psy_cfg);
- if (result)
+ if (IS_ERR(battery->bat)) {
+ int result = PTR_ERR(battery->bat);
+
+ battery->bat = NULL;
return result;
- return device_create_file(battery->bat.dev, &alarm_attr);
+ }
+ return device_create_file(&battery->bat->dev, &alarm_attr);
}
static void sysfs_remove_battery(struct acpi_battery *battery)
{
mutex_lock(&battery->sysfs_lock);
- if (!battery->bat.dev) {
+ if (!battery->bat) {
mutex_unlock(&battery->sysfs_lock);
return;
}
- device_remove_file(battery->bat.dev, &alarm_attr);
- power_supply_unregister(&battery->bat);
- battery->bat.dev = NULL;
+ device_remove_file(&battery->bat->dev, &alarm_attr);
+ power_supply_unregister(battery->bat);
+ battery->bat = NULL;
mutex_unlock(&battery->sysfs_lock);
}
return result;
acpi_battery_init_alarm(battery);
}
- if (!battery->bat.dev) {
+ if (!battery->bat) {
result = sysfs_add_battery(battery);
if (result)
return result;
{
int power_unit;
- if (!battery->bat.dev)
+ if (!battery->bat)
return;
power_unit = battery->power_unit;
static void acpi_battery_notify(struct acpi_device *device, u32 event)
{
struct acpi_battery *battery = acpi_driver_data(device);
- struct device *old;
+ struct power_supply *old;
if (!battery)
return;
- old = battery->bat.dev;
+ old = battery->bat;
/*
* On Acer Aspire V5-573G notifications are sometimes triggered too
* early. For example, when AC is unplugged and notification is
acpi_battery_present(battery));
acpi_notifier_call_chain(device, event, acpi_battery_present(battery));
/* acpi_battery_update could remove power_supply object */
- if (old && battery->bat.dev)
- power_supply_changed(&battery->bat);
+ if (old && battery->bat)
+ power_supply_changed(battery->bat);
}
static int battery_notify(struct notifier_block *nb,
if (!acpi_battery_present(battery))
return 0;
- if (!battery->bat.dev) {
+ if (battery->bat) {
result = acpi_battery_get_info(battery);
if (result)
return result;
if (!pin || !dev->irq_managed || dev->irq <= 0)
return;
+ /* Keep IOAPIC pin configuration when suspending */
+ if (dev->dev.power.is_prepared)
+ return;
+#ifdef CONFIG_PM
+ if (dev->dev.power.runtime_status == RPM_SUSPENDING)
+ return;
+#endif
+
entry = acpi_pci_irq_lookup(dev, pin);
if (!entry)
return;
if (gsi >= 0) {
acpi_unregister_gsi(gsi);
dev->irq_managed = 0;
- dev->irq = 0;
}
}
}
EXPORT_SYMBOL(acpi_pci_osc_control_set);
-static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm,
- int *clear_aspm)
+static void negotiate_os_control(struct acpi_pci_root *root, int *no_aspm)
{
u32 support, control, requested;
acpi_status status;
decode_osc_control(root, "OS now controls", control);
if (acpi_gbl_FADT.boot_flags & ACPI_FADT_NO_ASPM) {
/*
- * We have ASPM control, but the FADT indicates
- * that it's unsupported. Clear it.
+ * We have ASPM control, but the FADT indicates that
+ * it's unsupported. Leave existing configuration
+ * intact and prevent the OS from touching it.
*/
- *clear_aspm = 1;
+ dev_info(&device->dev, "FADT indicates ASPM is unsupported, using BIOS configuration\n");
+ *no_aspm = 1;
}
} else {
decode_osc_control(root, "OS requested", requested);
int result;
struct acpi_pci_root *root;
acpi_handle handle = device->handle;
- int no_aspm = 0, clear_aspm = 0;
+ int no_aspm = 0;
bool hotadd = system_state != SYSTEM_BOOTING;
root = kzalloc(sizeof(struct acpi_pci_root), GFP_KERNEL);
root->mcfg_addr = acpi_pci_root_get_mcfg_addr(handle);
- negotiate_os_control(root, &no_aspm, &clear_aspm);
+ negotiate_os_control(root, &no_aspm);
/*
* TBD: Need PCI interface for enumeration/configuration of roots.
goto remove_dmar;
}
- if (clear_aspm) {
- dev_info(&device->dev, "Disabling ASPM (FADT indicates it is unsupported)\n");
- pcie_clear_aspm(root->bus);
- }
if (no_aspm)
pcie_no_aspm();
#include <linux/acpi.h>
#include <linux/dmi.h>
#include <linux/sched.h> /* need_resched() */
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <linux/cpuidle.h>
#include <linux/syscore_ops.h>
#include <acpi/processor.h>
static void __lapic_timer_propagate_broadcast(void *arg)
{
struct acpi_processor *pr = (struct acpi_processor *) arg;
- unsigned long reason;
- reason = pr->power.timer_broadcast_on_state < INT_MAX ?
- CLOCK_EVT_NOTIFY_BROADCAST_ON : CLOCK_EVT_NOTIFY_BROADCAST_OFF;
-
- clockevents_notify(reason, &pr->id);
+ if (pr->power.timer_broadcast_on_state < INT_MAX)
+ tick_broadcast_enable();
+ else
+ tick_broadcast_disable();
}
static void lapic_timer_propagate_broadcast(struct acpi_processor *pr)
int state = cx - pr->power.states;
if (state >= pr->power.timer_broadcast_on_state) {
- unsigned long reason;
-
- reason = broadcast ? CLOCK_EVT_NOTIFY_BROADCAST_ENTER :
- CLOCK_EVT_NOTIFY_BROADCAST_EXIT;
- clockevents_notify(reason, &pr->id);
+ if (broadcast)
+ tick_broadcast_enter();
+ else
+ tick_broadcast_exit();
}
}
return -EINVAL;
drv->safe_state_index = -1;
- for (i = 0; i < CPUIDLE_STATE_MAX; i++) {
+ for (i = CPUIDLE_DRIVER_STATE_START; i < CPUIDLE_STATE_MAX; i++) {
drv->states[i].name[0] = '\0';
drv->states[i].desc[0] = '\0';
}
* CHECKME: len might be required to check versus a minimum
* length as well. 1 for io is fine, but for memory it does
* not make any sense at all.
+ * Note: some BIOSes report incorrect length for ACPI address space
+ * descriptor, so remove check of 'reslen == len' to avoid regression.
*/
- if (len && reslen && reslen == len && start <= end)
+ if (len && reslen && start <= end)
return true;
pr_debug("ACPI: invalid or unassigned resource %s [%016llx - %016llx] length [%016llx]\n",
MODULE_DEVICE_TABLE(acpi, sbs_device_ids);
struct acpi_battery {
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct acpi_sbs *sbs;
unsigned long update_time;
char name[8];
u8 have_sysfs_alarm:1;
};
-#define to_acpi_battery(x) container_of(x, struct acpi_battery, bat)
+#define to_acpi_battery(x) power_supply_get_drvdata(x)
struct acpi_sbs {
- struct power_supply charger;
+ struct power_supply *charger;
struct acpi_device *device;
struct acpi_smb_hc *hc;
struct mutex lock;
u8 charger_exists:1;
};
-#define to_acpi_sbs(x) container_of(x, struct acpi_sbs, charger)
+#define to_acpi_sbs(x) power_supply_get_drvdata(x)
static int acpi_sbs_remove(struct acpi_device *device);
static int acpi_battery_get_state(struct acpi_battery *battery);
POWER_SUPPLY_PROP_MANUFACTURER,
};
+static const struct power_supply_desc acpi_sbs_charger_desc = {
+ .name = "sbs-charger",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = sbs_ac_props,
+ .num_properties = ARRAY_SIZE(sbs_ac_props),
+ .get_property = sbs_get_ac_property,
+};
/* --------------------------------------------------------------------------
Smart Battery System Management
static int acpi_battery_add(struct acpi_sbs *sbs, int id)
{
struct acpi_battery *battery = &sbs->battery[id];
+ struct power_supply_config psy_cfg = { .drv_data = battery, };
int result;
battery->id = id;
return result;
sprintf(battery->name, ACPI_BATTERY_DIR_NAME, id);
- battery->bat.name = battery->name;
- battery->bat.type = POWER_SUPPLY_TYPE_BATTERY;
+ battery->bat_desc.name = battery->name;
+ battery->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
if (!acpi_battery_mode(battery)) {
- battery->bat.properties = sbs_charge_battery_props;
- battery->bat.num_properties =
+ battery->bat_desc.properties = sbs_charge_battery_props;
+ battery->bat_desc.num_properties =
ARRAY_SIZE(sbs_charge_battery_props);
} else {
- battery->bat.properties = sbs_energy_battery_props;
- battery->bat.num_properties =
+ battery->bat_desc.properties = sbs_energy_battery_props;
+ battery->bat_desc.num_properties =
ARRAY_SIZE(sbs_energy_battery_props);
}
- battery->bat.get_property = acpi_sbs_battery_get_property;
- result = power_supply_register(&sbs->device->dev, &battery->bat);
- if (result)
+ battery->bat_desc.get_property = acpi_sbs_battery_get_property;
+ battery->bat = power_supply_register(&sbs->device->dev,
+ &battery->bat_desc, &psy_cfg);
+ if (IS_ERR(battery->bat)) {
+ result = PTR_ERR(battery->bat);
+ battery->bat = NULL;
goto end;
+ }
- result = device_create_file(battery->bat.dev, &alarm_attr);
+ result = device_create_file(&battery->bat->dev, &alarm_attr);
if (result)
goto end;
battery->have_sysfs_alarm = 1;
{
struct acpi_battery *battery = &sbs->battery[id];
- if (battery->bat.dev) {
+ if (battery->bat) {
if (battery->have_sysfs_alarm)
- device_remove_file(battery->bat.dev, &alarm_attr);
- power_supply_unregister(&battery->bat);
+ device_remove_file(&battery->bat->dev, &alarm_attr);
+ power_supply_unregister(battery->bat);
}
}
static int acpi_charger_add(struct acpi_sbs *sbs)
{
int result;
+ struct power_supply_config psy_cfg = { .drv_data = sbs, };
result = acpi_ac_get_present(sbs);
if (result)
goto end;
sbs->charger_exists = 1;
- sbs->charger.name = "sbs-charger";
- sbs->charger.type = POWER_SUPPLY_TYPE_MAINS;
- sbs->charger.properties = sbs_ac_props;
- sbs->charger.num_properties = ARRAY_SIZE(sbs_ac_props);
- sbs->charger.get_property = sbs_get_ac_property;
- power_supply_register(&sbs->device->dev, &sbs->charger);
+ sbs->charger = power_supply_register(&sbs->device->dev,
+ &acpi_sbs_charger_desc, &psy_cfg);
+ if (IS_ERR(sbs->charger)) {
+ result = PTR_ERR(sbs->charger);
+ sbs->charger = NULL;
+ }
printk(KERN_INFO PREFIX "%s [%s]: AC Adapter [%s] (%s)\n",
ACPI_SBS_DEVICE_NAME, acpi_device_bid(sbs->device),
ACPI_AC_DIR_NAME, sbs->charger_present ? "on-line" : "off-line");
static void acpi_charger_remove(struct acpi_sbs *sbs)
{
- if (sbs->charger.dev)
- power_supply_unregister(&sbs->charger);
+ if (sbs->charger)
+ power_supply_unregister(sbs->charger);
}
static void acpi_sbs_callback(void *context)
if (sbs->charger_exists) {
acpi_ac_get_present(sbs);
if (sbs->charger_present != saved_charger_state)
- kobject_uevent(&sbs->charger.dev->kobj, KOBJ_CHANGE);
+ kobject_uevent(&sbs->charger->dev.kobj, KOBJ_CHANGE);
}
if (sbs->manager_present) {
acpi_battery_read(bat);
if (saved_battery_state == bat->present)
continue;
- kobject_uevent(&bat->bat.dev->kobj, KOBJ_CHANGE);
+ kobject_uevent(&bat->bat->dev.kobj, KOBJ_CHANGE);
}
}
}
int acpi_video_register(void)
{
- int result = 0;
+ int ret;
+
if (register_count) {
/*
* if the function of acpi_video_register is already called,
mutex_init(&video_list_lock);
INIT_LIST_HEAD(&video_bus_head);
- result = acpi_bus_register_driver(&acpi_video_bus);
- if (result < 0)
- return -ENODEV;
+ ret = acpi_bus_register_driver(&acpi_video_bus);
+ if (ret)
+ return ret;
/*
* When the acpi_video_bus is loaded successfully, increase
static int __init acpi_video_init(void)
{
+ /*
+ * Let the module load even if ACPI is disabled (e.g. due to
+ * a broken BIOS) so that i915.ko can still be loaded on such
+ * old systems without an AcpiOpRegion.
+ *
+ * acpi_video_register() will report -ENODEV later as well due
+ * to acpi_disabled when i915.ko tries to register itself afterwards.
+ */
+ if (acpi_disabled)
+ return 0;
+
dmi_check_system(video_dmi_table);
if (intel_opregion_present())
{
void *page_addr;
unsigned long user_page_addr;
- struct vm_struct tmp_area;
struct page **page;
struct mm_struct *mm;
proc->pid, page_addr);
goto err_alloc_page_failed;
}
- tmp_area.addr = page_addr;
- tmp_area.size = PAGE_SIZE + PAGE_SIZE /* guard page? */;
- ret = map_vm_area(&tmp_area, PAGE_KERNEL, page);
- if (ret) {
+ ret = map_kernel_range_noflush((unsigned long)page_addr,
+ PAGE_SIZE, PAGE_KERNEL, page);
+ flush_cache_vmap((unsigned long)page_addr,
+ (unsigned long)page_addr + PAGE_SIZE);
+ if (ret != 1) {
pr_err("%d: binder_alloc_buf failed to map page at %p in kernel\n",
proc->pid, page_addr);
goto err_map_kernel_failed;
# Should be last libata driver
obj-$(CONFIG_PATA_LEGACY) += pata_legacy.o
-libata-y := libata-core.o libata-scsi.o libata-eh.o libata-transport.o
+libata-y := libata-core.o libata-scsi.o libata-eh.o \
+ libata-transport.o libata-trace.o
libata-$(CONFIG_ATA_SFF) += libata-sff.o
libata-$(CONFIG_SATA_PMP) += libata-pmp.o
libata-$(CONFIG_ATA_ACPI) += libata-acpi.o
int rc;
if (using_dac &&
- !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ !dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
if (rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"64-bit DMA enable failed\n");
}
}
} else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"32-bit consistent DMA enable failed\n");
return 0;
if (using_dac &&
- !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ !dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
if (rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"64-bit DMA enable failed\n");
}
}
} else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"32-bit consistent DMA enable failed\n");
}
}
- st_ahci_configure_oob(drv_data->hpriv->mmio);
-
if (drv_data->sw_rst) {
err = reset_control_deassert(drv_data->sw_rst);
if (err) {
if (err)
return err;
+ st_ahci_configure_oob(drv_data->hpriv->mmio);
+
err = ahci_platform_init_host(pdev, hpriv, &st_ahci_port_info,
&ahci_platform_sht);
if (err) {
return err;
}
+ st_ahci_configure_oob(drv_data->hpriv->mmio);
+
return ahci_platform_resume_host(dev);
}
#endif
* NOTE: PM support is not currently available.
*
*/
+#include <linux/acpi.h>
#include <linux/module.h>
#include <linux/platform_device.h>
#include <linux/ahci_platform.h>
return rc;
}
+#ifdef CONFIG_ACPI
+static const struct acpi_device_id xgene_ahci_acpi_match[] = {
+ { "APMC0D0D", },
+ { }
+};
+MODULE_DEVICE_TABLE(acpi, xgene_ahci_acpi_match);
+#endif
+
static const struct of_device_id xgene_ahci_of_match[] = {
{.compatible = "apm,xgene-ahci"},
{},
.driver = {
.name = DRV_NAME,
.of_match_table = xgene_ahci_of_match,
+ .acpi_match_table = ACPI_PTR(xgene_ahci_acpi_match),
},
};
#include <linux/pm_runtime.h>
#include <linux/platform_device.h>
+#define CREATE_TRACE_POINTS
+#include <trace/events/libata.h>
+
#include "libata.h"
#include "libata-transport.h"
* RETURNS:
* Block address read from @tf.
*/
-u64 ata_tf_read_block(struct ata_taskfile *tf, struct ata_device *dev)
+u64 ata_tf_read_block(const struct ata_taskfile *tf, struct ata_device *dev)
{
u64 block = 0;
- if (tf->flags & ATA_TFLAG_LBA) {
+ if (!dev || tf->flags & ATA_TFLAG_LBA) {
if (tf->flags & ATA_TFLAG_LBA48) {
block |= (u64)tf->hob_lbah << 40;
block |= (u64)tf->hob_lbam << 32;
return 0;
}
+static void ata_dev_config_sense_reporting(struct ata_device *dev)
+{
+ unsigned int err_mask;
+
+ if (!ata_id_has_sense_reporting(dev->id))
+ return;
+
+ if (ata_id_sense_reporting_enabled(dev->id))
+ return;
+
+ err_mask = ata_dev_set_feature(dev, SETFEATURE_SENSE_DATA, 0x1);
+ if (err_mask) {
+ ata_dev_dbg(dev,
+ "failed to enable Sense Data Reporting, Emask 0x%x\n",
+ err_mask);
+ }
+}
+
/**
* ata_dev_configure - Configure the specified ATA/ATAPI device
* @dev: Target device to configure
dev->devslp_timing[i] = sata_setting[j];
}
}
-
+ ata_dev_config_sense_reporting(dev);
dev->cdb_len = 16;
}
{ "PIONEER DVD-RW DVR-216D", NULL, ATA_HORKAGE_NOSETXFER },
/* devices that don't properly handle queued TRIM commands */
- { "Micron_M[56]*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ { "Micron_M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Crucial_CT*M500*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Micron_M5[15]0*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Crucial_CT*M550*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Crucial_CT*MX100*", "MU01", ATA_HORKAGE_NO_NCQ_TRIM |
+ ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Samsung SSD 850 PRO*", NULL, ATA_HORKAGE_NO_NCQ_TRIM |
ATA_HORKAGE_ZERO_AFTER_TRIM, },
- { "Crucial_CT*SSD*", NULL, ATA_HORKAGE_NO_NCQ_TRIM, },
/*
* As defined, the DRAT (Deterministic Read After Trim) and RZAT
*/
{ "INTEL*SSDSC2MH*", NULL, 0, },
+ { "Micron*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM, },
+ { "Crucial*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM, },
{ "INTEL*SSD*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM, },
{ "SSD*INTEL*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM, },
{ "Samsung*SSD*", NULL, ATA_HORKAGE_ZERO_AFTER_TRIM, },
return NULL;
/* libsas case */
- if (!ap->scsi_host) {
+ if (ap->flags & ATA_FLAG_SAS_HOST) {
tag = ata_sas_allocate_tag(ap);
if (tag < 0)
return NULL;
tag = qc->tag;
if (likely(ata_tag_valid(tag))) {
qc->tag = ATA_TAG_POISON;
- if (!ap->scsi_host)
+ if (ap->flags & ATA_FLAG_SAS_HOST)
ata_sas_free_tag(tag, ap);
}
}
*/
if (unlikely(ata_tag_internal(qc->tag))) {
fill_result_tf(qc);
+ trace_ata_qc_complete_internal(qc);
__ata_qc_complete(qc);
return;
}
*/
if (unlikely(qc->flags & ATA_QCFLAG_FAILED)) {
fill_result_tf(qc);
+ trace_ata_qc_complete_failed(qc);
ata_qc_schedule_eh(qc);
return;
}
if (qc->flags & ATA_QCFLAG_RESULT_TF)
fill_result_tf(qc);
+ trace_ata_qc_complete_done(qc);
/* Some commands need post-processing after successful
* completion.
*/
}
ap->ops->qc_prep(qc);
-
+ trace_ata_qc_issue(qc);
qc->err_mask |= ap->ops->qc_issue(qc);
if (unlikely(qc->err_mask))
goto err;
#include <linux/libata.h>
+#include <trace/events/libata.h>
#include "libata.h"
enum {
DPRINTK("read log page - log 0x%x, page 0x%x\n", log, page);
ata_tf_init(dev, &tf);
- tf.command = ATA_CMD_READ_LOG_EXT;
+ if (dev->dma_mode && ata_id_has_read_log_dma_ext(dev->id)) {
+ tf.command = ATA_CMD_READ_LOG_DMA_EXT;
+ tf.protocol = ATA_PROT_DMA;
+ } else {
+ tf.command = ATA_CMD_READ_LOG_EXT;
+ tf.protocol = ATA_PROT_PIO;
+ }
tf.lbal = log;
tf.lbam = page;
tf.nsect = sectors;
tf.hob_nsect = sectors >> 8;
tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_LBA48 | ATA_TFLAG_DEVICE;
- tf.protocol = ATA_PROT_PIO;
err_mask = ata_exec_internal(dev, &tf, NULL, DMA_FROM_DEVICE,
buf, sectors * ATA_SECT_SIZE, 0);
tf->hob_lbah = buf[10];
tf->nsect = buf[12];
tf->hob_nsect = buf[13];
+ if (ata_id_has_ncq_autosense(dev->id))
+ tf->auxiliary = buf[14] << 16 | buf[15] << 8 | buf[16];
return 0;
}
return err_mask;
}
+/**
+ * ata_eh_request_sense - perform REQUEST_SENSE_DATA_EXT
+ * @dev: device to perform REQUEST_SENSE_SENSE_DATA_EXT to
+ * @sense_buf: result sense data buffer (SCSI_SENSE_BUFFERSIZE bytes long)
+ * @dfl_sense_key: default sense key to use
+ *
+ * Perform REQUEST_SENSE_DATA_EXT after the device reported CHECK
+ * SENSE. This function is EH helper.
+ *
+ * LOCKING:
+ * Kernel thread context (may sleep).
+ *
+ * RETURNS:
+ * encoded sense data on success, 0 on failure or if sense data
+ * is not available.
+ */
+static u32 ata_eh_request_sense(struct ata_queued_cmd *qc,
+ struct scsi_cmnd *cmd)
+{
+ struct ata_device *dev = qc->dev;
+ struct ata_taskfile tf;
+ unsigned int err_mask;
+
+ if (!cmd)
+ return 0;
+
+ DPRINTK("ATA request sense\n");
+ ata_dev_warn(dev, "request sense\n");
+ if (!ata_id_sense_reporting_enabled(dev->id)) {
+ ata_dev_warn(qc->dev, "sense data reporting disabled\n");
+ return 0;
+ }
+ ata_tf_init(dev, &tf);
+
+ tf.flags |= ATA_TFLAG_ISADDR | ATA_TFLAG_DEVICE;
+ tf.flags |= ATA_TFLAG_LBA | ATA_TFLAG_LBA48;
+ tf.command = ATA_CMD_REQ_SENSE_DATA;
+ tf.protocol = ATA_PROT_NODATA;
+
+ err_mask = ata_exec_internal(dev, &tf, NULL, DMA_NONE, NULL, 0, 0);
+ /*
+ * ACS-4 states:
+ * The device may set the SENSE DATA AVAILABLE bit to one in the
+ * STATUS field and clear the ERROR bit to zero in the STATUS field
+ * to indicate that the command returned completion without an error
+ * and the sense data described in table 306 is available.
+ *
+ * IOW the 'ATA_SENSE' bit might not be set even though valid
+ * sense data is available.
+ * So check for both.
+ */
+ if ((tf.command & ATA_SENSE) ||
+ tf.lbah != 0 || tf.lbam != 0 || tf.lbal != 0) {
+ ata_scsi_set_sense(cmd, tf.lbah, tf.lbam, tf.lbal);
+ qc->flags |= ATA_QCFLAG_SENSE_VALID;
+ ata_dev_warn(dev, "sense data %02x/%02x/%02x\n",
+ tf.lbah, tf.lbam, tf.lbal);
+ } else {
+ ata_dev_warn(dev, "request sense failed stat %02x emask %x\n",
+ tf.command, err_mask);
+ }
+ return err_mask;
+}
+
/**
* atapi_eh_request_sense - perform ATAPI REQUEST_SENSE
* @dev: device to perform REQUEST_SENSE to
memcpy(&qc->result_tf, &tf, sizeof(tf));
qc->result_tf.flags = ATA_TFLAG_ISADDR | ATA_TFLAG_LBA | ATA_TFLAG_LBA48;
qc->err_mask |= AC_ERR_DEV | AC_ERR_NCQ;
+ if (qc->result_tf.auxiliary) {
+ char sense_key, asc, ascq;
+
+ sense_key = (qc->result_tf.auxiliary >> 16) & 0xff;
+ asc = (qc->result_tf.auxiliary >> 8) & 0xff;
+ ascq = qc->result_tf.auxiliary & 0xff;
+ ata_dev_dbg(dev, "NCQ Autosense %02x/%02x/%02x\n",
+ sense_key, asc, ascq);
+ ata_scsi_set_sense(qc->scsicmd, sense_key, asc, ascq);
+ ata_scsi_set_sense_information(qc->scsicmd, &qc->result_tf);
+ qc->flags |= ATA_QCFLAG_SENSE_VALID;
+ }
+
ehc->i.err_mask &= ~AC_ERR_DEV;
}
return ATA_EH_RESET;
}
+ /*
+ * Sense data reporting does not work if the
+ * device fault bit is set.
+ */
+ if ((stat & ATA_SENSE) && !(stat & ATA_DF) &&
+ !(qc->flags & ATA_QCFLAG_SENSE_VALID)) {
+ if (!(qc->ap->pflags & ATA_PFLAG_FROZEN)) {
+ tmp = ata_eh_request_sense(qc, qc->scsicmd);
+ if (tmp)
+ qc->err_mask |= tmp;
+ else
+ ata_scsi_set_sense_information(qc->scsicmd, tf);
+ } else {
+ ata_dev_warn(qc->dev, "sense data available but port frozen\n");
+ }
+ }
+
+ /* Set by NCQ autosense or request sense above */
+ if (qc->flags & ATA_QCFLAG_SENSE_VALID)
+ return 0;
+
if (stat & (ATA_ERR | ATA_DF))
qc->err_mask |= AC_ERR_DEV;
else
all_err_mask |= qc->err_mask;
if (qc->flags & ATA_QCFLAG_IO)
eflags |= ATA_EFLAG_IS_IO;
+ trace_ata_eh_link_autopsy_qc(qc);
}
/* enforce default EH actions */
eflags |= ATA_EFLAG_DUBIOUS_XFER;
ehc->i.action |= ata_eh_speed_down(dev, eflags, all_err_mask);
}
-
+ trace_ata_eh_link_autopsy(dev, ehc->i.action, all_err_mask);
DPRINTK("EXIT\n");
}
const char *text;
} cmd_descr[] = {
{ ATA_CMD_DEV_RESET, "DEVICE RESET" },
- { ATA_CMD_CHK_POWER, "CHECK POWER MODE" },
- { ATA_CMD_STANDBY, "STANDBY" },
- { ATA_CMD_IDLE, "IDLE" },
- { ATA_CMD_EDD, "EXECUTE DEVICE DIAGNOSTIC" },
- { ATA_CMD_DOWNLOAD_MICRO, "DOWNLOAD MICROCODE" },
+ { ATA_CMD_CHK_POWER, "CHECK POWER MODE" },
+ { ATA_CMD_STANDBY, "STANDBY" },
+ { ATA_CMD_IDLE, "IDLE" },
+ { ATA_CMD_EDD, "EXECUTE DEVICE DIAGNOSTIC" },
+ { ATA_CMD_DOWNLOAD_MICRO, "DOWNLOAD MICROCODE" },
{ ATA_CMD_DOWNLOAD_MICRO_DMA, "DOWNLOAD MICROCODE DMA" },
{ ATA_CMD_NOP, "NOP" },
- { ATA_CMD_FLUSH, "FLUSH CACHE" },
- { ATA_CMD_FLUSH_EXT, "FLUSH CACHE EXT" },
- { ATA_CMD_ID_ATA, "IDENTIFY DEVICE" },
- { ATA_CMD_ID_ATAPI, "IDENTIFY PACKET DEVICE" },
- { ATA_CMD_SERVICE, "SERVICE" },
- { ATA_CMD_READ, "READ DMA" },
- { ATA_CMD_READ_EXT, "READ DMA EXT" },
- { ATA_CMD_READ_QUEUED, "READ DMA QUEUED" },
- { ATA_CMD_READ_STREAM_EXT, "READ STREAM EXT" },
+ { ATA_CMD_FLUSH, "FLUSH CACHE" },
+ { ATA_CMD_FLUSH_EXT, "FLUSH CACHE EXT" },
+ { ATA_CMD_ID_ATA, "IDENTIFY DEVICE" },
+ { ATA_CMD_ID_ATAPI, "IDENTIFY PACKET DEVICE" },
+ { ATA_CMD_SERVICE, "SERVICE" },
+ { ATA_CMD_READ, "READ DMA" },
+ { ATA_CMD_READ_EXT, "READ DMA EXT" },
+ { ATA_CMD_READ_QUEUED, "READ DMA QUEUED" },
+ { ATA_CMD_READ_STREAM_EXT, "READ STREAM EXT" },
{ ATA_CMD_READ_STREAM_DMA_EXT, "READ STREAM DMA EXT" },
- { ATA_CMD_WRITE, "WRITE DMA" },
- { ATA_CMD_WRITE_EXT, "WRITE DMA EXT" },
- { ATA_CMD_WRITE_QUEUED, "WRITE DMA QUEUED EXT" },
- { ATA_CMD_WRITE_STREAM_EXT, "WRITE STREAM EXT" },
+ { ATA_CMD_WRITE, "WRITE DMA" },
+ { ATA_CMD_WRITE_EXT, "WRITE DMA EXT" },
+ { ATA_CMD_WRITE_QUEUED, "WRITE DMA QUEUED EXT" },
+ { ATA_CMD_WRITE_STREAM_EXT, "WRITE STREAM EXT" },
{ ATA_CMD_WRITE_STREAM_DMA_EXT, "WRITE STREAM DMA EXT" },
{ ATA_CMD_WRITE_FUA_EXT, "WRITE DMA FUA EXT" },
{ ATA_CMD_WRITE_QUEUED_FUA_EXT, "WRITE DMA QUEUED FUA EXT" },
{ ATA_CMD_READ_MULTI_EXT, "READ MULTIPLE EXT" },
{ ATA_CMD_WRITE_MULTI, "WRITE MULTIPLE" },
{ ATA_CMD_WRITE_MULTI_EXT, "WRITE MULTIPLE EXT" },
- { ATA_CMD_WRITE_MULTI_FUA_EXT, "WRITE MULTIPLE FUA EXT" },
+ { ATA_CMD_WRITE_MULTI_FUA_EXT, "WRITE MULTIPLE FUA EXT" },
{ ATA_CMD_SET_FEATURES, "SET FEATURES" },
{ ATA_CMD_SET_MULTI, "SET MULTIPLE MODE" },
{ ATA_CMD_VERIFY, "READ VERIFY SECTOR(S)" },
{ ATA_CMD_READ_LOG_EXT, "READ LOG EXT" },
{ ATA_CMD_WRITE_LOG_EXT, "WRITE LOG EXT" },
{ ATA_CMD_READ_LOG_DMA_EXT, "READ LOG DMA EXT" },
- { ATA_CMD_WRITE_LOG_DMA_EXT, "WRITE LOG DMA EXT" },
+ { ATA_CMD_WRITE_LOG_DMA_EXT, "WRITE LOG DMA EXT" },
{ ATA_CMD_TRUSTED_NONDATA, "TRUSTED NON-DATA" },
{ ATA_CMD_TRUSTED_RCV, "TRUSTED RECEIVE" },
- { ATA_CMD_TRUSTED_RCV_DMA, "TRUSTED RECEIVE DMA" },
+ { ATA_CMD_TRUSTED_RCV_DMA, "TRUSTED RECEIVE DMA" },
{ ATA_CMD_TRUSTED_SND, "TRUSTED SEND" },
- { ATA_CMD_TRUSTED_SND_DMA, "TRUSTED SEND DMA" },
+ { ATA_CMD_TRUSTED_SND_DMA, "TRUSTED SEND DMA" },
{ ATA_CMD_PMP_READ, "READ BUFFER" },
{ ATA_CMD_PMP_READ_DMA, "READ BUFFER DMA" },
{ ATA_CMD_PMP_WRITE, "WRITE BUFFER" },
{ ATA_CMD_MEDIA_LOCK, "DOOR LOCK" },
{ ATA_CMD_MEDIA_UNLOCK, "DOOR UNLOCK" },
{ ATA_CMD_DSM, "DATA SET MANAGEMENT" },
- { ATA_CMD_CHK_MED_CRD_TYP, "CHECK MEDIA CARD TYPE" },
- { ATA_CMD_CFA_REQ_EXT_ERR, "CFA REQUEST EXTENDED ERROR" },
+ { ATA_CMD_CHK_MED_CRD_TYP, "CHECK MEDIA CARD TYPE" },
+ { ATA_CMD_CFA_REQ_EXT_ERR, "CFA REQUEST EXTENDED ERROR" },
{ ATA_CMD_CFA_WRITE_NE, "CFA WRITE SECTORS WITHOUT ERASE" },
{ ATA_CMD_CFA_TRANS_SECT, "CFA TRANSLATE SECTOR" },
{ ATA_CMD_CFA_ERASE, "CFA ERASE SECTORS" },
- { ATA_CMD_CFA_WRITE_MULT_NE, "CFA WRITE MULTIPLE WITHOUT ERASE" },
+ { ATA_CMD_CFA_WRITE_MULT_NE, "CFA WRITE MULTIPLE WITHOUT ERASE" },
{ ATA_CMD_REQ_SENSE_DATA, "REQUEST SENSE DATA EXT" },
{ ATA_CMD_SANITIZE_DEVICE, "SANITIZE DEVICE" },
{ ATA_CMD_READ_LONG, "READ LONG (with retries)" },
#ifdef CONFIG_ATA_VERBOSE_ERROR
if (res->command & (ATA_BUSY | ATA_DRDY | ATA_DF | ATA_DRQ |
- ATA_ERR)) {
+ ATA_SENSE | ATA_ERR)) {
if (res->command & ATA_BUSY)
ata_dev_err(qc->dev, "status: { Busy }\n");
else
- ata_dev_err(qc->dev, "status: { %s%s%s%s}\n",
+ ata_dev_err(qc->dev, "status: { %s%s%s%s%s}\n",
res->command & ATA_DRDY ? "DRDY " : "",
res->command & ATA_DF ? "DF " : "",
res->command & ATA_DRQ ? "DRQ " : "",
+ res->command & ATA_SENSE ? "SENSE " : "",
res->command & ATA_ERR ? "ERR " : "");
}
ata_scsi_park_show, ata_scsi_park_store);
EXPORT_SYMBOL_GPL(dev_attr_unload_heads);
-static void ata_scsi_set_sense(struct scsi_cmnd *cmd, u8 sk, u8 asc, u8 ascq)
+void ata_scsi_set_sense(struct scsi_cmnd *cmd, u8 sk, u8 asc, u8 ascq)
{
+ if (!cmd)
+ return;
+
cmd->result = (DRIVER_SENSE << 24) | SAM_STAT_CHECK_CONDITION;
scsi_build_sense_buffer(0, cmd->sense_buffer, sk, asc, ascq);
}
+void ata_scsi_set_sense_information(struct scsi_cmnd *cmd,
+ const struct ata_taskfile *tf)
+{
+ u64 information;
+
+ if (!cmd)
+ return;
+
+ information = ata_tf_read_block(tf, NULL);
+ scsi_set_sense_information(cmd->sense_buffer, information);
+}
+
static ssize_t
ata_scsi_em_message_store(struct device *dev, struct device_attribute *attr,
const char *buf, size_t count)
if (stat & ATA_BUSY) {
printk("Busy }\n"); /* Data is not valid in this case */
} else {
- if (stat & 0x40) printk("DriveReady ");
- if (stat & 0x20) printk("DeviceFault ");
- if (stat & 0x10) printk("SeekComplete ");
- if (stat & 0x08) printk("DataRequest ");
- if (stat & 0x04) printk("CorrectedError ");
- if (stat & 0x02) printk("Index ");
- if (stat & 0x01) printk("Error ");
+ if (stat & ATA_DRDY) printk("DriveReady ");
+ if (stat & ATA_DF) printk("DeviceFault ");
+ if (stat & ATA_DSC) printk("SeekComplete ");
+ if (stat & ATA_DRQ) printk("DataRequest ");
+ if (stat & ATA_CORR) printk("CorrectedError ");
+ if (stat & ATA_SENSE) printk("Sense ");
+ if (stat & ATA_ERR) printk("Error ");
printk("}\n");
if (err) {
printk(KERN_WARNING "ata%u: error=0x%02x { ", id, err);
- if (err & 0x04) printk("DriveStatusError ");
- if (err & 0x80) {
- if (err & 0x04) printk("BadCRC ");
+ if (err & ATA_ABORTED) printk("DriveStatusError ");
+ if (err & ATA_ICRC) {
+ if (err & ATA_ABORTED)
+ printk("BadCRC ");
else printk("Sector ");
}
- if (err & 0x40) printk("UncorrectableError ");
- if (err & 0x10) printk("SectorIdNotFound ");
- if (err & 0x02) printk("TrackZeroNotFound ");
- if (err & 0x01) printk("AddrMarkNotFound ");
+ if (err & ATA_UNC) printk("UncorrectableError ");
+ if (err & ATA_IDNF) printk("SectorIdNotFound ");
+ if (err & ATA_TRK0NF) printk("TrackZeroNotFound ");
+ if (err & ATA_AMNF) printk("AddrMarkNotFound ");
printk("}\n");
}
}
/* Based on the 3ware driver translation table */
static const unsigned char sense_table[][4] = {
/* BBD|ECC|ID|MAR */
- {0xd1, ABORTED_COMMAND, 0x00, 0x00}, // Device busy Aborted command
+ {0xd1, ABORTED_COMMAND, 0x00, 0x00},
+ // Device busy Aborted command
/* BBD|ECC|ID */
- {0xd0, ABORTED_COMMAND, 0x00, 0x00}, // Device busy Aborted command
+ {0xd0, ABORTED_COMMAND, 0x00, 0x00},
+ // Device busy Aborted command
/* ECC|MC|MARK */
- {0x61, HARDWARE_ERROR, 0x00, 0x00}, // Device fault Hardware error
+ {0x61, HARDWARE_ERROR, 0x00, 0x00},
+ // Device fault Hardware error
/* ICRC|ABRT */ /* NB: ICRC & !ABRT is BBD */
- {0x84, ABORTED_COMMAND, 0x47, 0x00}, // Data CRC error SCSI parity error
+ {0x84, ABORTED_COMMAND, 0x47, 0x00},
+ // Data CRC error SCSI parity error
/* MC|ID|ABRT|TRK0|MARK */
- {0x37, NOT_READY, 0x04, 0x00}, // Unit offline Not ready
+ {0x37, NOT_READY, 0x04, 0x00},
+ // Unit offline Not ready
/* MCR|MARK */
- {0x09, NOT_READY, 0x04, 0x00}, // Unrecovered disk error Not ready
+ {0x09, NOT_READY, 0x04, 0x00},
+ // Unrecovered disk error Not ready
/* Bad address mark */
- {0x01, MEDIUM_ERROR, 0x13, 0x00}, // Address mark not found Address mark not found for data field
- /* TRK0 */
- {0x02, HARDWARE_ERROR, 0x00, 0x00}, // Track 0 not found Hardware error
+ {0x01, MEDIUM_ERROR, 0x13, 0x00},
+ // Address mark not found for data field
+ /* TRK0 - Track 0 not found */
+ {0x02, HARDWARE_ERROR, 0x00, 0x00},
+ // Hardware error
/* Abort: 0x04 is not translated here, see below */
/* Media change request */
- {0x08, NOT_READY, 0x04, 0x00}, // Media change request FIXME: faking offline
- /* SRV/IDNF */
- {0x10, ILLEGAL_REQUEST, 0x21, 0x00}, // ID not found Logical address out of range
- /* MC */
- {0x20, UNIT_ATTENTION, 0x28, 0x00}, // Media Changed Not ready to ready change, medium may have changed
- /* ECC */
- {0x40, MEDIUM_ERROR, 0x11, 0x04}, // Uncorrectable ECC error Unrecovered read error
+ {0x08, NOT_READY, 0x04, 0x00},
+ // FIXME: faking offline
+ /* SRV/IDNF - ID not found */
+ {0x10, ILLEGAL_REQUEST, 0x21, 0x00},
+ // Logical address out of range
+ /* MC - Media Changed */
+ {0x20, UNIT_ATTENTION, 0x28, 0x00},
+ // Not ready to ready change, medium may have changed
+ /* ECC - Uncorrectable ECC error */
+ {0x40, MEDIUM_ERROR, 0x11, 0x04},
+ // Unrecovered read error
/* BBD - block marked bad */
- {0x80, MEDIUM_ERROR, 0x11, 0x04}, // Block marked bad Medium error, unrecovered read error
+ {0x80, MEDIUM_ERROR, 0x11, 0x04},
+ // Block marked bad Medium error, unrecovered read error
{0xFF, 0xFF, 0xFF, 0xFF}, // END mark
};
static const unsigned char stat_table[][4] = {
/* Must be first because BUSY means no other bits valid */
- {0x80, ABORTED_COMMAND, 0x47, 0x00}, // Busy, fake parity for now
- {0x20, HARDWARE_ERROR, 0x44, 0x00}, // Device fault, internal target failure
- {0x08, ABORTED_COMMAND, 0x47, 0x00}, // Timed out in xfer, fake parity for now
- {0x04, RECOVERED_ERROR, 0x11, 0x00}, // Recovered ECC error Medium error, recovered
+ {0x80, ABORTED_COMMAND, 0x47, 0x00},
+ // Busy, fake parity for now
+ {0x40, ILLEGAL_REQUEST, 0x21, 0x04},
+ // Device ready, unaligned write command
+ {0x20, HARDWARE_ERROR, 0x44, 0x00},
+ // Device fault, internal target failure
+ {0x08, ABORTED_COMMAND, 0x47, 0x00},
+ // Timed out in xfer, fake parity for now
+ {0x04, RECOVERED_ERROR, 0x11, 0x00},
+ // Recovered ECC error Medium error, recovered
{0xFF, 0xFF, 0xFF, 0xFF}, // END mark
};
((cdb[2] & 0x20) || need_sense)) {
ata_gen_passthru_sense(qc);
} else {
- if (!need_sense) {
+ if (qc->flags & ATA_QCFLAG_SENSE_VALID) {
+ cmd->result = SAM_STAT_CHECK_CONDITION;
+ } else if (!need_sense) {
cmd->result = SAM_STAT_GOOD;
} else {
/* TODO: decide which descriptor format to use
unsigned int i, tag;
for (i = 0, tag = ap->sas_last_tag + 1; i < max_queue; i++, tag++) {
- if (ap->flags & ATA_FLAG_LOWTAG)
- tag = 1;
- else
- tag = tag < max_queue ? tag : 0;
+ tag = tag < max_queue ? tag : 0;
/* the last tag is reserved for internal command. */
if (tag == ATA_TAG_INTERNAL)
* ->sff_irq_clear method. Try to initialize bmdma_addr
* regardless of dma masks.
*/
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
ata_bmdma_nodma(host, "failed to set dma mask");
if (!rc) {
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
ata_bmdma_nodma(host,
"failed to set consistent dma mask");
--- /dev/null
+/*
+ * libata-trace.c - trace functions for libata
+ *
+ * Copyright 2015 Hannes Reinecke
+ * Copyright 2015 SUSE Linux GmbH
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; see the file COPYING. If not, write to
+ * the Free Software Foundation, 675 Mass Ave, Cambridge, MA 02139, USA.
+ */
+
+#include <linux/kernel.h>
+#include <linux/trace_seq.h>
+#include <trace/events/libata.h>
+
+const char *
+libata_trace_parse_status(struct trace_seq *p, unsigned char status)
+{
+ const char *ret = trace_seq_buffer_ptr(p);
+
+ trace_seq_printf(p, "{ ");
+ if (status & ATA_BUSY)
+ trace_seq_printf(p, "BUSY ");
+ if (status & ATA_DRDY)
+ trace_seq_printf(p, "DRDY ");
+ if (status & ATA_DF)
+ trace_seq_printf(p, "DF ");
+ if (status & ATA_DSC)
+ trace_seq_printf(p, "DSC ");
+ if (status & ATA_DRQ)
+ trace_seq_printf(p, "DRQ ");
+ if (status & ATA_CORR)
+ trace_seq_printf(p, "CORR ");
+ if (status & ATA_SENSE)
+ trace_seq_printf(p, "SENSE ");
+ if (status & ATA_ERR)
+ trace_seq_printf(p, "ERR ");
+ trace_seq_putc(p, '}');
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+const char *
+libata_trace_parse_eh_action(struct trace_seq *p, unsigned int eh_action)
+{
+ const char *ret = trace_seq_buffer_ptr(p);
+
+ trace_seq_printf(p, "%x", eh_action);
+ if (eh_action) {
+ trace_seq_printf(p, "{ ");
+ if (eh_action & ATA_EH_REVALIDATE)
+ trace_seq_printf(p, "REVALIDATE ");
+ if (eh_action & (ATA_EH_SOFTRESET | ATA_EH_HARDRESET))
+ trace_seq_printf(p, "RESET ");
+ else if (eh_action & ATA_EH_SOFTRESET)
+ trace_seq_printf(p, "SOFTRESET ");
+ else if (eh_action & ATA_EH_HARDRESET)
+ trace_seq_printf(p, "HARDRESET ");
+ if (eh_action & ATA_EH_ENABLE_LINK)
+ trace_seq_printf(p, "ENABLE_LINK ");
+ if (eh_action & ATA_EH_PARK)
+ trace_seq_printf(p, "PARK ");
+ trace_seq_putc(p, '}');
+ }
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+const char *
+libata_trace_parse_eh_err_mask(struct trace_seq *p, unsigned int eh_err_mask)
+{
+ const char *ret = trace_seq_buffer_ptr(p);
+
+ trace_seq_printf(p, "%x", eh_err_mask);
+ if (eh_err_mask) {
+ trace_seq_printf(p, "{ ");
+ if (eh_err_mask & AC_ERR_DEV)
+ trace_seq_printf(p, "DEV ");
+ if (eh_err_mask & AC_ERR_HSM)
+ trace_seq_printf(p, "HSM ");
+ if (eh_err_mask & AC_ERR_TIMEOUT)
+ trace_seq_printf(p, "TIMEOUT ");
+ if (eh_err_mask & AC_ERR_MEDIA)
+ trace_seq_printf(p, "MEDIA ");
+ if (eh_err_mask & AC_ERR_ATA_BUS)
+ trace_seq_printf(p, "ATA_BUS ");
+ if (eh_err_mask & AC_ERR_HOST_BUS)
+ trace_seq_printf(p, "HOST_BUS ");
+ if (eh_err_mask & AC_ERR_SYSTEM)
+ trace_seq_printf(p, "SYSTEM ");
+ if (eh_err_mask & AC_ERR_INVALID)
+ trace_seq_printf(p, "INVALID ");
+ if (eh_err_mask & AC_ERR_OTHER)
+ trace_seq_printf(p, "OTHER ");
+ if (eh_err_mask & AC_ERR_NODEV_HINT)
+ trace_seq_printf(p, "NODEV_HINT ");
+ if (eh_err_mask & AC_ERR_NCQ)
+ trace_seq_printf(p, "NCQ ");
+ trace_seq_putc(p, '}');
+ }
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
+
+const char *
+libata_trace_parse_qc_flags(struct trace_seq *p, unsigned int qc_flags)
+{
+ const char *ret = trace_seq_buffer_ptr(p);
+
+ trace_seq_printf(p, "%x", qc_flags);
+ if (qc_flags) {
+ trace_seq_printf(p, "{ ");
+ if (qc_flags & ATA_QCFLAG_ACTIVE)
+ trace_seq_printf(p, "ACTIVE ");
+ if (qc_flags & ATA_QCFLAG_DMAMAP)
+ trace_seq_printf(p, "DMAMAP ");
+ if (qc_flags & ATA_QCFLAG_IO)
+ trace_seq_printf(p, "IO ");
+ if (qc_flags & ATA_QCFLAG_RESULT_TF)
+ trace_seq_printf(p, "RESULT_TF ");
+ if (qc_flags & ATA_QCFLAG_CLEAR_EXCL)
+ trace_seq_printf(p, "CLEAR_EXCL ");
+ if (qc_flags & ATA_QCFLAG_QUIET)
+ trace_seq_printf(p, "QUIET ");
+ if (qc_flags & ATA_QCFLAG_RETRY)
+ trace_seq_printf(p, "RETRY ");
+ if (qc_flags & ATA_QCFLAG_FAILED)
+ trace_seq_printf(p, "FAILED ");
+ if (qc_flags & ATA_QCFLAG_SENSE_VALID)
+ trace_seq_printf(p, "SENSE_VALID ");
+ if (qc_flags & ATA_QCFLAG_EH_SCHEDULED)
+ trace_seq_printf(p, "EH_SCHEDULED ");
+ trace_seq_putc(p, '}');
+ }
+ trace_seq_putc(p, 0);
+
+ return ret;
+}
extern int ata_build_rw_tf(struct ata_taskfile *tf, struct ata_device *dev,
u64 block, u32 n_block, unsigned int tf_flags,
unsigned int tag);
-extern u64 ata_tf_read_block(struct ata_taskfile *tf, struct ata_device *dev);
+extern u64 ata_tf_read_block(const struct ata_taskfile *tf,
+ struct ata_device *dev);
extern unsigned ata_exec_internal(struct ata_device *dev,
struct ata_taskfile *tf, const u8 *cdb,
int dma_dir, void *buf, unsigned int buflen,
struct scsi_host_template *sht);
extern void ata_scsi_scan_host(struct ata_port *ap, int sync);
extern int ata_scsi_offline_dev(struct ata_device *dev);
+extern void ata_scsi_set_sense(struct scsi_cmnd *cmd, u8 sk, u8 asc, u8 ascq);
+extern void ata_scsi_set_sense_information(struct scsi_cmnd *cmd,
+ const struct ata_taskfile *tf);
extern void ata_scsi_media_change_notify(struct ata_device *dev);
extern void ata_scsi_hotplug(struct work_struct *work);
extern void ata_schedule_scsi_eh(struct Scsi_Host *shost);
atp867x_fixup(host);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
return rc;
}
return -ENODEV;
}
- if (pci_set_dma_mask(pdev, DMA_BIT_MASK(32))) {
+ if (dma_set_mask(&pdev->dev, DMA_BIT_MASK(32))) {
printk(KERN_ERR DRV_NAME ": unable to configure DMA mask.\n");
return -ENODEV;
}
- if (pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32))) {
+ if (dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32))) {
printk(KERN_ERR DRV_NAME ": unable to configure consistent DMA mask.\n");
return -ENODEV;
}
if (rc)
return rc;
host->iomap = pcim_iomap_table(pdev);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
return rc;
host->iomap = pcim_iomap_table(dev);
- rc = pci_set_dma_mask(dev, ATA_DMA_MASK);
+ rc = dma_set_mask(&dev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(dev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&dev->dev, ATA_DMA_MASK);
if (rc)
return rc;
pci_set_master(dev);
return rc;
host->iomap = pcim_iomap_table(pdev);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
if (rc)
return rc;
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
host->iomap = pcim_iomap_table(pdev);
/* Setup DMA masks */
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
pci_set_master(pdev);
{
int rc;
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit consistent DMA enable failed\n");
return rc;
#include <linux/platform_device.h>
#include <linux/libata.h>
#include <linux/slab.h>
+
#include "libata.h"
#include <scsi/scsi_host.h>
#include <scsi/scsi_cmnd.h>
+/* Supported DMA engine drivers */
+#include <linux/platform_data/dma-dw.h>
+#include <linux/dma/dw.h>
+
/* These two are defined in "libata.h" */
#undef DRV_NAME
#undef DRV_VERSION
#define NO_IRQ 0
#endif
-/* SATA DMA driver Globals */
-#define DMA_NUM_CHANS 1
-#define DMA_NUM_CHAN_REGS 8
-
-/* SATA DMA Register definitions */
#define AHB_DMA_BRST_DFLT 64 /* 16 data items burst length*/
-struct dmareg {
- u32 low; /* Low bits 0-31 */
- u32 high; /* High bits 32-63 */
-};
-
-/* DMA Per Channel registers */
-struct dma_chan_regs {
- struct dmareg sar; /* Source Address */
- struct dmareg dar; /* Destination address */
- struct dmareg llp; /* Linked List Pointer */
- struct dmareg ctl; /* Control */
- struct dmareg sstat; /* Source Status not implemented in core */
- struct dmareg dstat; /* Destination Status not implemented in core*/
- struct dmareg sstatar; /* Source Status Address not impl in core */
- struct dmareg dstatar; /* Destination Status Address not implemente */
- struct dmareg cfg; /* Config */
- struct dmareg sgr; /* Source Gather */
- struct dmareg dsr; /* Destination Scatter */
-};
-
-/* Generic Interrupt Registers */
-struct dma_interrupt_regs {
- struct dmareg tfr; /* Transfer Interrupt */
- struct dmareg block; /* Block Interrupt */
- struct dmareg srctran; /* Source Transfer Interrupt */
- struct dmareg dsttran; /* Dest Transfer Interrupt */
- struct dmareg error; /* Error */
-};
-
-struct ahb_dma_regs {
- struct dma_chan_regs chan_regs[DMA_NUM_CHAN_REGS];
- struct dma_interrupt_regs interrupt_raw; /* Raw Interrupt */
- struct dma_interrupt_regs interrupt_status; /* Interrupt Status */
- struct dma_interrupt_regs interrupt_mask; /* Interrupt Mask */
- struct dma_interrupt_regs interrupt_clear; /* Interrupt Clear */
- struct dmareg statusInt; /* Interrupt combined*/
- struct dmareg rq_srcreg; /* Src Trans Req */
- struct dmareg rq_dstreg; /* Dst Trans Req */
- struct dmareg rq_sgl_srcreg; /* Sngl Src Trans Req*/
- struct dmareg rq_sgl_dstreg; /* Sngl Dst Trans Req*/
- struct dmareg rq_lst_srcreg; /* Last Src Trans Req*/
- struct dmareg rq_lst_dstreg; /* Last Dst Trans Req*/
- struct dmareg dma_cfg; /* DMA Config */
- struct dmareg dma_chan_en; /* DMA Channel Enable*/
- struct dmareg dma_id; /* DMA ID */
- struct dmareg dma_test; /* DMA Test */
- struct dmareg res1; /* reserved */
- struct dmareg res2; /* reserved */
- /*
- * DMA Comp Params
- * Param 6 = dma_param[0], Param 5 = dma_param[1],
- * Param 4 = dma_param[2] ...
- */
- struct dmareg dma_params[6];
-};
-
-/* Data structure for linked list item */
-struct lli {
- u32 sar; /* Source Address */
- u32 dar; /* Destination address */
- u32 llp; /* Linked List Pointer */
- struct dmareg ctl; /* Control */
- struct dmareg dstat; /* Destination Status */
-};
-
-enum {
- SATA_DWC_DMAC_LLI_SZ = (sizeof(struct lli)),
- SATA_DWC_DMAC_LLI_NUM = 256,
- SATA_DWC_DMAC_LLI_TBL_SZ = (SATA_DWC_DMAC_LLI_SZ * \
- SATA_DWC_DMAC_LLI_NUM),
- SATA_DWC_DMAC_TWIDTH_BYTES = 4,
- SATA_DWC_DMAC_CTRL_TSIZE_MAX = (0x00000800 * \
- SATA_DWC_DMAC_TWIDTH_BYTES),
-};
-
-/* DMA Register Operation Bits */
-enum {
- DMA_EN = 0x00000001, /* Enable AHB DMA */
- DMA_CTL_LLP_SRCEN = 0x10000000, /* Blk chain enable Src */
- DMA_CTL_LLP_DSTEN = 0x08000000, /* Blk chain enable Dst */
-};
-
-#define DMA_CTL_BLK_TS(size) ((size) & 0x000000FFF) /* Blk Transfer size */
-#define DMA_CHANNEL(ch) (0x00000001 << (ch)) /* Select channel */
- /* Enable channel */
-#define DMA_ENABLE_CHAN(ch) ((0x00000001 << (ch)) | \
- ((0x000000001 << (ch)) << 8))
- /* Disable channel */
-#define DMA_DISABLE_CHAN(ch) (0x00000000 | ((0x000000001 << (ch)) << 8))
- /* Transfer Type & Flow Controller */
-#define DMA_CTL_TTFC(type) (((type) & 0x7) << 20)
-#define DMA_CTL_SMS(num) (((num) & 0x3) << 25) /* Src Master Select */
-#define DMA_CTL_DMS(num) (((num) & 0x3) << 23)/* Dst Master Select */
- /* Src Burst Transaction Length */
-#define DMA_CTL_SRC_MSIZE(size) (((size) & 0x7) << 14)
- /* Dst Burst Transaction Length */
-#define DMA_CTL_DST_MSIZE(size) (((size) & 0x7) << 11)
- /* Source Transfer Width */
-#define DMA_CTL_SRC_TRWID(size) (((size) & 0x7) << 4)
- /* Destination Transfer Width */
-#define DMA_CTL_DST_TRWID(size) (((size) & 0x7) << 1)
-
-/* Assign HW handshaking interface (x) to destination / source peripheral */
-#define DMA_CFG_HW_HS_DEST(int_num) (((int_num) & 0xF) << 11)
-#define DMA_CFG_HW_HS_SRC(int_num) (((int_num) & 0xF) << 7)
-#define DMA_CFG_HW_CH_PRIOR(int_num) (((int_num) & 0xF) << 5)
-#define DMA_LLP_LMS(addr, master) (((addr) & 0xfffffffc) | (master))
-
-/*
- * This define is used to set block chaining disabled in the control low
- * register. It is already in little endian format so it can be &'d dirctly.
- * It is essentially: cpu_to_le32(~(DMA_CTL_LLP_SRCEN | DMA_CTL_LLP_DSTEN))
- */
enum {
- DMA_CTL_LLP_DISABLE_LE32 = 0xffffffe7,
- DMA_CTL_TTFC_P2M_DMAC = 0x00000002, /* Per to mem, DMAC cntr */
- DMA_CTL_TTFC_M2P_PER = 0x00000003, /* Mem to per, peripheral cntr */
- DMA_CTL_SINC_INC = 0x00000000, /* Source Address Increment */
- DMA_CTL_SINC_DEC = 0x00000200,
- DMA_CTL_SINC_NOCHANGE = 0x00000400,
- DMA_CTL_DINC_INC = 0x00000000, /* Destination Address Increment */
- DMA_CTL_DINC_DEC = 0x00000080,
- DMA_CTL_DINC_NOCHANGE = 0x00000100,
- DMA_CTL_INT_EN = 0x00000001, /* Interrupt Enable */
-
-/* Channel Configuration Register high bits */
- DMA_CFG_FCMOD_REQ = 0x00000001, /* Flow Control - request based */
- DMA_CFG_PROTCTL = (0x00000003 << 2),/* Protection Control */
-
-/* Channel Configuration Register low bits */
- DMA_CFG_RELD_DST = 0x80000000, /* Reload Dest / Src Addr */
- DMA_CFG_RELD_SRC = 0x40000000,
- DMA_CFG_HS_SELSRC = 0x00000800, /* Software handshake Src/ Dest */
- DMA_CFG_HS_SELDST = 0x00000400,
- DMA_CFG_FIFOEMPTY = (0x00000001 << 9), /* FIFO Empty bit */
-
-/* Channel Linked List Pointer Register */
- DMA_LLP_AHBMASTER1 = 0, /* List Master Select */
- DMA_LLP_AHBMASTER2 = 1,
-
SATA_DWC_MAX_PORTS = 1,
SATA_DWC_SCR_OFFSET = 0x24,
struct ata_host *host;
u8 __iomem *reg_base;
struct sata_dwc_regs *sata_dwc_regs; /* DW Synopsys SATA specific */
- int irq_dma;
+ struct dw_dma_chip *dma;
};
#define SATA_DWC_QCMD_MAX 32
struct sata_dwc_device_port {
struct sata_dwc_device *hsdev;
int cmd_issued[SATA_DWC_QCMD_MAX];
- struct lli *llit[SATA_DWC_QCMD_MAX]; /* DMA LLI table */
- dma_addr_t llit_dma[SATA_DWC_QCMD_MAX];
- u32 dma_chan[SATA_DWC_QCMD_MAX];
int dma_pending[SATA_DWC_QCMD_MAX];
+
+ /* DMA info */
+ struct dw_dma_slave *dws;
+ struct dma_chan *chan;
+ struct dma_async_tx_descriptor *desc[SATA_DWC_QCMD_MAX];
+ u32 dma_interrupt_count;
};
/*
void __iomem *scr_addr_sstatus;
u32 sata_dwc_sactive_issued ;
u32 sata_dwc_sactive_queued ;
- u32 dma_interrupt_count;
- struct ahb_dma_regs *sata_dma_regs;
- struct device *dwc_dev;
- int dma_channel;
};
static struct sata_dwc_host_priv host_pvt;
+static struct dw_dma_slave sata_dwc_dma_dws = {
+ .src_id = 0,
+ .dst_id = 0,
+ .src_master = 0,
+ .dst_master = 1,
+};
+
/*
* Prototypes
*/
static void sata_dwc_dma_xfer_complete(struct ata_port *ap, u32 check_status);
static void sata_dwc_port_stop(struct ata_port *ap);
static void sata_dwc_clear_dmacr(struct sata_dwc_device_port *hsdevp, u8 tag);
-static int dma_dwc_init(struct sata_dwc_device *hsdev, int irq);
-static void dma_dwc_exit(struct sata_dwc_device *hsdev);
-static int dma_dwc_xfer_setup(struct scatterlist *sg, int num_elems,
- struct lli *lli, dma_addr_t dma_lli,
- void __iomem *addr, int dir);
-static void dma_dwc_xfer_start(int dma_ch);
static const char *get_prot_descript(u8 protocol)
{
}
}
-static void sata_dwc_tf_dump(struct ata_taskfile *tf)
+static void sata_dwc_tf_dump(struct ata_port *ap, struct ata_taskfile *tf)
{
- dev_vdbg(host_pvt.dwc_dev, "taskfile cmd: 0x%02x protocol: %s flags:"
- "0x%lx device: %x\n", tf->command,
- get_prot_descript(tf->protocol), tf->flags, tf->device);
- dev_vdbg(host_pvt.dwc_dev, "feature: 0x%02x nsect: 0x%x lbal: 0x%x "
- "lbam: 0x%x lbah: 0x%x\n", tf->feature, tf->nsect, tf->lbal,
- tf->lbam, tf->lbah);
- dev_vdbg(host_pvt.dwc_dev, "hob_feature: 0x%02x hob_nsect: 0x%x "
- "hob_lbal: 0x%x hob_lbam: 0x%x hob_lbah: 0x%x\n",
+ dev_vdbg(ap->dev,
+ "taskfile cmd: 0x%02x protocol: %s flags: 0x%lx device: %x\n",
+ tf->command, get_prot_descript(tf->protocol), tf->flags,
+ tf->device);
+ dev_vdbg(ap->dev,
+ "feature: 0x%02x nsect: 0x%x lbal: 0x%x lbam: 0x%x lbah: 0x%x\n",
+ tf->feature, tf->nsect, tf->lbal, tf->lbam, tf->lbah);
+ dev_vdbg(ap->dev,
+ "hob_feature: 0x%02x hob_nsect: 0x%x hob_lbal: 0x%x hob_lbam: 0x%x hob_lbah: 0x%x\n",
tf->hob_feature, tf->hob_nsect, tf->hob_lbal, tf->hob_lbam,
tf->hob_lbah);
}
-/*
- * Function: get_burst_length_encode
- * arguments: datalength: length in bytes of data
- * returns value to be programmed in register corresponding to data length
- * This value is effectively the log(base 2) of the length
- */
-static int get_burst_length_encode(int datalength)
-{
- int items = datalength >> 2; /* div by 4 to get lword count */
-
- if (items >= 64)
- return 5;
-
- if (items >= 32)
- return 4;
-
- if (items >= 16)
- return 3;
-
- if (items >= 8)
- return 2;
-
- if (items >= 4)
- return 1;
-
- return 0;
-}
-
-static void clear_chan_interrupts(int c)
+static void dma_dwc_xfer_done(void *hsdev_instance)
{
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear.tfr.low),
- DMA_CHANNEL(c));
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear.block.low),
- DMA_CHANNEL(c));
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear.srctran.low),
- DMA_CHANNEL(c));
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear.dsttran.low),
- DMA_CHANNEL(c));
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear.error.low),
- DMA_CHANNEL(c));
-}
-
-/*
- * Function: dma_request_channel
- * arguments: None
- * returns channel number if available else -1
- * This function assigns the next available DMA channel from the list to the
- * requester
- */
-static int dma_request_channel(void)
-{
- /* Check if the channel is not currently in use */
- if (!(in_le32(&(host_pvt.sata_dma_regs->dma_chan_en.low)) &
- DMA_CHANNEL(host_pvt.dma_channel)))
- return host_pvt.dma_channel;
- dev_err(host_pvt.dwc_dev, "%s Channel %d is currently in use\n",
- __func__, host_pvt.dma_channel);
- return -1;
-}
-
-/*
- * Function: dma_dwc_interrupt
- * arguments: irq, dev_id, pt_regs
- * returns channel number if available else -1
- * Interrupt Handler for DW AHB SATA DMA
- */
-static irqreturn_t dma_dwc_interrupt(int irq, void *hsdev_instance)
-{
- int chan;
- u32 tfr_reg, err_reg;
unsigned long flags;
struct sata_dwc_device *hsdev = hsdev_instance;
struct ata_host *host = (struct ata_host *)hsdev->host;
hsdevp = HSDEVP_FROM_AP(ap);
tag = ap->link.active_tag;
- tfr_reg = in_le32(&(host_pvt.sata_dma_regs->interrupt_status.tfr\
- .low));
- err_reg = in_le32(&(host_pvt.sata_dma_regs->interrupt_status.error\
- .low));
-
- dev_dbg(ap->dev, "eot=0x%08x err=0x%08x pending=%d active port=%d\n",
- tfr_reg, err_reg, hsdevp->dma_pending[tag], port);
-
- chan = host_pvt.dma_channel;
- if (chan >= 0) {
- /* Check for end-of-transfer interrupt. */
- if (tfr_reg & DMA_CHANNEL(chan)) {
- /*
- * Each DMA command produces 2 interrupts. Only
- * complete the command after both interrupts have been
- * seen. (See sata_dwc_isr())
- */
- host_pvt.dma_interrupt_count++;
- sata_dwc_clear_dmacr(hsdevp, tag);
-
- if (hsdevp->dma_pending[tag] ==
- SATA_DWC_DMA_PENDING_NONE) {
- dev_err(ap->dev, "DMA not pending eot=0x%08x "
- "err=0x%08x tag=0x%02x pending=%d\n",
- tfr_reg, err_reg, tag,
- hsdevp->dma_pending[tag]);
- }
-
- if ((host_pvt.dma_interrupt_count % 2) == 0)
- sata_dwc_dma_xfer_complete(ap, 1);
-
- /* Clear the interrupt */
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear\
- .tfr.low),
- DMA_CHANNEL(chan));
- }
-
- /* Check for error interrupt. */
- if (err_reg & DMA_CHANNEL(chan)) {
- /* TODO Need error handler ! */
- dev_err(ap->dev, "error interrupt err_reg=0x%08x\n",
- err_reg);
-
- /* Clear the interrupt. */
- out_le32(&(host_pvt.sata_dma_regs->interrupt_clear\
- .error.low),
- DMA_CHANNEL(chan));
- }
- }
- spin_unlock_irqrestore(&host->lock, flags);
- return IRQ_HANDLED;
-}
-
-/*
- * Function: dma_request_interrupts
- * arguments: hsdev
- * returns status
- * This function registers ISR for a particular DMA channel interrupt
- */
-static int dma_request_interrupts(struct sata_dwc_device *hsdev, int irq)
-{
- int retval = 0;
- int chan = host_pvt.dma_channel;
-
- if (chan >= 0) {
- /* Unmask error interrupt */
- out_le32(&(host_pvt.sata_dma_regs)->interrupt_mask.error.low,
- DMA_ENABLE_CHAN(chan));
-
- /* Unmask end-of-transfer interrupt */
- out_le32(&(host_pvt.sata_dma_regs)->interrupt_mask.tfr.low,
- DMA_ENABLE_CHAN(chan));
- }
-
- retval = request_irq(irq, dma_dwc_interrupt, 0, "SATA DMA", hsdev);
- if (retval) {
- dev_err(host_pvt.dwc_dev, "%s: could not get IRQ %d\n",
- __func__, irq);
- return -ENODEV;
- }
-
- /* Mark this interrupt as requested */
- hsdev->irq_dma = irq;
- return 0;
-}
-
-/*
- * Function: map_sg_to_lli
- * The Synopsis driver has a comment proposing that better performance
- * is possible by only enabling interrupts on the last item in the linked list.
- * However, it seems that could be a problem if an error happened on one of the
- * first items. The transfer would halt, but no error interrupt would occur.
- * Currently this function sets interrupts enabled for each linked list item:
- * DMA_CTL_INT_EN.
- */
-static int map_sg_to_lli(struct scatterlist *sg, int num_elems,
- struct lli *lli, dma_addr_t dma_lli,
- void __iomem *dmadr_addr, int dir)
-{
- int i, idx = 0;
- int fis_len = 0;
- dma_addr_t next_llp;
- int bl;
- int sms_val, dms_val;
-
- sms_val = 0;
- dms_val = 1 + host_pvt.dma_channel;
- dev_dbg(host_pvt.dwc_dev,
- "%s: sg=%p nelem=%d lli=%p dma_lli=0x%pad dmadr=0x%p\n",
- __func__, sg, num_elems, lli, &dma_lli, dmadr_addr);
-
- bl = get_burst_length_encode(AHB_DMA_BRST_DFLT);
-
- for (i = 0; i < num_elems; i++, sg++) {
- u32 addr, offset;
- u32 sg_len, len;
-
- addr = (u32) sg_dma_address(sg);
- sg_len = sg_dma_len(sg);
-
- dev_dbg(host_pvt.dwc_dev, "%s: elem=%d sg_addr=0x%x sg_len"
- "=%d\n", __func__, i, addr, sg_len);
-
- while (sg_len) {
- if (idx >= SATA_DWC_DMAC_LLI_NUM) {
- /* The LLI table is not large enough. */
- dev_err(host_pvt.dwc_dev, "LLI table overrun "
- "(idx=%d)\n", idx);
- break;
- }
- len = (sg_len > SATA_DWC_DMAC_CTRL_TSIZE_MAX) ?
- SATA_DWC_DMAC_CTRL_TSIZE_MAX : sg_len;
-
- offset = addr & 0xffff;
- if ((offset + sg_len) > 0x10000)
- len = 0x10000 - offset;
-
- /*
- * Make sure a LLI block is not created that will span
- * 8K max FIS boundary. If the block spans such a FIS
- * boundary, there is a chance that a DMA burst will
- * cross that boundary -- this results in an error in
- * the host controller.
- */
- if (fis_len + len > 8192) {
- dev_dbg(host_pvt.dwc_dev, "SPLITTING: fis_len="
- "%d(0x%x) len=%d(0x%x)\n", fis_len,
- fis_len, len, len);
- len = 8192 - fis_len;
- fis_len = 0;
- } else {
- fis_len += len;
- }
- if (fis_len == 8192)
- fis_len = 0;
-
- /*
- * Set DMA addresses and lower half of control register
- * based on direction.
- */
- if (dir == DMA_FROM_DEVICE) {
- lli[idx].dar = cpu_to_le32(addr);
- lli[idx].sar = cpu_to_le32((u32)dmadr_addr);
-
- lli[idx].ctl.low = cpu_to_le32(
- DMA_CTL_TTFC(DMA_CTL_TTFC_P2M_DMAC) |
- DMA_CTL_SMS(sms_val) |
- DMA_CTL_DMS(dms_val) |
- DMA_CTL_SRC_MSIZE(bl) |
- DMA_CTL_DST_MSIZE(bl) |
- DMA_CTL_SINC_NOCHANGE |
- DMA_CTL_SRC_TRWID(2) |
- DMA_CTL_DST_TRWID(2) |
- DMA_CTL_INT_EN |
- DMA_CTL_LLP_SRCEN |
- DMA_CTL_LLP_DSTEN);
- } else { /* DMA_TO_DEVICE */
- lli[idx].sar = cpu_to_le32(addr);
- lli[idx].dar = cpu_to_le32((u32)dmadr_addr);
-
- lli[idx].ctl.low = cpu_to_le32(
- DMA_CTL_TTFC(DMA_CTL_TTFC_M2P_PER) |
- DMA_CTL_SMS(dms_val) |
- DMA_CTL_DMS(sms_val) |
- DMA_CTL_SRC_MSIZE(bl) |
- DMA_CTL_DST_MSIZE(bl) |
- DMA_CTL_DINC_NOCHANGE |
- DMA_CTL_SRC_TRWID(2) |
- DMA_CTL_DST_TRWID(2) |
- DMA_CTL_INT_EN |
- DMA_CTL_LLP_SRCEN |
- DMA_CTL_LLP_DSTEN);
- }
-
- dev_dbg(host_pvt.dwc_dev, "%s setting ctl.high len: "
- "0x%08x val: 0x%08x\n", __func__,
- len, DMA_CTL_BLK_TS(len / 4));
-
- /* Program the LLI CTL high register */
- lli[idx].ctl.high = cpu_to_le32(DMA_CTL_BLK_TS\
- (len / 4));
-
- /* Program the next pointer. The next pointer must be
- * the physical address, not the virtual address.
- */
- next_llp = (dma_lli + ((idx + 1) * sizeof(struct \
- lli)));
-
- /* The last 2 bits encode the list master select. */
- next_llp = DMA_LLP_LMS(next_llp, DMA_LLP_AHBMASTER2);
-
- lli[idx].llp = cpu_to_le32(next_llp);
- idx++;
- sg_len -= len;
- addr += len;
- }
- }
-
/*
- * The last next ptr has to be zero and the last control low register
- * has to have LLP_SRC_EN and LLP_DST_EN (linked list pointer source
- * and destination enable) set back to 0 (disabled.) This is what tells
- * the core that this is the last item in the linked list.
+ * Each DMA command produces 2 interrupts. Only
+ * complete the command after both interrupts have been
+ * seen. (See sata_dwc_isr())
*/
- if (idx) {
- lli[idx-1].llp = 0x00000000;
- lli[idx-1].ctl.low &= DMA_CTL_LLP_DISABLE_LE32;
+ hsdevp->dma_interrupt_count++;
+ sata_dwc_clear_dmacr(hsdevp, tag);
- /* Flush cache to memory */
- dma_cache_sync(NULL, lli, (sizeof(struct lli) * idx),
- DMA_BIDIRECTIONAL);
+ if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_NONE) {
+ dev_err(ap->dev, "DMA not pending tag=0x%02x pending=%d\n",
+ tag, hsdevp->dma_pending[tag]);
}
- return idx;
-}
+ if ((hsdevp->dma_interrupt_count % 2) == 0)
+ sata_dwc_dma_xfer_complete(ap, 1);
-/*
- * Function: dma_dwc_xfer_start
- * arguments: Channel number
- * Return : None
- * Enables the DMA channel
- */
-static void dma_dwc_xfer_start(int dma_ch)
-{
- /* Enable the DMA channel */
- out_le32(&(host_pvt.sata_dma_regs->dma_chan_en.low),
- in_le32(&(host_pvt.sata_dma_regs->dma_chan_en.low)) |
- DMA_ENABLE_CHAN(dma_ch));
+ spin_unlock_irqrestore(&host->lock, flags);
}
-static int dma_dwc_xfer_setup(struct scatterlist *sg, int num_elems,
- struct lli *lli, dma_addr_t dma_lli,
- void __iomem *addr, int dir)
+static struct dma_async_tx_descriptor *dma_dwc_xfer_setup(struct ata_queued_cmd *qc)
{
- int dma_ch;
- int num_lli;
- /* Acquire DMA channel */
- dma_ch = dma_request_channel();
- if (dma_ch == -1) {
- dev_err(host_pvt.dwc_dev, "%s: dma channel unavailable\n",
- __func__);
- return -EAGAIN;
+ struct ata_port *ap = qc->ap;
+ struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
+ struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
+ dma_addr_t addr = (dma_addr_t)&hsdev->sata_dwc_regs->dmadr;
+ struct dma_slave_config sconf;
+ struct dma_async_tx_descriptor *desc;
+
+ if (qc->dma_dir == DMA_DEV_TO_MEM) {
+ sconf.src_addr = addr;
+ sconf.device_fc = true;
+ } else { /* DMA_MEM_TO_DEV */
+ sconf.dst_addr = addr;
+ sconf.device_fc = false;
}
- /* Convert SG list to linked list of items (LLIs) for AHB DMA */
- num_lli = map_sg_to_lli(sg, num_elems, lli, dma_lli, addr, dir);
-
- dev_dbg(host_pvt.dwc_dev, "%s sg: 0x%p, count: %d lli: %p dma_lli:"
- " 0x%0xlx addr: %p lli count: %d\n", __func__, sg, num_elems,
- lli, (u32)dma_lli, addr, num_lli);
-
- clear_chan_interrupts(dma_ch);
+ sconf.direction = qc->dma_dir;
+ sconf.src_maxburst = AHB_DMA_BRST_DFLT;
+ sconf.dst_maxburst = AHB_DMA_BRST_DFLT;
+ sconf.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ sconf.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
- /* Program the CFG register. */
- out_le32(&(host_pvt.sata_dma_regs->chan_regs[dma_ch].cfg.high),
- DMA_CFG_HW_HS_SRC(dma_ch) | DMA_CFG_HW_HS_DEST(dma_ch) |
- DMA_CFG_PROTCTL | DMA_CFG_FCMOD_REQ);
- out_le32(&(host_pvt.sata_dma_regs->chan_regs[dma_ch].cfg.low),
- DMA_CFG_HW_CH_PRIOR(dma_ch));
+ dmaengine_slave_config(hsdevp->chan, &sconf);
- /* Program the address of the linked list */
- out_le32(&(host_pvt.sata_dma_regs->chan_regs[dma_ch].llp.low),
- DMA_LLP_LMS(dma_lli, DMA_LLP_AHBMASTER2));
-
- /* Program the CTL register with src enable / dst enable */
- out_le32(&(host_pvt.sata_dma_regs->chan_regs[dma_ch].ctl.low),
- DMA_CTL_LLP_SRCEN | DMA_CTL_LLP_DSTEN);
- return dma_ch;
-}
-
-/*
- * Function: dma_dwc_exit
- * arguments: None
- * returns status
- * This function exits the SATA DMA driver
- */
-static void dma_dwc_exit(struct sata_dwc_device *hsdev)
-{
- dev_dbg(host_pvt.dwc_dev, "%s:\n", __func__);
- if (host_pvt.sata_dma_regs) {
- iounmap((void __iomem *)host_pvt.sata_dma_regs);
- host_pvt.sata_dma_regs = NULL;
- }
-
- if (hsdev->irq_dma) {
- free_irq(hsdev->irq_dma, hsdev);
- hsdev->irq_dma = 0;
- }
-}
-
-/*
- * Function: dma_dwc_init
- * arguments: hsdev
- * returns status
- * This function initializes the SATA DMA driver
- */
-static int dma_dwc_init(struct sata_dwc_device *hsdev, int irq)
-{
- int err;
+ /* Convert SG list to linked list of items (LLIs) for AHB DMA */
+ desc = dmaengine_prep_slave_sg(hsdevp->chan, qc->sg, qc->n_elem,
+ qc->dma_dir,
+ DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
- err = dma_request_interrupts(hsdev, irq);
- if (err) {
- dev_err(host_pvt.dwc_dev, "%s: dma_request_interrupts returns"
- " %d\n", __func__, err);
- return err;
- }
+ if (!desc)
+ return NULL;
- /* Enabe DMA */
- out_le32(&(host_pvt.sata_dma_regs->dma_cfg.low), DMA_EN);
+ desc->callback = dma_dwc_xfer_done;
+ desc->callback_param = hsdev;
- dev_notice(host_pvt.dwc_dev, "DMA initialized\n");
- dev_dbg(host_pvt.dwc_dev, "SATA DMA registers=0x%p\n", host_pvt.\
- sata_dma_regs);
+ dev_dbg(hsdev->dev, "%s sg: 0x%p, count: %d addr: %pad\n",
+ __func__, qc->sg, qc->n_elem, &addr);
- return 0;
+ return desc;
}
static int sata_dwc_scr_read(struct ata_link *link, unsigned int scr, u32 *val)
struct ata_queued_cmd *qc;
u32 serror;
u8 status, tag;
- u32 err_reg;
ata_ehi_clear_desc(ehi);
serror = core_scr_read(SCR_ERROR);
status = ap->ops->sff_check_status(ap);
- err_reg = in_le32(&(host_pvt.sata_dma_regs->interrupt_status.error.\
- low));
tag = ap->link.active_tag;
- dev_err(ap->dev, "%s SCR_ERROR=0x%08x intpr=0x%08x status=0x%08x "
- "dma_intp=%d pending=%d issued=%d dma_err_status=0x%08x\n",
- __func__, serror, intpr, status, host_pvt.dma_interrupt_count,
- hsdevp->dma_pending[tag], hsdevp->cmd_issued[tag], err_reg);
+ dev_err(ap->dev,
+ "%s SCR_ERROR=0x%08x intpr=0x%08x status=0x%08x dma_intp=%d pending=%d issued=%d",
+ __func__, serror, intpr, status, hsdevp->dma_interrupt_count,
+ hsdevp->dma_pending[tag], hsdevp->cmd_issued[tag]);
/* Clear error register and interrupt bit */
clear_serror();
/* DEV interrupt w/ no active qc? */
if (unlikely(!qc || (qc->tf.flags & ATA_TFLAG_POLLING))) {
- dev_err(ap->dev, "%s interrupt with no active qc "
- "qc=%p\n", __func__, qc);
+ dev_err(ap->dev,
+ "%s interrupt with no active qc qc=%p\n",
+ __func__, qc);
ap->ops->sff_check_status(ap);
handled = 1;
goto DONE;
* operation done interrupt. The command should be
* completed only after both interrupts are seen.
*/
- host_pvt.dma_interrupt_count++;
+ hsdevp->dma_interrupt_count++;
if (hsdevp->dma_pending[tag] == \
SATA_DWC_DMA_PENDING_NONE) {
- dev_err(ap->dev, "%s: DMA not pending "
- "intpr=0x%08x status=0x%08x pending"
- "=%d\n", __func__, intpr, status,
+ dev_err(ap->dev,
+ "%s: DMA not pending intpr=0x%08x status=0x%08x pending=%d\n",
+ __func__, intpr, status,
hsdevp->dma_pending[tag]);
}
- if ((host_pvt.dma_interrupt_count % 2) == 0)
+ if ((hsdevp->dma_interrupt_count % 2) == 0)
sata_dwc_dma_xfer_complete(ap, 1);
} else if (ata_is_pio(qc->tf.protocol)) {
ata_sff_hsm_move(ap, qc, status, 0);
if (sactive != 0 || (host_pvt.sata_dwc_sactive_issued) > 1 || \
tag_mask > 1) {
- dev_dbg(ap->dev, "%s NCQ:sactive=0x%08x sactive_issued=0x%08x"
- "tag_mask=0x%08x\n", __func__, sactive,
- host_pvt.sata_dwc_sactive_issued, tag_mask);
+ dev_dbg(ap->dev,
+ "%s NCQ:sactive=0x%08x sactive_issued=0x%08x tag_mask=0x%08x\n",
+ __func__, sactive, host_pvt.sata_dwc_sactive_issued,
+ tag_mask);
}
if ((tag_mask | (host_pvt.sata_dwc_sactive_issued)) != \
(host_pvt.sata_dwc_sactive_issued)) {
- dev_warn(ap->dev, "Bad tag mask? sactive=0x%08x "
- "(host_pvt.sata_dwc_sactive_issued)=0x%08x tag_mask"
- "=0x%08x\n", sactive, host_pvt.sata_dwc_sactive_issued,
- tag_mask);
+ dev_warn(ap->dev,
+ "Bad tag mask? sactive=0x%08x (host_pvt.sata_dwc_sactive_issued)=0x%08x tag_mask=0x%08x\n",
+ sactive, host_pvt.sata_dwc_sactive_issued, tag_mask);
}
/* read just to clear ... not bad if currently still busy */
dev_dbg(ap->dev, "%s NCQ command, protocol: %s\n", __func__,
get_prot_descript(qc->tf.protocol));
if (ata_is_dma(qc->tf.protocol)) {
- host_pvt.dma_interrupt_count++;
+ hsdevp->dma_interrupt_count++;
if (hsdevp->dma_pending[tag] == \
SATA_DWC_DMA_PENDING_NONE)
dev_warn(ap->dev, "%s: DMA not pending?\n",
__func__);
- if ((host_pvt.dma_interrupt_count % 2) == 0)
+ if ((hsdevp->dma_interrupt_count % 2) == 0)
sata_dwc_dma_xfer_complete(ap, 1);
} else {
if (unlikely(sata_dwc_qc_complete(ap, qc, 1)))
*/
sactive2 = core_scr_read(SCR_ACTIVE);
if (sactive2 != sactive) {
- dev_dbg(ap->dev, "More completed - sactive=0x%x sactive2"
- "=0x%x\n", sactive, sactive2);
+ dev_dbg(ap->dev,
+ "More completed - sactive=0x%x sactive2=0x%x\n",
+ sactive, sactive2);
}
handled = 1;
* This should not happen, it indicates the driver is out of
* sync. If it does happen, clear dmacr anyway.
*/
- dev_err(host_pvt.dwc_dev, "%s DMA protocol RX and"
- "TX DMA not pending tag=0x%02x pending=%d"
- " dmacr: 0x%08x\n", __func__, tag,
- hsdevp->dma_pending[tag],
- in_le32(&(hsdev->sata_dwc_regs->dmacr)));
+ dev_err(hsdev->dev,
+ "%s DMA protocol RX and TX DMA not pending tag=0x%02x pending=%d dmacr: 0x%08x\n",
+ __func__, tag, hsdevp->dma_pending[tag],
+ in_le32(&hsdev->sata_dwc_regs->dmacr));
out_le32(&(hsdev->sata_dwc_regs->dmacr),
SATA_DWC_DMACR_TXRXCH_CLEAR);
}
#ifdef DEBUG_NCQ
if (tag > 0) {
- dev_info(ap->dev, "%s tag=%u cmd=0x%02x dma dir=%s proto=%s "
- "dmacr=0x%08x\n", __func__, qc->tag, qc->tf.command,
+ dev_info(ap->dev,
+ "%s tag=%u cmd=0x%02x dma dir=%s proto=%s dmacr=0x%08x\n",
+ __func__, qc->tag, qc->tf.command,
get_dma_dir_descript(qc->dma_dir),
get_prot_descript(qc->tf.protocol),
in_le32(&(hsdev->sata_dwc_regs->dmacr)));
if (ata_is_dma(qc->tf.protocol)) {
if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_NONE) {
- dev_err(ap->dev, "%s DMA protocol RX and TX DMA not "
- "pending dmacr: 0x%08x\n", __func__,
+ dev_err(ap->dev,
+ "%s DMA protocol RX and TX DMA not pending dmacr: 0x%08x\n",
+ __func__,
in_le32(&(hsdev->sata_dwc_regs->dmacr)));
}
dev_err(ap->dev, "TX DMA PENDING\n");
else if (hsdevp->dma_pending[tag] == SATA_DWC_DMA_PENDING_RX)
dev_err(ap->dev, "RX DMA PENDING\n");
- dev_dbg(ap->dev, "QC complete cmd=0x%02x status=0x%02x ata%u:"
- " protocol=%d\n", qc->tf.command, status, ap->print_id,
- qc->tf.protocol);
+ dev_dbg(ap->dev,
+ "QC complete cmd=0x%02x status=0x%02x ata%u: protocol=%d\n",
+ qc->tf.command, status, ap->print_id, qc->tf.protocol);
/* clear active bit */
mask = (~(qcmd_tag_to_mask(tag)));
*/
out_le32(&hsdev->sata_dwc_regs->errmr, SATA_DWC_SERROR_ERR_BITS);
- dev_dbg(host_pvt.dwc_dev, "%s: INTMR = 0x%08x, ERRMR = 0x%08x\n",
+ dev_dbg(hsdev->dev, "%s: INTMR = 0x%08x, ERRMR = 0x%08x\n",
__func__, in_le32(&hsdev->sata_dwc_regs->intmr),
in_le32(&hsdev->sata_dwc_regs->errmr));
}
+static bool sata_dwc_dma_filter(struct dma_chan *chan, void *param)
+{
+ struct sata_dwc_device_port *hsdevp = param;
+ struct dw_dma_slave *dws = hsdevp->dws;
+
+ if (dws->dma_dev != chan->device->dev)
+ return false;
+
+ chan->private = dws;
+ return true;
+}
+
static void sata_dwc_setup_port(struct ata_ioports *port, unsigned long base)
{
port->cmd_addr = (void __iomem *)base + 0x00;
struct sata_dwc_device *hsdev;
struct sata_dwc_device_port *hsdevp = NULL;
struct device *pdev;
+ dma_cap_mask_t mask;
int i;
hsdev = HSDEV_FROM_AP(ap);
}
hsdevp->hsdev = hsdev;
+ hsdevp->dws = &sata_dwc_dma_dws;
+ hsdevp->dws->dma_dev = hsdev->dev;
+
+ dma_cap_zero(mask);
+ dma_cap_set(DMA_SLAVE, mask);
+
+ /* Acquire DMA channel */
+ hsdevp->chan = dma_request_channel(mask, sata_dwc_dma_filter, hsdevp);
+ if (!hsdevp->chan) {
+ dev_err(hsdev->dev, "%s: dma channel unavailable\n",
+ __func__);
+ err = -EAGAIN;
+ goto CLEANUP_ALLOC;
+ }
+
for (i = 0; i < SATA_DWC_QCMD_MAX; i++)
hsdevp->cmd_issued[i] = SATA_DWC_CMD_ISSUED_NOT;
ap->bmdma_prd = NULL; /* set these so libata doesn't use them */
ap->bmdma_prd_dma = 0;
- /*
- * DMA - Assign scatter gather LLI table. We can't use the libata
- * version since it's PRD is IDE PCI specific.
- */
- for (i = 0; i < SATA_DWC_QCMD_MAX; i++) {
- hsdevp->llit[i] = dma_alloc_coherent(pdev,
- SATA_DWC_DMAC_LLI_TBL_SZ,
- &(hsdevp->llit_dma[i]),
- GFP_ATOMIC);
- if (!hsdevp->llit[i]) {
- dev_err(ap->dev, "%s: dma_alloc_coherent failed\n",
- __func__);
- err = -ENOMEM;
- goto CLEANUP_ALLOC;
- }
- }
-
if (ap->port_no == 0) {
dev_dbg(ap->dev, "%s: clearing TXCHEN, RXCHEN in DMAC\n",
__func__);
static void sata_dwc_port_stop(struct ata_port *ap)
{
- int i;
- struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
dev_dbg(ap->dev, "%s: ap->id = %d\n", __func__, ap->print_id);
- if (hsdevp && hsdev) {
- /* deallocate LLI table */
- for (i = 0; i < SATA_DWC_QCMD_MAX; i++) {
- dma_free_coherent(ap->host->dev,
- SATA_DWC_DMAC_LLI_TBL_SZ,
- hsdevp->llit[i], hsdevp->llit_dma[i]);
- }
+ dmaengine_terminate_all(hsdevp->chan);
+ dma_release_channel(hsdevp->chan);
- kfree(hsdevp);
- }
+ kfree(hsdevp);
ap->private_data = NULL;
}
static void sata_dwc_bmdma_start_by_tag(struct ata_queued_cmd *qc, u8 tag)
{
int start_dma;
- u32 reg, dma_chan;
+ u32 reg;
struct sata_dwc_device *hsdev = HSDEV_FROM_QC(qc);
struct ata_port *ap = qc->ap;
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
+ struct dma_async_tx_descriptor *desc = hsdevp->desc[tag];
int dir = qc->dma_dir;
- dma_chan = hsdevp->dma_chan[tag];
if (hsdevp->cmd_issued[tag] != SATA_DWC_CMD_ISSUED_NOT) {
start_dma = 1;
else
hsdevp->dma_pending[tag] = SATA_DWC_DMA_PENDING_RX;
} else {
- dev_err(ap->dev, "%s: Command not pending cmd_issued=%d "
- "(tag=%d) DMA NOT started\n", __func__,
- hsdevp->cmd_issued[tag], tag);
+ dev_err(ap->dev,
+ "%s: Command not pending cmd_issued=%d (tag=%d) DMA NOT started\n",
+ __func__, hsdevp->cmd_issued[tag], tag);
start_dma = 0;
}
- dev_dbg(ap->dev, "%s qc=%p tag: %x cmd: 0x%02x dma_dir: %s "
- "start_dma? %x\n", __func__, qc, tag, qc->tf.command,
+ dev_dbg(ap->dev,
+ "%s qc=%p tag: %x cmd: 0x%02x dma_dir: %s start_dma? %x\n",
+ __func__, qc, tag, qc->tf.command,
get_dma_dir_descript(qc->dma_dir), start_dma);
- sata_dwc_tf_dump(&(qc->tf));
+ sata_dwc_tf_dump(ap, &qc->tf);
if (start_dma) {
reg = core_scr_read(SCR_ERROR);
SATA_DWC_DMACR_RXCHEN);
/* Enable AHB DMA transfer on the specified channel */
- dma_dwc_xfer_start(dma_chan);
+ dmaengine_submit(desc);
+ dma_async_issue_pending(hsdevp->chan);
}
}
*/
static void sata_dwc_qc_prep_by_tag(struct ata_queued_cmd *qc, u8 tag)
{
- struct scatterlist *sg = qc->sg;
+ struct dma_async_tx_descriptor *desc;
struct ata_port *ap = qc->ap;
- int dma_chan;
- struct sata_dwc_device *hsdev = HSDEV_FROM_AP(ap);
struct sata_dwc_device_port *hsdevp = HSDEVP_FROM_AP(ap);
dev_dbg(ap->dev, "%s: port=%d dma dir=%s n_elem=%d\n",
__func__, ap->port_no, get_dma_dir_descript(qc->dma_dir),
qc->n_elem);
- dma_chan = dma_dwc_xfer_setup(sg, qc->n_elem, hsdevp->llit[tag],
- hsdevp->llit_dma[tag],
- (void __iomem *)&hsdev->sata_dwc_regs->dmadr,
- qc->dma_dir);
- if (dma_chan < 0) {
- dev_err(ap->dev, "%s: dma_dwc_xfer_setup returns err %d\n",
- __func__, dma_chan);
+ desc = dma_dwc_xfer_setup(qc);
+ if (!desc) {
+ dev_err(ap->dev, "%s: dma_dwc_xfer_setup returns NULL\n",
+ __func__);
return;
}
- hsdevp->dma_chan[tag] = dma_chan;
+ hsdevp->desc[tag] = desc;
}
static unsigned int sata_dwc_qc_issue(struct ata_queued_cmd *qc)
#ifdef DEBUG_NCQ
if (qc->tag > 0 || ap->link.sactive > 1)
- dev_info(ap->dev, "%s ap id=%d cmd(0x%02x)=%s qc tag=%d "
- "prot=%s ap active_tag=0x%08x ap sactive=0x%08x\n",
+ dev_info(ap->dev,
+ "%s ap id=%d cmd(0x%02x)=%s qc tag=%d prot=%s ap active_tag=0x%08x ap sactive=0x%08x\n",
__func__, ap->print_id, qc->tf.command,
ata_get_cmd_descript(qc->tf.command),
qc->tag, get_prot_descript(qc->tf.protocol),
sactive |= (0x00000001 << tag);
core_scr_write(SCR_ACTIVE, sactive);
- dev_dbg(qc->ap->dev, "%s: tag=%d ap->link.sactive = 0x%08x "
- "sactive=0x%08x\n", __func__, tag, qc->ap->link.sactive,
- sactive);
+ dev_dbg(qc->ap->dev,
+ "%s: tag=%d ap->link.sactive = 0x%08x sactive=0x%08x\n",
+ __func__, tag, qc->ap->link.sactive, sactive);
ap->ops->sff_tf_load(ap, &qc->tf);
sata_dwc_exec_command_by_tag(ap, &qc->tf, qc->tag,
struct ata_port_info pi = sata_dwc_port_info[0];
const struct ata_port_info *ppi[] = { &pi, NULL };
struct device_node *np = ofdev->dev.of_node;
- u32 dma_chan;
/* Allocate DWC SATA device */
host = ata_host_alloc_pinfo(&ofdev->dev, ppi, SATA_DWC_MAX_PORTS);
host->private_data = hsdev;
- if (of_property_read_u32(np, "dma-channel", &dma_chan)) {
- dev_warn(&ofdev->dev, "no dma-channel property set."
- " Use channel 0\n");
- dma_chan = 0;
- }
- host_pvt.dma_channel = dma_chan;
-
/* Ioremap SATA registers */
base = of_iomap(np, 0);
if (!base) {
- dev_err(&ofdev->dev, "ioremap failed for SATA register"
- " address\n");
+ dev_err(&ofdev->dev,
+ "ioremap failed for SATA register address\n");
return -ENODEV;
}
hsdev->reg_base = base;
idr, ver[0], ver[1], ver[2]);
/* Get SATA DMA interrupt number */
- irq = irq_of_parse_and_map(np, 1);
- if (irq == NO_IRQ) {
+ hsdev->dma->irq = irq_of_parse_and_map(np, 1);
+ if (hsdev->dma->irq == NO_IRQ) {
dev_err(&ofdev->dev, "no SATA DMA irq\n");
err = -ENODEV;
goto error_iomap;
}
/* Get physical SATA DMA register base address */
- host_pvt.sata_dma_regs = (void *)of_iomap(np, 1);
- if (!(host_pvt.sata_dma_regs)) {
- dev_err(&ofdev->dev, "ioremap failed for AHBDMA register"
- " address\n");
+ hsdev->dma->regs = of_iomap(np, 1);
+ if (!hsdev->dma->regs) {
+ dev_err(&ofdev->dev,
+ "ioremap failed for AHBDMA register address\n");
err = -ENODEV;
goto error_iomap;
}
/* Save dev for later use in dev_xxx() routines */
- host_pvt.dwc_dev = &ofdev->dev;
+ hsdev->dev = &ofdev->dev;
+
+ hsdev->dma->dev = &ofdev->dev;
/* Initialize AHB DMAC */
- err = dma_dwc_init(hsdev, irq);
+ err = dw_dma_probe(hsdev->dma, NULL);
if (err)
goto error_dma_iomap;
error_out:
/* Free SATA DMA resources */
- dma_dwc_exit(hsdev);
+ dw_dma_remove(hsdev->dma);
error_dma_iomap:
- iounmap((void __iomem *)host_pvt.sata_dma_regs);
+ iounmap(hsdev->dma->regs);
error_iomap:
iounmap(base);
return err;
ata_host_detach(host);
/* Free SATA DMA resources */
- dma_dwc_exit(hsdev);
+ dw_dma_remove(hsdev->dma);
- iounmap((void __iomem *)host_pvt.sata_dma_regs);
+ iounmap(hsdev->dma->regs);
iounmap(hsdev->reg_base);
dev_dbg(&ofdev->dev, "done\n");
return 0;
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Mark Miesfeld <mmiesfeld@amcc.com>");
-MODULE_DESCRIPTION("DesignWare Cores SATA controller low lever driver");
+MODULE_DESCRIPTION("DesignWare Cores SATA controller low level driver");
MODULE_VERSION(DRV_VERSION);
*/
ata_msleep(ap, 1);
+ sata_set_spd(link);
+
/*
* Now, bring the host controller online again, this can take time
* as PHY reset and communication establishment, 1st D2H FIS and
}
/* Set dma_mask. This devices doesn't support 64bit addressing. */
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit consistent DMA enable failed\n");
return rc;
MV5_PHY_CTL = 0x0C,
SATA_IFCFG = 0x050,
LP_PHY_CTL = 0x058,
+ LP_PHY_CTL_PIN_PU_PLL = (1 << 0),
+ LP_PHY_CTL_PIN_PU_RX = (1 << 1),
+ LP_PHY_CTL_PIN_PU_TX = (1 << 2),
+ LP_PHY_CTL_GEN_TX_3G = (1 << 5),
+ LP_PHY_CTL_GEN_RX_3G = (1 << 9),
MV_M2_PREAMP_MASK = 0x7e0,
/*
* Set PHY speed according to SControl speed.
*/
- if ((val & 0xf0) == 0x10)
- writelfl(0x7, lp_phy_addr);
- else
- writelfl(0x227, lp_phy_addr);
+ u32 lp_phy_val =
+ LP_PHY_CTL_PIN_PU_PLL |
+ LP_PHY_CTL_PIN_PU_RX |
+ LP_PHY_CTL_PIN_PU_TX;
+
+ if ((val & 0xf0) != 0x10)
+ lp_phy_val |=
+ LP_PHY_CTL_GEN_TX_3G |
+ LP_PHY_CTL_GEN_RX_3G;
+
+ writelfl(lp_phy_val, lp_phy_addr);
}
}
writelfl(val, addr);
{
int rc;
- if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
if (rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"64-bit DMA enable failed\n");
}
}
} else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"32-bit consistent DMA enable failed\n");
blk_queue_bounce_limit(sdev1->request_queue,
ATA_DMA_MASK);
- pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ dma_set_mask(&pdev->dev, ATA_DMA_MASK);
} else {
/** This shouldn't fail as it was set to this value before */
- pci_set_dma_mask(pdev, pp->adma_dma_mask);
+ dma_set_mask(&pdev->dev, pp->adma_dma_mask);
if (sdev0)
blk_queue_bounce_limit(sdev0->request_queue,
pp->adma_dma_mask);
/* Ensure DMA mask is set to 32-bit before allocating legacy PRD and
pad buffers */
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc)
return rc;
These are allowed to fail since we store the value that ends up
being used to set as the bounce limit in slave_config later if
needed. */
- pci_set_dma_mask(pdev, DMA_BIT_MASK(64));
- pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ dma_set_mask(&pdev->dev, DMA_BIT_MASK(64));
+ dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
pp->adma_dma_mask = *dev->dma_mask;
mem = dmam_alloc_coherent(dev, NV_ADMA_PORT_PRIV_DMA_SZ,
/* initialize adapter */
pdc_host_init(host);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
int rc, have_64bit_bus = (bus_info & QS_HPHY_64BIT);
if (have_64bit_bus &&
- !pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ !dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
if (rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"64-bit DMA enable failed\n");
}
}
} else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"32-bit consistent DMA enable failed\n");
return rc;
host->iomap = pcim_iomap_table(pdev);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
/* host flags */
SIL24_COMMON_FLAGS = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA |
ATA_FLAG_NCQ | ATA_FLAG_ACPI_SATA |
- ATA_FLAG_AN | ATA_FLAG_PMP | ATA_FLAG_LOWTAG,
+ ATA_FLAG_AN | ATA_FLAG_PMP,
SIL24_FLAG_PCIX_IRQ_WOC = (1 << 24), /* IRQ loss errata on PCI-X */
IRQ_STAT_4PORTS = 0xf,
host->iomap = iomap;
/* configure and activate the device */
- if (!pci_set_dma_mask(pdev, DMA_BIT_MASK(64))) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(64));
+ if (!dma_set_mask(&pdev->dev, DMA_BIT_MASK(64))) {
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(64));
if (rc) {
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"64-bit DMA enable failed\n");
}
}
} else {
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev, "32-bit DMA enable failed\n");
return rc;
}
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc) {
dev_err(&pdev->dev,
"32-bit consistent DMA enable failed\n");
ata_port_pbar_desc(ap, 5, offset, "port");
}
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
}
/* configure and activate */
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
for (i = 0; i < host->n_ports; i++)
vt6421_init_addrs(host->ports[i]);
- rc = pci_set_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, ATA_DMA_MASK);
+ rc = dma_set_coherent_mask(&pdev->dev, ATA_DMA_MASK);
if (rc)
return rc;
/*
* Use 32 bit DMA mask, because 64 bit address support is poor.
*/
- rc = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc)
return rc;
- rc = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
+ rc = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (rc)
return rc;
}
static int pm_genpd_summary_one(struct seq_file *s,
- struct generic_pm_domain *gpd)
+ struct generic_pm_domain *genpd)
{
static const char * const status_lookup[] = {
[GPD_STATE_ACTIVE] = "on",
struct gpd_link *link;
int ret;
- ret = mutex_lock_interruptible(&gpd->lock);
+ ret = mutex_lock_interruptible(&genpd->lock);
if (ret)
return -ERESTARTSYS;
- if (WARN_ON(gpd->status >= ARRAY_SIZE(status_lookup)))
+ if (WARN_ON(genpd->status >= ARRAY_SIZE(status_lookup)))
goto exit;
- seq_printf(s, "%-30s %-15s ", gpd->name, status_lookup[gpd->status]);
+ seq_printf(s, "%-30s %-15s ", genpd->name, status_lookup[genpd->status]);
/*
* Modifications on the list require holding locks on both
* master and slave, so we are safe.
- * Also gpd->name is immutable.
+ * Also genpd->name is immutable.
*/
- list_for_each_entry(link, &gpd->master_links, master_node) {
+ list_for_each_entry(link, &genpd->master_links, master_node) {
seq_printf(s, "%s", link->slave->name);
- if (!list_is_last(&link->master_node, &gpd->master_links))
+ if (!list_is_last(&link->master_node, &genpd->master_links))
seq_puts(s, ", ");
}
- list_for_each_entry(pm_data, &gpd->dev_list, list_node) {
+ list_for_each_entry(pm_data, &genpd->dev_list, list_node) {
kobj_path = kobject_get_path(&pm_data->dev->kobj, GFP_KERNEL);
if (kobj_path == NULL)
continue;
seq_puts(s, "\n");
exit:
- mutex_unlock(&gpd->lock);
+ mutex_unlock(&genpd->lock);
return 0;
}
static int pm_genpd_summary_show(struct seq_file *s, void *data)
{
- struct generic_pm_domain *gpd;
+ struct generic_pm_domain *genpd;
int ret = 0;
seq_puts(s, " domain status slaves\n");
if (ret)
return -ERESTARTSYS;
- list_for_each_entry(gpd, &gpd_list, gpd_list_node) {
- ret = pm_genpd_summary_one(s, gpd);
+ list_for_each_entry(genpd, &gpd_list, gpd_list_node) {
+ ret = pm_genpd_summary_one(s, genpd);
if (ret)
break;
}
pm_abort_suspend = true;
freeze_wake();
}
+EXPORT_SYMBOL_GPL(pm_system_wakeup);
void pm_wakeup_clear(void)
{
+# For include/trace/define_trace.h to include trace.h
+CFLAGS_regmap.o := -I$(src)
+
obj-$(CONFIG_REGMAP) += regmap.o regcache.o
obj-$(CONFIG_REGMAP) += regcache-rbtree.o regcache-lzo.o regcache-flat.o
obj-$(CONFIG_DEBUG_FS) += regmap-debugfs.o
extern struct regcache_ops regcache_lzo_ops;
extern struct regcache_ops regcache_flat_ops;
+static inline const char *regmap_name(const struct regmap *map)
+{
+ if (map->dev)
+ return dev_name(map->dev);
+
+ return map->name;
+}
+
#endif
if (pos == 0) {
memmove(blk + offset * map->cache_word_size,
blk, rbnode->blklen * map->cache_word_size);
- bitmap_shift_right(present, present, offset, blklen);
+ bitmap_shift_left(present, present, offset, blklen);
}
/* update the rbnode block, its size and the base register */
#include <linux/export.h>
#include <linux/slab.h>
#include <linux/sort.h>
-#include <trace/events/regmap.h>
+#include "trace.h"
#include "internal.h"
static const struct regcache_ops *cache_types[] = {
ret = map->cache_ops->read(map, reg, value);
if (ret == 0)
- trace_regmap_reg_read_cache(map->dev, reg, *value);
+ trace_regmap_reg_read_cache(map, reg, *value);
return ret;
}
dev_dbg(map->dev, "Syncing %s cache\n",
map->cache_ops->name);
name = map->cache_ops->name;
- trace_regcache_sync(map->dev, name, "start");
+ trace_regcache_sync(map, name, "start");
if (!map->cache_dirty)
goto out;
regmap_async_complete(map);
- trace_regcache_sync(map->dev, name, "stop");
+ trace_regcache_sync(map, name, "stop");
return ret;
}
name = map->cache_ops->name;
dev_dbg(map->dev, "Syncing %s cache from %d-%d\n", name, min, max);
- trace_regcache_sync(map->dev, name, "start region");
+ trace_regcache_sync(map, name, "start region");
if (!map->cache_dirty)
goto out;
regmap_async_complete(map);
- trace_regcache_sync(map->dev, name, "stop region");
+ trace_regcache_sync(map, name, "stop region");
return ret;
}
map->lock(map->lock_arg);
- trace_regcache_drop_region(map->dev, min, max);
+ trace_regcache_drop_region(map, min, max);
ret = map->cache_ops->drop(map, min, max);
map->lock(map->lock_arg);
WARN_ON(map->cache_bypass && enable);
map->cache_only = enable;
- trace_regmap_cache_only(map->dev, enable);
+ trace_regmap_cache_only(map, enable);
map->unlock(map->lock_arg);
}
EXPORT_SYMBOL_GPL(regcache_cache_only);
map->lock(map->lock_arg);
WARN_ON(map->cache_only && enable);
map->cache_bypass = enable;
- trace_regmap_cache_bypass(map->dev, enable);
+ trace_regmap_cache_bypass(map, enable);
map->unlock(map->lock_arg);
}
EXPORT_SYMBOL_GPL(regcache_cache_bypass);
for (i = start; i < end; i++) {
regtmp = block_base + (i * map->reg_stride);
- if (!regcache_reg_present(cache_present, i))
+ if (!regcache_reg_present(cache_present, i) ||
+ !regmap_writeable(map, regtmp))
continue;
val = regcache_get_val(map, block, i);
for (i = start; i < end; i++) {
regtmp = block_base + (i * map->reg_stride);
- if (!regcache_reg_present(cache_present, i)) {
+ if (!regcache_reg_present(cache_present, i) ||
+ !regmap_writeable(map, regtmp)) {
ret = regcache_sync_block_raw_flush(map, &data,
base, regtmp);
if (ret != 0)
goto err_alloc;
}
- ret = request_threaded_irq(irq, NULL, regmap_irq_thread, irq_flags,
+ ret = request_threaded_irq(irq, NULL, regmap_irq_thread,
+ irq_flags | IRQF_ONESHOT,
chip->name, d);
if (ret != 0) {
dev_err(map->dev, "Failed to request IRQ %d for %s: %d\n",
#include <linux/sched.h>
#define CREATE_TRACE_POINTS
-#include <trace/events/regmap.h>
+#include "trace.h"
#include "internal.h"
if (map->async && map->bus->async_write) {
struct regmap_async *async;
- trace_regmap_async_write_start(map->dev, reg, val_len);
+ trace_regmap_async_write_start(map, reg, val_len);
spin_lock_irqsave(&map->async_lock, flags);
async = list_first_entry_or_null(&map->async_free,
return ret;
}
- trace_regmap_hw_write_start(map->dev, reg,
- val_len / map->format.val_bytes);
+ trace_regmap_hw_write_start(map, reg, val_len / map->format.val_bytes);
/* If we're doing a single register write we can probably just
* send the work_buf directly, otherwise try to do a gather
kfree(buf);
}
- trace_regmap_hw_write_done(map->dev, reg,
- val_len / map->format.val_bytes);
+ trace_regmap_hw_write_done(map, reg, val_len / map->format.val_bytes);
return ret;
}
map->format.format_write(map, reg, val);
- trace_regmap_hw_write_start(map->dev, reg, 1);
+ trace_regmap_hw_write_start(map, reg, 1);
ret = map->bus->write(map->bus_context, map->work_buf,
map->format.buf_size);
- trace_regmap_hw_write_done(map->dev, reg, 1);
+ trace_regmap_hw_write_done(map, reg, 1);
return ret;
}
dev_info(map->dev, "%x <= %x\n", reg, val);
#endif
- trace_regmap_reg_write(map->dev, reg, val);
+ trace_regmap_reg_write(map, reg, val);
return map->reg_write(context, reg, val);
}
for (i = 0; i < num_regs; i++) {
int reg = regs[i].reg;
int val = regs[i].def;
- trace_regmap_hw_write_start(map->dev, reg, 1);
+ trace_regmap_hw_write_start(map, reg, 1);
map->format.format_reg(u8, reg, map->reg_shift);
u8 += reg_bytes + pad_bytes;
map->format.format_val(u8, val, 0);
for (i = 0; i < num_regs; i++) {
int reg = regs[i].reg;
- trace_regmap_hw_write_done(map->dev, reg, 1);
+ trace_regmap_hw_write_done(map, reg, 1);
}
return ret;
}
*/
u8[0] |= map->read_flag_mask;
- trace_regmap_hw_read_start(map->dev, reg,
- val_len / map->format.val_bytes);
+ trace_regmap_hw_read_start(map, reg, val_len / map->format.val_bytes);
ret = map->bus->read(map->bus_context, map->work_buf,
map->format.reg_bytes + map->format.pad_bytes,
val, val_len);
- trace_regmap_hw_read_done(map->dev, reg,
- val_len / map->format.val_bytes);
+ trace_regmap_hw_read_done(map, reg, val_len / map->format.val_bytes);
return ret;
}
dev_info(map->dev, "%x => %x\n", reg, *val);
#endif
- trace_regmap_reg_read(map->dev, reg, *val);
+ trace_regmap_reg_read(map, reg, *val);
if (!map->cache_bypass)
regcache_write(map, reg, *val);
struct regmap *map = async->map;
bool wake;
- trace_regmap_async_io_complete(map->dev);
+ trace_regmap_async_io_complete(map);
spin_lock(&map->async_lock);
list_move(&async->list, &map->async_free);
if (!map->bus || !map->bus->async_write)
return 0;
- trace_regmap_async_complete_start(map->dev);
+ trace_regmap_async_complete_start(map);
wait_event(map->async_waitq, regmap_async_is_done(map));
map->async_ret = 0;
spin_unlock_irqrestore(&map->async_lock, flags);
- trace_regmap_async_complete_done(map->dev);
+ trace_regmap_async_complete_done(map);
return ret;
}
#include <linux/ktime.h>
#include <linux/tracepoint.h>
-struct device;
-struct regmap;
+#include "internal.h"
/*
* Log register events
*/
DECLARE_EVENT_CLASS(regmap_reg,
- TP_PROTO(struct device *dev, unsigned int reg,
+ TP_PROTO(struct regmap *map, unsigned int reg,
unsigned int val),
- TP_ARGS(dev, reg, val),
+ TP_ARGS(map, reg, val),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
- __field( unsigned int, reg )
- __field( unsigned int, val )
+ __string( name, regmap_name(map) )
+ __field( unsigned int, reg )
+ __field( unsigned int, val )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
__entry->reg = reg;
__entry->val = val;
),
DEFINE_EVENT(regmap_reg, regmap_reg_write,
- TP_PROTO(struct device *dev, unsigned int reg,
+ TP_PROTO(struct regmap *map, unsigned int reg,
unsigned int val),
- TP_ARGS(dev, reg, val)
+ TP_ARGS(map, reg, val)
);
DEFINE_EVENT(regmap_reg, regmap_reg_read,
- TP_PROTO(struct device *dev, unsigned int reg,
+ TP_PROTO(struct regmap *map, unsigned int reg,
unsigned int val),
- TP_ARGS(dev, reg, val)
+ TP_ARGS(map, reg, val)
);
DEFINE_EVENT(regmap_reg, regmap_reg_read_cache,
- TP_PROTO(struct device *dev, unsigned int reg,
+ TP_PROTO(struct regmap *map, unsigned int reg,
unsigned int val),
- TP_ARGS(dev, reg, val)
+ TP_ARGS(map, reg, val)
);
DECLARE_EVENT_CLASS(regmap_block,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count),
+ TP_ARGS(map, reg, count),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
- __field( unsigned int, reg )
- __field( int, count )
+ __string( name, regmap_name(map) )
+ __field( unsigned int, reg )
+ __field( int, count )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
__entry->reg = reg;
__entry->count = count;
),
DEFINE_EVENT(regmap_block, regmap_hw_read_start,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count)
+ TP_ARGS(map, reg, count)
);
DEFINE_EVENT(regmap_block, regmap_hw_read_done,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count)
+ TP_ARGS(map, reg, count)
);
DEFINE_EVENT(regmap_block, regmap_hw_write_start,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count)
+ TP_ARGS(map, reg, count)
);
DEFINE_EVENT(regmap_block, regmap_hw_write_done,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count)
+ TP_ARGS(map, reg, count)
);
TRACE_EVENT(regcache_sync,
- TP_PROTO(struct device *dev, const char *type,
+ TP_PROTO(struct regmap *map, const char *type,
const char *status),
- TP_ARGS(dev, type, status),
+ TP_ARGS(map, type, status),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
- __string( status, status )
- __string( type, type )
- __field( int, type )
+ __string( name, regmap_name(map) )
+ __string( status, status )
+ __string( type, type )
+ __field( int, type )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
__assign_str(status, status);
__assign_str(type, type);
),
DECLARE_EVENT_CLASS(regmap_bool,
- TP_PROTO(struct device *dev, bool flag),
+ TP_PROTO(struct regmap *map, bool flag),
- TP_ARGS(dev, flag),
+ TP_ARGS(map, flag),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
- __field( int, flag )
+ __string( name, regmap_name(map) )
+ __field( int, flag )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
__entry->flag = flag;
),
DEFINE_EVENT(regmap_bool, regmap_cache_only,
- TP_PROTO(struct device *dev, bool flag),
+ TP_PROTO(struct regmap *map, bool flag),
- TP_ARGS(dev, flag)
+ TP_ARGS(map, flag)
);
DEFINE_EVENT(regmap_bool, regmap_cache_bypass,
- TP_PROTO(struct device *dev, bool flag),
+ TP_PROTO(struct regmap *map, bool flag),
- TP_ARGS(dev, flag)
+ TP_ARGS(map, flag)
);
DECLARE_EVENT_CLASS(regmap_async,
- TP_PROTO(struct device *dev),
+ TP_PROTO(struct regmap *map),
- TP_ARGS(dev),
+ TP_ARGS(map),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
+ __string( name, regmap_name(map) )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
),
TP_printk("%s", __get_str(name))
DEFINE_EVENT(regmap_block, regmap_async_write_start,
- TP_PROTO(struct device *dev, unsigned int reg, int count),
+ TP_PROTO(struct regmap *map, unsigned int reg, int count),
- TP_ARGS(dev, reg, count)
+ TP_ARGS(map, reg, count)
);
DEFINE_EVENT(regmap_async, regmap_async_io_complete,
- TP_PROTO(struct device *dev),
+ TP_PROTO(struct regmap *map),
- TP_ARGS(dev)
+ TP_ARGS(map)
);
DEFINE_EVENT(regmap_async, regmap_async_complete_start,
- TP_PROTO(struct device *dev),
+ TP_PROTO(struct regmap *map),
- TP_ARGS(dev)
+ TP_ARGS(map)
);
DEFINE_EVENT(regmap_async, regmap_async_complete_done,
- TP_PROTO(struct device *dev),
+ TP_PROTO(struct regmap *map),
- TP_ARGS(dev)
+ TP_ARGS(map)
);
TRACE_EVENT(regcache_drop_region,
- TP_PROTO(struct device *dev, unsigned int from,
+ TP_PROTO(struct regmap *map, unsigned int from,
unsigned int to),
- TP_ARGS(dev, from, to),
+ TP_ARGS(map, from, to),
TP_STRUCT__entry(
- __string( name, dev_name(dev) )
- __field( unsigned int, from )
- __field( unsigned int, to )
+ __string( name, regmap_name(map) )
+ __field( unsigned int, from )
+ __field( unsigned int, to )
),
TP_fast_assign(
- __assign_str(name, dev_name(dev));
+ __assign_str(name, regmap_name(map));
__entry->from = from;
__entry->to = to;
),
#endif /* _TRACE_REGMAP_H */
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+
+#undef TRACE_INCLUDE_FILE
+#define TRACE_INCLUDE_FILE trace
+
/* This part must be outside protection */
#include <trace/define_trace.h>
goto Enomem4;
}
hba[i]->access.set_intr_mask(hba[i], 0);
- if (request_irq(hba[i]->intr, do_ida_intr,
- IRQF_DISABLED|IRQF_SHARED, hba[i]->devname, hba[i]))
+ if (request_irq(hba[i]->intr, do_ida_intr, IRQF_SHARED,
+ hba[i]->devname, hba[i]))
{
printk(KERN_ERR "cpqarray: Unable to get irq %d for %s\n",
hba[i]->intr, hba[i]->devname);
return -EINVAL;
}
- nbd_dev = kcalloc(nbds_max, sizeof(*nbd_dev), GFP_KERNEL);
- if (!nbd_dev)
- return -ENOMEM;
-
part_shift = 0;
if (max_part > 0) {
part_shift = fls(max_part);
if (nbds_max > 1UL << (MINORBITS - part_shift))
return -EINVAL;
+ nbd_dev = kcalloc(nbds_max, sizeof(*nbd_dev), GFP_KERNEL);
+ if (!nbd_dev)
+ return -ENOMEM;
+
for (i = 0; i < nbds_max; i++) {
struct gendisk *disk = alloc_disk(1 << part_shift);
if (!disk)
}
}
+#ifdef CONFIG_BLK_DEV_INTEGRITY
static void nvme_dif_prep(u32 p, u32 v, struct t10_pi_tuple *pi)
{
if (be32_to_cpu(pi->ref_tag) == v)
kunmap_atomic(pmap);
}
+static int nvme_noop_verify(struct blk_integrity_iter *iter)
+{
+ return 0;
+}
+
+static int nvme_noop_generate(struct blk_integrity_iter *iter)
+{
+ return 0;
+}
+
+struct blk_integrity nvme_meta_noop = {
+ .name = "NVME_META_NOOP",
+ .generate_fn = nvme_noop_generate,
+ .verify_fn = nvme_noop_verify,
+};
+
+static void nvme_init_integrity(struct nvme_ns *ns)
+{
+ struct blk_integrity integrity;
+
+ switch (ns->pi_type) {
+ case NVME_NS_DPS_PI_TYPE3:
+ integrity = t10_pi_type3_crc;
+ break;
+ case NVME_NS_DPS_PI_TYPE1:
+ case NVME_NS_DPS_PI_TYPE2:
+ integrity = t10_pi_type1_crc;
+ break;
+ default:
+ integrity = nvme_meta_noop;
+ break;
+ }
+ integrity.tuple_size = ns->ms;
+ blk_integrity_register(ns->disk, &integrity);
+ blk_queue_max_integrity_segments(ns->queue, 1);
+}
+#else /* CONFIG_BLK_DEV_INTEGRITY */
+static void nvme_dif_remap(struct request *req,
+ void (*dif_swap)(u32 p, u32 v, struct t10_pi_tuple *pi))
+{
+}
+static void nvme_dif_prep(u32 p, u32 v, struct t10_pi_tuple *pi)
+{
+}
+static void nvme_dif_complete(u32 p, u32 v, struct t10_pi_tuple *pi)
+{
+}
+static void nvme_init_integrity(struct nvme_ns *ns)
+{
+}
+#endif
+
static void req_completion(struct nvme_queue *nvmeq, void *ctx,
struct nvme_completion *cqe)
{
queue_flag_set_unlocked(QUEUE_FLAG_DISCARD, ns->queue);
}
-static int nvme_noop_verify(struct blk_integrity_iter *iter)
-{
- return 0;
-}
-
-static int nvme_noop_generate(struct blk_integrity_iter *iter)
-{
- return 0;
-}
-
-struct blk_integrity nvme_meta_noop = {
- .name = "NVME_META_NOOP",
- .generate_fn = nvme_noop_generate,
- .verify_fn = nvme_noop_verify,
-};
-
-static void nvme_init_integrity(struct nvme_ns *ns)
-{
- struct blk_integrity integrity;
-
- switch (ns->pi_type) {
- case NVME_NS_DPS_PI_TYPE3:
- integrity = t10_pi_type3_crc;
- break;
- case NVME_NS_DPS_PI_TYPE1:
- case NVME_NS_DPS_PI_TYPE2:
- integrity = t10_pi_type1_crc;
- break;
- default:
- integrity = nvme_meta_noop;
- break;
- }
- integrity.tuple_size = ns->ms;
- blk_integrity_register(ns->disk, &integrity);
- blk_queue_max_integrity_segments(ns->queue, 1);
-}
-
static int nvme_revalidate_disk(struct gendisk *disk)
{
struct nvme_ns *ns = disk->private_data;
pi_type = ns->ms == sizeof(struct t10_pi_tuple) ?
id->dps & NVME_NS_DPS_PI_MASK : 0;
- if (disk->integrity && (ns->pi_type != pi_type || ns->ms != old_ms ||
+ if (blk_get_integrity(disk) && (ns->pi_type != pi_type ||
+ ns->ms != old_ms ||
bs != queue_logical_block_size(disk->queue) ||
(ns->ms && id->flbas & NVME_NS_FLBAS_META_EXT)))
blk_integrity_unregister(disk);
ns->pi_type = pi_type;
blk_queue_logical_block_size(ns->queue, bs);
- if (ns->ms && !disk->integrity && (disk->flags & GENHD_FL_UP) &&
+ if (ns->ms && !blk_get_integrity(disk) && (disk->flags & GENHD_FL_UP) &&
!(id->flbas & NVME_NS_FLBAS_META_EXT))
nvme_init_integrity(ns);
- if (id->ncap == 0 || (ns->ms && !disk->integrity))
+ if (id->ncap == 0 || (ns->ms && !blk_get_integrity(disk)))
set_capacity(disk, 0);
else
set_capacity(disk, le64_to_cpup(&id->nsze) << (ns->lba_shift - 9));
list_for_each_entry(ns, &dev->namespaces, list) {
if (ns->disk->flags & GENHD_FL_UP) {
- if (ns->disk->integrity)
+ if (blk_get_integrity(ns->disk))
blk_integrity_unregister(ns->disk);
del_gendisk(ns->disk);
}
}
get_device(dev->device);
+ INIT_LIST_HEAD(&dev->node);
INIT_WORK(&dev->probe_work, nvme_async_probe);
schedule_work(&dev->probe_work);
return 0;
static inline void update_used_max(struct zram *zram,
const unsigned long pages)
{
- int old_max, cur_max;
+ unsigned long old_max, cur_max;
old_max = atomic_long_read(&zram->stats.max_used_pages);
{ USB_DEVICE(0x1286, 0x2046), .driver_info = BTUSB_MARVELL },
/* Intel Bluetooth devices */
+ { USB_DEVICE(0x8087, 0x07da), .driver_info = BTUSB_CSR },
{ USB_DEVICE(0x8087, 0x07dc), .driver_info = BTUSB_INTEL },
{ USB_DEVICE(0x8087, 0x0a2a), .driver_info = BTUSB_INTEL },
{ USB_DEVICE(0x8087, 0x0a2b), .driver_info = BTUSB_INTEL_NEW },
*/
l3->debug_irq = platform_get_irq(pdev, 0);
ret = devm_request_irq(l3->dev, l3->debug_irq, l3_interrupt_handler,
- IRQF_DISABLED, "l3-dbg-irq", l3);
+ 0x0, "l3-dbg-irq", l3);
if (ret) {
dev_err(l3->dev, "request_irq failed for %d\n",
l3->debug_irq);
l3->app_irq = platform_get_irq(pdev, 1);
ret = devm_request_irq(l3->dev, l3->app_irq, l3_interrupt_handler,
- IRQF_DISABLED, "l3-app-irq", l3);
+ 0x0, "l3-app-irq", l3);
if (ret)
dev_err(l3->dev, "request_irq failed for %d\n", l3->app_irq);
}
l3->debug_irq = platform_get_irq(pdev, 0);
- ret = request_irq(l3->debug_irq, omap3_l3_app_irq,
- IRQF_DISABLED | IRQF_TRIGGER_RISING,
- "l3-debug-irq", l3);
+ ret = request_irq(l3->debug_irq, omap3_l3_app_irq, IRQF_TRIGGER_RISING,
+ "l3-debug-irq", l3);
if (ret) {
dev_err(&pdev->dev, "couldn't request debug irq\n");
goto err1;
}
l3->app_irq = platform_get_irq(pdev, 1);
- ret = request_irq(l3->app_irq, omap3_l3_app_irq,
- IRQF_DISABLED | IRQF_TRIGGER_RISING,
- "l3-app-irq", l3);
+ ret = request_irq(l3->app_irq, omap3_l3_app_irq, IRQF_TRIGGER_RISING,
+ "l3-app-irq", l3);
if (ret) {
dev_err(&pdev->dev, "couldn't request app irq\n");
goto err2;
spin_lock_irqsave(&smi->msg_lock, flags);
if (!smi->cur_msg) {
+ spin_unlock_irqrestore(&smi->msg_lock, flags);
pr_warn("no current message?\n");
return 0;
}
*/
bool supports_event_msg_buff;
+ /*
+ * Can we clear the global enables receive irq bit?
+ */
+ bool cannot_clear_recv_irq_bit;
+
/*
* Did we get an attention that we did not handle?
*/
* allocate messages, we just leave them in the BMC and run the system
* polled until we can allocate some memory. Once we have some
* memory, we will re-enable the interrupt.
+ *
+ * Note that we cannot just use disable_irq(), since the interrupt may
+ * be shared.
*/
static inline bool disable_si_irq(struct smi_info *smi_info)
{
if (smi_info->supports_event_msg_buff)
enables |= IPMI_BMC_EVT_MSG_BUFF;
- else
- enables &= ~IPMI_BMC_EVT_MSG_BUFF;
- if (smi_info->irq && !smi_info->interrupt_disabled)
+ if ((smi_info->irq && !smi_info->interrupt_disabled) ||
+ smi_info->cannot_clear_recv_irq_bit)
enables |= IPMI_BMC_RCV_MSG_INTR;
- else
- enables &= ~IPMI_BMC_RCV_MSG_INTR;
if (smi_info->supports_event_msg_buff &&
smi_info->irq && !smi_info->interrupt_disabled)
enables |= IPMI_BMC_EVT_MSG_INTR;
- else
- enables &= ~IPMI_BMC_EVT_MSG_INTR;
*irq_on = enables & (IPMI_BMC_EVT_MSG_INTR | IPMI_BMC_RCV_MSG_INTR);
return rv;
}
+/*
+ * Some BMCs do not support clearing the receive irq bit in the global
+ * enables (even if they don't support interrupts on the BMC). Check
+ * for this and handle it properly.
+ */
+static void check_clr_rcv_irq(struct smi_info *smi_info)
+{
+ unsigned char msg[3];
+ unsigned char *resp;
+ unsigned long resp_len;
+ int rv;
+
+ resp = kmalloc(IPMI_MAX_MSG_LENGTH, GFP_KERNEL);
+ if (!resp) {
+ printk(KERN_WARNING PFX "Out of memory allocating response for"
+ " global enables command, cannot check recv irq bit"
+ " handling.\n");
+ return;
+ }
+
+ msg[0] = IPMI_NETFN_APP_REQUEST << 2;
+ msg[1] = IPMI_GET_BMC_GLOBAL_ENABLES_CMD;
+ smi_info->handlers->start_transaction(smi_info->si_sm, msg, 2);
+
+ rv = wait_for_msg_done(smi_info);
+ if (rv) {
+ printk(KERN_WARNING PFX "Error getting response from get"
+ " global enables command, cannot check recv irq bit"
+ " handling.\n");
+ goto out;
+ }
+
+ resp_len = smi_info->handlers->get_result(smi_info->si_sm,
+ resp, IPMI_MAX_MSG_LENGTH);
+
+ if (resp_len < 4 ||
+ resp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2 ||
+ resp[1] != IPMI_GET_BMC_GLOBAL_ENABLES_CMD ||
+ resp[2] != 0) {
+ printk(KERN_WARNING PFX "Invalid return from get global"
+ " enables command, cannot check recv irq bit"
+ " handling.\n");
+ rv = -EINVAL;
+ goto out;
+ }
+
+ if ((resp[3] & IPMI_BMC_RCV_MSG_INTR) == 0)
+ /* Already clear, should work ok. */
+ goto out;
+
+ msg[0] = IPMI_NETFN_APP_REQUEST << 2;
+ msg[1] = IPMI_SET_BMC_GLOBAL_ENABLES_CMD;
+ msg[2] = resp[3] & ~IPMI_BMC_RCV_MSG_INTR;
+ smi_info->handlers->start_transaction(smi_info->si_sm, msg, 3);
+
+ rv = wait_for_msg_done(smi_info);
+ if (rv) {
+ printk(KERN_WARNING PFX "Error getting response from set"
+ " global enables command, cannot check recv irq bit"
+ " handling.\n");
+ goto out;
+ }
+
+ resp_len = smi_info->handlers->get_result(smi_info->si_sm,
+ resp, IPMI_MAX_MSG_LENGTH);
+
+ if (resp_len < 3 ||
+ resp[0] != (IPMI_NETFN_APP_REQUEST | 1) << 2 ||
+ resp[1] != IPMI_SET_BMC_GLOBAL_ENABLES_CMD) {
+ printk(KERN_WARNING PFX "Invalid return from get global"
+ " enables command, cannot check recv irq bit"
+ " handling.\n");
+ rv = -EINVAL;
+ goto out;
+ }
+
+ if (resp[2] != 0) {
+ /*
+ * An error when setting the event buffer bit means
+ * clearing the bit is not supported.
+ */
+ printk(KERN_WARNING PFX "The BMC does not support clearing"
+ " the recv irq bit, compensating, but the BMC needs to"
+ " be fixed.\n");
+ smi_info->cannot_clear_recv_irq_bit = true;
+ }
+ out:
+ kfree(resp);
+}
+
static int try_enable_event_buffer(struct smi_info *smi_info)
{
unsigned char msg[3];
goto out_err;
}
+ check_clr_rcv_irq(new_smi);
+
setup_oem_data_handler(new_smi);
setup_xaction_handlers(new_smi);
int result;
/* Wait for something to do */
- wait_for_completion(&ssif_info->wake_thread);
- init_completion(&ssif_info->wake_thread);
-
+ result = wait_for_completion_interruptible(
+ &ssif_info->wake_thread);
if (ssif_info->stopping)
break;
+ if (result == -ERESTARTSYS)
+ continue;
+ init_completion(&ssif_info->wake_thread);
if (ssif_info->i2c_read_write == I2C_SMBUS_WRITE) {
result = i2c_smbus_write_block_data(
{
int rc;
- rc = device_add(&chip->dev);
+ rc = cdev_add(&chip->cdev, chip->dev.devt, 1);
if (rc) {
dev_err(&chip->dev,
- "unable to device_register() %s, major %d, minor %d, err=%d\n",
+ "unable to cdev_add() %s, major %d, minor %d, err=%d\n",
chip->devname, MAJOR(chip->dev.devt),
MINOR(chip->dev.devt), rc);
+ device_unregister(&chip->dev);
return rc;
}
- rc = cdev_add(&chip->cdev, chip->dev.devt, 1);
+ rc = device_add(&chip->dev);
if (rc) {
dev_err(&chip->dev,
- "unable to cdev_add() %s, major %d, minor %d, err=%d\n",
+ "unable to device_register() %s, major %d, minor %d, err=%d\n",
chip->devname, MAJOR(chip->dev.devt),
MINOR(chip->dev.devt), rc);
- device_unregister(&chip->dev);
return rc;
}
* tpm_chip_register() - create a character device for the TPM chip
* @chip: TPM chip to use.
*
- * Creates a character device for the TPM chip and adds sysfs interfaces for
- * the device, PPI and TCPA. As the last step this function adds the
- * chip to the list of TPM chips available for use.
+ * Creates a character device for the TPM chip and adds sysfs attributes for
+ * the device. As the last step this function adds the chip to the list of TPM
+ * chips available for in-kernel use.
*
- * NOTE: This function should be only called after the chip initialization
- * is complete.
- *
- * Called from tpm_<specific>.c probe function only for devices
- * the driver has determined it should claim. Prior to calling
- * this function the specific probe function has called pci_enable_device
- * upon errant exit from this function specific probe function should call
- * pci_disable_device
+ * This function should be only called after the chip initialization is
+ * complete.
*/
int tpm_chip_register(struct tpm_chip *chip)
{
int rc;
- rc = tpm_dev_add_device(chip);
- if (rc)
- return rc;
-
/* Populate sysfs for TPM1 devices. */
if (!(chip->flags & TPM_CHIP_FLAG_TPM2)) {
rc = tpm_sysfs_add_device(chip);
chip->bios_dir = tpm_bios_log_setup(chip->devname);
}
+ rc = tpm_dev_add_device(chip);
+ if (rc)
+ return rc;
+
/* Make the chip available. */
spin_lock(&driver_lock);
list_add_rcu(&chip->list, &tpm_chip_list);
{
struct ibmvtpm_dev *ibmvtpm;
struct ibmvtpm_crq crq;
- u64 *word = (u64 *) &crq;
+ __be64 *word = (__be64 *)&crq;
int rc;
ibmvtpm = (struct ibmvtpm_dev *)TPM_VPRIV(chip);
memcpy((void *)ibmvtpm->rtce_buf, (void *)buf, count);
crq.valid = (u8)IBMVTPM_VALID_CMD;
crq.msg = (u8)VTPM_TPM_COMMAND;
- crq.len = (u16)count;
- crq.data = ibmvtpm->rtce_dma_handle;
+ crq.len = cpu_to_be16(count);
+ crq.data = cpu_to_be32(ibmvtpm->rtce_dma_handle);
- rc = ibmvtpm_send_crq(ibmvtpm->vdev, cpu_to_be64(word[0]),
- cpu_to_be64(word[1]));
+ rc = ibmvtpm_send_crq(ibmvtpm->vdev, be64_to_cpu(word[0]),
+ be64_to_cpu(word[1]));
if (rc != H_SUCCESS) {
dev_err(ibmvtpm->dev, "tpm_ibmvtpm_send failed rc=%d\n", rc);
rc = 0;
struct ibmvtpm_crq {
u8 valid;
u8 msg;
- u16 len;
- u32 data;
- u64 reserved;
+ __be16 len;
+ __be32 data;
+ __be64 reserved;
} __attribute__((packed, aligned(8)));
struct ibmvtpm_crq_queue {
* notification
*/
struct work_struct control_work;
+ struct work_struct config_work;
struct list_head ports;
portdev = vdev->priv;
+ if (!use_multiport(portdev))
+ schedule_work(&portdev->config_work);
+}
+
+static void config_work_handler(struct work_struct *work)
+{
+ struct ports_device *portdev;
+
+ portdev = container_of(work, struct ports_device, control_work);
if (!use_multiport(portdev)) {
+ struct virtio_device *vdev;
struct port *port;
u16 rows, cols;
+ vdev = portdev->vdev;
virtio_cread(vdev, struct virtio_console_config, cols, &cols);
virtio_cread(vdev, struct virtio_console_config, rows, &rows);
virtio_device_ready(portdev->vdev);
+ INIT_WORK(&portdev->config_work, &config_work_handler);
+ INIT_WORK(&portdev->control_work, &control_work_handler);
+
if (multiport) {
unsigned int nr_added_bufs;
spin_lock_init(&portdev->c_ivq_lock);
spin_lock_init(&portdev->c_ovq_lock);
- INIT_WORK(&portdev->control_work, &control_work_handler);
nr_added_bufs = fill_queue(portdev->c_ivq,
&portdev->c_ivq_lock);
/* Finish up work that's lined up */
if (use_multiport(portdev))
cancel_work_sync(&portdev->control_work);
+ else
+ cancel_work_sync(&portdev->config_work);
list_for_each_entry_safe(port, port2, &portdev->ports, list)
unplug_port(port);
virtqueue_disable_cb(portdev->c_ivq);
cancel_work_sync(&portdev->control_work);
+ cancel_work_sync(&portdev->config_work);
/*
* Once more: if control_work_handler() was running, it would
* enable the cb as the last step.
return 0;
}
+static void pmc_irq_suspend(struct irq_data *d)
+{
+ struct at91_pmc *pmc = irq_data_get_irq_chip_data(d);
+
+ pmc->imr = pmc_read(pmc, AT91_PMC_IMR);
+ pmc_write(pmc, AT91_PMC_IDR, pmc->imr);
+}
+
+static void pmc_irq_resume(struct irq_data *d)
+{
+ struct at91_pmc *pmc = irq_data_get_irq_chip_data(d);
+
+ pmc_write(pmc, AT91_PMC_IER, pmc->imr);
+}
+
static struct irq_chip pmc_irq = {
.name = "PMC",
.irq_disable = pmc_irq_mask,
.irq_mask = pmc_irq_mask,
.irq_unmask = pmc_irq_unmask,
.irq_set_type = pmc_irq_set_type,
+ .irq_suspend = pmc_irq_suspend,
+ .irq_resume = pmc_irq_resume,
};
static struct lock_class_key pmc_lock_class;
goto out_free_pmc;
pmc_write(pmc, AT91_PMC_IDR, 0xffffffff);
- if (request_irq(pmc->virq, pmc_irq_handler, IRQF_SHARED, "pmc", pmc))
+ if (request_irq(pmc->virq, pmc_irq_handler,
+ IRQF_SHARED | IRQF_COND_SUSPEND, "pmc", pmc))
goto out_remove_irqdomain;
return pmc;
spinlock_t lock;
const struct at91_pmc_caps *caps;
struct irq_domain *irqdomain;
+ u32 imr;
};
static inline void pmc_lock(struct at91_pmc *pmc)
divider->flags);
}
-/*
- * The reverse of DIV_ROUND_UP: The maximum number which
- * divided by m is r
- */
-#define MULT_ROUND_UP(r, m) ((r) * (m) + (m) - 1)
-
static bool _is_valid_table_div(const struct clk_div_table *table,
unsigned int div)
{
unsigned long parent_rate, unsigned long rate,
unsigned long flags)
{
- int up, down, div;
+ int up, down;
+ unsigned long up_rate, down_rate;
- up = down = div = DIV_ROUND_CLOSEST(parent_rate, rate);
+ up = DIV_ROUND_UP(parent_rate, rate);
+ down = parent_rate / rate;
if (flags & CLK_DIVIDER_POWER_OF_TWO) {
- up = __roundup_pow_of_two(div);
- down = __rounddown_pow_of_two(div);
+ up = __roundup_pow_of_two(up);
+ down = __rounddown_pow_of_two(down);
} else if (table) {
- up = _round_up_table(table, div);
- down = _round_down_table(table, div);
+ up = _round_up_table(table, up);
+ down = _round_down_table(table, down);
}
- return (up - div) <= (div - down) ? up : down;
+ up_rate = DIV_ROUND_UP(parent_rate, up);
+ down_rate = DIV_ROUND_UP(parent_rate, down);
+
+ return (rate - up_rate) <= (down_rate - rate) ? up : down;
}
static int _div_round(const struct clk_div_table *table,
return i;
}
parent_rate = __clk_round_rate(__clk_get_parent(hw->clk),
- MULT_ROUND_UP(rate, i));
+ rate * i);
now = DIV_ROUND_UP(parent_rate, i);
if (_is_best_div(rate, now, best, flags)) {
bestdiv = i;
bestdiv = readl(divider->reg) >> divider->shift;
bestdiv &= div_mask(divider->width);
bestdiv = _get_div(divider->table, bestdiv, divider->flags);
- return bestdiv;
+ return DIV_ROUND_UP(*prate, bestdiv);
}
return divider_round_rate(hw, rate, prate, divider->table,
return rate;
}
-EXPORT_SYMBOL_GPL(clk_core_get_rate);
/**
* clk_get_rate - return the rate of clk
return clk_core_get_phase(clk->core);
}
+/**
+ * clk_is_match - check if two clk's point to the same hardware clock
+ * @p: clk compared against q
+ * @q: clk compared against p
+ *
+ * Returns true if the two struct clk pointers both point to the same hardware
+ * clock node. Put differently, returns true if struct clk *p and struct clk *q
+ * share the same struct clk_core object.
+ *
+ * Returns false otherwise. Note that two NULL clks are treated as matching.
+ */
+bool clk_is_match(const struct clk *p, const struct clk *q)
+{
+ /* trivial case: identical struct clk's or both NULL */
+ if (p == q)
+ return true;
+
+ /* true if clk->core pointers match. Avoid derefing garbage */
+ if (!IS_ERR_OR_NULL(p) && !IS_ERR_OR_NULL(q))
+ if (p->core == q->core)
+ return true;
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(clk_is_match);
+
/**
* __clk_init - initialize the data structures in a struct clk
* @dev: device initializing this clk, placeholder for now
},
};
+static struct clk_regmap pll4_vote = {
+ .enable_reg = 0x34c0,
+ .enable_mask = BIT(4),
+ .hw.init = &(struct clk_init_data){
+ .name = "pll4_vote",
+ .parent_names = (const char *[]){ "pll4" },
+ .num_parents = 1,
+ .ops = &clk_pll_vote_ops,
+ },
+};
+
static struct clk_pll pll8 = {
.l_reg = 0x3144,
.m_reg = 0x3148,
static struct clk_regmap *gcc_msm8960_clks[] = {
[PLL3] = &pll3.clkr,
+ [PLL4_VOTE] = &pll4_vote,
[PLL8] = &pll8.clkr,
[PLL8_VOTE] = &pll8_vote,
[PLL14] = &pll14.clkr,
static struct clk_regmap *gcc_apq8064_clks[] = {
[PLL3] = &pll3.clkr,
+ [PLL4_VOTE] = &pll4_vote,
[PLL8] = &pll8.clkr,
[PLL8_VOTE] = &pll8_vote,
[PLL14] = &pll14.clkr,
.remove = lcc_ipq806x_remove,
.driver = {
.name = "lcc-ipq806x",
- .owner = THIS_MODULE,
.of_match_table = lcc_ipq806x_match_table,
},
};
.mnctr_en_bit = 8,
.mnctr_reset_bit = 7,
.mnctr_mode_shift = 5,
- .n_val_shift = 16,
- .m_val_shift = 16,
+ .n_val_shift = 24,
+ .m_val_shift = 8,
.width = 8,
},
.p = {
return PTR_ERR(regmap);
/* Use the correct frequency plan depending on speed of PLL4 */
- val = regmap_read(regmap, 0x4, &val);
+ regmap_read(regmap, 0x4, &val);
if (val == 0x12) {
slimbus_src.freq_tbl = clk_tbl_aif_osr_492;
mi2s_osr_src.freq_tbl = clk_tbl_aif_osr_492;
.remove = lcc_msm8960_remove,
.driver = {
.name = "lcc-msm8960",
- .owner = THIS_MODULE,
.of_match_table = lcc_msm8960_match_table,
},
};
struct fapll_data *fd = to_fapll(hw);
u32 v = readl_relaxed(fd->base);
- v |= (1 << FAPLL_MAIN_PLLEN);
+ v |= FAPLL_MAIN_PLLEN;
writel_relaxed(v, fd->base);
return 0;
struct fapll_data *fd = to_fapll(hw);
u32 v = readl_relaxed(fd->base);
- v &= ~(1 << FAPLL_MAIN_PLLEN);
+ v &= ~FAPLL_MAIN_PLLEN;
writel_relaxed(v, fd->base);
}
struct fapll_data *fd = to_fapll(hw);
u32 v = readl_relaxed(fd->base);
- return v & (1 << FAPLL_MAIN_PLLEN);
+ return v & FAPLL_MAIN_PLLEN;
}
static unsigned long ti_fapll_recalc_rate(struct clk_hw *hw,
config CADENCE_TTC_TIMER
bool
+config ASM9260_TIMER
+ bool
+ select CLKSRC_MMIO
+ select CLKSRC_OF
+
config CLKSRC_NOMADIK_MTU
bool
depends on (ARCH_NOMADIK || ARCH_U8500)
config SH_TIMER_CMT
bool "Renesas CMT timer driver" if COMPILE_TEST
depends on GENERIC_CLOCKEVENTS
+ depends on HAS_IOMEM
default SYS_SUPPORTS_SH_CMT
help
This enables build of a clocksource and clockevent driver for
config SH_TIMER_MTU2
bool "Renesas MTU2 timer driver" if COMPILE_TEST
depends on GENERIC_CLOCKEVENTS
+ depends on HAS_IOMEM
default SYS_SUPPORTS_SH_MTU2
help
This enables build of a clockevent driver for the Multi-Function
config SH_TIMER_TMU
bool "Renesas TMU timer driver" if COMPILE_TEST
depends on GENERIC_CLOCKEVENTS
+ depends on HAS_IOMEM
default SYS_SUPPORTS_SH_TMU
help
This enables build of a clocksource and clockevent driver for
help
This enables OST0 support available on PXA and SA-11x0
platforms.
-
-config ASM9260_TIMER
- bool "Alphascale ASM9260 timer driver"
- depends on GENERIC_CLOCKEVENTS
- select CLKSRC_MMIO
- select CLKSRC_OF
- default y if MACH_ASM9260
- help
- This enables build of a clocksource and clockevent driver for
- the 32-bit System Timer hardware available on a Alphascale ASM9260.
-
endmenu
};
static bool __init
-arch_timer_probed(int type, const struct of_device_id *matches)
+arch_timer_needs_probing(int type, const struct of_device_id *matches)
{
struct device_node *dn;
- bool probed = true;
+ bool needs_probing = false;
dn = of_find_matching_node(NULL, matches);
if (dn && of_device_is_available(dn) && !(arch_timers_present & type))
- probed = false;
+ needs_probing = true;
of_node_put(dn);
- return probed;
+ return needs_probing;
}
static void __init arch_timer_common_init(void)
/* Wait until both nodes are probed if we have two timers */
if ((arch_timers_present & mask) != mask) {
- if (!arch_timer_probed(ARCH_MEM_TIMER, arch_timer_mem_of_match))
+ if (arch_timer_needs_probing(ARCH_MEM_TIMER, arch_timer_mem_of_match))
return;
- if (!arch_timer_probed(ARCH_CP15_TIMER, arch_timer_of_match))
+ if (arch_timer_needs_probing(ARCH_CP15_TIMER, arch_timer_of_match))
return;
}
static u64 notrace read_sched_clock(void)
{
- return ~__raw_readl(sched_io_base);
+ return ~readl_relaxed(sched_io_base);
}
static const struct of_device_id sptimer_ids[] __initconst = {
ret = em_sti_start(p, USER_CLOCKSOURCE);
if (!ret)
- __clocksource_updatefreq_hz(cs, p->rate);
+ __clocksource_update_freq_hz(cs, p->rate);
return ret;
}
}
rate = clk_get_rate(clk);
+ mtk_timer_global_reset(evt);
+
if (request_irq(evt->dev.irq, mtk_timer_interrupt,
IRQF_TIMER | IRQF_IRQPOLL, "mtk_timer", evt)) {
pr_warn("failed to setup irq %d\n", evt->dev.irq);
evt->ticks_per_jiffy = DIV_ROUND_UP(rate, HZ);
- mtk_timer_global_reset(evt);
-
/* Configure clock source */
mtk_timer_setup(evt, GPT_CLK_SRC, TIMER_CTRL_OP_FREERUN);
clocksource_mmio_init(evt->gpt_base + TIMER_CNT_REG(GPT_CLK_SRC),
/* Configure clock event */
mtk_timer_setup(evt, GPT_CLK_EVT, TIMER_CTRL_OP_REPEAT);
- mtk_timer_enable_irq(evt, GPT_CLK_EVT);
-
clockevents_config_and_register(&evt->dev, rate, 0x3,
0xffffffff);
+
+ mtk_timer_enable_irq(evt, GPT_CLK_EVT);
+
return;
err_clk_disable:
.dev_id = &ckevt_pxa_osmr0,
};
-static void pxa_timer_common_init(int irq, unsigned long clock_tick_rate)
+static void __init pxa_timer_common_init(int irq, unsigned long clock_tick_rate)
{
timer_writel(0, OIER);
timer_writel(OSSR_M0 | OSSR_M1 | OSSR_M2 | OSSR_M3, OSSR);
ret = sh_cmt_start(ch, FLAG_CLOCKSOURCE);
if (!ret) {
- __clocksource_updatefreq_hz(cs, ch->rate);
+ __clocksource_update_freq_hz(cs, ch->rate);
ch->cs_enabled = true;
}
return ret;
ret = sh_tmu_enable(ch);
if (!ret) {
- __clocksource_updatefreq_hz(cs, ch->rate);
+ __clocksource_update_freq_hz(cs, ch->rate);
ch->cs_enabled = true;
}
TIMER_CTL_CLK_SRC(TIMER_CTL_CLK_SRC_OSC24M),
timer_base + TIMER_CTL_REG(1));
- sched_clock_register(sun4i_timer_sched_read, 32, rate);
+ /*
+ * sched_clock_register does not have priorities, and on sun6i and
+ * later there is a better sched_clock registered by arm_arch_timer.c
+ */
+ if (of_machine_is_compatible("allwinner,sun4i-a10") ||
+ of_machine_is_compatible("allwinner,sun5i-a13") ||
+ of_machine_is_compatible("allwinner,sun5i-a10s"))
+ sched_clock_register(sun4i_timer_sched_read, 32, rate);
+
clocksource_mmio_init(timer_base + TIMER_CNTVAL_REG(1), node->name,
rate, 350, 32, clocksource_mmio_readl_down);
static void __iomem *timer_reg_base;
static void __iomem *rtc_base;
-static struct timespec persistent_ts;
+static struct timespec64 persistent_ts;
static u64 persistent_ms, last_persistent_ms;
static struct delay_timer tegra_delay_timer;
#define timer_writel(value, reg) \
- __raw_writel(value, timer_reg_base + (reg))
+ writel_relaxed(value, timer_reg_base + (reg))
#define timer_readl(reg) \
- __raw_readl(timer_reg_base + (reg))
+ readl_relaxed(timer_reg_base + (reg))
static int tegra_timer_set_next_event(unsigned long cycles,
struct clock_event_device *evt)
}
/*
- * tegra_read_persistent_clock - Return time from a persistent clock.
+ * tegra_read_persistent_clock64 - Return time from a persistent clock.
*
* Reads the time from a source which isn't disabled during PM, the
* 32k sync timer. Convert the cycles elapsed since last read into
- * nsecs and adds to a monotonically increasing timespec.
+ * nsecs and adds to a monotonically increasing timespec64.
* Care must be taken that this funciton is not called while the
* tegra_rtc driver could be executing to avoid race conditions
* on the RTC shadow register
*/
-static void tegra_read_persistent_clock(struct timespec *ts)
+static void tegra_read_persistent_clock64(struct timespec64 *ts)
{
u64 delta;
- struct timespec *tsp = &persistent_ts;
last_persistent_ms = persistent_ms;
persistent_ms = tegra_rtc_read_ms();
delta = persistent_ms - last_persistent_ms;
- timespec_add_ns(tsp, delta * NSEC_PER_MSEC);
- *ts = *tsp;
+ timespec64_add_ns(&persistent_ts, delta * NSEC_PER_MSEC);
+ *ts = persistent_ts;
}
static unsigned long tegra_delay_timer_read_counter_long(void)
else
clk_prepare_enable(clk);
- register_persistent_clock(NULL, tegra_read_persistent_clock);
+ register_persistent_clock(NULL, tegra_read_persistent_clock64);
}
CLOCKSOURCE_OF_DECLARE(tegra20_rtc, "nvidia,tegra20-rtc", tegra20_init_rtc);
static struct efm32_clock_event_ddata clock_event_ddata = {
.evtdev = {
.name = "efm32 clockevent",
- .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_MODE_PERIODIC,
+ .features = CLOCK_EVT_FEAT_ONESHOT | CLOCK_EVT_FEAT_PERIODIC,
.set_mode = efm32_clock_event_set_mode,
.set_next_event = efm32_clock_event_set_next_event,
.rating = 200,
clock_event_ddata.base = base;
clock_event_ddata.periodic_top = DIV_ROUND_CLOSEST(rate, 1024 * HZ);
- setup_irq(irq, &efm32_clock_event_irq);
-
clockevents_config_and_register(&clock_event_ddata.evtdev,
DIV_ROUND_CLOSEST(rate, 1024),
0xf, 0xffff);
+ setup_irq(irq, &efm32_clock_event_irq);
+
return 0;
err_get_irq:
static inline unsigned int pit_read(void __iomem *base, unsigned int reg_offset)
{
- return __raw_readl(base + reg_offset);
+ return readl_relaxed(base + reg_offset);
}
static inline void pit_write(void __iomem *base, unsigned int reg_offset, unsigned long value)
{
- __raw_writel(value, base + reg_offset);
+ writel_relaxed(value, base + reg_offset);
}
/*
#include <linux/irq.h>
#include <linux/irqreturn.h>
#include <linux/reset.h>
-#include <linux/sched_clock.h>
+#include <linux/slab.h>
#include <linux/of.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#define TIMER_SYNC_TICKS 3
-static void __iomem *timer_base;
-static u32 ticks_per_jiffy;
+struct sun5i_timer {
+ void __iomem *base;
+ struct clk *clk;
+ struct notifier_block clk_rate_cb;
+ u32 ticks_per_jiffy;
+};
+
+#define to_sun5i_timer(x) \
+ container_of(x, struct sun5i_timer, clk_rate_cb)
+
+struct sun5i_timer_clksrc {
+ struct sun5i_timer timer;
+ struct clocksource clksrc;
+};
+
+#define to_sun5i_timer_clksrc(x) \
+ container_of(x, struct sun5i_timer_clksrc, clksrc)
+
+struct sun5i_timer_clkevt {
+ struct sun5i_timer timer;
+ struct clock_event_device clkevt;
+};
+
+#define to_sun5i_timer_clkevt(x) \
+ container_of(x, struct sun5i_timer_clkevt, clkevt)
/*
* When we disable a timer, we need to wait at least for 2 cycles of
* that is already setup and runs at the same frequency than the other
* timers, and we never will be disabled.
*/
-static void sun5i_clkevt_sync(void)
+static void sun5i_clkevt_sync(struct sun5i_timer_clkevt *ce)
{
- u32 old = readl(timer_base + TIMER_CNTVAL_LO_REG(1));
+ u32 old = readl(ce->timer.base + TIMER_CNTVAL_LO_REG(1));
- while ((old - readl(timer_base + TIMER_CNTVAL_LO_REG(1))) < TIMER_SYNC_TICKS)
+ while ((old - readl(ce->timer.base + TIMER_CNTVAL_LO_REG(1))) < TIMER_SYNC_TICKS)
cpu_relax();
}
-static void sun5i_clkevt_time_stop(u8 timer)
+static void sun5i_clkevt_time_stop(struct sun5i_timer_clkevt *ce, u8 timer)
{
- u32 val = readl(timer_base + TIMER_CTL_REG(timer));
- writel(val & ~TIMER_CTL_ENABLE, timer_base + TIMER_CTL_REG(timer));
+ u32 val = readl(ce->timer.base + TIMER_CTL_REG(timer));
+ writel(val & ~TIMER_CTL_ENABLE, ce->timer.base + TIMER_CTL_REG(timer));
- sun5i_clkevt_sync();
+ sun5i_clkevt_sync(ce);
}
-static void sun5i_clkevt_time_setup(u8 timer, u32 delay)
+static void sun5i_clkevt_time_setup(struct sun5i_timer_clkevt *ce, u8 timer, u32 delay)
{
- writel(delay, timer_base + TIMER_INTVAL_LO_REG(timer));
+ writel(delay, ce->timer.base + TIMER_INTVAL_LO_REG(timer));
}
-static void sun5i_clkevt_time_start(u8 timer, bool periodic)
+static void sun5i_clkevt_time_start(struct sun5i_timer_clkevt *ce, u8 timer, bool periodic)
{
- u32 val = readl(timer_base + TIMER_CTL_REG(timer));
+ u32 val = readl(ce->timer.base + TIMER_CTL_REG(timer));
if (periodic)
val &= ~TIMER_CTL_ONESHOT;
val |= TIMER_CTL_ONESHOT;
writel(val | TIMER_CTL_ENABLE | TIMER_CTL_RELOAD,
- timer_base + TIMER_CTL_REG(timer));
+ ce->timer.base + TIMER_CTL_REG(timer));
}
static void sun5i_clkevt_mode(enum clock_event_mode mode,
- struct clock_event_device *clk)
+ struct clock_event_device *clkevt)
{
+ struct sun5i_timer_clkevt *ce = to_sun5i_timer_clkevt(clkevt);
+
switch (mode) {
case CLOCK_EVT_MODE_PERIODIC:
- sun5i_clkevt_time_stop(0);
- sun5i_clkevt_time_setup(0, ticks_per_jiffy);
- sun5i_clkevt_time_start(0, true);
+ sun5i_clkevt_time_stop(ce, 0);
+ sun5i_clkevt_time_setup(ce, 0, ce->timer.ticks_per_jiffy);
+ sun5i_clkevt_time_start(ce, 0, true);
break;
case CLOCK_EVT_MODE_ONESHOT:
- sun5i_clkevt_time_stop(0);
- sun5i_clkevt_time_start(0, false);
+ sun5i_clkevt_time_stop(ce, 0);
+ sun5i_clkevt_time_start(ce, 0, false);
break;
case CLOCK_EVT_MODE_UNUSED:
case CLOCK_EVT_MODE_SHUTDOWN:
default:
- sun5i_clkevt_time_stop(0);
+ sun5i_clkevt_time_stop(ce, 0);
break;
}
}
static int sun5i_clkevt_next_event(unsigned long evt,
- struct clock_event_device *unused)
+ struct clock_event_device *clkevt)
{
- sun5i_clkevt_time_stop(0);
- sun5i_clkevt_time_setup(0, evt - TIMER_SYNC_TICKS);
- sun5i_clkevt_time_start(0, false);
+ struct sun5i_timer_clkevt *ce = to_sun5i_timer_clkevt(clkevt);
+
+ sun5i_clkevt_time_stop(ce, 0);
+ sun5i_clkevt_time_setup(ce, 0, evt - TIMER_SYNC_TICKS);
+ sun5i_clkevt_time_start(ce, 0, false);
return 0;
}
-static struct clock_event_device sun5i_clockevent = {
- .name = "sun5i_tick",
- .rating = 340,
- .features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT,
- .set_mode = sun5i_clkevt_mode,
- .set_next_event = sun5i_clkevt_next_event,
-};
-
-
static irqreturn_t sun5i_timer_interrupt(int irq, void *dev_id)
{
- struct clock_event_device *evt = (struct clock_event_device *)dev_id;
+ struct sun5i_timer_clkevt *ce = (struct sun5i_timer_clkevt *)dev_id;
- writel(0x1, timer_base + TIMER_IRQ_ST_REG);
- evt->event_handler(evt);
+ writel(0x1, ce->timer.base + TIMER_IRQ_ST_REG);
+ ce->clkevt.event_handler(&ce->clkevt);
return IRQ_HANDLED;
}
-static struct irqaction sun5i_timer_irq = {
- .name = "sun5i_timer0",
- .flags = IRQF_TIMER | IRQF_IRQPOLL,
- .handler = sun5i_timer_interrupt,
- .dev_id = &sun5i_clockevent,
-};
+static cycle_t sun5i_clksrc_read(struct clocksource *clksrc)
+{
+ struct sun5i_timer_clksrc *cs = to_sun5i_timer_clksrc(clksrc);
+
+ return ~readl(cs->timer.base + TIMER_CNTVAL_LO_REG(1));
+}
-static u64 sun5i_timer_sched_read(void)
+static int sun5i_rate_cb_clksrc(struct notifier_block *nb,
+ unsigned long event, void *data)
{
- return ~readl(timer_base + TIMER_CNTVAL_LO_REG(1));
+ struct clk_notifier_data *ndata = data;
+ struct sun5i_timer *timer = to_sun5i_timer(nb);
+ struct sun5i_timer_clksrc *cs = container_of(timer, struct sun5i_timer_clksrc, timer);
+
+ switch (event) {
+ case PRE_RATE_CHANGE:
+ clocksource_unregister(&cs->clksrc);
+ break;
+
+ case POST_RATE_CHANGE:
+ clocksource_register_hz(&cs->clksrc, ndata->new_rate);
+ break;
+
+ default:
+ break;
+ }
+
+ return NOTIFY_DONE;
+}
+
+static int __init sun5i_setup_clocksource(struct device_node *node,
+ void __iomem *base,
+ struct clk *clk, int irq)
+{
+ struct sun5i_timer_clksrc *cs;
+ unsigned long rate;
+ int ret;
+
+ cs = kzalloc(sizeof(*cs), GFP_KERNEL);
+ if (!cs)
+ return -ENOMEM;
+
+ ret = clk_prepare_enable(clk);
+ if (ret) {
+ pr_err("Couldn't enable parent clock\n");
+ goto err_free;
+ }
+
+ rate = clk_get_rate(clk);
+
+ cs->timer.base = base;
+ cs->timer.clk = clk;
+ cs->timer.clk_rate_cb.notifier_call = sun5i_rate_cb_clksrc;
+ cs->timer.clk_rate_cb.next = NULL;
+
+ ret = clk_notifier_register(clk, &cs->timer.clk_rate_cb);
+ if (ret) {
+ pr_err("Unable to register clock notifier.\n");
+ goto err_disable_clk;
+ }
+
+ writel(~0, base + TIMER_INTVAL_LO_REG(1));
+ writel(TIMER_CTL_ENABLE | TIMER_CTL_RELOAD,
+ base + TIMER_CTL_REG(1));
+
+ cs->clksrc.name = node->name;
+ cs->clksrc.rating = 340;
+ cs->clksrc.read = sun5i_clksrc_read;
+ cs->clksrc.mask = CLOCKSOURCE_MASK(32);
+ cs->clksrc.flags = CLOCK_SOURCE_IS_CONTINUOUS;
+
+ ret = clocksource_register_hz(&cs->clksrc, rate);
+ if (ret) {
+ pr_err("Couldn't register clock source.\n");
+ goto err_remove_notifier;
+ }
+
+ return 0;
+
+err_remove_notifier:
+ clk_notifier_unregister(clk, &cs->timer.clk_rate_cb);
+err_disable_clk:
+ clk_disable_unprepare(clk);
+err_free:
+ kfree(cs);
+ return ret;
+}
+
+static int sun5i_rate_cb_clkevt(struct notifier_block *nb,
+ unsigned long event, void *data)
+{
+ struct clk_notifier_data *ndata = data;
+ struct sun5i_timer *timer = to_sun5i_timer(nb);
+ struct sun5i_timer_clkevt *ce = container_of(timer, struct sun5i_timer_clkevt, timer);
+
+ if (event == POST_RATE_CHANGE) {
+ clockevents_update_freq(&ce->clkevt, ndata->new_rate);
+ ce->timer.ticks_per_jiffy = DIV_ROUND_UP(ndata->new_rate, HZ);
+ }
+
+ return NOTIFY_DONE;
+}
+
+static int __init sun5i_setup_clockevent(struct device_node *node, void __iomem *base,
+ struct clk *clk, int irq)
+{
+ struct sun5i_timer_clkevt *ce;
+ unsigned long rate;
+ int ret;
+ u32 val;
+
+ ce = kzalloc(sizeof(*ce), GFP_KERNEL);
+ if (!ce)
+ return -ENOMEM;
+
+ ret = clk_prepare_enable(clk);
+ if (ret) {
+ pr_err("Couldn't enable parent clock\n");
+ goto err_free;
+ }
+
+ rate = clk_get_rate(clk);
+
+ ce->timer.base = base;
+ ce->timer.ticks_per_jiffy = DIV_ROUND_UP(rate, HZ);
+ ce->timer.clk = clk;
+ ce->timer.clk_rate_cb.notifier_call = sun5i_rate_cb_clkevt;
+ ce->timer.clk_rate_cb.next = NULL;
+
+ ret = clk_notifier_register(clk, &ce->timer.clk_rate_cb);
+ if (ret) {
+ pr_err("Unable to register clock notifier.\n");
+ goto err_disable_clk;
+ }
+
+ ce->clkevt.name = node->name;
+ ce->clkevt.features = CLOCK_EVT_FEAT_PERIODIC | CLOCK_EVT_FEAT_ONESHOT;
+ ce->clkevt.set_next_event = sun5i_clkevt_next_event;
+ ce->clkevt.set_mode = sun5i_clkevt_mode;
+ ce->clkevt.rating = 340;
+ ce->clkevt.irq = irq;
+ ce->clkevt.cpumask = cpu_possible_mask;
+
+ /* Enable timer0 interrupt */
+ val = readl(base + TIMER_IRQ_EN_REG);
+ writel(val | TIMER_IRQ_EN(0), base + TIMER_IRQ_EN_REG);
+
+ clockevents_config_and_register(&ce->clkevt, rate,
+ TIMER_SYNC_TICKS, 0xffffffff);
+
+ ret = request_irq(irq, sun5i_timer_interrupt, IRQF_TIMER | IRQF_IRQPOLL,
+ "sun5i_timer0", ce);
+ if (ret) {
+ pr_err("Unable to register interrupt\n");
+ goto err_remove_notifier;
+ }
+
+ return 0;
+
+err_remove_notifier:
+ clk_notifier_unregister(clk, &ce->timer.clk_rate_cb);
+err_disable_clk:
+ clk_disable_unprepare(clk);
+err_free:
+ kfree(ce);
+ return ret;
}
static void __init sun5i_timer_init(struct device_node *node)
{
struct reset_control *rstc;
- unsigned long rate;
+ void __iomem *timer_base;
struct clk *clk;
- int ret, irq;
- u32 val;
+ int irq;
- timer_base = of_iomap(node, 0);
+ timer_base = of_io_request_and_map(node, 0, of_node_full_name(node));
if (!timer_base)
panic("Can't map registers");
clk = of_clk_get(node, 0);
if (IS_ERR(clk))
panic("Can't get timer clock");
- clk_prepare_enable(clk);
- rate = clk_get_rate(clk);
rstc = of_reset_control_get(node, NULL);
if (!IS_ERR(rstc))
reset_control_deassert(rstc);
- writel(~0, timer_base + TIMER_INTVAL_LO_REG(1));
- writel(TIMER_CTL_ENABLE | TIMER_CTL_RELOAD,
- timer_base + TIMER_CTL_REG(1));
-
- sched_clock_register(sun5i_timer_sched_read, 32, rate);
- clocksource_mmio_init(timer_base + TIMER_CNTVAL_LO_REG(1), node->name,
- rate, 340, 32, clocksource_mmio_readl_down);
-
- ticks_per_jiffy = DIV_ROUND_UP(rate, HZ);
-
- ret = setup_irq(irq, &sun5i_timer_irq);
- if (ret)
- pr_warn("failed to setup irq %d\n", irq);
-
- /* Enable timer0 interrupt */
- val = readl(timer_base + TIMER_IRQ_EN_REG);
- writel(val | TIMER_IRQ_EN(0), timer_base + TIMER_IRQ_EN_REG);
-
- sun5i_clockevent.cpumask = cpu_possible_mask;
- sun5i_clockevent.irq = irq;
-
- clockevents_config_and_register(&sun5i_clockevent, rate,
- TIMER_SYNC_TICKS, 0xffffffff);
+ sun5i_setup_clocksource(node, timer_base, clk, irq);
+ sun5i_setup_clockevent(node, timer_base, clk, irq);
}
CLOCKSOURCE_OF_DECLARE(sun5i_a13, "allwinner,sun5i-a13-hstimer",
sun5i_timer_init);
|| __cpufreq_governor(policy, CPUFREQ_GOV_LIMITS))
pr_err("%s: Failed to start governor for policy: %p\n",
__func__, policy);
-
- /*
- * schedule call cpufreq_update_policy() for boot CPU, i.e. last
- * policy in list. It will verify that the current freq is in
- * sync with what we believe it to be.
- */
- if (list_is_last(&policy->policy_list, &cpufreq_policy_list))
- schedule_work(&policy->update);
}
+
+ /*
+ * schedule call cpufreq_update_policy() for first-online CPU, as that
+ * wouldn't be hotplugged-out on suspend. It will verify that the
+ * current freq is in sync with what we believe it to be.
+ */
+ policy = cpufreq_cpu_get_raw(cpumask_first(cpu_online_mask));
+ if (WARN_ON(!policy))
+ return;
+
+ schedule_work(&policy->update);
}
/**
static int exynos_cpufreq_probe(struct platform_device *pdev)
{
- struct device_node *cpus, *np;
+ struct device_node *cpu0;
int ret = -EINVAL;
exynos_info = kzalloc(sizeof(*exynos_info), GFP_KERNEL);
if (ret)
goto err_cpufreq_reg;
- cpus = of_find_node_by_path("/cpus");
- if (!cpus) {
- pr_err("failed to find cpus node\n");
+ cpu0 = of_get_cpu_node(0, NULL);
+ if (!cpu0) {
+ pr_err("failed to find cpu0 node\n");
return 0;
}
- np = of_get_next_child(cpus, NULL);
- if (!np) {
- pr_err("failed to find cpus child node\n");
- of_node_put(cpus);
- return 0;
- }
-
- if (of_find_property(np, "#cooling-cells", NULL)) {
- cdev = of_cpufreq_cooling_register(np,
+ if (of_find_property(cpu0, "#cooling-cells", NULL)) {
+ cdev = of_cpufreq_cooling_register(cpu0,
cpu_present_mask);
if (IS_ERR(cdev))
pr_err("running cpufreq without cooling device: %ld\n",
PTR_ERR(cdev));
}
- of_node_put(np);
- of_node_put(cpus);
return 0;
#include <linux/smp.h>
#include <sysdev/fsl_soc.h>
+#include <asm/smp.h> /* for get_hard_smp_processor_id() in UP configs */
+
/**
* struct cpu_data - per CPU data struct
* @parent: the parent node of cpu clock
deepidle = true;
ret = mvebu_v7_cpu_suspend(deepidle);
+ cpu_pm_exit();
+
if (ret)
return ret;
- cpu_pm_exit();
-
return index;
}
.states[0] = ARM_CPUIDLE_WFI_STATE,
.states[1] = {
.enter = mvebu_v7_enter_idle,
- .exit_latency = 10,
+ .exit_latency = 100,
.power_usage = 50,
- .target_residency = 100,
+ .target_residency = 1000,
.name = "MV CPU IDLE",
.desc = "CPU power down",
},
.states[2] = {
.enter = mvebu_v7_enter_idle,
- .exit_latency = 100,
+ .exit_latency = 1000,
.power_usage = 5,
- .target_residency = 1000,
+ .target_residency = 10000,
.flags = MVEBU_V7_FLAG_DEEP_IDLE,
.name = "MV CPU DEEP IDLE",
.desc = "CPU and L2 Fabric power down",
off = 1;
}
+bool cpuidle_not_available(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev)
+{
+ return off || !initialized || !drv || !dev || !dev->enabled;
+}
+
/**
* cpuidle_play_dead - cpu off-lining
*
return -ENODEV;
}
-/**
- * cpuidle_find_deepest_state - Find deepest state meeting specific conditions.
- * @drv: cpuidle driver for the given CPU.
- * @dev: cpuidle device for the given CPU.
- * @freeze: Whether or not the state should be suitable for suspend-to-idle.
- */
-static int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
- struct cpuidle_device *dev, bool freeze)
+static int find_deepest_state(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev, bool freeze)
{
unsigned int latency_req = 0;
int i, ret = freeze ? -1 : CPUIDLE_DRIVER_STATE_START - 1;
return ret;
}
+/**
+ * cpuidle_find_deepest_state - Find the deepest available idle state.
+ * @drv: cpuidle driver for the given CPU.
+ * @dev: cpuidle device for the given CPU.
+ */
+int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev)
+{
+ return find_deepest_state(drv, dev, false);
+}
+
static void enter_freeze_proper(struct cpuidle_driver *drv,
struct cpuidle_device *dev, int index)
{
/**
* cpuidle_enter_freeze - Enter an idle state suitable for suspend-to-idle.
+ * @drv: cpuidle driver for the given CPU.
+ * @dev: cpuidle device for the given CPU.
*
* If there are states with the ->enter_freeze callback, find the deepest of
- * them and enter it with frozen tick. Otherwise, find the deepest state
- * available and enter it normally.
+ * them and enter it with frozen tick.
*/
-void cpuidle_enter_freeze(void)
+int cpuidle_enter_freeze(struct cpuidle_driver *drv, struct cpuidle_device *dev)
{
- struct cpuidle_device *dev = __this_cpu_read(cpuidle_devices);
- struct cpuidle_driver *drv = cpuidle_get_cpu_driver(dev);
int index;
/*
* that interrupts won't be enabled when it exits and allows the tick to
* be frozen safely.
*/
- index = cpuidle_find_deepest_state(drv, dev, true);
- if (index >= 0) {
- enter_freeze_proper(drv, dev, index);
- return;
- }
-
- /*
- * It is not safe to freeze the tick, find the deepest state available
- * at all and try to enter it normally.
- */
- index = cpuidle_find_deepest_state(drv, dev, false);
+ index = find_deepest_state(drv, dev, true);
if (index >= 0)
- cpuidle_enter(drv, dev, index);
- else
- arch_cpu_idle();
+ enter_freeze_proper(drv, dev, index);
- /* Interrupts are enabled again here. */
- local_irq_disable();
+ return index;
}
/**
*/
int cpuidle_select(struct cpuidle_driver *drv, struct cpuidle_device *dev)
{
- if (off || !initialized)
- return -ENODEV;
-
- if (!drv || !dev || !dev->enabled)
- return -EBUSY;
-
return cpuidle_curr_governor->select(drv, dev);
}
if (!dev->registered)
return -EINVAL;
- if (!dev->state_count)
- dev->state_count = drv->state_count;
-
ret = cpuidle_add_device_sysfs(dev);
if (ret)
return ret;
#include <linux/sched.h>
#include <linux/cpuidle.h>
#include <linux/cpumask.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include "cpuidle.h"
#endif
/**
- * cpuidle_setup_broadcast_timer - enable/disable the broadcast timer
+ * cpuidle_setup_broadcast_timer - enable/disable the broadcast timer on a cpu
* @arg: a void pointer used to match the SMP cross call API
*
- * @arg is used as a value of type 'long' with one of the two values:
- * - CLOCK_EVT_NOTIFY_BROADCAST_ON
- * - CLOCK_EVT_NOTIFY_BROADCAST_OFF
+ * If @arg is NULL broadcast is disabled otherwise enabled
*
- * Set the broadcast timer notification for the current CPU. This function
- * is executed per CPU by an SMP cross call. It not supposed to be called
- * directly.
+ * This function is executed per CPU by an SMP cross call. It's not
+ * supposed to be called directly.
*/
static void cpuidle_setup_broadcast_timer(void *arg)
{
- int cpu = smp_processor_id();
- clockevents_notify((long)(arg), &cpu);
+ if (arg)
+ tick_broadcast_enable();
+ else
+ tick_broadcast_disable();
}
/**
if (drv->bctimer)
on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer,
- (void *)CLOCK_EVT_NOTIFY_BROADCAST_ON, 1);
+ (void *)1, 1);
poll_idle_init(drv);
if (drv->bctimer) {
drv->bctimer = 0;
on_each_cpu_mask(drv->cpumask, cpuidle_setup_broadcast_timer,
- (void *)CLOCK_EVT_NOTIFY_BROADCAST_OFF, 1);
+ NULL, 1);
}
__cpuidle_unset_driver(drv);
struct cpuidle_driver *drv = cpuidle_get_cpu_driver(device);
/* state statistics */
- for (i = 0; i < device->state_count; i++) {
+ for (i = 0; i < drv->state_count; i++) {
kobj = kzalloc(sizeof(struct cpuidle_state_kobj), GFP_KERNEL);
if (!kobj)
goto error_state;
*/
static void cpuidle_remove_state_sysfs(struct cpuidle_device *device)
{
+ struct cpuidle_driver *drv = cpuidle_get_cpu_driver(device);
int i;
- for (i = 0; i < device->state_count; i++)
+ for (i = 0; i < drv->state_count; i++)
cpuidle_free_state_kobj(device, i);
}
if (WARN_ON(timeout < 0))
return -EINVAL;
+ if (timeout == 0)
+ return fence_is_signaled(fence);
+
trace_fence_wait_start(fence);
ret = fence->ops->wait(fence, intr, timeout);
trace_fence_wait_end(fence);
unsigned seq, shared_count, i = 0;
long ret = timeout;
+ if (!timeout)
+ return reservation_object_test_signaled_rcu(obj, wait_all);
+
retry:
fence = NULL;
shared_count = 0;
int ret = 1;
if (!test_bit(FENCE_FLAG_SIGNALED_BIT, &lfence->flags)) {
- int ret;
-
fence = fence_get_rcu(lfence);
if (!fence)
return -1;
OS and tools for MIC to use with this driver are available from
<http://software.intel.com/en-us/mic-developer>.
-config INTEL_MID_DMAC
- tristate "Intel MID DMA support for Peripheral DMA controllers"
- depends on PCI && X86
- select DMA_ENGINE
- default n
- help
- Enable support for the Intel(R) MID DMA engine present
- in Intel MID chipsets.
-
- Say Y here if you have such a chipset.
-
- If unsure, say N.
-
config ASYNC_TX_ENABLE_CHANNEL_SWITCH
bool
obj-$(CONFIG_DMA_ACPI) += acpi-dma.o
obj-$(CONFIG_DMA_OF) += of-dma.o
-obj-$(CONFIG_INTEL_MID_DMAC) += intel_mid_dma.o
obj-$(CONFIG_DMATEST) += dmatest.o
obj-$(CONFIG_INTEL_IOATDMA) += ioat/
obj-$(CONFIG_INTEL_IOP_ADMA) += iop-adma.o
#define DRIVER_NAME "pl08xdmac"
+#define PL80X_DMA_BUSWIDTHS \
+ BIT(DMA_SLAVE_BUSWIDTH_UNDEFINED) | \
+ BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \
+ BIT(DMA_SLAVE_BUSWIDTH_4_BYTES)
+
static struct amba_driver pl08x_amba_driver;
struct pl08x_driver_data;
pl08x->memcpy.device_pause = pl08x_pause;
pl08x->memcpy.device_resume = pl08x_resume;
pl08x->memcpy.device_terminate_all = pl08x_terminate_all;
+ pl08x->memcpy.src_addr_widths = PL80X_DMA_BUSWIDTHS;
+ pl08x->memcpy.dst_addr_widths = PL80X_DMA_BUSWIDTHS;
+ pl08x->memcpy.directions = BIT(DMA_MEM_TO_MEM);
+ pl08x->memcpy.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
/* Initialize slave engine */
dma_cap_set(DMA_SLAVE, pl08x->slave.cap_mask);
pl08x->slave.device_pause = pl08x_pause;
pl08x->slave.device_resume = pl08x_resume;
pl08x->slave.device_terminate_all = pl08x_terminate_all;
+ pl08x->slave.src_addr_widths = PL80X_DMA_BUSWIDTHS;
+ pl08x->slave.dst_addr_widths = PL80X_DMA_BUSWIDTHS;
+ pl08x->slave.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ pl08x->slave.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
/* Get the platform data */
pl08x->pd = dev_get_platdata(&adev->dev);
}
/*
- * atc_get_current_descriptors -
- * locate the descriptor which equal to physical address in DSCR
- * @atchan: the channel we want to start
- * @dscr_addr: physical descriptor address in DSCR
+ * atc_get_desc_by_cookie - get the descriptor of a cookie
+ * @atchan: the DMA channel
+ * @cookie: the cookie to get the descriptor for
*/
-static struct at_desc *atc_get_current_descriptors(struct at_dma_chan *atchan,
- u32 dscr_addr)
+static struct at_desc *atc_get_desc_by_cookie(struct at_dma_chan *atchan,
+ dma_cookie_t cookie)
{
- struct at_desc *desc, *_desc, *child, *desc_cur = NULL;
+ struct at_desc *desc, *_desc;
- list_for_each_entry_safe(desc, _desc, &atchan->active_list, desc_node) {
- if (desc->lli.dscr == dscr_addr) {
- desc_cur = desc;
- break;
- }
+ list_for_each_entry_safe(desc, _desc, &atchan->queue, desc_node) {
+ if (desc->txd.cookie == cookie)
+ return desc;
+ }
- list_for_each_entry(child, &desc->tx_list, desc_node) {
- if (child->lli.dscr == dscr_addr) {
- desc_cur = child;
- break;
- }
- }
+ list_for_each_entry_safe(desc, _desc, &atchan->active_list, desc_node) {
+ if (desc->txd.cookie == cookie)
+ return desc;
}
- return desc_cur;
+ return NULL;
}
-/*
- * atc_get_bytes_left -
- * Get the number of bytes residue in dma buffer,
- * @chan: the channel we want to start
+/**
+ * atc_calc_bytes_left - calculates the number of bytes left according to the
+ * value read from CTRLA.
+ *
+ * @current_len: the number of bytes left before reading CTRLA
+ * @ctrla: the value of CTRLA
+ * @desc: the descriptor containing the transfer width
+ */
+static inline int atc_calc_bytes_left(int current_len, u32 ctrla,
+ struct at_desc *desc)
+{
+ return current_len - ((ctrla & ATC_BTSIZE_MAX) << desc->tx_width);
+}
+
+/**
+ * atc_calc_bytes_left_from_reg - calculates the number of bytes left according
+ * to the current value of CTRLA.
+ *
+ * @current_len: the number of bytes left before reading CTRLA
+ * @atchan: the channel to read CTRLA for
+ * @desc: the descriptor containing the transfer width
+ */
+static inline int atc_calc_bytes_left_from_reg(int current_len,
+ struct at_dma_chan *atchan, struct at_desc *desc)
+{
+ u32 ctrla = channel_readl(atchan, CTRLA);
+
+ return atc_calc_bytes_left(current_len, ctrla, desc);
+}
+
+/**
+ * atc_get_bytes_left - get the number of bytes residue for a cookie
+ * @chan: DMA channel
+ * @cookie: transaction identifier to check status of
*/
-static int atc_get_bytes_left(struct dma_chan *chan)
+static int atc_get_bytes_left(struct dma_chan *chan, dma_cookie_t cookie)
{
struct at_dma_chan *atchan = to_at_dma_chan(chan);
- struct at_dma *atdma = to_at_dma(chan->device);
- int chan_id = atchan->chan_common.chan_id;
struct at_desc *desc_first = atc_first_active(atchan);
- struct at_desc *desc_cur;
- int ret = 0, count = 0;
+ struct at_desc *desc;
+ int ret;
+ u32 ctrla, dscr;
/*
- * Initialize necessary values in the first time.
- * remain_desc record remain desc length.
+ * If the cookie doesn't match to the currently running transfer then
+ * we can return the total length of the associated DMA transfer,
+ * because it is still queued.
*/
- if (atchan->remain_desc == 0)
- /* First descriptor embedds the transaction length */
- atchan->remain_desc = desc_first->len;
+ desc = atc_get_desc_by_cookie(atchan, cookie);
+ if (desc == NULL)
+ return -EINVAL;
+ else if (desc != desc_first)
+ return desc->total_len;
- /*
- * This happens when current descriptor transfer complete.
- * The residual buffer size should reduce current descriptor length.
- */
- if (unlikely(test_bit(ATC_IS_BTC, &atchan->status))) {
- clear_bit(ATC_IS_BTC, &atchan->status);
- desc_cur = atc_get_current_descriptors(atchan,
- channel_readl(atchan, DSCR));
- if (!desc_cur) {
- ret = -EINVAL;
- goto out;
- }
+ /* cookie matches to the currently running transfer */
+ ret = desc_first->total_len;
- count = (desc_cur->lli.ctrla & ATC_BTSIZE_MAX)
- << desc_first->tx_width;
- if (atchan->remain_desc < count) {
- ret = -EINVAL;
- goto out;
+ if (desc_first->lli.dscr) {
+ /* hardware linked list transfer */
+
+ /*
+ * Calculate the residue by removing the length of the child
+ * descriptors already transferred from the total length.
+ * To get the current child descriptor we can use the value of
+ * the channel's DSCR register and compare it against the value
+ * of the hardware linked list structure of each child
+ * descriptor.
+ */
+
+ ctrla = channel_readl(atchan, CTRLA);
+ rmb(); /* ensure CTRLA is read before DSCR */
+ dscr = channel_readl(atchan, DSCR);
+
+ /* for the first descriptor we can be more accurate */
+ if (desc_first->lli.dscr == dscr)
+ return atc_calc_bytes_left(ret, ctrla, desc_first);
+
+ ret -= desc_first->len;
+ list_for_each_entry(desc, &desc_first->tx_list, desc_node) {
+ if (desc->lli.dscr == dscr)
+ break;
+
+ ret -= desc->len;
}
- atchan->remain_desc -= count;
- ret = atchan->remain_desc;
- } else {
/*
- * Get residual bytes when current
- * descriptor transfer in progress.
+ * For the last descriptor in the chain we can calculate
+ * the remaining bytes using the channel's register.
+ * Note that the transfer width of the first and last
+ * descriptor may differ.
*/
- count = (channel_readl(atchan, CTRLA) & ATC_BTSIZE_MAX)
- << (desc_first->tx_width);
- ret = atchan->remain_desc - count;
+ if (!desc->lli.dscr)
+ ret = atc_calc_bytes_left_from_reg(ret, atchan, desc);
+ } else {
+ /* single transfer */
+ ret = atc_calc_bytes_left_from_reg(ret, atchan, desc_first);
}
- /*
- * Check fifo empty.
- */
- if (!(dma_readl(atdma, CHSR) & AT_DMA_EMPT(chan_id)))
- atc_issue_pending(chan);
-out:
return ret;
}
/* Give information to tasklet */
set_bit(ATC_IS_ERROR, &atchan->status);
}
- if (pending & AT_DMA_BTC(i))
- set_bit(ATC_IS_BTC, &atchan->status);
tasklet_schedule(&atchan->tasklet);
ret = IRQ_HANDLED;
}
desc->lli.ctrlb = ctrlb;
desc->txd.cookie = 0;
+ desc->len = xfer_count << src_width;
atc_desc_chain(&first, &prev, desc);
}
/* First descriptor of the chain embedds additional information */
first->txd.cookie = -EBUSY;
- first->len = len;
+ first->total_len = len;
+
+ /* set transfer width for the calculation of the residue */
first->tx_width = src_width;
+ prev->tx_width = src_width;
/* set end-of-link to the last link descriptor of list*/
set_desc_eol(desc);
| ATC_SRC_WIDTH(mem_width)
| len >> mem_width;
desc->lli.ctrlb = ctrlb;
+ desc->len = len;
atc_desc_chain(&first, &prev, desc);
total_len += len;
| ATC_DST_WIDTH(mem_width)
| len >> reg_width;
desc->lli.ctrlb = ctrlb;
+ desc->len = len;
atc_desc_chain(&first, &prev, desc);
total_len += len;
/* First descriptor of the chain embedds additional information */
first->txd.cookie = -EBUSY;
- first->len = total_len;
+ first->total_len = total_len;
+
+ /* set transfer width for the calculation of the residue */
first->tx_width = reg_width;
+ prev->tx_width = reg_width;
/* first link descriptor of list is responsible of flags */
first->txd.flags = flags; /* client is in control of this ack */
| ATC_FC_MEM2PER
| ATC_SIF(atchan->mem_if)
| ATC_DIF(atchan->per_if);
+ desc->len = period_len;
break;
case DMA_DEV_TO_MEM:
| ATC_FC_PER2MEM
| ATC_SIF(atchan->per_if)
| ATC_DIF(atchan->mem_if);
+ desc->len = period_len;
break;
default:
/* First descriptor of the chain embedds additional information */
first->txd.cookie = -EBUSY;
- first->len = buf_len;
+ first->total_len = buf_len;
first->tx_width = reg_width;
return &first->txd;
spin_lock_irqsave(&atchan->lock, flags);
/* Get number of bytes left in the active transactions */
- bytes = atc_get_bytes_left(chan);
+ bytes = atc_get_bytes_left(chan, cookie);
spin_unlock_irqrestore(&atchan->lock, flags);
spin_lock_irqsave(&atchan->lock, flags);
atchan->descs_allocated = i;
- atchan->remain_desc = 0;
list_splice(&tmp_list, &atchan->free_list);
dma_cookie_init(chan);
spin_unlock_irqrestore(&atchan->lock, flags);
list_splice_init(&atchan->free_list, &list);
atchan->descs_allocated = 0;
atchan->status = 0;
- atchan->remain_desc = 0;
dev_vdbg(chan2dev(chan), "free_chan_resources: done\n");
}
* @at_lli: hardware lli structure
* @txd: support for the async_tx api
* @desc_node: node on the channed descriptors list
- * @len: total transaction bytecount
+ * @len: descriptor byte count
* @tx_width: transfer width
+ * @total_len: total transaction byte count
*/
struct at_desc {
/* FIRST values the hardware uses */
struct list_head desc_node;
size_t len;
u32 tx_width;
+ size_t total_len;
};
static inline struct at_desc *
enum atc_status {
ATC_IS_ERROR = 0,
ATC_IS_PAUSED = 1,
- ATC_IS_BTC = 2,
ATC_IS_CYCLIC = 24,
};
* @save_cfg: configuration register that is saved on suspend/resume cycle
* @save_dscr: for cyclic operations, preserve next descriptor address in
* the cyclic list on suspend/resume cycle
- * @remain_desc: to save remain desc length
* @dma_sconfig: configuration for slave transfers, passed via
* .device_config
* @lock: serializes enqueue/dequeue operations to descriptors lists
struct tasklet_struct tasklet;
u32 save_cfg;
u32 save_dscr;
- u32 remain_desc;
struct dma_slave_config dma_sconfig;
spinlock_t lock;
struct at_xdmac_desc *first = NULL, *prev = NULL;
unsigned int periods = buf_len / period_len;
int i;
- u32 cfg;
dev_dbg(chan2dev(chan), "%s: buf_addr=%pad, buf_len=%zd, period_len=%zd, dir=%s, flags=0x%lx\n",
__func__, &buf_addr, buf_len, period_len,
if (direction == DMA_DEV_TO_MEM) {
desc->lld.mbr_sa = atchan->per_src_addr;
desc->lld.mbr_da = buf_addr + i * period_len;
- cfg = atchan->cfg[AT_XDMAC_DEV_TO_MEM_CFG];
+ desc->lld.mbr_cfg = atchan->cfg[AT_XDMAC_DEV_TO_MEM_CFG];
} else {
desc->lld.mbr_sa = buf_addr + i * period_len;
desc->lld.mbr_da = atchan->per_dst_addr;
- cfg = atchan->cfg[AT_XDMAC_MEM_TO_DEV_CFG];
+ desc->lld.mbr_cfg = atchan->cfg[AT_XDMAC_MEM_TO_DEV_CFG];
}
desc->lld.mbr_ubc = AT_XDMAC_MBR_UBC_NDV1
| AT_XDMAC_MBR_UBC_NDEN
| AT_XDMAC_MBR_UBC_NSEN
| AT_XDMAC_MBR_UBC_NDE
- | period_len >> at_xdmac_get_dwidth(cfg);
+ | period_len >> at_xdmac_get_dwidth(desc->lld.mbr_cfg);
dev_dbg(chan2dev(chan),
"%s: lld: mbr_sa=%pad, mbr_da=%pad, mbr_ubc=0x%08x\n",
* c->desc is NULL and exit.)
*/
if (c->desc) {
+ bcm2835_dma_desc_free(&c->desc->vd);
c->desc = NULL;
bcm2835_dma_abort(c->chan_base);
return of_id->data;
}
+#define CPPI41_DMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | \
+ BIT(DMA_SLAVE_BUSWIDTH_3_BYTES) | \
+ BIT(DMA_SLAVE_BUSWIDTH_4_BYTES))
+
static int cppi41_dma_probe(struct platform_device *pdev)
{
struct cppi41_dd *cdd;
cdd->ddev.device_issue_pending = cppi41_dma_issue_pending;
cdd->ddev.device_prep_slave_sg = cppi41_dma_prep_slave_sg;
cdd->ddev.device_terminate_all = cppi41_stop_chan;
+ cdd->ddev.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ cdd->ddev.src_addr_widths = CPPI41_DMA_BUSWIDTHS;
+ cdd->ddev.dst_addr_widths = CPPI41_DMA_BUSWIDTHS;
+ cdd->ddev.residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
cdd->ddev.dev = dev;
INIT_LIST_HEAD(&cdd->ddev.channels);
cpp41_dma_info.dma_cap = cdd->ddev.cap_mask;
kfree(container_of(vdesc, struct jz4740_dma_desc, vdesc));
}
+#define JZ4740_DMA_BUSWIDTHS (BIT(DMA_SLAVE_BUSWIDTH_1_BYTE) | \
+ BIT(DMA_SLAVE_BUSWIDTH_2_BYTES) | BIT(DMA_SLAVE_BUSWIDTH_4_BYTES))
+
static int jz4740_dma_probe(struct platform_device *pdev)
{
struct jz4740_dmaengine_chan *chan;
dd->device_prep_dma_cyclic = jz4740_dma_prep_dma_cyclic;
dd->device_config = jz4740_dma_slave_config;
dd->device_terminate_all = jz4740_dma_terminate_all;
+ dd->src_addr_widths = JZ4740_DMA_BUSWIDTHS;
+ dd->dst_addr_widths = JZ4740_DMA_BUSWIDTHS;
+ dd->directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ dd->residue_granularity = DMA_RESIDUE_GRANULARITY_BURST;
dd->dev = &pdev->dev;
INIT_LIST_HEAD(&dd->channels);
BUG_ON(!device->device_issue_pending);
BUG_ON(!device->dev);
- WARN(dma_has_cap(DMA_SLAVE, device->cap_mask) && !device->directions,
- "this driver doesn't support generic slave capabilities reporting\n");
-
/* note: this only matters in the
* CONFIG_ASYNC_TX_ENABLE_CHANNEL_SWITCH=n case
*/
dev_vdbg(dw->dma.dev, "%s: status=0x%x\n", __func__, status);
/* Check if we have any interrupt from the DMAC */
- if (!status)
+ if (!status || !dw->in_use)
return IRQ_NONE;
/*
#include "internal.h"
+#define DRV_NAME "dw_dmac"
+
static struct dma_chan *dw_dma_of_xlate(struct of_phandle_args *dma_spec,
struct of_dma *ofdma)
{
.remove = dw_remove,
.shutdown = dw_shutdown,
.driver = {
- .name = "dw_dmac",
+ .name = DRV_NAME,
.pm = &dw_dev_pm_ops,
.of_match_table = of_match_ptr(dw_dma_of_id_table),
.acpi_match_table = ACPI_PTR(dw_dma_acpi_id_table),
MODULE_LICENSE("GPL v2");
MODULE_DESCRIPTION("Synopsys DesignWare DMA Controller platform driver");
+MODULE_ALIAS("platform:" DRV_NAME);
*/
if (echan->edesc) {
int cyclic = echan->edesc->cyclic;
+
+ /*
+ * free the running request descriptor
+ * since it is not in any of the vdesc lists
+ */
+ edma_desc_free(&echan->edesc->vdesc);
+
echan->edesc = NULL;
edma_stop(echan->ch_num);
/* Move the cyclic channel back to default queue */
dev_err(sdma->dev, "Timeout waiting for CH0 ready\n");
}
+ /* Set bits of CONFIG register with dynamic context switching */
+ if (readl(sdma->regs + SDMA_H_CONFIG) == 0)
+ writel_relaxed(SDMA_H_CONFIG_CSM, sdma->regs + SDMA_H_CONFIG);
+
return ret ? 0 : -ETIMEDOUT;
}
writel_relaxed(ccb_phys, sdma->regs + SDMA_H_C0PTR);
- /* Set bits of CONFIG register with given context switching mode */
- writel_relaxed(SDMA_H_CONFIG_CSM, sdma->regs + SDMA_H_CONFIG);
-
/* Initializes channel's priorities */
sdma_set_channel_priority(&sdma->channel[0], 7);
+++ /dev/null
-/*
- * intel_mid_dma.c - Intel Langwell DMA Drivers
- *
- * Copyright (C) 2008-10 Intel Corp
- * Author: Vinod Koul <vinod.koul@intel.com>
- * The driver design is based on dw_dmac driver
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
- *
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- *
- */
-#include <linux/pci.h>
-#include <linux/interrupt.h>
-#include <linux/pm_runtime.h>
-#include <linux/intel_mid_dma.h>
-#include <linux/module.h>
-
-#include "dmaengine.h"
-
-#define MAX_CHAN 4 /*max ch across controllers*/
-#include "intel_mid_dma_regs.h"
-
-#define INTEL_MID_DMAC1_ID 0x0814
-#define INTEL_MID_DMAC2_ID 0x0813
-#define INTEL_MID_GP_DMAC2_ID 0x0827
-#define INTEL_MFLD_DMAC1_ID 0x0830
-#define LNW_PERIPHRAL_MASK_BASE 0xFFAE8008
-#define LNW_PERIPHRAL_MASK_SIZE 0x10
-#define LNW_PERIPHRAL_STATUS 0x0
-#define LNW_PERIPHRAL_MASK 0x8
-
-struct intel_mid_dma_probe_info {
- u8 max_chan;
- u8 ch_base;
- u16 block_size;
- u32 pimr_mask;
-};
-
-#define INFO(_max_chan, _ch_base, _block_size, _pimr_mask) \
- ((kernel_ulong_t)&(struct intel_mid_dma_probe_info) { \
- .max_chan = (_max_chan), \
- .ch_base = (_ch_base), \
- .block_size = (_block_size), \
- .pimr_mask = (_pimr_mask), \
- })
-
-/*****************************************************************************
-Utility Functions*/
-/**
- * get_ch_index - convert status to channel
- * @status: status mask
- * @base: dma ch base value
- *
- * Modify the status mask and return the channel index needing
- * attention (or -1 if neither)
- */
-static int get_ch_index(int *status, unsigned int base)
-{
- int i;
- for (i = 0; i < MAX_CHAN; i++) {
- if (*status & (1 << (i + base))) {
- *status = *status & ~(1 << (i + base));
- pr_debug("MDMA: index %d New status %x\n", i, *status);
- return i;
- }
- }
- return -1;
-}
-
-/**
- * get_block_ts - calculates dma transaction length
- * @len: dma transfer length
- * @tx_width: dma transfer src width
- * @block_size: dma controller max block size
- *
- * Based on src width calculate the DMA trsaction length in data items
- * return data items or FFFF if exceeds max length for block
- */
-static int get_block_ts(int len, int tx_width, int block_size)
-{
- int byte_width = 0, block_ts = 0;
-
- switch (tx_width) {
- case DMA_SLAVE_BUSWIDTH_1_BYTE:
- byte_width = 1;
- break;
- case DMA_SLAVE_BUSWIDTH_2_BYTES:
- byte_width = 2;
- break;
- case DMA_SLAVE_BUSWIDTH_4_BYTES:
- default:
- byte_width = 4;
- break;
- }
-
- block_ts = len/byte_width;
- if (block_ts > block_size)
- block_ts = 0xFFFF;
- return block_ts;
-}
-
-/*****************************************************************************
-DMAC1 interrupt Functions*/
-
-/**
- * dmac1_mask_periphral_intr - mask the periphral interrupt
- * @mid: dma device for which masking is required
- *
- * Masks the DMA periphral interrupt
- * this is valid for DMAC1 family controllers only
- * This controller should have periphral mask registers already mapped
- */
-static void dmac1_mask_periphral_intr(struct middma_device *mid)
-{
- u32 pimr;
-
- if (mid->pimr_mask) {
- pimr = readl(mid->mask_reg + LNW_PERIPHRAL_MASK);
- pimr |= mid->pimr_mask;
- writel(pimr, mid->mask_reg + LNW_PERIPHRAL_MASK);
- }
- return;
-}
-
-/**
- * dmac1_unmask_periphral_intr - unmask the periphral interrupt
- * @midc: dma channel for which masking is required
- *
- * UnMasks the DMA periphral interrupt,
- * this is valid for DMAC1 family controllers only
- * This controller should have periphral mask registers already mapped
- */
-static void dmac1_unmask_periphral_intr(struct intel_mid_dma_chan *midc)
-{
- u32 pimr;
- struct middma_device *mid = to_middma_device(midc->chan.device);
-
- if (mid->pimr_mask) {
- pimr = readl(mid->mask_reg + LNW_PERIPHRAL_MASK);
- pimr &= ~mid->pimr_mask;
- writel(pimr, mid->mask_reg + LNW_PERIPHRAL_MASK);
- }
- return;
-}
-
-/**
- * enable_dma_interrupt - enable the periphral interrupt
- * @midc: dma channel for which enable interrupt is required
- *
- * Enable the DMA periphral interrupt,
- * this is valid for DMAC1 family controllers only
- * This controller should have periphral mask registers already mapped
- */
-static void enable_dma_interrupt(struct intel_mid_dma_chan *midc)
-{
- dmac1_unmask_periphral_intr(midc);
-
- /*en ch interrupts*/
- iowrite32(UNMASK_INTR_REG(midc->ch_id), midc->dma_base + MASK_TFR);
- iowrite32(UNMASK_INTR_REG(midc->ch_id), midc->dma_base + MASK_ERR);
- return;
-}
-
-/**
- * disable_dma_interrupt - disable the periphral interrupt
- * @midc: dma channel for which disable interrupt is required
- *
- * Disable the DMA periphral interrupt,
- * this is valid for DMAC1 family controllers only
- * This controller should have periphral mask registers already mapped
- */
-static void disable_dma_interrupt(struct intel_mid_dma_chan *midc)
-{
- /*Check LPE PISR, make sure fwd is disabled*/
- iowrite32(MASK_INTR_REG(midc->ch_id), midc->dma_base + MASK_BLOCK);
- iowrite32(MASK_INTR_REG(midc->ch_id), midc->dma_base + MASK_TFR);
- iowrite32(MASK_INTR_REG(midc->ch_id), midc->dma_base + MASK_ERR);
- return;
-}
-
-/*****************************************************************************
-DMA channel helper Functions*/
-/**
- * mid_desc_get - get a descriptor
- * @midc: dma channel for which descriptor is required
- *
- * Obtain a descriptor for the channel. Returns NULL if none are free.
- * Once the descriptor is returned it is private until put on another
- * list or freed
- */
-static struct intel_mid_dma_desc *midc_desc_get(struct intel_mid_dma_chan *midc)
-{
- struct intel_mid_dma_desc *desc, *_desc;
- struct intel_mid_dma_desc *ret = NULL;
-
- spin_lock_bh(&midc->lock);
- list_for_each_entry_safe(desc, _desc, &midc->free_list, desc_node) {
- if (async_tx_test_ack(&desc->txd)) {
- list_del(&desc->desc_node);
- ret = desc;
- break;
- }
- }
- spin_unlock_bh(&midc->lock);
- return ret;
-}
-
-/**
- * mid_desc_put - put a descriptor
- * @midc: dma channel for which descriptor is required
- * @desc: descriptor to put
- *
- * Return a descriptor from lwn_desc_get back to the free pool
- */
-static void midc_desc_put(struct intel_mid_dma_chan *midc,
- struct intel_mid_dma_desc *desc)
-{
- if (desc) {
- spin_lock_bh(&midc->lock);
- list_add_tail(&desc->desc_node, &midc->free_list);
- spin_unlock_bh(&midc->lock);
- }
-}
-/**
- * midc_dostart - begin a DMA transaction
- * @midc: channel for which txn is to be started
- * @first: first descriptor of series
- *
- * Load a transaction into the engine. This must be called with midc->lock
- * held and bh disabled.
- */
-static void midc_dostart(struct intel_mid_dma_chan *midc,
- struct intel_mid_dma_desc *first)
-{
- struct middma_device *mid = to_middma_device(midc->chan.device);
-
- /* channel is idle */
- if (midc->busy && test_ch_en(midc->dma_base, midc->ch_id)) {
- /*error*/
- pr_err("ERR_MDMA: channel is busy in start\n");
- /* The tasklet will hopefully advance the queue... */
- return;
- }
- midc->busy = true;
- /*write registers and en*/
- iowrite32(first->sar, midc->ch_regs + SAR);
- iowrite32(first->dar, midc->ch_regs + DAR);
- iowrite32(first->lli_phys, midc->ch_regs + LLP);
- iowrite32(first->cfg_hi, midc->ch_regs + CFG_HIGH);
- iowrite32(first->cfg_lo, midc->ch_regs + CFG_LOW);
- iowrite32(first->ctl_lo, midc->ch_regs + CTL_LOW);
- iowrite32(first->ctl_hi, midc->ch_regs + CTL_HIGH);
- pr_debug("MDMA:TX SAR %x,DAR %x,CFGL %x,CFGH %x,CTLH %x, CTLL %x\n",
- (int)first->sar, (int)first->dar, first->cfg_hi,
- first->cfg_lo, first->ctl_hi, first->ctl_lo);
- first->status = DMA_IN_PROGRESS;
-
- iowrite32(ENABLE_CHANNEL(midc->ch_id), mid->dma_base + DMA_CHAN_EN);
-}
-
-/**
- * midc_descriptor_complete - process completed descriptor
- * @midc: channel owning the descriptor
- * @desc: the descriptor itself
- *
- * Process a completed descriptor and perform any callbacks upon
- * the completion. The completion handling drops the lock during the
- * callbacks but must be called with the lock held.
- */
-static void midc_descriptor_complete(struct intel_mid_dma_chan *midc,
- struct intel_mid_dma_desc *desc)
- __releases(&midc->lock) __acquires(&midc->lock)
-{
- struct dma_async_tx_descriptor *txd = &desc->txd;
- dma_async_tx_callback callback_txd = NULL;
- struct intel_mid_dma_lli *llitem;
- void *param_txd = NULL;
-
- dma_cookie_complete(txd);
- callback_txd = txd->callback;
- param_txd = txd->callback_param;
-
- if (desc->lli != NULL) {
- /*clear the DONE bit of completed LLI in memory*/
- llitem = desc->lli + desc->current_lli;
- llitem->ctl_hi &= CLEAR_DONE;
- if (desc->current_lli < desc->lli_length-1)
- (desc->current_lli)++;
- else
- desc->current_lli = 0;
- }
- spin_unlock_bh(&midc->lock);
- if (callback_txd) {
- pr_debug("MDMA: TXD callback set ... calling\n");
- callback_txd(param_txd);
- }
- if (midc->raw_tfr) {
- desc->status = DMA_COMPLETE;
- if (desc->lli != NULL) {
- pci_pool_free(desc->lli_pool, desc->lli,
- desc->lli_phys);
- pci_pool_destroy(desc->lli_pool);
- desc->lli = NULL;
- }
- list_move(&desc->desc_node, &midc->free_list);
- midc->busy = false;
- }
- spin_lock_bh(&midc->lock);
-
-}
-/**
- * midc_scan_descriptors - check the descriptors in channel
- * mark completed when tx is completete
- * @mid: device
- * @midc: channel to scan
- *
- * Walk the descriptor chain for the device and process any entries
- * that are complete.
- */
-static void midc_scan_descriptors(struct middma_device *mid,
- struct intel_mid_dma_chan *midc)
-{
- struct intel_mid_dma_desc *desc = NULL, *_desc = NULL;
-
- /*tx is complete*/
- list_for_each_entry_safe(desc, _desc, &midc->active_list, desc_node) {
- if (desc->status == DMA_IN_PROGRESS)
- midc_descriptor_complete(midc, desc);
- }
- return;
- }
-/**
- * midc_lli_fill_sg - Helper function to convert
- * SG list to Linked List Items.
- *@midc: Channel
- *@desc: DMA descriptor
- *@sglist: Pointer to SG list
- *@sglen: SG list length
- *@flags: DMA transaction flags
- *
- * Walk through the SG list and convert the SG list into Linked
- * List Items (LLI).
- */
-static int midc_lli_fill_sg(struct intel_mid_dma_chan *midc,
- struct intel_mid_dma_desc *desc,
- struct scatterlist *sglist,
- unsigned int sglen,
- unsigned int flags)
-{
- struct intel_mid_dma_slave *mids;
- struct scatterlist *sg;
- dma_addr_t lli_next, sg_phy_addr;
- struct intel_mid_dma_lli *lli_bloc_desc;
- union intel_mid_dma_ctl_lo ctl_lo;
- union intel_mid_dma_ctl_hi ctl_hi;
- int i;
-
- pr_debug("MDMA: Entered midc_lli_fill_sg\n");
- mids = midc->mid_slave;
-
- lli_bloc_desc = desc->lli;
- lli_next = desc->lli_phys;
-
- ctl_lo.ctl_lo = desc->ctl_lo;
- ctl_hi.ctl_hi = desc->ctl_hi;
- for_each_sg(sglist, sg, sglen, i) {
- /*Populate CTL_LOW and LLI values*/
- if (i != sglen - 1) {
- lli_next = lli_next +
- sizeof(struct intel_mid_dma_lli);
- } else {
- /*Check for circular list, otherwise terminate LLI to ZERO*/
- if (flags & DMA_PREP_CIRCULAR_LIST) {
- pr_debug("MDMA: LLI is configured in circular mode\n");
- lli_next = desc->lli_phys;
- } else {
- lli_next = 0;
- ctl_lo.ctlx.llp_dst_en = 0;
- ctl_lo.ctlx.llp_src_en = 0;
- }
- }
- /*Populate CTL_HI values*/
- ctl_hi.ctlx.block_ts = get_block_ts(sg_dma_len(sg),
- desc->width,
- midc->dma->block_size);
- /*Populate SAR and DAR values*/
- sg_phy_addr = sg_dma_address(sg);
- if (desc->dirn == DMA_MEM_TO_DEV) {
- lli_bloc_desc->sar = sg_phy_addr;
- lli_bloc_desc->dar = mids->dma_slave.dst_addr;
- } else if (desc->dirn == DMA_DEV_TO_MEM) {
- lli_bloc_desc->sar = mids->dma_slave.src_addr;
- lli_bloc_desc->dar = sg_phy_addr;
- }
- /*Copy values into block descriptor in system memroy*/
- lli_bloc_desc->llp = lli_next;
- lli_bloc_desc->ctl_lo = ctl_lo.ctl_lo;
- lli_bloc_desc->ctl_hi = ctl_hi.ctl_hi;
-
- lli_bloc_desc++;
- }
- /*Copy very first LLI values to descriptor*/
- desc->ctl_lo = desc->lli->ctl_lo;
- desc->ctl_hi = desc->lli->ctl_hi;
- desc->sar = desc->lli->sar;
- desc->dar = desc->lli->dar;
-
- return 0;
-}
-/*****************************************************************************
-DMA engine callback Functions*/
-/**
- * intel_mid_dma_tx_submit - callback to submit DMA transaction
- * @tx: dma engine descriptor
- *
- * Submit the DMA transaction for this descriptor, start if ch idle
- */
-static dma_cookie_t intel_mid_dma_tx_submit(struct dma_async_tx_descriptor *tx)
-{
- struct intel_mid_dma_desc *desc = to_intel_mid_dma_desc(tx);
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(tx->chan);
- dma_cookie_t cookie;
-
- spin_lock_bh(&midc->lock);
- cookie = dma_cookie_assign(tx);
-
- if (list_empty(&midc->active_list))
- list_add_tail(&desc->desc_node, &midc->active_list);
- else
- list_add_tail(&desc->desc_node, &midc->queue);
-
- midc_dostart(midc, desc);
- spin_unlock_bh(&midc->lock);
-
- return cookie;
-}
-
-/**
- * intel_mid_dma_issue_pending - callback to issue pending txn
- * @chan: chan where pending trascation needs to be checked and submitted
- *
- * Call for scan to issue pending descriptors
- */
-static void intel_mid_dma_issue_pending(struct dma_chan *chan)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
-
- spin_lock_bh(&midc->lock);
- if (!list_empty(&midc->queue))
- midc_scan_descriptors(to_middma_device(chan->device), midc);
- spin_unlock_bh(&midc->lock);
-}
-
-/**
- * intel_mid_dma_tx_status - Return status of txn
- * @chan: chan for where status needs to be checked
- * @cookie: cookie for txn
- * @txstate: DMA txn state
- *
- * Return status of DMA txn
- */
-static enum dma_status intel_mid_dma_tx_status(struct dma_chan *chan,
- dma_cookie_t cookie,
- struct dma_tx_state *txstate)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
- enum dma_status ret;
-
- ret = dma_cookie_status(chan, cookie, txstate);
- if (ret != DMA_COMPLETE) {
- spin_lock_bh(&midc->lock);
- midc_scan_descriptors(to_middma_device(chan->device), midc);
- spin_unlock_bh(&midc->lock);
-
- ret = dma_cookie_status(chan, cookie, txstate);
- }
-
- return ret;
-}
-
-static int intel_mid_dma_config(struct dma_chan *chan,
- struct dma_slave_config *slave)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
- struct intel_mid_dma_slave *mid_slave;
-
- BUG_ON(!midc);
- BUG_ON(!slave);
- pr_debug("MDMA: slave control called\n");
-
- mid_slave = to_intel_mid_dma_slave(slave);
-
- BUG_ON(!mid_slave);
-
- midc->mid_slave = mid_slave;
- return 0;
-}
-
-static int intel_mid_dma_terminate_all(struct dma_chan *chan)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
- struct middma_device *mid = to_middma_device(chan->device);
- struct intel_mid_dma_desc *desc, *_desc;
- union intel_mid_dma_cfg_lo cfg_lo;
-
- spin_lock_bh(&midc->lock);
- if (midc->busy == false) {
- spin_unlock_bh(&midc->lock);
- return 0;
- }
- /*Suspend and disable the channel*/
- cfg_lo.cfg_lo = ioread32(midc->ch_regs + CFG_LOW);
- cfg_lo.cfgx.ch_susp = 1;
- iowrite32(cfg_lo.cfg_lo, midc->ch_regs + CFG_LOW);
- iowrite32(DISABLE_CHANNEL(midc->ch_id), mid->dma_base + DMA_CHAN_EN);
- midc->busy = false;
- /* Disable interrupts */
- disable_dma_interrupt(midc);
- midc->descs_allocated = 0;
-
- spin_unlock_bh(&midc->lock);
- list_for_each_entry_safe(desc, _desc, &midc->active_list, desc_node) {
- if (desc->lli != NULL) {
- pci_pool_free(desc->lli_pool, desc->lli,
- desc->lli_phys);
- pci_pool_destroy(desc->lli_pool);
- desc->lli = NULL;
- }
- list_move(&desc->desc_node, &midc->free_list);
- }
- return 0;
-}
-
-
-/**
- * intel_mid_dma_prep_memcpy - Prep memcpy txn
- * @chan: chan for DMA transfer
- * @dest: destn address
- * @src: src address
- * @len: DMA transfer len
- * @flags: DMA flags
- *
- * Perform a DMA memcpy. Note we support slave periphral DMA transfers only
- * The periphral txn details should be filled in slave structure properly
- * Returns the descriptor for this txn
- */
-static struct dma_async_tx_descriptor *intel_mid_dma_prep_memcpy(
- struct dma_chan *chan, dma_addr_t dest,
- dma_addr_t src, size_t len, unsigned long flags)
-{
- struct intel_mid_dma_chan *midc;
- struct intel_mid_dma_desc *desc = NULL;
- struct intel_mid_dma_slave *mids;
- union intel_mid_dma_ctl_lo ctl_lo;
- union intel_mid_dma_ctl_hi ctl_hi;
- union intel_mid_dma_cfg_lo cfg_lo;
- union intel_mid_dma_cfg_hi cfg_hi;
- enum dma_slave_buswidth width;
-
- pr_debug("MDMA: Prep for memcpy\n");
- BUG_ON(!chan);
- if (!len)
- return NULL;
-
- midc = to_intel_mid_dma_chan(chan);
- BUG_ON(!midc);
-
- mids = midc->mid_slave;
- BUG_ON(!mids);
-
- pr_debug("MDMA:called for DMA %x CH %d Length %zu\n",
- midc->dma->pci_id, midc->ch_id, len);
- pr_debug("MDMA:Cfg passed Mode %x, Dirn %x, HS %x, Width %x\n",
- mids->cfg_mode, mids->dma_slave.direction,
- mids->hs_mode, mids->dma_slave.src_addr_width);
-
- /*calculate CFG_LO*/
- if (mids->hs_mode == LNW_DMA_SW_HS) {
- cfg_lo.cfg_lo = 0;
- cfg_lo.cfgx.hs_sel_dst = 1;
- cfg_lo.cfgx.hs_sel_src = 1;
- } else if (mids->hs_mode == LNW_DMA_HW_HS)
- cfg_lo.cfg_lo = 0x00000;
-
- /*calculate CFG_HI*/
- if (mids->cfg_mode == LNW_DMA_MEM_TO_MEM) {
- /*SW HS only*/
- cfg_hi.cfg_hi = 0;
- } else {
- cfg_hi.cfg_hi = 0;
- if (midc->dma->pimr_mask) {
- cfg_hi.cfgx.protctl = 0x0; /*default value*/
- cfg_hi.cfgx.fifo_mode = 1;
- if (mids->dma_slave.direction == DMA_MEM_TO_DEV) {
- cfg_hi.cfgx.src_per = 0;
- if (mids->device_instance == 0)
- cfg_hi.cfgx.dst_per = 3;
- if (mids->device_instance == 1)
- cfg_hi.cfgx.dst_per = 1;
- } else if (mids->dma_slave.direction == DMA_DEV_TO_MEM) {
- if (mids->device_instance == 0)
- cfg_hi.cfgx.src_per = 2;
- if (mids->device_instance == 1)
- cfg_hi.cfgx.src_per = 0;
- cfg_hi.cfgx.dst_per = 0;
- }
- } else {
- cfg_hi.cfgx.protctl = 0x1; /*default value*/
- cfg_hi.cfgx.src_per = cfg_hi.cfgx.dst_per =
- midc->ch_id - midc->dma->chan_base;
- }
- }
-
- /*calculate CTL_HI*/
- ctl_hi.ctlx.reser = 0;
- ctl_hi.ctlx.done = 0;
- width = mids->dma_slave.src_addr_width;
-
- ctl_hi.ctlx.block_ts = get_block_ts(len, width, midc->dma->block_size);
- pr_debug("MDMA:calc len %d for block size %d\n",
- ctl_hi.ctlx.block_ts, midc->dma->block_size);
- /*calculate CTL_LO*/
- ctl_lo.ctl_lo = 0;
- ctl_lo.ctlx.int_en = 1;
- ctl_lo.ctlx.dst_msize = mids->dma_slave.src_maxburst;
- ctl_lo.ctlx.src_msize = mids->dma_slave.dst_maxburst;
-
- /*
- * Here we need some translation from "enum dma_slave_buswidth"
- * to the format for our dma controller
- * standard intel_mid_dmac's format
- * 1 Byte 0b000
- * 2 Bytes 0b001
- * 4 Bytes 0b010
- */
- ctl_lo.ctlx.dst_tr_width = mids->dma_slave.dst_addr_width / 2;
- ctl_lo.ctlx.src_tr_width = mids->dma_slave.src_addr_width / 2;
-
- if (mids->cfg_mode == LNW_DMA_MEM_TO_MEM) {
- ctl_lo.ctlx.tt_fc = 0;
- ctl_lo.ctlx.sinc = 0;
- ctl_lo.ctlx.dinc = 0;
- } else {
- if (mids->dma_slave.direction == DMA_MEM_TO_DEV) {
- ctl_lo.ctlx.sinc = 0;
- ctl_lo.ctlx.dinc = 2;
- ctl_lo.ctlx.tt_fc = 1;
- } else if (mids->dma_slave.direction == DMA_DEV_TO_MEM) {
- ctl_lo.ctlx.sinc = 2;
- ctl_lo.ctlx.dinc = 0;
- ctl_lo.ctlx.tt_fc = 2;
- }
- }
-
- pr_debug("MDMA:Calc CTL LO %x, CTL HI %x, CFG LO %x, CFG HI %x\n",
- ctl_lo.ctl_lo, ctl_hi.ctl_hi, cfg_lo.cfg_lo, cfg_hi.cfg_hi);
-
- enable_dma_interrupt(midc);
-
- desc = midc_desc_get(midc);
- if (desc == NULL)
- goto err_desc_get;
- desc->sar = src;
- desc->dar = dest ;
- desc->len = len;
- desc->cfg_hi = cfg_hi.cfg_hi;
- desc->cfg_lo = cfg_lo.cfg_lo;
- desc->ctl_lo = ctl_lo.ctl_lo;
- desc->ctl_hi = ctl_hi.ctl_hi;
- desc->width = width;
- desc->dirn = mids->dma_slave.direction;
- desc->lli_phys = 0;
- desc->lli = NULL;
- desc->lli_pool = NULL;
- return &desc->txd;
-
-err_desc_get:
- pr_err("ERR_MDMA: Failed to get desc\n");
- midc_desc_put(midc, desc);
- return NULL;
-}
-/**
- * intel_mid_dma_prep_slave_sg - Prep slave sg txn
- * @chan: chan for DMA transfer
- * @sgl: scatter gather list
- * @sg_len: length of sg txn
- * @direction: DMA transfer dirtn
- * @flags: DMA flags
- * @context: transfer context (ignored)
- *
- * Prepares LLI based periphral transfer
- */
-static struct dma_async_tx_descriptor *intel_mid_dma_prep_slave_sg(
- struct dma_chan *chan, struct scatterlist *sgl,
- unsigned int sg_len, enum dma_transfer_direction direction,
- unsigned long flags, void *context)
-{
- struct intel_mid_dma_chan *midc = NULL;
- struct intel_mid_dma_slave *mids = NULL;
- struct intel_mid_dma_desc *desc = NULL;
- struct dma_async_tx_descriptor *txd = NULL;
- union intel_mid_dma_ctl_lo ctl_lo;
-
- pr_debug("MDMA: Prep for slave SG\n");
-
- if (!sg_len) {
- pr_err("MDMA: Invalid SG length\n");
- return NULL;
- }
- midc = to_intel_mid_dma_chan(chan);
- BUG_ON(!midc);
-
- mids = midc->mid_slave;
- BUG_ON(!mids);
-
- if (!midc->dma->pimr_mask) {
- /* We can still handle sg list with only one item */
- if (sg_len == 1) {
- txd = intel_mid_dma_prep_memcpy(chan,
- mids->dma_slave.dst_addr,
- mids->dma_slave.src_addr,
- sg_dma_len(sgl),
- flags);
- return txd;
- } else {
- pr_warn("MDMA: SG list is not supported by this controller\n");
- return NULL;
- }
- }
-
- pr_debug("MDMA: SG Length = %d, direction = %d, Flags = %#lx\n",
- sg_len, direction, flags);
-
- txd = intel_mid_dma_prep_memcpy(chan, 0, 0, sg_dma_len(sgl), flags);
- if (NULL == txd) {
- pr_err("MDMA: Prep memcpy failed\n");
- return NULL;
- }
-
- desc = to_intel_mid_dma_desc(txd);
- desc->dirn = direction;
- ctl_lo.ctl_lo = desc->ctl_lo;
- ctl_lo.ctlx.llp_dst_en = 1;
- ctl_lo.ctlx.llp_src_en = 1;
- desc->ctl_lo = ctl_lo.ctl_lo;
- desc->lli_length = sg_len;
- desc->current_lli = 0;
- /* DMA coherent memory pool for LLI descriptors*/
- desc->lli_pool = pci_pool_create("intel_mid_dma_lli_pool",
- midc->dma->pdev,
- (sizeof(struct intel_mid_dma_lli)*sg_len),
- 32, 0);
- if (NULL == desc->lli_pool) {
- pr_err("MID_DMA:LLI pool create failed\n");
- return NULL;
- }
-
- desc->lli = pci_pool_alloc(desc->lli_pool, GFP_KERNEL, &desc->lli_phys);
- if (!desc->lli) {
- pr_err("MID_DMA: LLI alloc failed\n");
- pci_pool_destroy(desc->lli_pool);
- return NULL;
- }
-
- midc_lli_fill_sg(midc, desc, sgl, sg_len, flags);
- if (flags & DMA_PREP_INTERRUPT) {
- iowrite32(UNMASK_INTR_REG(midc->ch_id),
- midc->dma_base + MASK_BLOCK);
- pr_debug("MDMA:Enabled Block interrupt\n");
- }
- return &desc->txd;
-}
-
-/**
- * intel_mid_dma_free_chan_resources - Frees dma resources
- * @chan: chan requiring attention
- *
- * Frees the allocated resources on this DMA chan
- */
-static void intel_mid_dma_free_chan_resources(struct dma_chan *chan)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
- struct middma_device *mid = to_middma_device(chan->device);
- struct intel_mid_dma_desc *desc, *_desc;
-
- if (true == midc->busy) {
- /*trying to free ch in use!!!!!*/
- pr_err("ERR_MDMA: trying to free ch in use\n");
- }
- spin_lock_bh(&midc->lock);
- midc->descs_allocated = 0;
- list_for_each_entry_safe(desc, _desc, &midc->active_list, desc_node) {
- list_del(&desc->desc_node);
- pci_pool_free(mid->dma_pool, desc, desc->txd.phys);
- }
- list_for_each_entry_safe(desc, _desc, &midc->free_list, desc_node) {
- list_del(&desc->desc_node);
- pci_pool_free(mid->dma_pool, desc, desc->txd.phys);
- }
- list_for_each_entry_safe(desc, _desc, &midc->queue, desc_node) {
- list_del(&desc->desc_node);
- pci_pool_free(mid->dma_pool, desc, desc->txd.phys);
- }
- spin_unlock_bh(&midc->lock);
- midc->in_use = false;
- midc->busy = false;
- /* Disable CH interrupts */
- iowrite32(MASK_INTR_REG(midc->ch_id), mid->dma_base + MASK_BLOCK);
- iowrite32(MASK_INTR_REG(midc->ch_id), mid->dma_base + MASK_ERR);
- pm_runtime_put(&mid->pdev->dev);
-}
-
-/**
- * intel_mid_dma_alloc_chan_resources - Allocate dma resources
- * @chan: chan requiring attention
- *
- * Allocates DMA resources on this chan
- * Return the descriptors allocated
- */
-static int intel_mid_dma_alloc_chan_resources(struct dma_chan *chan)
-{
- struct intel_mid_dma_chan *midc = to_intel_mid_dma_chan(chan);
- struct middma_device *mid = to_middma_device(chan->device);
- struct intel_mid_dma_desc *desc;
- dma_addr_t phys;
- int i = 0;
-
- pm_runtime_get_sync(&mid->pdev->dev);
-
- if (mid->state == SUSPENDED) {
- if (dma_resume(&mid->pdev->dev)) {
- pr_err("ERR_MDMA: resume failed");
- return -EFAULT;
- }
- }
-
- /* ASSERT: channel is idle */
- if (test_ch_en(mid->dma_base, midc->ch_id)) {
- /*ch is not idle*/
- pr_err("ERR_MDMA: ch not idle\n");
- pm_runtime_put(&mid->pdev->dev);
- return -EIO;
- }
- dma_cookie_init(chan);
-
- spin_lock_bh(&midc->lock);
- while (midc->descs_allocated < DESCS_PER_CHANNEL) {
- spin_unlock_bh(&midc->lock);
- desc = pci_pool_alloc(mid->dma_pool, GFP_KERNEL, &phys);
- if (!desc) {
- pr_err("ERR_MDMA: desc failed\n");
- pm_runtime_put(&mid->pdev->dev);
- return -ENOMEM;
- /*check*/
- }
- dma_async_tx_descriptor_init(&desc->txd, chan);
- desc->txd.tx_submit = intel_mid_dma_tx_submit;
- desc->txd.flags = DMA_CTRL_ACK;
- desc->txd.phys = phys;
- spin_lock_bh(&midc->lock);
- i = ++midc->descs_allocated;
- list_add_tail(&desc->desc_node, &midc->free_list);
- }
- spin_unlock_bh(&midc->lock);
- midc->in_use = true;
- midc->busy = false;
- pr_debug("MID_DMA: Desc alloc done ret: %d desc\n", i);
- return i;
-}
-
-/**
- * midc_handle_error - Handle DMA txn error
- * @mid: controller where error occurred
- * @midc: chan where error occurred
- *
- * Scan the descriptor for error
- */
-static void midc_handle_error(struct middma_device *mid,
- struct intel_mid_dma_chan *midc)
-{
- midc_scan_descriptors(mid, midc);
-}
-
-/**
- * dma_tasklet - DMA interrupt tasklet
- * @data: tasklet arg (the controller structure)
- *
- * Scan the controller for interrupts for completion/error
- * Clear the interrupt and call for handling completion/error
- */
-static void dma_tasklet(unsigned long data)
-{
- struct middma_device *mid = NULL;
- struct intel_mid_dma_chan *midc = NULL;
- u32 status, raw_tfr, raw_block;
- int i;
-
- mid = (struct middma_device *)data;
- if (mid == NULL) {
- pr_err("ERR_MDMA: tasklet Null param\n");
- return;
- }
- pr_debug("MDMA: in tasklet for device %x\n", mid->pci_id);
- raw_tfr = ioread32(mid->dma_base + RAW_TFR);
- raw_block = ioread32(mid->dma_base + RAW_BLOCK);
- status = raw_tfr | raw_block;
- status &= mid->intr_mask;
- while (status) {
- /*txn interrupt*/
- i = get_ch_index(&status, mid->chan_base);
- if (i < 0) {
- pr_err("ERR_MDMA:Invalid ch index %x\n", i);
- return;
- }
- midc = &mid->ch[i];
- if (midc == NULL) {
- pr_err("ERR_MDMA:Null param midc\n");
- return;
- }
- pr_debug("MDMA:Tx complete interrupt %x, Ch No %d Index %d\n",
- status, midc->ch_id, i);
- midc->raw_tfr = raw_tfr;
- midc->raw_block = raw_block;
- spin_lock_bh(&midc->lock);
- /*clearing this interrupts first*/
- iowrite32((1 << midc->ch_id), mid->dma_base + CLEAR_TFR);
- if (raw_block) {
- iowrite32((1 << midc->ch_id),
- mid->dma_base + CLEAR_BLOCK);
- }
- midc_scan_descriptors(mid, midc);
- pr_debug("MDMA:Scan of desc... complete, unmasking\n");
- iowrite32(UNMASK_INTR_REG(midc->ch_id),
- mid->dma_base + MASK_TFR);
- if (raw_block) {
- iowrite32(UNMASK_INTR_REG(midc->ch_id),
- mid->dma_base + MASK_BLOCK);
- }
- spin_unlock_bh(&midc->lock);
- }
-
- status = ioread32(mid->dma_base + RAW_ERR);
- status &= mid->intr_mask;
- while (status) {
- /*err interrupt*/
- i = get_ch_index(&status, mid->chan_base);
- if (i < 0) {
- pr_err("ERR_MDMA:Invalid ch index %x\n", i);
- return;
- }
- midc = &mid->ch[i];
- if (midc == NULL) {
- pr_err("ERR_MDMA:Null param midc\n");
- return;
- }
- pr_debug("MDMA:Tx complete interrupt %x, Ch No %d Index %d\n",
- status, midc->ch_id, i);
-
- iowrite32((1 << midc->ch_id), mid->dma_base + CLEAR_ERR);
- spin_lock_bh(&midc->lock);
- midc_handle_error(mid, midc);
- iowrite32(UNMASK_INTR_REG(midc->ch_id),
- mid->dma_base + MASK_ERR);
- spin_unlock_bh(&midc->lock);
- }
- pr_debug("MDMA:Exiting takslet...\n");
- return;
-}
-
-static void dma_tasklet1(unsigned long data)
-{
- pr_debug("MDMA:in takslet1...\n");
- return dma_tasklet(data);
-}
-
-static void dma_tasklet2(unsigned long data)
-{
- pr_debug("MDMA:in takslet2...\n");
- return dma_tasklet(data);
-}
-
-/**
- * intel_mid_dma_interrupt - DMA ISR
- * @irq: IRQ where interrupt occurred
- * @data: ISR cllback data (the controller structure)
- *
- * See if this is our interrupt if so then schedule the tasklet
- * otherwise ignore
- */
-static irqreturn_t intel_mid_dma_interrupt(int irq, void *data)
-{
- struct middma_device *mid = data;
- u32 tfr_status, err_status;
- int call_tasklet = 0;
-
- tfr_status = ioread32(mid->dma_base + RAW_TFR);
- err_status = ioread32(mid->dma_base + RAW_ERR);
- if (!tfr_status && !err_status)
- return IRQ_NONE;
-
- /*DMA Interrupt*/
- pr_debug("MDMA:Got an interrupt on irq %d\n", irq);
- pr_debug("MDMA: Status %x, Mask %x\n", tfr_status, mid->intr_mask);
- tfr_status &= mid->intr_mask;
- if (tfr_status) {
- /*need to disable intr*/
- iowrite32((tfr_status << INT_MASK_WE), mid->dma_base + MASK_TFR);
- iowrite32((tfr_status << INT_MASK_WE), mid->dma_base + MASK_BLOCK);
- pr_debug("MDMA: Calling tasklet %x\n", tfr_status);
- call_tasklet = 1;
- }
- err_status &= mid->intr_mask;
- if (err_status) {
- iowrite32((err_status << INT_MASK_WE),
- mid->dma_base + MASK_ERR);
- call_tasklet = 1;
- }
- if (call_tasklet)
- tasklet_schedule(&mid->tasklet);
-
- return IRQ_HANDLED;
-}
-
-static irqreturn_t intel_mid_dma_interrupt1(int irq, void *data)
-{
- return intel_mid_dma_interrupt(irq, data);
-}
-
-static irqreturn_t intel_mid_dma_interrupt2(int irq, void *data)
-{
- return intel_mid_dma_interrupt(irq, data);
-}
-
-/**
- * mid_setup_dma - Setup the DMA controller
- * @pdev: Controller PCI device structure
- *
- * Initialize the DMA controller, channels, registers with DMA engine,
- * ISR. Initialize DMA controller channels.
- */
-static int mid_setup_dma(struct pci_dev *pdev)
-{
- struct middma_device *dma = pci_get_drvdata(pdev);
- int err, i;
-
- /* DMA coherent memory pool for DMA descriptor allocations */
- dma->dma_pool = pci_pool_create("intel_mid_dma_desc_pool", pdev,
- sizeof(struct intel_mid_dma_desc),
- 32, 0);
- if (NULL == dma->dma_pool) {
- pr_err("ERR_MDMA:pci_pool_create failed\n");
- err = -ENOMEM;
- goto err_dma_pool;
- }
-
- INIT_LIST_HEAD(&dma->common.channels);
- dma->pci_id = pdev->device;
- if (dma->pimr_mask) {
- dma->mask_reg = ioremap(LNW_PERIPHRAL_MASK_BASE,
- LNW_PERIPHRAL_MASK_SIZE);
- if (dma->mask_reg == NULL) {
- pr_err("ERR_MDMA:Can't map periphral intr space !!\n");
- err = -ENOMEM;
- goto err_ioremap;
- }
- } else
- dma->mask_reg = NULL;
-
- pr_debug("MDMA:Adding %d channel for this controller\n", dma->max_chan);
- /*init CH structures*/
- dma->intr_mask = 0;
- dma->state = RUNNING;
- for (i = 0; i < dma->max_chan; i++) {
- struct intel_mid_dma_chan *midch = &dma->ch[i];
-
- midch->chan.device = &dma->common;
- dma_cookie_init(&midch->chan);
- midch->ch_id = dma->chan_base + i;
- pr_debug("MDMA:Init CH %d, ID %d\n", i, midch->ch_id);
-
- midch->dma_base = dma->dma_base;
- midch->ch_regs = dma->dma_base + DMA_CH_SIZE * midch->ch_id;
- midch->dma = dma;
- dma->intr_mask |= 1 << (dma->chan_base + i);
- spin_lock_init(&midch->lock);
-
- INIT_LIST_HEAD(&midch->active_list);
- INIT_LIST_HEAD(&midch->queue);
- INIT_LIST_HEAD(&midch->free_list);
- /*mask interrupts*/
- iowrite32(MASK_INTR_REG(midch->ch_id),
- dma->dma_base + MASK_BLOCK);
- iowrite32(MASK_INTR_REG(midch->ch_id),
- dma->dma_base + MASK_SRC_TRAN);
- iowrite32(MASK_INTR_REG(midch->ch_id),
- dma->dma_base + MASK_DST_TRAN);
- iowrite32(MASK_INTR_REG(midch->ch_id),
- dma->dma_base + MASK_ERR);
- iowrite32(MASK_INTR_REG(midch->ch_id),
- dma->dma_base + MASK_TFR);
-
- disable_dma_interrupt(midch);
- list_add_tail(&midch->chan.device_node, &dma->common.channels);
- }
- pr_debug("MDMA: Calc Mask as %x for this controller\n", dma->intr_mask);
-
- /*init dma structure*/
- dma_cap_zero(dma->common.cap_mask);
- dma_cap_set(DMA_MEMCPY, dma->common.cap_mask);
- dma_cap_set(DMA_SLAVE, dma->common.cap_mask);
- dma_cap_set(DMA_PRIVATE, dma->common.cap_mask);
- dma->common.dev = &pdev->dev;
-
- dma->common.device_alloc_chan_resources =
- intel_mid_dma_alloc_chan_resources;
- dma->common.device_free_chan_resources =
- intel_mid_dma_free_chan_resources;
-
- dma->common.device_tx_status = intel_mid_dma_tx_status;
- dma->common.device_prep_dma_memcpy = intel_mid_dma_prep_memcpy;
- dma->common.device_issue_pending = intel_mid_dma_issue_pending;
- dma->common.device_prep_slave_sg = intel_mid_dma_prep_slave_sg;
- dma->common.device_config = intel_mid_dma_config;
- dma->common.device_terminate_all = intel_mid_dma_terminate_all;
-
- /*enable dma cntrl*/
- iowrite32(REG_BIT0, dma->dma_base + DMA_CFG);
-
- /*register irq */
- if (dma->pimr_mask) {
- pr_debug("MDMA:Requesting irq shared for DMAC1\n");
- err = request_irq(pdev->irq, intel_mid_dma_interrupt1,
- IRQF_SHARED, "INTEL_MID_DMAC1", dma);
- if (0 != err)
- goto err_irq;
- } else {
- dma->intr_mask = 0x03;
- pr_debug("MDMA:Requesting irq for DMAC2\n");
- err = request_irq(pdev->irq, intel_mid_dma_interrupt2,
- IRQF_SHARED, "INTEL_MID_DMAC2", dma);
- if (0 != err)
- goto err_irq;
- }
- /*register device w/ engine*/
- err = dma_async_device_register(&dma->common);
- if (0 != err) {
- pr_err("ERR_MDMA:device_register failed: %d\n", err);
- goto err_engine;
- }
- if (dma->pimr_mask) {
- pr_debug("setting up tasklet1 for DMAC1\n");
- tasklet_init(&dma->tasklet, dma_tasklet1, (unsigned long)dma);
- } else {
- pr_debug("setting up tasklet2 for DMAC2\n");
- tasklet_init(&dma->tasklet, dma_tasklet2, (unsigned long)dma);
- }
- return 0;
-
-err_engine:
- free_irq(pdev->irq, dma);
-err_irq:
- if (dma->mask_reg)
- iounmap(dma->mask_reg);
-err_ioremap:
- pci_pool_destroy(dma->dma_pool);
-err_dma_pool:
- pr_err("ERR_MDMA:setup_dma failed: %d\n", err);
- return err;
-
-}
-
-/**
- * middma_shutdown - Shutdown the DMA controller
- * @pdev: Controller PCI device structure
- *
- * Called by remove
- * Unregister DMa controller, clear all structures and free interrupt
- */
-static void middma_shutdown(struct pci_dev *pdev)
-{
- struct middma_device *device = pci_get_drvdata(pdev);
-
- dma_async_device_unregister(&device->common);
- pci_pool_destroy(device->dma_pool);
- if (device->mask_reg)
- iounmap(device->mask_reg);
- if (device->dma_base)
- iounmap(device->dma_base);
- free_irq(pdev->irq, device);
- return;
-}
-
-/**
- * intel_mid_dma_probe - PCI Probe
- * @pdev: Controller PCI device structure
- * @id: pci device id structure
- *
- * Initialize the PCI device, map BARs, query driver data.
- * Call setup_dma to complete contoller and chan initilzation
- */
-static int intel_mid_dma_probe(struct pci_dev *pdev,
- const struct pci_device_id *id)
-{
- struct middma_device *device;
- u32 base_addr, bar_size;
- struct intel_mid_dma_probe_info *info;
- int err;
-
- pr_debug("MDMA: probe for %x\n", pdev->device);
- info = (void *)id->driver_data;
- pr_debug("MDMA: CH %d, base %d, block len %d, Periphral mask %x\n",
- info->max_chan, info->ch_base,
- info->block_size, info->pimr_mask);
-
- err = pci_enable_device(pdev);
- if (err)
- goto err_enable_device;
-
- err = pci_request_regions(pdev, "intel_mid_dmac");
- if (err)
- goto err_request_regions;
-
- err = pci_set_dma_mask(pdev, DMA_BIT_MASK(32));
- if (err)
- goto err_set_dma_mask;
-
- err = pci_set_consistent_dma_mask(pdev, DMA_BIT_MASK(32));
- if (err)
- goto err_set_dma_mask;
-
- device = kzalloc(sizeof(*device), GFP_KERNEL);
- if (!device) {
- pr_err("ERR_MDMA:kzalloc failed probe\n");
- err = -ENOMEM;
- goto err_kzalloc;
- }
- device->pdev = pci_dev_get(pdev);
-
- base_addr = pci_resource_start(pdev, 0);
- bar_size = pci_resource_len(pdev, 0);
- device->dma_base = ioremap_nocache(base_addr, DMA_REG_SIZE);
- if (!device->dma_base) {
- pr_err("ERR_MDMA:ioremap failed\n");
- err = -ENOMEM;
- goto err_ioremap;
- }
- pci_set_drvdata(pdev, device);
- pci_set_master(pdev);
- device->max_chan = info->max_chan;
- device->chan_base = info->ch_base;
- device->block_size = info->block_size;
- device->pimr_mask = info->pimr_mask;
-
- err = mid_setup_dma(pdev);
- if (err)
- goto err_dma;
-
- pm_runtime_put_noidle(&pdev->dev);
- pm_runtime_allow(&pdev->dev);
- return 0;
-
-err_dma:
- iounmap(device->dma_base);
-err_ioremap:
- pci_dev_put(pdev);
- kfree(device);
-err_kzalloc:
-err_set_dma_mask:
- pci_release_regions(pdev);
- pci_disable_device(pdev);
-err_request_regions:
-err_enable_device:
- pr_err("ERR_MDMA:Probe failed %d\n", err);
- return err;
-}
-
-/**
- * intel_mid_dma_remove - PCI remove
- * @pdev: Controller PCI device structure
- *
- * Free up all resources and data
- * Call shutdown_dma to complete contoller and chan cleanup
- */
-static void intel_mid_dma_remove(struct pci_dev *pdev)
-{
- struct middma_device *device = pci_get_drvdata(pdev);
-
- pm_runtime_get_noresume(&pdev->dev);
- pm_runtime_forbid(&pdev->dev);
- middma_shutdown(pdev);
- pci_dev_put(pdev);
- kfree(device);
- pci_release_regions(pdev);
- pci_disable_device(pdev);
-}
-
-/* Power Management */
-/*
-* dma_suspend - PCI suspend function
-*
-* @pci: PCI device structure
-* @state: PM message
-*
-* This function is called by OS when a power event occurs
-*/
-static int dma_suspend(struct device *dev)
-{
- struct pci_dev *pci = to_pci_dev(dev);
- int i;
- struct middma_device *device = pci_get_drvdata(pci);
- pr_debug("MDMA: dma_suspend called\n");
-
- for (i = 0; i < device->max_chan; i++) {
- if (device->ch[i].in_use)
- return -EAGAIN;
- }
- dmac1_mask_periphral_intr(device);
- device->state = SUSPENDED;
- pci_save_state(pci);
- pci_disable_device(pci);
- pci_set_power_state(pci, PCI_D3hot);
- return 0;
-}
-
-/**
-* dma_resume - PCI resume function
-*
-* @pci: PCI device structure
-*
-* This function is called by OS when a power event occurs
-*/
-int dma_resume(struct device *dev)
-{
- struct pci_dev *pci = to_pci_dev(dev);
- int ret;
- struct middma_device *device = pci_get_drvdata(pci);
-
- pr_debug("MDMA: dma_resume called\n");
- pci_set_power_state(pci, PCI_D0);
- pci_restore_state(pci);
- ret = pci_enable_device(pci);
- if (ret) {
- pr_err("MDMA: device can't be enabled for %x\n", pci->device);
- return ret;
- }
- device->state = RUNNING;
- iowrite32(REG_BIT0, device->dma_base + DMA_CFG);
- return 0;
-}
-
-static int dma_runtime_suspend(struct device *dev)
-{
- struct pci_dev *pci_dev = to_pci_dev(dev);
- struct middma_device *device = pci_get_drvdata(pci_dev);
-
- device->state = SUSPENDED;
- return 0;
-}
-
-static int dma_runtime_resume(struct device *dev)
-{
- struct pci_dev *pci_dev = to_pci_dev(dev);
- struct middma_device *device = pci_get_drvdata(pci_dev);
-
- device->state = RUNNING;
- iowrite32(REG_BIT0, device->dma_base + DMA_CFG);
- return 0;
-}
-
-static int dma_runtime_idle(struct device *dev)
-{
- struct pci_dev *pdev = to_pci_dev(dev);
- struct middma_device *device = pci_get_drvdata(pdev);
- int i;
-
- for (i = 0; i < device->max_chan; i++) {
- if (device->ch[i].in_use)
- return -EAGAIN;
- }
-
- return 0;
-}
-
-/******************************************************************************
-* PCI stuff
-*/
-static struct pci_device_id intel_mid_dma_ids[] = {
- { PCI_VDEVICE(INTEL, INTEL_MID_DMAC1_ID), INFO(2, 6, 4095, 0x200020)},
- { PCI_VDEVICE(INTEL, INTEL_MID_DMAC2_ID), INFO(2, 0, 2047, 0)},
- { PCI_VDEVICE(INTEL, INTEL_MID_GP_DMAC2_ID), INFO(2, 0, 2047, 0)},
- { PCI_VDEVICE(INTEL, INTEL_MFLD_DMAC1_ID), INFO(4, 0, 4095, 0x400040)},
- { 0, }
-};
-MODULE_DEVICE_TABLE(pci, intel_mid_dma_ids);
-
-static const struct dev_pm_ops intel_mid_dma_pm = {
- .runtime_suspend = dma_runtime_suspend,
- .runtime_resume = dma_runtime_resume,
- .runtime_idle = dma_runtime_idle,
- .suspend = dma_suspend,
- .resume = dma_resume,
-};
-
-static struct pci_driver intel_mid_dma_pci_driver = {
- .name = "Intel MID DMA",
- .id_table = intel_mid_dma_ids,
- .probe = intel_mid_dma_probe,
- .remove = intel_mid_dma_remove,
-#ifdef CONFIG_PM
- .driver = {
- .pm = &intel_mid_dma_pm,
- },
-#endif
-};
-
-static int __init intel_mid_dma_init(void)
-{
- pr_debug("INFO_MDMA: LNW DMA Driver Version %s\n",
- INTEL_MID_DMA_DRIVER_VERSION);
- return pci_register_driver(&intel_mid_dma_pci_driver);
-}
-fs_initcall(intel_mid_dma_init);
-
-static void __exit intel_mid_dma_exit(void)
-{
- pci_unregister_driver(&intel_mid_dma_pci_driver);
-}
-module_exit(intel_mid_dma_exit);
-
-MODULE_AUTHOR("Vinod Koul <vinod.koul@intel.com>");
-MODULE_DESCRIPTION("Intel (R) MID DMAC Driver");
-MODULE_LICENSE("GPL v2");
-MODULE_VERSION(INTEL_MID_DMA_DRIVER_VERSION);
+++ /dev/null
-/*
- * intel_mid_dma_regs.h - Intel MID DMA Drivers
- *
- * Copyright (C) 2008-10 Intel Corp
- * Author: Vinod Koul <vinod.koul@intel.com>
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
- *
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- *
- */
-#ifndef __INTEL_MID_DMAC_REGS_H__
-#define __INTEL_MID_DMAC_REGS_H__
-
-#include <linux/dmaengine.h>
-#include <linux/dmapool.h>
-#include <linux/pci_ids.h>
-
-#define INTEL_MID_DMA_DRIVER_VERSION "1.1.0"
-
-#define REG_BIT0 0x00000001
-#define REG_BIT8 0x00000100
-#define INT_MASK_WE 0x8
-#define CLEAR_DONE 0xFFFFEFFF
-#define UNMASK_INTR_REG(chan_num) \
- ((REG_BIT0 << chan_num) | (REG_BIT8 << chan_num))
-#define MASK_INTR_REG(chan_num) (REG_BIT8 << chan_num)
-
-#define ENABLE_CHANNEL(chan_num) \
- ((REG_BIT0 << chan_num) | (REG_BIT8 << chan_num))
-
-#define DISABLE_CHANNEL(chan_num) \
- (REG_BIT8 << chan_num)
-
-#define DESCS_PER_CHANNEL 16
-/*DMA Registers*/
-/*registers associated with channel programming*/
-#define DMA_REG_SIZE 0x400
-#define DMA_CH_SIZE 0x58
-
-/*CH X REG = (DMA_CH_SIZE)*CH_NO + REG*/
-#define SAR 0x00 /* Source Address Register*/
-#define DAR 0x08 /* Destination Address Register*/
-#define LLP 0x10 /* Linked List Pointer Register*/
-#define CTL_LOW 0x18 /* Control Register*/
-#define CTL_HIGH 0x1C /* Control Register*/
-#define CFG_LOW 0x40 /* Configuration Register Low*/
-#define CFG_HIGH 0x44 /* Configuration Register high*/
-
-#define STATUS_TFR 0x2E8
-#define STATUS_BLOCK 0x2F0
-#define STATUS_ERR 0x308
-
-#define RAW_TFR 0x2C0
-#define RAW_BLOCK 0x2C8
-#define RAW_ERR 0x2E0
-
-#define MASK_TFR 0x310
-#define MASK_BLOCK 0x318
-#define MASK_SRC_TRAN 0x320
-#define MASK_DST_TRAN 0x328
-#define MASK_ERR 0x330
-
-#define CLEAR_TFR 0x338
-#define CLEAR_BLOCK 0x340
-#define CLEAR_SRC_TRAN 0x348
-#define CLEAR_DST_TRAN 0x350
-#define CLEAR_ERR 0x358
-
-#define INTR_STATUS 0x360
-#define DMA_CFG 0x398
-#define DMA_CHAN_EN 0x3A0
-
-/*DMA channel control registers*/
-union intel_mid_dma_ctl_lo {
- struct {
- u32 int_en:1; /*enable or disable interrupts*/
- /*should be 0*/
- u32 dst_tr_width:3; /*destination transfer width*/
- /*usually 32 bits = 010*/
- u32 src_tr_width:3; /*source transfer width*/
- /*usually 32 bits = 010*/
- u32 dinc:2; /*destination address inc/dec*/
- /*For mem:INC=00, Periphral NoINC=11*/
- u32 sinc:2; /*source address inc or dec, as above*/
- u32 dst_msize:3; /*destination burst transaction length*/
- /*always = 16 ie 011*/
- u32 src_msize:3; /*source burst transaction length*/
- /*always = 16 ie 011*/
- u32 reser1:3;
- u32 tt_fc:3; /*transfer type and flow controller*/
- /*M-M = 000
- P-M = 010
- M-P = 001*/
- u32 dms:2; /*destination master select = 0*/
- u32 sms:2; /*source master select = 0*/
- u32 llp_dst_en:1; /*enable/disable destination LLP = 0*/
- u32 llp_src_en:1; /*enable/disable source LLP = 0*/
- u32 reser2:3;
- } ctlx;
- u32 ctl_lo;
-};
-
-union intel_mid_dma_ctl_hi {
- struct {
- u32 block_ts:12; /*block transfer size*/
- u32 done:1; /*Done - updated by DMAC*/
- u32 reser:19; /*configured by DMAC*/
- } ctlx;
- u32 ctl_hi;
-
-};
-
-/*DMA channel configuration registers*/
-union intel_mid_dma_cfg_lo {
- struct {
- u32 reser1:5;
- u32 ch_prior:3; /*channel priority = 0*/
- u32 ch_susp:1; /*channel suspend = 0*/
- u32 fifo_empty:1; /*FIFO empty or not R bit = 0*/
- u32 hs_sel_dst:1; /*select HW/SW destn handshaking*/
- /*HW = 0, SW = 1*/
- u32 hs_sel_src:1; /*select HW/SW src handshaking*/
- u32 reser2:6;
- u32 dst_hs_pol:1; /*dest HS interface polarity*/
- u32 src_hs_pol:1; /*src HS interface polarity*/
- u32 max_abrst:10; /*max AMBA burst len = 0 (no sw limit*/
- u32 reload_src:1; /*auto reload src addr =1 if src is P*/
- u32 reload_dst:1; /*AR destn addr =1 if dstn is P*/
- } cfgx;
- u32 cfg_lo;
-};
-
-union intel_mid_dma_cfg_hi {
- struct {
- u32 fcmode:1; /*flow control mode = 1*/
- u32 fifo_mode:1; /*FIFO mode select = 1*/
- u32 protctl:3; /*protection control = 0*/
- u32 rsvd:2;
- u32 src_per:4; /*src hw HS interface*/
- u32 dst_per:4; /*dstn hw HS interface*/
- u32 reser2:17;
- } cfgx;
- u32 cfg_hi;
-};
-
-
-/**
- * struct intel_mid_dma_chan - internal mid representation of a DMA channel
- * @chan: dma_chan strcture represetation for mid chan
- * @ch_regs: MMIO register space pointer to channel register
- * @dma_base: MMIO register space DMA engine base pointer
- * @ch_id: DMA channel id
- * @lock: channel spinlock
- * @active_list: current active descriptors
- * @queue: current queued up descriptors
- * @free_list: current free descriptors
- * @slave: dma slave structure
- * @descs_allocated: total number of descriptors allocated
- * @dma: dma device structure pointer
- * @busy: bool representing if ch is busy (active txn) or not
- * @in_use: bool representing if ch is in use or not
- * @raw_tfr: raw trf interrupt received
- * @raw_block: raw block interrupt received
- */
-struct intel_mid_dma_chan {
- struct dma_chan chan;
- void __iomem *ch_regs;
- void __iomem *dma_base;
- int ch_id;
- spinlock_t lock;
- struct list_head active_list;
- struct list_head queue;
- struct list_head free_list;
- unsigned int descs_allocated;
- struct middma_device *dma;
- bool busy;
- bool in_use;
- u32 raw_tfr;
- u32 raw_block;
- struct intel_mid_dma_slave *mid_slave;
-};
-
-static inline struct intel_mid_dma_chan *to_intel_mid_dma_chan(
- struct dma_chan *chan)
-{
- return container_of(chan, struct intel_mid_dma_chan, chan);
-}
-
-enum intel_mid_dma_state {
- RUNNING = 0,
- SUSPENDED,
-};
-/**
- * struct middma_device - internal representation of a DMA device
- * @pdev: PCI device
- * @dma_base: MMIO register space pointer of DMA
- * @dma_pool: for allocating DMA descriptors
- * @common: embedded struct dma_device
- * @tasklet: dma tasklet for processing interrupts
- * @ch: per channel data
- * @pci_id: DMA device PCI ID
- * @intr_mask: Interrupt mask to be used
- * @mask_reg: MMIO register for periphral mask
- * @chan_base: Base ch index (read from driver data)
- * @max_chan: max number of chs supported (from drv_data)
- * @block_size: Block size of DMA transfer supported (from drv_data)
- * @pimr_mask: MMIO register addr for periphral interrupt (from drv_data)
- * @state: dma PM device state
- */
-struct middma_device {
- struct pci_dev *pdev;
- void __iomem *dma_base;
- struct pci_pool *dma_pool;
- struct dma_device common;
- struct tasklet_struct tasklet;
- struct intel_mid_dma_chan ch[MAX_CHAN];
- unsigned int pci_id;
- unsigned int intr_mask;
- void __iomem *mask_reg;
- int chan_base;
- int max_chan;
- int block_size;
- unsigned int pimr_mask;
- enum intel_mid_dma_state state;
-};
-
-static inline struct middma_device *to_middma_device(struct dma_device *common)
-{
- return container_of(common, struct middma_device, common);
-}
-
-struct intel_mid_dma_desc {
- void __iomem *block; /*ch ptr*/
- struct list_head desc_node;
- struct dma_async_tx_descriptor txd;
- size_t len;
- dma_addr_t sar;
- dma_addr_t dar;
- u32 cfg_hi;
- u32 cfg_lo;
- u32 ctl_lo;
- u32 ctl_hi;
- struct pci_pool *lli_pool;
- struct intel_mid_dma_lli *lli;
- dma_addr_t lli_phys;
- unsigned int lli_length;
- unsigned int current_lli;
- dma_addr_t next;
- enum dma_transfer_direction dirn;
- enum dma_status status;
- enum dma_slave_buswidth width; /*width of DMA txn*/
- enum intel_mid_dma_mode cfg_mode; /*mode configuration*/
-
-};
-
-struct intel_mid_dma_lli {
- dma_addr_t sar;
- dma_addr_t dar;
- dma_addr_t llp;
- u32 ctl_lo;
- u32 ctl_hi;
-} __attribute__ ((packed));
-
-static inline int test_ch_en(void __iomem *dma, u32 ch_no)
-{
- u32 en_reg = ioread32(dma + DMA_CHAN_EN);
- return (en_reg >> ch_no) & 0x1;
-}
-
-static inline struct intel_mid_dma_desc *to_intel_mid_dma_desc
- (struct dma_async_tx_descriptor *txd)
-{
- return container_of(txd, struct intel_mid_dma_desc, txd);
-}
-
-static inline struct intel_mid_dma_slave *to_intel_mid_dma_slave
- (struct dma_slave_config *slave)
-{
- return container_of(slave, struct intel_mid_dma_slave, dma_slave);
-}
-
-
-int dma_resume(struct device *dev);
-
-#endif /*__INTEL_MID_DMAC_REGS_H__*/
switch (pdev->device) {
case PCI_DEVICE_ID_INTEL_IOAT_BWD2:
case PCI_DEVICE_ID_INTEL_IOAT_BWD3:
+ case PCI_DEVICE_ID_INTEL_IOAT_BDXDE0:
+ case PCI_DEVICE_ID_INTEL_IOAT_BDXDE1:
+ case PCI_DEVICE_ID_INTEL_IOAT_BDXDE2:
+ case PCI_DEVICE_ID_INTEL_IOAT_BDXDE3:
return true;
default:
return false;
while (dint) {
i = __ffs(dint);
+ /* only handle interrupts belonging to pdma driver*/
+ if (i >= pdev->dma_channels)
+ break;
dint &= (dint - 1);
phy = &pdev->phy[i];
ret = mmp_pdma_chan_handler(irq, phy);
struct resource *iores;
int i, ret, irq = 0;
int dma_channels = 0, irq_num = 0;
+ const enum dma_slave_buswidth widths =
+ DMA_SLAVE_BUSWIDTH_1_BYTE | DMA_SLAVE_BUSWIDTH_2_BYTES |
+ DMA_SLAVE_BUSWIDTH_4_BYTES;
pdev = devm_kzalloc(&op->dev, sizeof(*pdev), GFP_KERNEL);
if (!pdev)
pdev->device.device_config = mmp_pdma_config;
pdev->device.device_terminate_all = mmp_pdma_terminate_all;
pdev->device.copy_align = PDMA_ALIGNMENT;
+ pdev->device.src_addr_widths = widths;
+ pdev->device.dst_addr_widths = widths;
+ pdev->device.directions = BIT(DMA_MEM_TO_DEV) | BIT(DMA_DEV_TO_MEM);
+ pdev->device.residue_granularity = DMA_RESIDUE_GRANULARITY_DESCRIPTOR;
if (pdev->dev->coherent_dma_mask)
dma_set_mask(pdev->dev, pdev->dev->coherent_dma_mask);
struct tasklet_struct tasklet;
struct mmp_tdma_desc *desc_arr;
- phys_addr_t desc_arr_phys;
+ dma_addr_t desc_arr_phys;
int desc_num;
enum dma_transfer_direction dir;
dma_addr_t dev_addr;
static int mmp_tdma_disable_chan(struct dma_chan *chan)
{
struct mmp_tdma_chan *tdmac = to_mmp_tdma_chan(chan);
+ u32 tdcr;
- writel(readl(tdmac->reg_base + TDCR) & ~TDCR_CHANEN,
- tdmac->reg_base + TDCR);
+ tdcr = readl(tdmac->reg_base + TDCR);
+ tdcr |= TDCR_ABR;
+ tdcr &= ~TDCR_CHANEN;
+ writel(tdcr, tdmac->reg_base + TDCR);
tdmac->status = DMA_COMPLETE;
return -EAGAIN;
}
+static size_t mmp_tdma_get_pos(struct mmp_tdma_chan *tdmac)
+{
+ size_t reg;
+
+ if (tdmac->idx == 0) {
+ reg = __raw_readl(tdmac->reg_base + TDSAR);
+ reg -= tdmac->desc_arr[0].src_addr;
+ } else if (tdmac->idx == 1) {
+ reg = __raw_readl(tdmac->reg_base + TDDAR);
+ reg -= tdmac->desc_arr[0].dst_addr;
+ } else
+ return -EINVAL;
+
+ return reg;
+}
+
static irqreturn_t mmp_tdma_chan_handler(int irq, void *dev_id)
{
struct mmp_tdma_chan *tdmac = dev_id;
if (mmp_tdma_clear_chan_irq(tdmac) == 0) {
- tdmac->pos = (tdmac->pos + tdmac->period_len) % tdmac->buf_len;
tasklet_schedule(&tdmac->tasklet);
return IRQ_HANDLED;
} else
int size = tdmac->desc_num * sizeof(struct mmp_tdma_desc);
gpool = tdmac->pool;
- if (tdmac->desc_arr)
+ if (gpool && tdmac->desc_arr)
gen_pool_free(gpool, (unsigned long)tdmac->desc_arr,
size);
tdmac->desc_arr = NULL;
{
struct mmp_tdma_chan *tdmac = to_mmp_tdma_chan(chan);
+ tdmac->pos = mmp_tdma_get_pos(tdmac);
dma_set_tx_state(txstate, chan->completed_cookie, chan->cookie,
tdmac->buf_len - tdmac->pos);
int i, ret;
int irq = 0, irq_num = 0;
int chan_num = TDMA_CHANNEL_NUM;
- struct gen_pool *pool;
+ struct gen_pool *pool = NULL;
of_id = of_match_device(mmp_tdma_dt_ids, &pdev->dev);
if (of_id)
spin_lock_irqsave(&ch->vc.lock, flags);
- if (ch->desc)
+ if (ch->desc) {
+ moxart_dma_desc_free(&ch->desc->vd);
ch->desc = NULL;
+ }
ctrl = readl(ch->base + REG_OFF_CTRL);
ctrl &= ~(APB_DMA_ENABLE | APB_DMA_FIN_INT_EN | APB_DMA_ERR_INT_EN);
* c->desc is NULL and exit.)
*/
if (c->desc) {
+ omap_dma_desc_free(&c->desc->vd);
c->desc = NULL;
/* Avoid stopping the dma twice */
if (!c->paused)
[BAM_P_IRQ_STTS] = { 0x1010, 0x1000, 0x00, 0x00 },
[BAM_P_IRQ_CLR] = { 0x1014, 0x1000, 0x00, 0x00 },
[BAM_P_IRQ_EN] = { 0x1018, 0x1000, 0x00, 0x00 },
- [BAM_P_EVNT_DEST_ADDR] = { 0x102C, 0x00, 0x1000, 0x00 },
- [BAM_P_EVNT_REG] = { 0x1018, 0x00, 0x1000, 0x00 },
- [BAM_P_SW_OFSTS] = { 0x1000, 0x00, 0x1000, 0x00 },
+ [BAM_P_EVNT_DEST_ADDR] = { 0x182C, 0x00, 0x1000, 0x00 },
+ [BAM_P_EVNT_REG] = { 0x1818, 0x00, 0x1000, 0x00 },
+ [BAM_P_SW_OFSTS] = { 0x1800, 0x00, 0x1000, 0x00 },
[BAM_P_DATA_FIFO_ADDR] = { 0x1824, 0x00, 0x1000, 0x00 },
[BAM_P_DESC_FIFO_ADDR] = { 0x181C, 0x00, 0x1000, 0x00 },
[BAM_P_EVNT_GEN_TRSHLD] = { 0x1828, 0x00, 0x1000, 0x00 },
dma_cap_set(DMA_SLAVE, bdev->common.cap_mask);
/* initialize dmaengine apis */
+ bdev->common.directions = BIT(DMA_DEV_TO_MEM) | BIT(DMA_MEM_TO_DEV);
+ bdev->common.residue_granularity = DMA_RESIDUE_GRANULARITY_SEGMENT;
+ bdev->common.src_addr_widths = DMA_SLAVE_BUSWIDTH_4_BYTES;
+ bdev->common.dst_addr_widths = DMA_SLAVE_BUSWIDTH_4_BYTES;
bdev->common.device_alloc_chan_resources = bam_alloc_chan;
bdev->common.device_free_chan_resources = bam_free_chan;
bdev->common.device_prep_slave_sg = bam_prep_slave_sg;
}
}
-static void sh_dmae_shutdown(struct platform_device *pdev)
-{
- struct sh_dmae_device *shdev = platform_get_drvdata(pdev);
- sh_dmae_ctl_stop(shdev);
-}
-
#ifdef CONFIG_PM
static int sh_dmae_runtime_suspend(struct device *dev)
{
+ struct sh_dmae_device *shdev = dev_get_drvdata(dev);
+
+ sh_dmae_ctl_stop(shdev);
return 0;
}
#ifdef CONFIG_PM_SLEEP
static int sh_dmae_suspend(struct device *dev)
{
+ struct sh_dmae_device *shdev = dev_get_drvdata(dev);
+
+ sh_dmae_ctl_stop(shdev);
return 0;
}
}
static struct platform_driver sh_dmae_driver = {
- .driver = {
+ .driver = {
.pm = &sh_dmae_pm,
.name = SH_DMAE_DRV_NAME,
.of_match_table = sh_dmae_of_match,
},
.remove = sh_dmae_remove,
- .shutdown = sh_dmae_shutdown,
};
static int __init sh_dmae_init(void)
*/
static atomic_t drv_instances = ATOMIC_INIT(0);
-/* Per-node driver instances */
-static struct mem_ctl_info **mcis;
+/* Per-node stuff */
static struct ecc_settings **ecc_stngs;
/*
/* On F10h and later ErrAddr is MC4_ADDR[47:1] */
static u64 get_error_address(struct amd64_pvt *pvt, struct mce *m)
{
- u64 addr;
+ u16 mce_nid = amd_get_nb_id(m->extcpu);
+ struct mem_ctl_info *mci;
u8 start_bit = 1;
u8 end_bit = 47;
+ u64 addr;
+
+ mci = edac_mc_find(mce_nid);
+ if (!mci)
+ return 0;
+
+ pvt = mci->pvt_info;
if (pvt->fam == 0xf) {
start_bit = 3;
* Erratum 637 workaround
*/
if (pvt->fam == 0x15) {
- struct amd64_pvt *pvt;
u64 cc6_base, tmp_addr;
u32 tmp;
- u16 mce_nid;
u8 intlv_en;
if ((addr & GENMASK_ULL(47, 24)) >> 24 != 0x00fdf7)
return addr;
- mce_nid = amd_get_nb_id(m->extcpu);
- pvt = mcis[mce_nid]->pvt_info;
amd64_read_pci_cfg(pvt->F1, DRAM_LOCAL_NODE_LIM, &tmp);
intlv_en = tmp >> 21 & 0x7;
int cs_found = -EINVAL;
int csrow;
- mci = mcis[nid];
+ mci = edac_mc_find(nid);
if (!mci)
return cs_found;
return true;
}
-static int set_mc_sysfs_attrs(struct mem_ctl_info *mci)
-{
- struct amd64_pvt *pvt = mci->pvt_info;
- int rc;
-
- rc = amd64_create_sysfs_dbg_files(mci);
- if (rc < 0)
- return rc;
-
- if (pvt->fam >= 0x10) {
- rc = amd64_create_sysfs_inject_files(mci);
- if (rc < 0)
- return rc;
- }
-
- return 0;
-}
-
-static void del_mc_sysfs_attrs(struct mem_ctl_info *mci)
-{
- struct amd64_pvt *pvt = mci->pvt_info;
-
- amd64_remove_sysfs_dbg_files(mci);
-
- if (pvt->fam >= 0x10)
- amd64_remove_sysfs_inject_files(mci);
-}
-
static void setup_mci_misc_attrs(struct mem_ctl_info *mci,
struct amd64_family_type *fam)
{
return fam_type;
}
+static const struct attribute_group *amd64_edac_attr_groups[] = {
+#ifdef CONFIG_EDAC_DEBUG
+ &amd64_edac_dbg_group,
+#endif
+#ifdef CONFIG_EDAC_AMD64_ERROR_INJECTION
+ &amd64_edac_inj_group,
+#endif
+ NULL
+};
+
static int init_one_instance(struct pci_dev *F2)
{
struct amd64_pvt *pvt = NULL;
mci->edac_cap = EDAC_FLAG_NONE;
ret = -ENODEV;
- if (edac_mc_add_mc(mci)) {
+ if (edac_mc_add_mc_with_groups(mci, amd64_edac_attr_groups)) {
edac_dbg(1, "failed edac_mc_add_mc()\n");
goto err_add_mc;
}
- if (set_mc_sysfs_attrs(mci)) {
- edac_dbg(1, "failed edac_mc_add_mc()\n");
- goto err_add_sysfs;
- }
/* register stuff with EDAC MCE */
if (report_gart_errors)
amd_register_ecc_decoder(decode_bus_error);
- mcis[nid] = mci;
-
atomic_inc(&drv_instances);
return 0;
-err_add_sysfs:
- edac_mc_del_mc(mci->pdev);
err_add_mc:
edac_mc_free(mci);
mci = find_mci_by_dev(&pdev->dev);
WARN_ON(!mci);
- del_mc_sysfs_attrs(mci);
/* Remove from EDAC CORE tracking list */
mci = edac_mc_del_mc(&pdev->dev);
if (!mci)
/* Free the EDAC CORE resources */
mci->pvt_info = NULL;
- mcis[nid] = NULL;
kfree(pvt);
edac_mc_free(mci);
if (pci_ctl)
return;
- mci = mcis[0];
+ mci = edac_mc_find(0);
if (!mci)
return;
goto err_ret;
err = -ENOMEM;
- mcis = kzalloc(amd_nb_num() * sizeof(mcis[0]), GFP_KERNEL);
ecc_stngs = kzalloc(amd_nb_num() * sizeof(ecc_stngs[0]), GFP_KERNEL);
- if (!(mcis && ecc_stngs))
+ if (!ecc_stngs)
goto err_free;
msrs = msrs_alloc();
msrs = NULL;
err_free:
- kfree(mcis);
- mcis = NULL;
-
kfree(ecc_stngs);
ecc_stngs = NULL;
kfree(ecc_stngs);
ecc_stngs = NULL;
- kfree(mcis);
- mcis = NULL;
-
msrs_free(msrs);
msrs = NULL;
}
};
#ifdef CONFIG_EDAC_DEBUG
-int amd64_create_sysfs_dbg_files(struct mem_ctl_info *mci);
-void amd64_remove_sysfs_dbg_files(struct mem_ctl_info *mci);
-
-#else
-static inline int amd64_create_sysfs_dbg_files(struct mem_ctl_info *mci)
-{
- return 0;
-}
-static void inline amd64_remove_sysfs_dbg_files(struct mem_ctl_info *mci)
-{
-}
+extern const struct attribute_group amd64_edac_dbg_group;
#endif
#ifdef CONFIG_EDAC_AMD64_ERROR_INJECTION
-int amd64_create_sysfs_inject_files(struct mem_ctl_info *mci);
-void amd64_remove_sysfs_inject_files(struct mem_ctl_info *mci);
-
-#else
-static inline int amd64_create_sysfs_inject_files(struct mem_ctl_info *mci)
-{
- return 0;
-}
-static inline void amd64_remove_sysfs_inject_files(struct mem_ctl_info *mci)
-{
-}
+extern const struct attribute_group amd64_edac_inj_group;
#endif
/*
static DEVICE_ATTR(topmem2, S_IRUGO, amd64_top_mem2_show, NULL);
static DEVICE_ATTR(dram_hole, S_IRUGO, amd64_hole_show, NULL);
-int amd64_create_sysfs_dbg_files(struct mem_ctl_info *mci)
-{
- int rc;
-
- rc = device_create_file(&mci->dev, &dev_attr_dhar);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_dbam);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_topmem);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_topmem2);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_dram_hole);
- if (rc < 0)
- return rc;
-
- return 0;
-}
-
-void amd64_remove_sysfs_dbg_files(struct mem_ctl_info *mci)
-{
- device_remove_file(&mci->dev, &dev_attr_dhar);
- device_remove_file(&mci->dev, &dev_attr_dbam);
- device_remove_file(&mci->dev, &dev_attr_topmem);
- device_remove_file(&mci->dev, &dev_attr_topmem2);
- device_remove_file(&mci->dev, &dev_attr_dram_hole);
-}
+static struct attribute *amd64_edac_dbg_attrs[] = {
+ &dev_attr_dhar.attr,
+ &dev_attr_dbam.attr,
+ &dev_attr_topmem.attr,
+ &dev_attr_topmem2.attr,
+ &dev_attr_dram_hole.attr,
+ NULL
+};
+
+const struct attribute_group amd64_edac_dbg_group = {
+ .attrs = amd64_edac_dbg_attrs,
+};
static DEVICE_ATTR(inject_read, S_IWUSR,
NULL, amd64_inject_read_store);
-
-int amd64_create_sysfs_inject_files(struct mem_ctl_info *mci)
+static struct attribute *amd64_edac_inj_attrs[] = {
+ &dev_attr_inject_section.attr,
+ &dev_attr_inject_word.attr,
+ &dev_attr_inject_ecc_vector.attr,
+ &dev_attr_inject_write.attr,
+ &dev_attr_inject_read.attr,
+ NULL
+};
+
+static umode_t amd64_edac_inj_is_visible(struct kobject *kobj,
+ struct attribute *attr, int idx)
{
- int rc;
-
- rc = device_create_file(&mci->dev, &dev_attr_inject_section);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_word);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_ecc_vector);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_write);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_read);
- if (rc < 0)
- return rc;
-
- return 0;
-}
+ struct device *dev = kobj_to_dev(kobj);
+ struct mem_ctl_info *mci = container_of(dev, struct mem_ctl_info, dev);
+ struct amd64_pvt *pvt = mci->pvt_info;
-void amd64_remove_sysfs_inject_files(struct mem_ctl_info *mci)
-{
- device_remove_file(&mci->dev, &dev_attr_inject_section);
- device_remove_file(&mci->dev, &dev_attr_inject_word);
- device_remove_file(&mci->dev, &dev_attr_inject_ecc_vector);
- device_remove_file(&mci->dev, &dev_attr_inject_write);
- device_remove_file(&mci->dev, &dev_attr_inject_read);
+ if (pvt->fam < 0x10)
+ return 0;
+ return attr->mode;
}
+
+const struct attribute_group amd64_edac_inj_group = {
+ .attrs = amd64_edac_inj_attrs,
+ .is_visible = amd64_edac_inj_is_visible,
+};
unsigned n_layers,
struct edac_mc_layer *layers,
unsigned sz_pvt);
-extern int edac_mc_add_mc(struct mem_ctl_info *mci);
+extern int edac_mc_add_mc_with_groups(struct mem_ctl_info *mci,
+ const struct attribute_group **groups);
+#define edac_mc_add_mc(mci) edac_mc_add_mc_with_groups(mci, NULL)
extern void edac_mc_free(struct mem_ctl_info *mci);
extern struct mem_ctl_info *edac_mc_find(int idx);
extern struct mem_ctl_info *find_mci_by_dev(struct device *dev);
EXPORT_SYMBOL(edac_mc_find);
/**
- * edac_mc_add_mc: Insert the 'mci' structure into the mci global list and
- * create sysfs entries associated with mci structure
+ * edac_mc_add_mc_with_groups: Insert the 'mci' structure into the mci
+ * global list and create sysfs entries associated with mci structure
* @mci: pointer to the mci structure to be added to the list
+ * @groups: optional attribute groups for the driver-specific sysfs entries
*
* Return:
* 0 Success
*/
/* FIXME - should a warning be printed if no error detection? correction? */
-int edac_mc_add_mc(struct mem_ctl_info *mci)
+int edac_mc_add_mc_with_groups(struct mem_ctl_info *mci,
+ const struct attribute_group **groups)
{
int ret = -EINVAL;
edac_dbg(0, "\n");
mci->bus = &mc_bus[mci->mc_idx];
- if (edac_create_sysfs_mci_device(mci)) {
+ if (edac_create_sysfs_mci_device(mci, groups)) {
edac_mc_printk(mci, KERN_WARNING,
"failed to create sysfs device\n");
goto fail1;
mutex_unlock(&mem_ctls_mutex);
return ret;
}
-EXPORT_SYMBOL_GPL(edac_mc_add_mc);
+EXPORT_SYMBOL_GPL(edac_mc_add_mc_with_groups);
/**
* edac_mc_del_mc: Remove sysfs entries for specified mci structure and
channel_dimm_label_show, channel_dimm_label_store, 5);
/* Total possible dynamic DIMM Label attribute file table */
-static struct device_attribute *dynamic_csrow_dimm_attr[] = {
- &dev_attr_legacy_ch0_dimm_label.attr,
- &dev_attr_legacy_ch1_dimm_label.attr,
- &dev_attr_legacy_ch2_dimm_label.attr,
- &dev_attr_legacy_ch3_dimm_label.attr,
- &dev_attr_legacy_ch4_dimm_label.attr,
- &dev_attr_legacy_ch5_dimm_label.attr
+static struct attribute *dynamic_csrow_dimm_attr[] = {
+ &dev_attr_legacy_ch0_dimm_label.attr.attr,
+ &dev_attr_legacy_ch1_dimm_label.attr.attr,
+ &dev_attr_legacy_ch2_dimm_label.attr.attr,
+ &dev_attr_legacy_ch3_dimm_label.attr.attr,
+ &dev_attr_legacy_ch4_dimm_label.attr.attr,
+ &dev_attr_legacy_ch5_dimm_label.attr.attr,
+ NULL
};
/* possible dynamic channel ce_count attribute files */
channel_ce_count_show, NULL, 5);
/* Total possible dynamic ce_count attribute file table */
-static struct device_attribute *dynamic_csrow_ce_count_attr[] = {
- &dev_attr_legacy_ch0_ce_count.attr,
- &dev_attr_legacy_ch1_ce_count.attr,
- &dev_attr_legacy_ch2_ce_count.attr,
- &dev_attr_legacy_ch3_ce_count.attr,
- &dev_attr_legacy_ch4_ce_count.attr,
- &dev_attr_legacy_ch5_ce_count.attr
+static struct attribute *dynamic_csrow_ce_count_attr[] = {
+ &dev_attr_legacy_ch0_ce_count.attr.attr,
+ &dev_attr_legacy_ch1_ce_count.attr.attr,
+ &dev_attr_legacy_ch2_ce_count.attr.attr,
+ &dev_attr_legacy_ch3_ce_count.attr.attr,
+ &dev_attr_legacy_ch4_ce_count.attr.attr,
+ &dev_attr_legacy_ch5_ce_count.attr.attr,
+ NULL
+};
+
+static umode_t csrow_dev_is_visible(struct kobject *kobj,
+ struct attribute *attr, int idx)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct csrow_info *csrow = container_of(dev, struct csrow_info, dev);
+
+ if (idx >= csrow->nr_channels)
+ return 0;
+ /* Only expose populated DIMMs */
+ if (!csrow->channels[idx]->dimm->nr_pages)
+ return 0;
+ return attr->mode;
+}
+
+
+static const struct attribute_group csrow_dev_dimm_group = {
+ .attrs = dynamic_csrow_dimm_attr,
+ .is_visible = csrow_dev_is_visible,
+};
+
+static const struct attribute_group csrow_dev_ce_count_group = {
+ .attrs = dynamic_csrow_ce_count_attr,
+ .is_visible = csrow_dev_is_visible,
+};
+
+static const struct attribute_group *csrow_dev_groups[] = {
+ &csrow_dev_dimm_group,
+ &csrow_dev_ce_count_group,
+ NULL
};
static inline int nr_pages_per_csrow(struct csrow_info *csrow)
static int edac_create_csrow_object(struct mem_ctl_info *mci,
struct csrow_info *csrow, int index)
{
- int err, chan;
-
if (csrow->nr_channels > EDAC_NR_CHANNELS)
return -ENODEV;
csrow->dev.type = &csrow_attr_type;
csrow->dev.bus = mci->bus;
+ csrow->dev.groups = csrow_dev_groups;
device_initialize(&csrow->dev);
csrow->dev.parent = &mci->dev;
csrow->mci = mci;
edac_dbg(0, "creating (virtual) csrow node %s\n",
dev_name(&csrow->dev));
- err = device_add(&csrow->dev);
- if (err < 0)
- return err;
-
- for (chan = 0; chan < csrow->nr_channels; chan++) {
- /* Only expose populated DIMMs */
- if (!csrow->channels[chan]->dimm->nr_pages)
- continue;
- err = device_create_file(&csrow->dev,
- dynamic_csrow_dimm_attr[chan]);
- if (err < 0)
- goto error;
- err = device_create_file(&csrow->dev,
- dynamic_csrow_ce_count_attr[chan]);
- if (err < 0) {
- device_remove_file(&csrow->dev,
- dynamic_csrow_dimm_attr[chan]);
- goto error;
- }
- }
-
- return 0;
-
-error:
- for (--chan; chan >= 0; chan--) {
- device_remove_file(&csrow->dev,
- dynamic_csrow_dimm_attr[chan]);
- device_remove_file(&csrow->dev,
- dynamic_csrow_ce_count_attr[chan]);
- }
- put_device(&csrow->dev);
-
- return err;
+ return device_add(&csrow->dev);
}
/* Create a CSROW object under specifed edac_mc_device */
static int edac_create_csrow_objects(struct mem_ctl_info *mci)
{
- int err, i, chan;
+ int err, i;
struct csrow_info *csrow;
for (i = 0; i < mci->nr_csrows; i++) {
csrow = mci->csrows[i];
if (!nr_pages_per_csrow(csrow))
continue;
- for (chan = csrow->nr_channels - 1; chan >= 0; chan--) {
- if (!csrow->channels[chan]->dimm->nr_pages)
- continue;
- device_remove_file(&csrow->dev,
- dynamic_csrow_dimm_attr[chan]);
- device_remove_file(&csrow->dev,
- dynamic_csrow_ce_count_attr[chan]);
- }
put_device(&mci->csrows[i]->dev);
}
static void edac_delete_csrow_objects(struct mem_ctl_info *mci)
{
- int i, chan;
+ int i;
struct csrow_info *csrow;
for (i = mci->nr_csrows - 1; i >= 0; i--) {
csrow = mci->csrows[i];
if (!nr_pages_per_csrow(csrow))
continue;
- for (chan = csrow->nr_channels - 1; chan >= 0; chan--) {
- if (!csrow->channels[chan]->dimm->nr_pages)
- continue;
- edac_dbg(1, "Removing csrow %d channel %d sysfs nodes\n",
- i, chan);
- device_remove_file(&csrow->dev,
- dynamic_csrow_dimm_attr[chan]);
- device_remove_file(&csrow->dev,
- dynamic_csrow_ce_count_attr[chan]);
- }
device_unregister(&mci->csrows[i]->dev);
}
}
static DEVICE_ATTR(max_location, S_IRUGO, mci_max_location_show, NULL);
/* memory scrubber attribute file */
-static DEVICE_ATTR(sdram_scrub_rate, 0, NULL, NULL);
+DEVICE_ATTR(sdram_scrub_rate, 0, mci_sdram_scrub_rate_show,
+ mci_sdram_scrub_rate_store); /* umode set later in is_visible */
static struct attribute *mci_attrs[] = {
&dev_attr_reset_counters.attr,
&dev_attr_ue_count.attr,
&dev_attr_ce_count.attr,
&dev_attr_max_location.attr,
+ &dev_attr_sdram_scrub_rate.attr,
NULL
};
+static umode_t mci_attr_is_visible(struct kobject *kobj,
+ struct attribute *attr, int idx)
+{
+ struct device *dev = kobj_to_dev(kobj);
+ struct mem_ctl_info *mci = to_mci(dev);
+ umode_t mode = 0;
+
+ if (attr != &dev_attr_sdram_scrub_rate.attr)
+ return attr->mode;
+ if (mci->get_sdram_scrub_rate)
+ mode |= S_IRUGO;
+ if (mci->set_sdram_scrub_rate)
+ mode |= S_IWUSR;
+ return mode;
+}
+
static struct attribute_group mci_attr_grp = {
.attrs = mci_attrs,
+ .is_visible = mci_attr_is_visible,
};
static const struct attribute_group *mci_attr_groups[] = {
return 0;
}
-void __exit edac_debugfs_exit(void)
+void edac_debugfs_exit(void)
{
debugfs_remove(edac_debugfs);
}
* 0 Success
* !0 Failure
*/
-int edac_create_sysfs_mci_device(struct mem_ctl_info *mci)
+int edac_create_sysfs_mci_device(struct mem_ctl_info *mci,
+ const struct attribute_group **groups)
{
int i, err;
mci->dev.parent = mci_pdev;
mci->dev.bus = mci->bus;
+ mci->dev.groups = groups;
dev_set_name(&mci->dev, "mc%d", mci->mc_idx);
dev_set_drvdata(&mci->dev, mci);
pm_runtime_forbid(&mci->dev);
goto fail_unregister_bus;
}
- if (mci->set_sdram_scrub_rate || mci->get_sdram_scrub_rate) {
- if (mci->get_sdram_scrub_rate) {
- dev_attr_sdram_scrub_rate.attr.mode |= S_IRUGO;
- dev_attr_sdram_scrub_rate.show = &mci_sdram_scrub_rate_show;
- }
-
- if (mci->set_sdram_scrub_rate) {
- dev_attr_sdram_scrub_rate.attr.mode |= S_IWUSR;
- dev_attr_sdram_scrub_rate.store = &mci_sdram_scrub_rate_store;
- }
-
- err = device_create_file(&mci->dev, &dev_attr_sdram_scrub_rate);
- if (err) {
- edac_dbg(1, "failure: create sdram_scrub_rate\n");
- goto fail_unregister_dev;
- }
- }
/*
* Create the dimm/rank devices
*/
device_unregister(&dimm->dev);
}
-fail_unregister_dev:
device_unregister(&mci->dev);
fail_unregister_bus:
bus_unregister(mci->bus);
return err;
}
-void __exit edac_mc_sysfs_exit(void)
+void edac_mc_sysfs_exit(void)
{
device_unregister(mci_pdev);
edac_put_sysfs_subsys();
err = edac_mc_sysfs_init();
if (err)
- goto error;
+ goto err_sysfs;
edac_debugfs_init();
- /* Setup/Initialize the workq for this core */
err = edac_workqueue_setup();
if (err) {
- edac_printk(KERN_ERR, EDAC_MC, "init WorkQueue failure\n");
- goto error;
+ edac_printk(KERN_ERR, EDAC_MC, "Failure initializing workqueue\n");
+ goto err_wq;
}
return 0;
-error:
+err_wq:
+ edac_debugfs_exit();
+ edac_mc_sysfs_exit();
+
+err_sysfs:
return err;
}
/* on edac_mc_sysfs.c */
int edac_mc_sysfs_init(void);
void edac_mc_sysfs_exit(void);
-extern int edac_create_sysfs_mci_device(struct mem_ctl_info *mci);
+extern int edac_create_sysfs_mci_device(struct mem_ctl_info *mci,
+ const struct attribute_group **groups);
extern void edac_remove_sysfs_mci_device(struct mem_ctl_info *mci);
void edac_unregister_sysfs(struct mem_ctl_info *mci);
extern int edac_get_log_ue(void);
static DEVICE_ATTR(inject_ctrl, S_IWUSR, NULL, highbank_mc_inject_ctrl);
+static struct attribute *highbank_dev_attrs[] = {
+ &dev_attr_inject_ctrl.attr,
+ NULL
+};
+
+ATTRIBUTE_GROUPS(highbank_dev);
+
struct hb_mc_settings {
int err_offset;
int int_offset;
.int_offset = MW_DDR_ECC_INT_BASE,
};
-static struct of_device_id hb_ddr_ctrl_of_match[] = {
+static const struct of_device_id hb_ddr_ctrl_of_match[] = {
{ .compatible = "calxeda,hb-ddr-ctrl", .data = &hb_settings },
{ .compatible = "calxeda,ecx-2000-ddr-ctrl", .data = &mw_settings },
{},
dimm->mtype = MEM_DDR3;
dimm->edac_mode = EDAC_SECDED;
- res = edac_mc_add_mc(mci);
+ res = edac_mc_add_mc_with_groups(mci, highbank_dev_groups);
if (res < 0)
goto err;
goto err2;
}
- device_create_file(&mci->dev, &dev_attr_inject_ctrl);
-
devres_close_group(&pdev->dev, NULL);
return 0;
err2:
{
struct mem_ctl_info *mci = platform_get_drvdata(pdev);
- device_remove_file(&mci->dev, &dev_attr_inject_ctrl);
edac_mc_del_mc(&pdev->dev);
edac_mc_free(mci);
return 0;
static DEVICE_ATTR(inject_enable, S_IRUGO | S_IWUSR,
i7core_inject_enable_show, i7core_inject_enable_store);
+static struct attribute *i7core_dev_attrs[] = {
+ &dev_attr_inject_section.attr,
+ &dev_attr_inject_type.attr,
+ &dev_attr_inject_eccmask.attr,
+ &dev_attr_inject_enable.attr,
+ NULL
+};
+
+ATTRIBUTE_GROUPS(i7core_dev);
+
static int i7core_create_sysfs_devices(struct mem_ctl_info *mci)
{
struct i7core_pvt *pvt = mci->pvt_info;
int rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_section);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_type);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_eccmask);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_enable);
- if (rc < 0)
- return rc;
-
pvt->addrmatch_dev = kzalloc(sizeof(*pvt->addrmatch_dev), GFP_KERNEL);
if (!pvt->addrmatch_dev)
- return rc;
+ return -ENOMEM;
pvt->addrmatch_dev->type = &addrmatch_type;
pvt->addrmatch_dev->bus = mci->dev.bus;
if (!pvt->chancounts_dev) {
put_device(pvt->addrmatch_dev);
device_del(pvt->addrmatch_dev);
- return rc;
+ return -ENOMEM;
}
pvt->chancounts_dev->type = &all_channel_counts_type;
edac_dbg(1, "\n");
- device_remove_file(&mci->dev, &dev_attr_inject_section);
- device_remove_file(&mci->dev, &dev_attr_inject_type);
- device_remove_file(&mci->dev, &dev_attr_inject_eccmask);
- device_remove_file(&mci->dev, &dev_attr_inject_enable);
-
if (!pvt->is_registered) {
put_device(pvt->chancounts_dev);
device_del(pvt->chancounts_dev);
enable_sdram_scrub_setting(mci);
/* add this new MC control structure to EDAC's list of MCs */
- if (unlikely(edac_mc_add_mc(mci))) {
+ if (unlikely(edac_mc_add_mc_with_groups(mci, i7core_dev_groups))) {
edac_dbg(0, "MC: failed edac_mc_add_mc()\n");
/* FIXME: perhaps some code should go here that disables error
* reporting if we just enabled it
return -ENODEV;
}
-EXPORT_SYMBOL_GPL(i82443bxgx_edacmc_probe1);
-
/* returns count (>= 0), or negative on error */
static int i82443bxgx_edacmc_init_one(struct pci_dev *pdev,
const struct pci_device_id *ent)
edac_mc_free(mci);
}
-EXPORT_SYMBOL_GPL(i82443bxgx_edacmc_remove_one);
-
static const struct pci_device_id i82443bxgx_pci_tbl[] = {
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82443BX_0)},
{PCI_DEVICE(PCI_VENDOR_ID_INTEL, PCI_DEVICE_ID_INTEL_82443BX_2)},
pci_unregister_driver(&i82443bxgx_edacmc_driver);
fail0:
- if (mci_pdev != NULL)
- pci_dev_put(mci_pdev);
-
+ pci_dev_put(mci_pdev);
return pci_rc;
}
pci_unregister_driver(&i82860_driver);
fail0:
- if (mci_pdev != NULL)
- pci_dev_put(mci_pdev);
-
+ pci_dev_put(mci_pdev);
return pci_rc;
}
static void __exit i82860_exit(void)
{
edac_dbg(3, "\n");
-
pci_unregister_driver(&i82860_driver);
-
- if (mci_pdev != NULL)
- pci_dev_put(mci_pdev);
+ pci_dev_put(mci_pdev);
}
module_init(i82860_init);
pci_unregister_driver(&i82875p_driver);
fail0:
- if (mci_pdev != NULL)
- pci_dev_put(mci_pdev);
-
+ pci_dev_put(mci_pdev);
return pci_rc;
}
pci_unregister_driver(&i82975x_driver);
fail0:
- if (mci_pdev != NULL)
- pci_dev_put(mci_pdev);
-
+ pci_dev_put(mci_pdev);
return pci_rc;
}
DEVICE_ATTR(inject_ctrl, S_IRUGO | S_IWUSR,
mpc85xx_mc_inject_ctrl_show, mpc85xx_mc_inject_ctrl_store);
-static int mpc85xx_create_sysfs_attributes(struct mem_ctl_info *mci)
-{
- int rc;
-
- rc = device_create_file(&mci->dev, &dev_attr_inject_data_hi);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_data_lo);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_inject_ctrl);
- if (rc < 0)
- return rc;
+static struct attribute *mpc85xx_dev_attrs[] = {
+ &dev_attr_inject_data_hi.attr,
+ &dev_attr_inject_data_lo.attr,
+ &dev_attr_inject_ctrl.attr,
+ NULL
+};
- return 0;
-}
-
-static void mpc85xx_remove_sysfs_attributes(struct mem_ctl_info *mci)
-{
- device_remove_file(&mci->dev, &dev_attr_inject_data_hi);
- device_remove_file(&mci->dev, &dev_attr_inject_data_lo);
- device_remove_file(&mci->dev, &dev_attr_inject_ctrl);
-}
+ATTRIBUTE_GROUPS(mpc85xx_dev);
/**************************** PCI Err device ***************************/
#ifdef CONFIG_PCI
return 0;
}
-static struct of_device_id mpc85xx_l2_err_of_match[] = {
+static const struct of_device_id mpc85xx_l2_err_of_match[] = {
/* deprecate the fsl,85.. forms in the future, 2.6.30? */
{ .compatible = "fsl,8540-l2-cache-controller", },
{ .compatible = "fsl,8541-l2-cache-controller", },
/* clear all error bits */
out_be32(pdata->mc_vbase + MPC85XX_MC_ERR_DETECT, ~0);
- if (edac_mc_add_mc(mci)) {
- edac_dbg(3, "failed edac_mc_add_mc()\n");
- goto err;
- }
-
- if (mpc85xx_create_sysfs_attributes(mci)) {
- edac_mc_del_mc(mci->pdev);
+ if (edac_mc_add_mc_with_groups(mci, mpc85xx_dev_groups)) {
edac_dbg(3, "failed edac_mc_add_mc()\n");
goto err;
}
orig_ddr_err_disable);
out_be32(pdata->mc_vbase + MPC85XX_MC_ERR_SBE, orig_ddr_err_sbe);
- mpc85xx_remove_sysfs_attributes(mci);
edac_mc_del_mc(&op->dev);
edac_mc_free(mci);
return 0;
}
-static struct of_device_id mpc85xx_mc_err_of_match[] = {
+static const struct of_device_id mpc85xx_mc_err_of_match[] = {
/* deprecate the fsl,85.. forms in the future, 2.6.30? */
{ .compatible = "fsl,8540-memory-controller", },
{ .compatible = "fsl,8541-memory-controller", },
static DEVICE_ATTR(col, S_IRUGO | S_IWUSR,
octeon_mc_inject_col_show, octeon_mc_inject_col_store);
+static struct attribute *octeon_dev_attrs[] = {
+ &dev_attr_inject.attr,
+ &dev_attr_error_type.attr,
+ &dev_attr_dimm.attr,
+ &dev_attr_rank.attr,
+ &dev_attr_bank.attr,
+ &dev_attr_row.attr,
+ &dev_attr_col.attr,
+ NULL
+};
-static int octeon_set_mc_sysfs_attributes(struct mem_ctl_info *mci)
-{
- int rc;
-
- rc = device_create_file(&mci->dev, &dev_attr_inject);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_error_type);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_dimm);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_rank);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_bank);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_row);
- if (rc < 0)
- return rc;
- rc = device_create_file(&mci->dev, &dev_attr_col);
- if (rc < 0)
- return rc;
-
- return 0;
-}
+ATTRIBUTE_GROUPS(octeon_dev);
static int octeon_lmc_edac_probe(struct platform_device *pdev)
{
mci->ctl_name = "octeon-lmc-err";
mci->edac_check = octeon_lmc_edac_poll;
- if (edac_mc_add_mc(mci)) {
+ if (edac_mc_add_mc_with_groups(mci, octeon_dev_groups)) {
dev_err(&pdev->dev, "edac_mc_add_mc() failed\n");
edac_mc_free(mci);
return -ENXIO;
}
- if (octeon_set_mc_sysfs_attributes(mci)) {
- dev_err(&pdev->dev, "octeon_set_mc_sysfs_attributes() failed\n");
- return -ENXIO;
- }
-
-
cfg0.u64 = cvmx_read_csr(CVMX_LMCX_MEM_CFG0(mc));
cfg0.s.intr_ded_ena = 0; /* We poll */
cfg0.s.intr_sec_ena = 0;
mci->ctl_name = "co_lmc_err";
mci->edac_check = octeon_lmc_edac_poll_o2;
- if (edac_mc_add_mc(mci)) {
+ if (edac_mc_add_mc_with_groups(mci, octeon_dev_groups)) {
dev_err(&pdev->dev, "edac_mc_add_mc() failed\n");
edac_mc_free(mci);
return -ENXIO;
}
- if (octeon_set_mc_sysfs_attributes(mci)) {
- dev_err(&pdev->dev, "octeon_set_mc_sysfs_attributes() failed\n");
- return -ENXIO;
- }
-
-
en.u64 = cvmx_read_csr(CVMX_LMCX_MEM_CFG0(mc));
en.s.intr_ded_ena = 0; /* We poll */
en.s.intr_sec_ena = 0;
* Device tree node type and compatible tuples this driver can match
* on.
*/
-static struct of_device_id ppc4xx_edac_match[] = {
+static const struct of_device_id ppc4xx_edac_match[] = {
{
.compatible = "ibm,sdram-4xx-ddr2"
},
return 0;
}
-static struct of_device_id synps_edac_match[] = {
+static const struct of_device_id synps_edac_match[] = {
{ .compatible = "xlnx,zynq-ddrc-a05", },
{ /* end of table */ }
};
*/
static const char dmi_empty_string[] = " ";
-static u16 __initdata dmi_ver;
+static u32 dmi_ver __initdata;
+static u32 dmi_len;
+static u16 dmi_num;
/*
* Catch too early calls to dmi_check_system():
*/
* We have to be cautious here. We have seen BIOSes with DMI pointers
* pointing to completely the wrong place for example
*/
-static void dmi_table(u8 *buf, int len, int num,
+static void dmi_table(u8 *buf,
void (*decode)(const struct dmi_header *, void *),
void *private_data)
{
int i = 0;
/*
- * Stop when we see all the items the table claimed to have
- * OR we run off the end of the table (also happens)
+ * Stop when we have seen all the items the table claimed to have
+ * (SMBIOS < 3.0 only) OR we reach an end-of-table marker OR we run
+ * off the end of the table (should never happen but sometimes does
+ * on bogus implementations.)
*/
- while ((i < num) && (data - buf + sizeof(struct dmi_header)) <= len) {
+ while ((!dmi_num || i < dmi_num) &&
+ (data - buf + sizeof(struct dmi_header)) <= dmi_len) {
const struct dmi_header *dm = (const struct dmi_header *)data;
- /*
- * 7.45 End-of-Table (Type 127) [SMBIOS reference spec v3.0.0]
- */
- if (dm->type == DMI_ENTRY_END_OF_TABLE)
- break;
-
/*
* We want to know the total length (formatted area and
* strings) before decoding to make sure we won't run off the
* table in dmi_decode or dmi_string
*/
data += dm->length;
- while ((data - buf < len - 1) && (data[0] || data[1]))
+ while ((data - buf < dmi_len - 1) && (data[0] || data[1]))
data++;
- if (data - buf < len - 1)
+ if (data - buf < dmi_len - 1)
decode(dm, private_data);
+
+ /*
+ * 7.45 End-of-Table (Type 127) [SMBIOS reference spec v3.0.0]
+ */
+ if (dm->type == DMI_ENTRY_END_OF_TABLE)
+ break;
+
data += 2;
i++;
}
}
static phys_addr_t dmi_base;
-static u16 dmi_len;
-static u16 dmi_num;
static int __init dmi_walk_early(void (*decode)(const struct dmi_header *,
void *))
if (buf == NULL)
return -1;
- dmi_table(buf, dmi_len, dmi_num, decode, NULL);
+ dmi_table(buf, decode, NULL);
add_device_randomness(buf, dmi_len);
* the UUID are supposed to be little-endian encoded. The specification
* says that this is the defacto standard.
*/
- if (dmi_ver >= 0x0206)
+ if (dmi_ver >= 0x020600)
sprintf(s, "%pUL", d);
else
sprintf(s, "%pUB", d);
*/
static int __init dmi_present(const u8 *buf)
{
- int smbios_ver;
+ u32 smbios_ver;
if (memcmp(buf, "_SM_", 4) == 0 &&
buf[5] < 32 && dmi_checksum(buf, buf[5])) {
if (dmi_walk_early(dmi_decode) == 0) {
if (smbios_ver) {
dmi_ver = smbios_ver;
- pr_info("SMBIOS %d.%d present.\n",
- dmi_ver >> 8, dmi_ver & 0xFF);
+ pr_info("SMBIOS %d.%d%s present.\n",
+ dmi_ver >> 8, dmi_ver & 0xFF,
+ (dmi_ver < 0x0300) ? "" : ".x");
} else {
dmi_ver = (buf[14] & 0xF0) << 4 |
(buf[14] & 0x0F);
pr_info("Legacy DMI %d.%d present.\n",
dmi_ver >> 8, dmi_ver & 0xFF);
}
+ dmi_ver <<= 8;
dmi_format_ids(dmi_ids_string, sizeof(dmi_ids_string));
printk(KERN_DEBUG "DMI: %s\n", dmi_ids_string);
return 0;
{
if (memcmp(buf, "_SM3_", 5) == 0 &&
buf[6] < 32 && dmi_checksum(buf, buf[6])) {
- dmi_ver = get_unaligned_be16(buf + 7);
+ dmi_ver = get_unaligned_be32(buf + 6);
+ dmi_ver &= 0xFFFFFF;
+ dmi_num = 0; /* No longer specified */
dmi_len = get_unaligned_le32(buf + 12);
dmi_base = get_unaligned_le64(buf + 16);
- /*
- * The 64-bit SMBIOS 3.0 entry point no longer has a field
- * containing the number of structures present in the table.
- * Instead, it defines the table size as a maximum size, and
- * relies on the end-of-table structure type (#127) to be used
- * to signal the end of the table.
- * So let's define dmi_num as an upper bound as well: each
- * structure has a 4 byte header, so dmi_len / 4 is an upper
- * bound for the number of structures in the table.
- */
- dmi_num = dmi_len / 4;
-
if (dmi_walk_early(dmi_decode) == 0) {
- pr_info("SMBIOS %d.%d present.\n",
- dmi_ver >> 8, dmi_ver & 0xFF);
+ pr_info("SMBIOS %d.%d.%d present.\n",
+ dmi_ver >> 16, (dmi_ver >> 8) & 0xFF,
+ dmi_ver & 0xFF);
dmi_format_ids(dmi_ids_string, sizeof(dmi_ids_string));
pr_debug("DMI: %s\n", dmi_ids_string);
return 0;
if (buf == NULL)
return -1;
- dmi_table(buf, dmi_len, dmi_num, decode, private_data);
+ dmi_table(buf, decode, private_data);
dmi_unmap(buf);
return 0;
unsigned long initrd_addr;
u64 initrd_size = 0;
unsigned long fdt_addr = 0; /* Original DTB */
- u64 fdt_size = 0; /* We don't get size from configuration table */
+ unsigned long fdt_size = 0;
char *cmdline_ptr = NULL;
int cmdline_size = 0;
unsigned long new_fdt_addr;
} else {
status = handle_cmdline_files(sys_table, image, cmdline_ptr,
"dtb=",
- ~0UL, (unsigned long *)&fdt_addr,
- (unsigned long *)&fdt_size);
+ ~0UL, &fdt_addr, &fdt_size);
if (status != EFI_SUCCESS) {
pr_efi_err(sys_table, "Failed to load device tree!\n");
pr_efi(sys_table, "Using DTB from command line\n");
} else {
/* Look for a device tree configuration table entry. */
- fdt_addr = (uintptr_t)get_fdt(sys_table);
+ fdt_addr = (uintptr_t)get_fdt(sys_table, &fdt_size);
if (fdt_addr)
pr_efi(sys_table, "Using DTB from configuration table\n");
}
start = desc->phys_addr;
end = start + desc->num_pages * (1UL << EFI_PAGE_SHIFT);
- if ((start + size) > end || (start + size) > max)
- continue;
-
- if (end - size > max)
+ if (end > max)
end = max;
+ if ((start + size) > end)
+ continue;
+
if (round_down(end - size, align) < start)
continue;
unsigned long fdt_addr,
unsigned long fdt_size);
-void *get_fdt(efi_system_table_t *sys_table);
+void *get_fdt(efi_system_table_t *sys_table, unsigned long *fdt_size);
void efi_get_virtmap(efi_memory_desc_t *memory_map, unsigned long map_size,
unsigned long desc_size, efi_memory_desc_t *runtime_map,
return EFI_LOAD_ERROR;
}
-void *get_fdt(efi_system_table_t *sys_table)
+void *get_fdt(efi_system_table_t *sys_table, unsigned long *fdt_size)
{
efi_guid_t fdt_guid = DEVICE_TREE_GUID;
efi_config_table_t *tables;
for (i = 0; i < sys_table->nr_tables; i++)
if (efi_guidcmp(tables[i].guid, fdt_guid) == 0) {
fdt = (void *) tables[i].table;
+ if (fdt_check_header(fdt) != 0) {
+ pr_efi_err(sys_table, "Invalid header detected on UEFI supplied FDT, ignoring ...\n");
+ return NULL;
+ }
+ *fdt_size = fdt_totalsize(fdt);
break;
}
.xlate = irq_domain_xlate_twocell,
};
-static struct of_device_id mpc8xxx_gpio_ids[] __initdata = {
+static struct of_device_id mpc8xxx_gpio_ids[] = {
{ .compatible = "fsl,mpc8349-gpio", },
{ .compatible = "fsl,mpc8572-gpio", },
{ .compatible = "fsl,mpc8610-gpio", },
ret = of_property_read_u32_index(np, "gpio,syscon-dev", 2,
&priv->dir_reg_offset);
if (ret)
- dev_err(dev, "can't read the dir register offset!\n");
+ dev_dbg(dev, "can't read the dir register offset!\n");
priv->dir_reg_offset <<= 3;
}
struct gpio_chip gpio_chip;
};
+#define to_tgd(gc) container_of(gc, struct tps65912_gpio_data, gpio_chip)
+
static int tps65912_gpio_get(struct gpio_chip *gc, unsigned offset)
{
- struct tps65912 *tps65912 = container_of(gc, struct tps65912, gpio);
+ struct tps65912_gpio_data *tps65912_gpio = to_tgd(gc);
+ struct tps65912 *tps65912 = tps65912_gpio->tps65912;
int val;
val = tps65912_reg_read(tps65912, TPS65912_GPIO1 + offset);
static void tps65912_gpio_set(struct gpio_chip *gc, unsigned offset,
int value)
{
- struct tps65912 *tps65912 = container_of(gc, struct tps65912, gpio);
+ struct tps65912_gpio_data *tps65912_gpio = to_tgd(gc);
+ struct tps65912 *tps65912 = tps65912_gpio->tps65912;
if (value)
tps65912_set_bits(tps65912, TPS65912_GPIO1 + offset,
static int tps65912_gpio_output(struct gpio_chip *gc, unsigned offset,
int value)
{
- struct tps65912 *tps65912 = container_of(gc, struct tps65912, gpio);
+ struct tps65912_gpio_data *tps65912_gpio = to_tgd(gc);
+ struct tps65912 *tps65912 = tps65912_gpio->tps65912;
/* Set the initial value */
tps65912_gpio_set(gc, offset, value);
static int tps65912_gpio_input(struct gpio_chip *gc, unsigned offset)
{
- struct tps65912 *tps65912 = container_of(gc, struct tps65912, gpio);
+ struct tps65912_gpio_data *tps65912_gpio = to_tgd(gc);
+ struct tps65912 *tps65912 = tps65912_gpio->tps65912;
return tps65912_clear_bits(tps65912, TPS65912_GPIO1 + offset,
GPIO_CFG_MASK);
if (!handler)
return AE_BAD_PARAMETER;
+ pin = acpi_gpiochip_pin_to_gpio_offset(chip, pin);
+ if (pin < 0)
+ return AE_BAD_PARAMETER;
+
desc = gpiochip_request_own_desc(chip, pin, "ACPI:Event");
if (IS_ERR(desc)) {
dev_err(chip->dev, "Failed to request GPIO\n");
struct gpio_desc *desc;
bool found;
+ pin = acpi_gpiochip_pin_to_gpio_offset(chip, pin);
+ if (pin < 0) {
+ status = AE_BAD_PARAMETER;
+ goto out;
+ }
+
mutex_lock(&achip->conn_lock);
found = false;
ret = gc->of_xlate(gc, &gg_data->gpiospec, gg_data->flags);
if (ret < 0) {
- /* We've found the gpio chip, but the translation failed.
- * Return true to stop looking and return the translation
- * error via out_gpio
+ /* We've found a gpio chip, but the translation failed.
+ * Store translation error in out_gpio.
+ * Return false to keep looking, as more than one gpio chip
+ * could be registered per of-node.
*/
gg_data->out_gpio = ERR_PTR(ret);
- return true;
+ return false;
}
gg_data->out_gpio = gpiochip_get_desc(gc, ret);
return KFD_MQD_TYPE_CP;
}
-static inline unsigned int get_first_pipe(struct device_queue_manager *dqm)
+unsigned int get_first_pipe(struct device_queue_manager *dqm)
{
- BUG_ON(!dqm);
+ BUG_ON(!dqm || !dqm->dev);
return dqm->dev->shared_resources.first_compute_pipe;
}
+unsigned int get_pipes_num(struct device_queue_manager *dqm)
+{
+ BUG_ON(!dqm || !dqm->dev);
+ return dqm->dev->shared_resources.compute_pipe_count;
+}
+
static inline unsigned int get_pipes_num_cpsch(void)
{
return PIPE_PER_ME_CP_SCHEDULING;
pr_debug(" sdma queue id: %d\n", q->properties.sdma_queue_id);
pr_debug(" sdma engine id: %d\n", q->properties.sdma_engine_id);
+ init_sdma_vm(dqm, q, qpd);
retval = mqd->init_mqd(mqd, &q->mqd, &q->mqd_mem_obj,
&q->gart_mqd_addr, &q->properties);
if (retval != 0) {
return retval;
}
- init_sdma_vm(dqm, q, qpd);
+ retval = mqd->load_mqd(mqd, q->mqd, 0,
+ 0, NULL);
+ if (retval != 0) {
+ deallocate_sdma_queue(dqm, q->sdma_id);
+ mqd->uninit_mqd(mqd, q->mqd, q->mqd_mem_obj);
+ return retval;
+ }
+
return 0;
}
struct qcm_process_device *qpd);
int init_pipelines(struct device_queue_manager *dqm,
unsigned int pipes_num, unsigned int first_pipe);
+unsigned int get_first_pipe(struct device_queue_manager *dqm);
+unsigned int get_pipes_num(struct device_queue_manager *dqm);
extern inline unsigned int get_sh_mem_bases_32(struct kfd_process_device *pdd)
{
return (pdd->lds_base >> 60) & 0x0E;
}
-extern inline unsigned int get_pipes_num(struct device_queue_manager *dqm)
-{
- BUG_ON(!dqm || !dqm->dev);
- return dqm->dev->shared_resources.compute_pipe_count;
-}
-
#endif /* KFD_DEVICE_QUEUE_MANAGER_H_ */
static int initialize_cpsch_cik(struct device_queue_manager *dqm)
{
- return init_pipelines(dqm, get_pipes_num(dqm), 0);
+ return init_pipelines(dqm, get_pipes_num(dqm), get_first_pipe(dqm));
}
BUG_ON(!kq || !dev);
BUG_ON(type != KFD_QUEUE_TYPE_DIQ && type != KFD_QUEUE_TYPE_HIQ);
- pr_debug("kfd: In func %s initializing queue type %d size %d\n",
+ pr_debug("amdkfd: In func %s initializing queue type %d size %d\n",
__func__, KFD_QUEUE_TYPE_HIQ, queue_size);
nop.opcode = IT_NOP;
prop.doorbell_ptr = kfd_get_kernel_doorbell(dev, &prop.doorbell_off);
- if (prop.doorbell_ptr == NULL)
+ if (prop.doorbell_ptr == NULL) {
+ pr_err("amdkfd: error init doorbell");
goto err_get_kernel_doorbell;
+ }
retval = kfd_gtt_sa_allocate(dev, queue_size, &kq->pq);
- if (retval != 0)
+ if (retval != 0) {
+ pr_err("amdkfd: error init pq queues size (%d)\n", queue_size);
goto err_pq_allocate_vidmem;
+ }
kq->pq_kernel_addr = kq->pq->cpu_ptr;
kq->pq_gpu_addr = kq->pq->gpu_addr;
err_eop_allocate_vidmem:
kfd_gtt_sa_free(dev, kq->pq);
err_pq_allocate_vidmem:
- pr_err("kfd: error init pq\n");
kfd_release_kernel_doorbell(dev, prop.doorbell_ptr);
err_get_kernel_doorbell:
- pr_err("kfd: error init doorbell");
return false;
}
else if (kq->queue->properties.type == KFD_QUEUE_TYPE_DIQ)
kfd_gtt_sa_free(kq->dev, kq->fence_mem_obj);
+ kq->mqd->uninit_mqd(kq->mqd, kq->queue->mqd, kq->queue->mqd_mem_obj);
+
kfd_gtt_sa_free(kq->dev, kq->rptr_mem);
kfd_gtt_sa_free(kq->dev, kq->wptr_mem);
kq->ops_asic_specific.uninitialize(kq);
queue_address = (unsigned int *)kq->pq_kernel_addr;
queue_size_dwords = kq->queue->properties.queue_size / sizeof(uint32_t);
- pr_debug("kfd: In func %s\nrptr: %d\nwptr: %d\nqueue_address 0x%p\n",
+ pr_debug("amdkfd: In func %s\nrptr: %d\nwptr: %d\nqueue_address 0x%p\n",
__func__, rptr, wptr, queue_address);
available_size = (rptr - 1 - wptr + queue_size_dwords) %
}
if (kq->ops.initialize(kq, dev, type, KFD_KERNEL_QUEUE_SIZE) == false) {
- pr_err("kfd: failed to init kernel queue\n");
+ pr_err("amdkfd: failed to init kernel queue\n");
kfree(kq);
return NULL;
}
BUG_ON(!dev);
- pr_err("kfd: starting kernel queue test\n");
+ pr_err("amdkfd: starting kernel queue test\n");
kq = kernel_queue_init(dev, KFD_QUEUE_TYPE_HIQ);
BUG_ON(!kq);
buffer[i] = kq->nop_packet;
kq->ops.submit_packet(kq);
- pr_err("kfd: ending kernel queue test\n");
+ pr_err("amdkfd: ending kernel queue test\n");
}
(adj->crtc_hdisplay - 1) |
((adj->crtc_vdisplay - 1) << 16));
- cfg = ATMEL_HLCDC_CLKPOL;
+ cfg = 0;
prate = clk_get_rate(crtc->dc->hlcdc->sys_clk);
mode_rate = mode->crtc_clock * 1000;
pm_runtime_enable(dev->dev);
- pm_runtime_put_sync(dev->dev);
-
ret = atmel_hlcdc_dc_modeset_init(dev);
if (ret < 0) {
dev_err(dev->dev, "failed to initialize mode setting\n");
/* Disable the layer */
regmap_write(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_CHDR,
- ATMEL_HLCDC_LAYER_RST);
+ ATMEL_HLCDC_LAYER_RST | ATMEL_HLCDC_LAYER_A2Q |
+ ATMEL_HLCDC_LAYER_UPDATE);
/* Clear all pending interrupts */
regmap_read(regmap, desc->regs_offset + ATMEL_HLCDC_LAYER_ISR, &isr);
#include "drm_crtc_internal.h"
#include "drm_internal.h"
-static struct drm_framebuffer *add_framebuffer_internal(struct drm_device *dev,
- struct drm_mode_fb_cmd2 *r,
- struct drm_file *file_priv);
+static struct drm_framebuffer *
+internal_framebuffer_create(struct drm_device *dev,
+ struct drm_mode_fb_cmd2 *r,
+ struct drm_file *file_priv);
/* Avoid boilerplate. I'm tired of typing. */
#define DRM_ENUM_NAME_FN(fnname, list) \
}
EXPORT_SYMBOL(drm_framebuffer_reference);
-static void drm_framebuffer_free_bug(struct kref *kref)
-{
- BUG();
-}
-
-static void __drm_framebuffer_unreference(struct drm_framebuffer *fb)
-{
- DRM_DEBUG("%p: FB ID: %d (%d)\n", fb, fb->base.id, atomic_read(&fb->refcount.refcount));
- kref_put(&fb->refcount, drm_framebuffer_free_bug);
-}
-
/**
* drm_framebuffer_unregister_private - unregister a private fb from the lookup idr
* @fb: fb to unregister
return;
}
/* disconnect the plane from the fb and crtc: */
- __drm_framebuffer_unreference(plane->old_fb);
+ drm_framebuffer_unreference(plane->old_fb);
plane->old_fb = NULL;
plane->fb = NULL;
plane->crtc = NULL;
DRM_DEBUG_KMS("[CONNECTOR:%d:?]\n", out_resp->connector_id);
mutex_lock(&dev->mode_config.mutex);
- drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
connector = drm_connector_find(dev, out_resp->connector_id);
if (!connector) {
ret = -ENOENT;
- goto out;
+ goto out_unlock;
}
for (i = 0; i < DRM_CONNECTOR_MAX_ENCODER; i++)
out_resp->mm_height = connector->display_info.height_mm;
out_resp->subpixel = connector->display_info.subpixel_order;
out_resp->connection = connector->status;
+
+ drm_modeset_lock(&dev->mode_config.connection_mutex, NULL);
encoder = drm_connector_get_encoder(connector);
if (encoder)
out_resp->encoder_id = encoder->base.id;
out:
drm_modeset_unlock(&dev->mode_config.connection_mutex);
+
+out_unlock:
mutex_unlock(&dev->mode_config.mutex);
return ret;
*/
if (req->flags & DRM_MODE_CURSOR_BO) {
if (req->handle) {
- fb = add_framebuffer_internal(dev, &fbreq, file_priv);
+ fb = internal_framebuffer_create(dev, &fbreq, file_priv);
if (IS_ERR(fb)) {
DRM_DEBUG_KMS("failed to wrap cursor buffer in drm framebuffer\n");
return PTR_ERR(fb);
}
-
- drm_framebuffer_reference(fb);
} else {
fb = NULL;
}
return 0;
}
-static struct drm_framebuffer *add_framebuffer_internal(struct drm_device *dev,
- struct drm_mode_fb_cmd2 *r,
- struct drm_file *file_priv)
+static struct drm_framebuffer *
+internal_framebuffer_create(struct drm_device *dev,
+ struct drm_mode_fb_cmd2 *r,
+ struct drm_file *file_priv)
{
struct drm_mode_config *config = &dev->mode_config;
struct drm_framebuffer *fb;
return fb;
}
- mutex_lock(&file_priv->fbs_lock);
- r->fb_id = fb->base.id;
- list_add(&fb->filp_head, &file_priv->fbs);
- DRM_DEBUG_KMS("[FB:%d]\n", fb->base.id);
- mutex_unlock(&file_priv->fbs_lock);
-
return fb;
}
int drm_mode_addfb2(struct drm_device *dev,
void *data, struct drm_file *file_priv)
{
+ struct drm_mode_fb_cmd2 *r = data;
struct drm_framebuffer *fb;
if (!drm_core_check_feature(dev, DRIVER_MODESET))
return -EINVAL;
- fb = add_framebuffer_internal(dev, data, file_priv);
+ fb = internal_framebuffer_create(dev, r, file_priv);
if (IS_ERR(fb))
return PTR_ERR(fb);
+ /* Transfer ownership to the filp for reaping on close */
+
+ DRM_DEBUG_KMS("[FB:%d]\n", fb->base.id);
+ mutex_lock(&file_priv->fbs_lock);
+ r->fb_id = fb->base.id;
+ list_add(&fb->filp_head, &file_priv->fbs);
+ mutex_unlock(&file_priv->fbs_lock);
+
return 0;
}
struct drm_dp_sideband_msg_tx *txmsg)
{
bool ret;
- mutex_lock(&mgr->qlock);
+
+ /*
+ * All updates to txmsg->state are protected by mgr->qlock, and the two
+ * cases we check here are terminal states. For those the barriers
+ * provided by the wake_up/wait_event pair are enough.
+ */
ret = (txmsg->state == DRM_DP_SIDEBAND_TX_RX ||
txmsg->state == DRM_DP_SIDEBAND_TX_TIMEOUT);
- mutex_unlock(&mgr->qlock);
return ret;
}
return 0;
}
-/* must be called holding qlock */
static void process_single_down_tx_qlock(struct drm_dp_mst_topology_mgr *mgr)
{
struct drm_dp_sideband_msg_tx *txmsg;
int ret;
+ WARN_ON(!mutex_is_locked(&mgr->qlock));
+
/* construct a chunk from the first msg in the tx_msg queue */
if (list_empty(&mgr->tx_msg_downq)) {
mgr->tx_down_in_progress = false;
drm_mode_connector_update_edid_property(connector, edid);
ret = drm_add_edid_modes(connector, edid);
+ drm_edid_to_eld(connector, edid);
kfree(edid);
return ret;
*/
static struct drm_mm_node *drm_mm_search_free_generic(const struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
enum drm_mm_search_flags flags);
static struct drm_mm_node *drm_mm_search_free_in_range_generic(const struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
- unsigned long start,
- unsigned long end,
+ u64 start,
+ u64 end,
enum drm_mm_search_flags flags);
static void drm_mm_insert_helper(struct drm_mm_node *hole_node,
struct drm_mm_node *node,
- unsigned long size, unsigned alignment,
+ u64 size, unsigned alignment,
unsigned long color,
enum drm_mm_allocator_flags flags)
{
struct drm_mm *mm = hole_node->mm;
- unsigned long hole_start = drm_mm_hole_node_start(hole_node);
- unsigned long hole_end = drm_mm_hole_node_end(hole_node);
- unsigned long adj_start = hole_start;
- unsigned long adj_end = hole_end;
+ u64 hole_start = drm_mm_hole_node_start(hole_node);
+ u64 hole_end = drm_mm_hole_node_end(hole_node);
+ u64 adj_start = hole_start;
+ u64 adj_end = hole_end;
BUG_ON(node->allocated);
adj_start = adj_end - size;
if (alignment) {
- unsigned tmp = adj_start % alignment;
- if (tmp) {
+ u64 tmp = adj_start;
+ unsigned rem;
+
+ rem = do_div(tmp, alignment);
+ if (rem) {
if (flags & DRM_MM_CREATE_TOP)
- adj_start -= tmp;
+ adj_start -= rem;
else
- adj_start += alignment - tmp;
+ adj_start += alignment - rem;
}
}
int drm_mm_reserve_node(struct drm_mm *mm, struct drm_mm_node *node)
{
struct drm_mm_node *hole;
- unsigned long end = node->start + node->size;
- unsigned long hole_start;
- unsigned long hole_end;
+ u64 end = node->start + node->size;
+ u64 hole_start;
+ u64 hole_end;
BUG_ON(node == NULL);
* 0 on success, -ENOSPC if there's no suitable hole.
*/
int drm_mm_insert_node_generic(struct drm_mm *mm, struct drm_mm_node *node,
- unsigned long size, unsigned alignment,
+ u64 size, unsigned alignment,
unsigned long color,
enum drm_mm_search_flags sflags,
enum drm_mm_allocator_flags aflags)
static void drm_mm_insert_helper_range(struct drm_mm_node *hole_node,
struct drm_mm_node *node,
- unsigned long size, unsigned alignment,
+ u64 size, unsigned alignment,
unsigned long color,
- unsigned long start, unsigned long end,
+ u64 start, u64 end,
enum drm_mm_allocator_flags flags)
{
struct drm_mm *mm = hole_node->mm;
- unsigned long hole_start = drm_mm_hole_node_start(hole_node);
- unsigned long hole_end = drm_mm_hole_node_end(hole_node);
- unsigned long adj_start = hole_start;
- unsigned long adj_end = hole_end;
+ u64 hole_start = drm_mm_hole_node_start(hole_node);
+ u64 hole_end = drm_mm_hole_node_end(hole_node);
+ u64 adj_start = hole_start;
+ u64 adj_end = hole_end;
BUG_ON(!hole_node->hole_follows || node->allocated);
mm->color_adjust(hole_node, color, &adj_start, &adj_end);
if (alignment) {
- unsigned tmp = adj_start % alignment;
- if (tmp) {
+ u64 tmp = adj_start;
+ unsigned rem;
+
+ rem = do_div(tmp, alignment);
+ if (rem) {
if (flags & DRM_MM_CREATE_TOP)
- adj_start -= tmp;
+ adj_start -= rem;
else
- adj_start += alignment - tmp;
+ adj_start += alignment - rem;
}
}
* 0 on success, -ENOSPC if there's no suitable hole.
*/
int drm_mm_insert_node_in_range_generic(struct drm_mm *mm, struct drm_mm_node *node,
- unsigned long size, unsigned alignment,
+ u64 size, unsigned alignment,
unsigned long color,
- unsigned long start, unsigned long end,
+ u64 start, u64 end,
enum drm_mm_search_flags sflags,
enum drm_mm_allocator_flags aflags)
{
}
EXPORT_SYMBOL(drm_mm_remove_node);
-static int check_free_hole(unsigned long start, unsigned long end,
- unsigned long size, unsigned alignment)
+static int check_free_hole(u64 start, u64 end, u64 size, unsigned alignment)
{
if (end - start < size)
return 0;
if (alignment) {
- unsigned tmp = start % alignment;
- if (tmp)
- start += alignment - tmp;
+ u64 tmp = start;
+ unsigned rem;
+
+ rem = do_div(tmp, alignment);
+ if (rem)
+ start += alignment - rem;
}
return end >= start + size;
}
static struct drm_mm_node *drm_mm_search_free_generic(const struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
enum drm_mm_search_flags flags)
{
struct drm_mm_node *entry;
struct drm_mm_node *best;
- unsigned long adj_start;
- unsigned long adj_end;
- unsigned long best_size;
+ u64 adj_start;
+ u64 adj_end;
+ u64 best_size;
BUG_ON(mm->scanned_blocks);
__drm_mm_for_each_hole(entry, mm, adj_start, adj_end,
flags & DRM_MM_SEARCH_BELOW) {
- unsigned long hole_size = adj_end - adj_start;
+ u64 hole_size = adj_end - adj_start;
if (mm->color_adjust) {
mm->color_adjust(entry, color, &adj_start, &adj_end);
}
static struct drm_mm_node *drm_mm_search_free_in_range_generic(const struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
- unsigned long start,
- unsigned long end,
+ u64 start,
+ u64 end,
enum drm_mm_search_flags flags)
{
struct drm_mm_node *entry;
struct drm_mm_node *best;
- unsigned long adj_start;
- unsigned long adj_end;
- unsigned long best_size;
+ u64 adj_start;
+ u64 adj_end;
+ u64 best_size;
BUG_ON(mm->scanned_blocks);
__drm_mm_for_each_hole(entry, mm, adj_start, adj_end,
flags & DRM_MM_SEARCH_BELOW) {
- unsigned long hole_size = adj_end - adj_start;
+ u64 hole_size = adj_end - adj_start;
if (adj_start < start)
adj_start = start;
* adding/removing nodes to/from the scan list are allowed.
*/
void drm_mm_init_scan(struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color)
{
* adding/removing nodes to/from the scan list are allowed.
*/
void drm_mm_init_scan_with_range(struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
- unsigned long start,
- unsigned long end)
+ u64 start,
+ u64 end)
{
mm->scan_color = color;
mm->scan_alignment = alignment;
{
struct drm_mm *mm = node->mm;
struct drm_mm_node *prev_node;
- unsigned long hole_start, hole_end;
- unsigned long adj_start, adj_end;
+ u64 hole_start, hole_end;
+ u64 adj_start, adj_end;
mm->scanned_blocks++;
*
* Note that @mm must be cleared to 0 before calling this function.
*/
-void drm_mm_init(struct drm_mm * mm, unsigned long start, unsigned long size)
+void drm_mm_init(struct drm_mm * mm, u64 start, u64 size)
{
INIT_LIST_HEAD(&mm->hole_stack);
mm->scanned_blocks = 0;
}
EXPORT_SYMBOL(drm_mm_takedown);
-static unsigned long drm_mm_debug_hole(struct drm_mm_node *entry,
- const char *prefix)
+static u64 drm_mm_debug_hole(struct drm_mm_node *entry,
+ const char *prefix)
{
- unsigned long hole_start, hole_end, hole_size;
+ u64 hole_start, hole_end, hole_size;
if (entry->hole_follows) {
hole_start = drm_mm_hole_node_start(entry);
hole_end = drm_mm_hole_node_end(entry);
hole_size = hole_end - hole_start;
- printk(KERN_DEBUG "%s 0x%08lx-0x%08lx: %8lu: free\n",
- prefix, hole_start, hole_end,
- hole_size);
+ pr_debug("%s %#llx-%#llx: %llu: free\n", prefix, hole_start,
+ hole_end, hole_size);
return hole_size;
}
void drm_mm_debug_table(struct drm_mm *mm, const char *prefix)
{
struct drm_mm_node *entry;
- unsigned long total_used = 0, total_free = 0, total = 0;
+ u64 total_used = 0, total_free = 0, total = 0;
total_free += drm_mm_debug_hole(&mm->head_node, prefix);
drm_mm_for_each_node(entry, mm) {
- printk(KERN_DEBUG "%s 0x%08lx-0x%08lx: %8lu: used\n",
- prefix, entry->start, entry->start + entry->size,
- entry->size);
+ pr_debug("%s %#llx-%#llx: %llu: used\n", prefix, entry->start,
+ entry->start + entry->size, entry->size);
total_used += entry->size;
total_free += drm_mm_debug_hole(entry, prefix);
}
total = total_free + total_used;
- printk(KERN_DEBUG "%s total: %lu, used %lu free %lu\n", prefix, total,
- total_used, total_free);
+ pr_debug("%s total: %llu, used %llu free %llu\n", prefix, total,
+ total_used, total_free);
}
EXPORT_SYMBOL(drm_mm_debug_table);
#if defined(CONFIG_DEBUG_FS)
-static unsigned long drm_mm_dump_hole(struct seq_file *m, struct drm_mm_node *entry)
+static u64 drm_mm_dump_hole(struct seq_file *m, struct drm_mm_node *entry)
{
- unsigned long hole_start, hole_end, hole_size;
+ u64 hole_start, hole_end, hole_size;
if (entry->hole_follows) {
hole_start = drm_mm_hole_node_start(entry);
hole_end = drm_mm_hole_node_end(entry);
hole_size = hole_end - hole_start;
- seq_printf(m, "0x%08lx-0x%08lx: 0x%08lx: free\n",
- hole_start, hole_end, hole_size);
+ seq_printf(m, "%#llx-%#llx: %llu: free\n", hole_start,
+ hole_end, hole_size);
return hole_size;
}
int drm_mm_dump_table(struct seq_file *m, struct drm_mm *mm)
{
struct drm_mm_node *entry;
- unsigned long total_used = 0, total_free = 0, total = 0;
+ u64 total_used = 0, total_free = 0, total = 0;
total_free += drm_mm_dump_hole(m, &mm->head_node);
drm_mm_for_each_node(entry, mm) {
- seq_printf(m, "0x%08lx-0x%08lx: 0x%08lx: used\n",
- entry->start, entry->start + entry->size,
- entry->size);
+ seq_printf(m, "%#016llx-%#016llx: %llu: used\n", entry->start,
+ entry->start + entry->size, entry->size);
total_used += entry->size;
total_free += drm_mm_dump_hole(m, entry);
}
total = total_free + total_used;
- seq_printf(m, "total: %lu, used %lu free %lu\n", total, total_used, total_free);
+ seq_printf(m, "total: %llu, used %llu free %llu\n", total,
+ total_used, total_free);
return 0;
}
EXPORT_SYMBOL(drm_mm_dump_table);
struct edid *edid = (struct edid *) connector->edid_blob_ptr->data;
count = drm_add_edid_modes(connector, edid);
+ drm_edid_to_eld(connector, edid);
} else
count = (*connector_funcs->get_modes)(connector);
}
config DRM_EXYNOS_DP
bool "EXYNOS DRM DP driver support"
- depends on (DRM_EXYNOS_FIMD || DRM_EXYNOS7DECON) && ARCH_EXYNOS && (DRM_PTN3460=n || DRM_PTN3460=y || DRM_PTN3460=DRM_EXYNOS)
+ depends on (DRM_EXYNOS_FIMD || DRM_EXYNOS7_DECON) && ARCH_EXYNOS && (DRM_PTN3460=n || DRM_PTN3460=y || DRM_PTN3460=DRM_EXYNOS)
default DRM_EXYNOS
select DRM_PANEL
help
of_node_put(i80_if_timings);
ctx->regs = of_iomap(dev->of_node, 0);
- if (IS_ERR(ctx->regs)) {
- ret = PTR_ERR(ctx->regs);
+ if (!ctx->regs) {
+ ret = -ENOMEM;
goto err_del_component;
}
+++ /dev/null
-/*
- * Copyright (c) 2011 Samsung Electronics Co., Ltd.
- * Authors:
- * Inki Dae <inki.dae@samsung.com>
- * Joonyoung Shim <jy0922.shim@samsung.com>
- * Seung-Woo Kim <sw0312.kim@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#include <drm/drmP.h>
-#include <drm/drm_crtc_helper.h>
-
-#include <drm/exynos_drm.h>
-#include "exynos_drm_drv.h"
-#include "exynos_drm_encoder.h"
-#include "exynos_drm_connector.h"
-
-#define to_exynos_connector(x) container_of(x, struct exynos_drm_connector,\
- drm_connector)
-
-struct exynos_drm_connector {
- struct drm_connector drm_connector;
- uint32_t encoder_id;
- struct exynos_drm_display *display;
-};
-
-static int exynos_drm_connector_get_modes(struct drm_connector *connector)
-{
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
- struct exynos_drm_display *display = exynos_connector->display;
- struct edid *edid = NULL;
- unsigned int count = 0;
- int ret;
-
- /*
- * if get_edid() exists then get_edid() callback of hdmi side
- * is called to get edid data through i2c interface else
- * get timing from the FIMD driver(display controller).
- *
- * P.S. in case of lcd panel, count is always 1 if success
- * because lcd panel has only one mode.
- */
- if (display->ops->get_edid) {
- edid = display->ops->get_edid(display, connector);
- if (IS_ERR_OR_NULL(edid)) {
- ret = PTR_ERR(edid);
- edid = NULL;
- DRM_ERROR("Panel operation get_edid failed %d\n", ret);
- goto out;
- }
-
- count = drm_add_edid_modes(connector, edid);
- if (!count) {
- DRM_ERROR("Add edid modes failed %d\n", count);
- goto out;
- }
-
- drm_mode_connector_update_edid_property(connector, edid);
- } else {
- struct exynos_drm_panel_info *panel;
- struct drm_display_mode *mode = drm_mode_create(connector->dev);
- if (!mode) {
- DRM_ERROR("failed to create a new display mode.\n");
- return 0;
- }
-
- if (display->ops->get_panel)
- panel = display->ops->get_panel(display);
- else {
- drm_mode_destroy(connector->dev, mode);
- return 0;
- }
-
- drm_display_mode_from_videomode(&panel->vm, mode);
- mode->width_mm = panel->width_mm;
- mode->height_mm = panel->height_mm;
- connector->display_info.width_mm = mode->width_mm;
- connector->display_info.height_mm = mode->height_mm;
-
- mode->type = DRM_MODE_TYPE_DRIVER | DRM_MODE_TYPE_PREFERRED;
- drm_mode_set_name(mode);
- drm_mode_probed_add(connector, mode);
-
- count = 1;
- }
-
-out:
- kfree(edid);
- return count;
-}
-
-static int exynos_drm_connector_mode_valid(struct drm_connector *connector,
- struct drm_display_mode *mode)
-{
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
- struct exynos_drm_display *display = exynos_connector->display;
- int ret = MODE_BAD;
-
- DRM_DEBUG_KMS("%s\n", __FILE__);
-
- if (display->ops->check_mode)
- if (!display->ops->check_mode(display, mode))
- ret = MODE_OK;
-
- return ret;
-}
-
-static struct drm_encoder *exynos_drm_best_encoder(
- struct drm_connector *connector)
-{
- struct drm_device *dev = connector->dev;
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
- return drm_encoder_find(dev, exynos_connector->encoder_id);
-}
-
-static struct drm_connector_helper_funcs exynos_connector_helper_funcs = {
- .get_modes = exynos_drm_connector_get_modes,
- .mode_valid = exynos_drm_connector_mode_valid,
- .best_encoder = exynos_drm_best_encoder,
-};
-
-static int exynos_drm_connector_fill_modes(struct drm_connector *connector,
- unsigned int max_width, unsigned int max_height)
-{
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
- struct exynos_drm_display *display = exynos_connector->display;
- unsigned int width, height;
-
- width = max_width;
- height = max_height;
-
- /*
- * if specific driver want to find desired_mode using maxmum
- * resolution then get max width and height from that driver.
- */
- if (display->ops->get_max_resol)
- display->ops->get_max_resol(display, &width, &height);
-
- return drm_helper_probe_single_connector_modes(connector, width,
- height);
-}
-
-/* get detection status of display device. */
-static enum drm_connector_status
-exynos_drm_connector_detect(struct drm_connector *connector, bool force)
-{
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
- struct exynos_drm_display *display = exynos_connector->display;
- enum drm_connector_status status = connector_status_disconnected;
-
- if (display->ops->is_connected) {
- if (display->ops->is_connected(display))
- status = connector_status_connected;
- else
- status = connector_status_disconnected;
- }
-
- return status;
-}
-
-static void exynos_drm_connector_destroy(struct drm_connector *connector)
-{
- struct exynos_drm_connector *exynos_connector =
- to_exynos_connector(connector);
-
- drm_connector_unregister(connector);
- drm_connector_cleanup(connector);
- kfree(exynos_connector);
-}
-
-static struct drm_connector_funcs exynos_connector_funcs = {
- .dpms = drm_helper_connector_dpms,
- .fill_modes = exynos_drm_connector_fill_modes,
- .detect = exynos_drm_connector_detect,
- .destroy = exynos_drm_connector_destroy,
-};
-
-struct drm_connector *exynos_drm_connector_create(struct drm_device *dev,
- struct drm_encoder *encoder)
-{
- struct exynos_drm_connector *exynos_connector;
- struct exynos_drm_display *display = exynos_drm_get_display(encoder);
- struct drm_connector *connector;
- int type;
- int err;
-
- exynos_connector = kzalloc(sizeof(*exynos_connector), GFP_KERNEL);
- if (!exynos_connector)
- return NULL;
-
- connector = &exynos_connector->drm_connector;
-
- switch (display->type) {
- case EXYNOS_DISPLAY_TYPE_HDMI:
- type = DRM_MODE_CONNECTOR_HDMIA;
- connector->interlace_allowed = true;
- connector->polled = DRM_CONNECTOR_POLL_HPD;
- break;
- case EXYNOS_DISPLAY_TYPE_VIDI:
- type = DRM_MODE_CONNECTOR_VIRTUAL;
- connector->polled = DRM_CONNECTOR_POLL_HPD;
- break;
- default:
- type = DRM_MODE_CONNECTOR_Unknown;
- break;
- }
-
- drm_connector_init(dev, connector, &exynos_connector_funcs, type);
- drm_connector_helper_add(connector, &exynos_connector_helper_funcs);
-
- err = drm_connector_register(connector);
- if (err)
- goto err_connector;
-
- exynos_connector->encoder_id = encoder->base.id;
- exynos_connector->display = display;
- connector->dpms = DRM_MODE_DPMS_OFF;
- connector->encoder = encoder;
-
- err = drm_mode_connector_attach_encoder(connector, encoder);
- if (err) {
- DRM_ERROR("failed to attach a connector to a encoder\n");
- goto err_sysfs;
- }
-
- DRM_DEBUG_KMS("connector has been created\n");
-
- return connector;
-
-err_sysfs:
- drm_connector_unregister(connector);
-err_connector:
- drm_connector_cleanup(connector);
- kfree(exynos_connector);
- return NULL;
-}
+++ /dev/null
-/*
- * Copyright (c) 2011 Samsung Electronics Co., Ltd.
- * Authors:
- * Inki Dae <inki.dae@samsung.com>
- * Joonyoung Shim <jy0922.shim@samsung.com>
- * Seung-Woo Kim <sw0312.kim@samsung.com>
- *
- * This program is free software; you can redistribute it and/or modify it
- * under the terms of the GNU General Public License as published by the
- * Free Software Foundation; either version 2 of the License, or (at your
- * option) any later version.
- */
-
-#ifndef _EXYNOS_DRM_CONNECTOR_H_
-#define _EXYNOS_DRM_CONNECTOR_H_
-
-struct drm_connector *exynos_drm_connector_create(struct drm_device *dev,
- struct drm_encoder *encoder);
-
-#endif
unsigned int ovl_height;
unsigned int fb_width;
unsigned int fb_height;
+ unsigned int fb_pitch;
unsigned int bpp;
unsigned int pixel_format;
dma_addr_t dma_addr;
}
}
-static int fimd_ctx_initialize(struct fimd_context *ctx,
+static int fimd_iommu_attach_devices(struct fimd_context *ctx,
struct drm_device *drm_dev)
{
- struct exynos_drm_private *priv;
- priv = drm_dev->dev_private;
-
- ctx->drm_dev = drm_dev;
- ctx->pipe = priv->pipe++;
/* attach this sub driver to iommu mapping if supported. */
if (is_drm_iommu_supported(ctx->drm_dev)) {
return 0;
}
-static void fimd_ctx_remove(struct fimd_context *ctx)
+static void fimd_iommu_detach_devices(struct fimd_context *ctx)
{
/* detach this sub driver from iommu mapping if supported. */
if (is_drm_iommu_supported(ctx->drm_dev))
win_data->offset_y = plane->crtc_y;
win_data->ovl_width = plane->crtc_width;
win_data->ovl_height = plane->crtc_height;
+ win_data->fb_pitch = plane->pitch;
win_data->fb_width = plane->fb_width;
win_data->fb_height = plane->fb_height;
win_data->dma_addr = plane->dma_addr[0] + offset;
win_data->bpp = plane->bpp;
win_data->pixel_format = plane->pixel_format;
- win_data->buf_offsize = (plane->fb_width - plane->crtc_width) *
- (plane->bpp >> 3);
+ win_data->buf_offsize =
+ plane->pitch - (plane->crtc_width * (plane->bpp >> 3));
win_data->line_size = plane->crtc_width * (plane->bpp >> 3);
DRM_DEBUG_KMS("offset_x = %d, offset_y = %d\n",
writel(val, ctx->regs + VIDWx_BUF_START(win, 0));
/* buffer end address */
- size = win_data->fb_width * win_data->ovl_height * (win_data->bpp >> 3);
+ size = win_data->fb_pitch * win_data->ovl_height * (win_data->bpp >> 3);
val = (unsigned long)(win_data->dma_addr + size);
writel(val, ctx->regs + VIDWx_BUF_END(win, 0));
{
struct fimd_context *ctx = dev_get_drvdata(dev);
struct drm_device *drm_dev = data;
+ struct exynos_drm_private *priv = drm_dev->dev_private;
int ret;
- ret = fimd_ctx_initialize(ctx, drm_dev);
- if (ret) {
- DRM_ERROR("fimd_ctx_initialize failed.\n");
- return ret;
- }
+ ctx->drm_dev = drm_dev;
+ ctx->pipe = priv->pipe++;
ctx->crtc = exynos_drm_crtc_create(drm_dev, ctx->pipe,
EXYNOS_DISPLAY_TYPE_LCD,
&fimd_crtc_ops, ctx);
- if (IS_ERR(ctx->crtc)) {
- fimd_ctx_remove(ctx);
- return PTR_ERR(ctx->crtc);
- }
if (ctx->display)
exynos_drm_create_enc_conn(drm_dev, ctx->display);
+ ret = fimd_iommu_attach_devices(ctx, drm_dev);
+ if (ret)
+ return ret;
+
return 0;
}
fimd_dpms(ctx->crtc, DRM_MODE_DPMS_OFF);
+ fimd_iommu_detach_devices(ctx);
+
if (ctx->display)
exynos_dpi_remove(ctx->display);
-
- fimd_ctx_remove(ctx);
}
static const struct component_ops fimd_component_ops = {
struct exynos_drm_plane *exynos_plane = to_exynos_plane(plane);
struct exynos_drm_crtc *exynos_crtc = to_exynos_crtc(plane->crtc);
- if (exynos_crtc->ops->win_disable)
+ if (exynos_crtc && exynos_crtc->ops->win_disable)
exynos_crtc->ops->win_disable(exynos_crtc,
exynos_plane->zpos);
unsigned int fb_x;
unsigned int fb_y;
unsigned int fb_width;
+ unsigned int fb_pitch;
unsigned int fb_height;
unsigned int src_width;
unsigned int src_height;
} else {
luma_addr[0] = win_data->dma_addr;
chroma_addr[0] = win_data->dma_addr
- + (win_data->fb_width * win_data->fb_height);
+ + (win_data->fb_pitch * win_data->fb_height);
}
if (win_data->scan_flags & DRM_MODE_FLAG_INTERLACE) {
luma_addr[1] = luma_addr[0] + 0x40;
chroma_addr[1] = chroma_addr[0] + 0x40;
} else {
- luma_addr[1] = luma_addr[0] + win_data->fb_width;
- chroma_addr[1] = chroma_addr[0] + win_data->fb_width;
+ luma_addr[1] = luma_addr[0] + win_data->fb_pitch;
+ chroma_addr[1] = chroma_addr[0] + win_data->fb_pitch;
}
} else {
ctx->interlace = false;
vp_reg_writemask(res, VP_MODE, val, VP_MODE_FMT_MASK);
/* setting size of input image */
- vp_reg_write(res, VP_IMG_SIZE_Y, VP_IMG_HSIZE(win_data->fb_width) |
+ vp_reg_write(res, VP_IMG_SIZE_Y, VP_IMG_HSIZE(win_data->fb_pitch) |
VP_IMG_VSIZE(win_data->fb_height));
/* chroma height has to reduced by 2 to avoid chroma distorions */
- vp_reg_write(res, VP_IMG_SIZE_C, VP_IMG_HSIZE(win_data->fb_width) |
+ vp_reg_write(res, VP_IMG_SIZE_C, VP_IMG_HSIZE(win_data->fb_pitch) |
VP_IMG_VSIZE(win_data->fb_height / 2));
vp_reg_write(res, VP_SRC_WIDTH, win_data->src_width);
/* converting dma address base and source offset */
dma_addr = win_data->dma_addr
+ (win_data->fb_x * win_data->bpp >> 3)
- + (win_data->fb_y * win_data->fb_width * win_data->bpp >> 3);
+ + (win_data->fb_y * win_data->fb_pitch);
src_x_offset = 0;
src_y_offset = 0;
MXR_GRP_CFG_FORMAT_VAL(fmt), MXR_GRP_CFG_FORMAT_MASK);
/* setup geometry */
- mixer_reg_write(res, MXR_GRAPHIC_SPAN(win), win_data->fb_width);
+ mixer_reg_write(res, MXR_GRAPHIC_SPAN(win),
+ win_data->fb_pitch / (win_data->bpp >> 3));
/* setup display size */
if (ctx->mxr_ver == MXR_VER_128_0_0_184 &&
win_data->fb_y = plane->fb_y;
win_data->fb_width = plane->fb_width;
win_data->fb_height = plane->fb_height;
+ win_data->fb_pitch = plane->pitch;
win_data->src_width = plane->src_width;
win_data->src_height = plane->src_height;
seq_puts(m, " (pp");
else
seq_puts(m, " (g");
- seq_printf(m, "gtt offset: %08lx, size: %08lx, type: %u)",
+ seq_printf(m, "gtt offset: %08llx, size: %08llx, type: %u)",
vma->node.start, vma->node.size,
vma->ggtt_view.type);
}
if (obj->stolen)
- seq_printf(m, " (stolen: %08lx)", obj->stolen->start);
+ seq_printf(m, " (stolen: %08llx)", obj->stolen->start);
if (obj->pin_mappable || obj->fault_mappable) {
char s[3], *t = s;
if (obj->pin_mappable)
return 0;
}
-static int i915_drm_suspend_late(struct drm_device *drm_dev)
+static int i915_drm_suspend_late(struct drm_device *drm_dev, bool hibernation)
{
struct drm_i915_private *dev_priv = drm_dev->dev_private;
int ret;
}
pci_disable_device(drm_dev->pdev);
- pci_set_power_state(drm_dev->pdev, PCI_D3hot);
+ /*
+ * During hibernation on some GEN4 platforms the BIOS may try to access
+ * the device even though it's already in D3 and hang the machine. So
+ * leave the device in D0 on those platforms and hope the BIOS will
+ * power down the device properly. Platforms where this was seen:
+ * Lenovo Thinkpad X301, X61s
+ */
+ if (!(hibernation &&
+ drm_dev->pdev->subsystem_vendor == PCI_VENDOR_ID_LENOVO &&
+ INTEL_INFO(dev_priv)->gen == 4))
+ pci_set_power_state(drm_dev->pdev, PCI_D3hot);
return 0;
}
if (error)
return error;
- return i915_drm_suspend_late(dev);
+ return i915_drm_suspend_late(dev, false);
}
static int i915_drm_resume(struct drm_device *dev)
if (drm_dev->switch_power_state == DRM_SWITCH_POWER_OFF)
return 0;
- return i915_drm_suspend_late(drm_dev);
+ return i915_drm_suspend_late(drm_dev, false);
+}
+
+static int i915_pm_poweroff_late(struct device *dev)
+{
+ struct drm_device *drm_dev = dev_to_i915(dev)->dev;
+
+ if (drm_dev->switch_power_state == DRM_SWITCH_POWER_OFF)
+ return 0;
+
+ return i915_drm_suspend_late(drm_dev, true);
}
static int i915_pm_resume_early(struct device *dev)
/* Gunit-Display CZ domain, 0x182028-0x1821CF */
s->gu_ctl0 = I915_READ(VLV_GU_CTL0);
s->gu_ctl1 = I915_READ(VLV_GU_CTL1);
+ s->pcbr = I915_READ(VLV_PCBR);
s->clock_gate_dis2 = I915_READ(VLV_GUNIT_CLOCK_GATE2);
/*
/* Gunit-Display CZ domain, 0x182028-0x1821CF */
I915_WRITE(VLV_GU_CTL0, s->gu_ctl0);
I915_WRITE(VLV_GU_CTL1, s->gu_ctl1);
+ I915_WRITE(VLV_PCBR, s->pcbr);
I915_WRITE(VLV_GUNIT_CLOCK_GATE2, s->clock_gate_dis2);
}
u32 val;
int err;
- val = I915_READ(VLV_GTLC_SURVIVABILITY_REG);
- WARN_ON(!!(val & VLV_GFX_CLK_FORCE_ON_BIT) == force_on);
-
#define COND (I915_READ(VLV_GTLC_SURVIVABILITY_REG) & VLV_GFX_CLK_STATUS_BIT)
- /* Wait for a previous force-off to settle */
- if (force_on) {
- err = wait_for(!COND, 20);
- if (err) {
- DRM_ERROR("timeout waiting for GFX clock force-off (%08x)\n",
- I915_READ(VLV_GTLC_SURVIVABILITY_REG));
- return err;
- }
- }
val = I915_READ(VLV_GTLC_SURVIVABILITY_REG);
val &= ~VLV_GFX_CLK_FORCE_ON_BIT;
.thaw_early = i915_pm_resume_early,
.thaw = i915_pm_resume,
.poweroff = i915_pm_suspend,
- .poweroff_late = i915_pm_suspend_late,
+ .poweroff_late = i915_pm_poweroff_late,
.restore_early = i915_pm_resume_early,
.restore = i915_pm_resume,
/* Display 2 CZ domain */
u32 gu_ctl0;
u32 gu_ctl1;
+ u32 pcbr;
u32 clock_gate_dis2;
};
* number comparisons on buffer last_read|write_seqno. It also allows an
* emission time to be associated with the request for tracking how far ahead
* of the GPU the submission is.
+ *
+ * The requests are reference counted, so upon creation they should have an
+ * initial reference taken using kref_init
*/
struct drm_i915_gem_request {
struct kref ref;
/** Position in the ringbuffer of the end of the whole request */
u32 tail;
- /** Context related to this request */
+ /**
+ * Context related to this request
+ * Contexts are refcounted, so when this request is associated with a
+ * context, we must increment the context's refcount, to guarantee that
+ * it persists while any request is linked to it. Requests themselves
+ * are also refcounted, so the request will only be freed when the last
+ * reference to it is dismissed, and the code in
+ * i915_gem_request_free() will then decrement the refcount on the
+ * context.
+ */
struct intel_context *ctx;
/** Batch buffer related to this request if any */
(INTEL_DEVID(dev) & 0xFF00) == 0x0C00)
#define IS_BDW_ULT(dev) (IS_BROADWELL(dev) && \
((INTEL_DEVID(dev) & 0xf) == 0x6 || \
+ (INTEL_DEVID(dev) & 0xf) == 0xb || \
(INTEL_DEVID(dev) & 0xf) == 0xe))
#define IS_BDW_GT3(dev) (IS_BROADWELL(dev) && \
(INTEL_DEVID(dev) & 0x00F0) == 0x0020)
if (submit_req->ctx != ring->default_context)
intel_lr_context_unpin(ring, submit_req->ctx);
- i915_gem_context_unreference(submit_req->ctx);
- kfree(submit_req);
+ i915_gem_request_unreference(submit_req);
}
/*
WARN_ON(i915_verify_lists(ring->dev));
- /* Move any buffers on the active list that are no longer referenced
- * by the ringbuffer to the flushing/inactive lists as appropriate,
- * before we free the context associated with the requests.
+ /* Retire requests first as we use it above for the early return.
+ * If we retire requests last, we may use a later seqno and so clear
+ * the requests lists without clearing the active list, leading to
+ * confusion.
*/
- while (!list_empty(&ring->active_list)) {
- struct drm_i915_gem_object *obj;
-
- obj = list_first_entry(&ring->active_list,
- struct drm_i915_gem_object,
- ring_list);
-
- if (!i915_gem_request_completed(obj->last_read_req, true))
- break;
-
- i915_gem_object_move_to_inactive(obj);
- }
-
-
while (!list_empty(&ring->request_list)) {
struct drm_i915_gem_request *request;
struct intel_ringbuffer *ringbuf;
i915_gem_free_request(request);
}
+ /* Move any buffers on the active list that are no longer referenced
+ * by the ringbuffer to the flushing/inactive lists as appropriate,
+ * before we free the context associated with the requests.
+ */
+ while (!list_empty(&ring->active_list)) {
+ struct drm_i915_gem_object *obj;
+
+ obj = list_first_entry(&ring->active_list,
+ struct drm_i915_gem_object,
+ ring_list);
+
+ if (!i915_gem_request_completed(obj->last_read_req, true))
+ break;
+
+ i915_gem_object_move_to_inactive(obj);
+ }
+
if (unlikely(ring->trace_irq_req &&
i915_gem_request_completed(ring->trace_irq_req, true))) {
ring->irq_put(ring);
req = obj->last_read_req;
/* Do this after OLR check to make sure we make forward progress polling
- * on this IOCTL with a timeout <=0 (like busy ioctl)
+ * on this IOCTL with a timeout == 0 (like busy ioctl)
*/
- if (args->timeout_ns <= 0) {
+ if (args->timeout_ns == 0) {
ret = -ETIME;
goto out;
}
i915_gem_request_reference(req);
mutex_unlock(&dev->struct_mutex);
- ret = __i915_wait_request(req, reset_counter, true, &args->timeout_ns,
+ ret = __i915_wait_request(req, reset_counter, true,
+ args->timeout_ns > 0 ? &args->timeout_ns : NULL,
file->driver_priv);
mutex_lock(&dev->struct_mutex);
i915_gem_request_unreference(req);
if (INTEL_INFO(dev)->gen < 6 && !intel_enable_gtt())
return -EIO;
+ /* Double layer security blanket, see i915_gem_init() */
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+
if (dev_priv->ellc_size)
I915_WRITE(HSW_IDICR, I915_READ(HSW_IDICR) | IDIHASHMSK(0xf));
for_each_ring(ring, dev_priv, i) {
ret = ring->init_hw(ring);
if (ret)
- return ret;
+ goto out;
}
for (i = 0; i < NUM_L3_SLICES(dev); i++)
DRM_ERROR("Context enable failed %d\n", ret);
i915_gem_cleanup_ringbuffer(dev);
- return ret;
+ goto out;
}
+out:
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
return ret;
}
dev_priv->gt.stop_ring = intel_logical_ring_stop;
}
+ /* This is just a security blanket to placate dragons.
+ * On some systems, we very sporadically observe that the first TLBs
+ * used by the CS may be stale, despite us poking the TLB reset. If
+ * we hold the forcewake during initialisation these problems
+ * just magically go away.
+ */
+ intel_uncore_forcewake_get(dev_priv, FORCEWAKE_ALL);
+
ret = i915_gem_init_userptr(dev);
if (ret)
goto out_unlock;
}
out_unlock:
+ intel_uncore_forcewake_put(dev_priv, FORCEWAKE_ALL);
mutex_unlock(&dev->struct_mutex);
return ret;
goto err;
}
- if (i915_needs_cmd_parser(ring)) {
+ if (i915_needs_cmd_parser(ring) && args->batch_len) {
batch_obj = i915_gem_execbuffer_parse(ring,
&shadow_exec_entry,
eb,
ppgtt->base.clear_range(&ppgtt->base, 0, ppgtt->base.total, true);
- DRM_DEBUG_DRIVER("Allocated pde space (%ldM) at GTT entry: %lx\n",
+ DRM_DEBUG_DRIVER("Allocated pde space (%lldM) at GTT entry: %llx\n",
ppgtt->node.size >> 20,
ppgtt->node.start / PAGE_SIZE);
static void i915_gtt_color_adjust(struct drm_mm_node *node,
unsigned long color,
- unsigned long *start,
- unsigned long *end)
+ u64 *start,
+ u64 *end)
{
if (node->color != color)
*start += 4096;
stolen_offset, gtt_offset, size);
/* KISS and expect everything to be page-aligned */
- BUG_ON(stolen_offset & 4095);
- BUG_ON(size & 4095);
-
- if (WARN_ON(size == 0))
+ if (WARN_ON(size == 0) || WARN_ON(size & 4095) ||
+ WARN_ON(stolen_offset & 4095))
return NULL;
stolen = kzalloc(sizeof(*stolen), GFP_KERNEL);
return -EINVAL;
}
+ mutex_lock(&dev->struct_mutex);
if (i915_gem_obj_is_pinned(obj) || obj->framebuffer_references) {
- drm_gem_object_unreference_unlocked(&obj->base);
- return -EBUSY;
+ ret = -EBUSY;
+ goto err;
}
if (args->tiling_mode == I915_TILING_NONE) {
}
}
- mutex_lock(&dev->struct_mutex);
if (args->tiling_mode != obj->tiling_mode ||
args->stride != obj->stride) {
/* We need to rebind the object if its current allocation
obj->bit_17 = NULL;
}
+err:
drm_gem_object_unreference(&obj->base);
mutex_unlock(&dev->struct_mutex);
u32 iir, gt_iir, pm_iir;
irqreturn_t ret = IRQ_NONE;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
while (true) {
/* Find, clear, then process each source of interrupt */
u32 master_ctl, iir;
irqreturn_t ret = IRQ_NONE;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
for (;;) {
master_ctl = I915_READ(GEN8_MASTER_IRQ) & ~GEN8_MASTER_IRQ_CONTROL;
iir = I915_READ(VLV_IIR);
u32 de_iir, gt_iir, de_ier, sde_ier = 0;
irqreturn_t ret = IRQ_NONE;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
/* We get interrupts on unclaimed registers, so check for this before we
* do any I915_{READ,WRITE}. */
intel_uncore_check_errors(dev);
enum pipe pipe;
u32 aux_mask = GEN8_AUX_CHANNEL_A;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
if (IS_GEN9(dev))
aux_mask |= GEN9_AUX_CHANNEL_B | GEN9_AUX_CHANNEL_C |
GEN9_AUX_CHANNEL_D;
I915_DISPLAY_PLANE_A_FLIP_PENDING_INTERRUPT |
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
iir = I915_READ16(IIR);
if (iir == 0)
return IRQ_NONE;
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
int pipe, ret = IRQ_NONE;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
iir = I915_READ(IIR);
do {
bool irq_received = (iir & ~flip_mask) != 0;
I915_DISPLAY_PLANE_A_FLIP_PENDING_INTERRUPT |
I915_DISPLAY_PLANE_B_FLIP_PENDING_INTERRUPT;
+ if (!intel_irqs_enabled(dev_priv))
+ return IRQ_NONE;
+
iir = I915_READ(IIR);
for (;;) {
{
dev_priv->dev->driver->irq_uninstall(dev_priv->dev);
dev_priv->pm.irqs_enabled = false;
+ synchronize_irq(dev_priv->dev->irq);
}
/**
#include <drm/i915_drm.h>
#include "i915_drv.h"
#include "i915_trace.h"
+#include <drm/drm_atomic.h>
#include <drm/drm_atomic_helper.h>
#include <drm/drm_dp_helper.h>
#include <drm/drm_crtc_helper.h>
struct drm_device *dev = crtc->base.dev;
struct drm_i915_gem_object *obj = NULL;
struct drm_mode_fb_cmd2 mode_cmd = { 0 };
- u32 base = plane_config->base;
+ u32 base_aligned = round_down(plane_config->base, PAGE_SIZE);
+ u32 size_aligned = round_up(plane_config->base + plane_config->size,
+ PAGE_SIZE);
+
+ size_aligned -= base_aligned;
if (plane_config->size == 0)
return false;
- obj = i915_gem_object_create_stolen_for_preallocated(dev, base, base,
- plane_config->size);
+ obj = i915_gem_object_create_stolen_for_preallocated(dev,
+ base_aligned,
+ base_aligned,
+ size_aligned);
if (!obj)
return false;
return false;
}
+/* Update plane->state->fb to match plane->fb after driver-internal updates */
+static void
+update_state_fb(struct drm_plane *plane)
+{
+ if (plane->fb != plane->state->fb)
+ drm_atomic_set_fb_for_plane(plane->state, plane->fb);
+}
+
static void
intel_find_plane_obj(struct intel_crtc *intel_crtc,
struct intel_initial_plane_config *plane_config)
if (!intel_crtc->base.primary->fb)
return;
- if (intel_alloc_plane_obj(intel_crtc, plane_config))
+ if (intel_alloc_plane_obj(intel_crtc, plane_config)) {
+ struct drm_plane *primary = intel_crtc->base.primary;
+
+ primary->state->crtc = &intel_crtc->base;
+ primary->crtc = &intel_crtc->base;
+ update_state_fb(primary);
+
return;
+ }
kfree(intel_crtc->base.primary->fb);
intel_crtc->base.primary->fb = NULL;
continue;
if (i915_gem_obj_ggtt_offset(obj) == plane_config->base) {
+ struct drm_plane *primary = intel_crtc->base.primary;
+
if (obj->tiling_mode != I915_TILING_NONE)
dev_priv->preserve_bios_swizzle = true;
drm_framebuffer_reference(c->primary->fb);
- intel_crtc->base.primary->fb = c->primary->fb;
+ primary->fb = c->primary->fb;
+ primary->state->crtc = &intel_crtc->base;
+ primary->crtc = &intel_crtc->base;
obj->frontbuffer_bits |= INTEL_FRONTBUFFER_PRIMARY(intel_crtc->pipe);
break;
}
}
+
+ update_state_fb(intel_crtc->base.primary);
}
static void i9xx_update_primary_plane(struct drm_crtc *crtc,
case DRM_FORMAT_XRGB8888:
plane_ctl |= PLANE_CTL_FORMAT_XRGB_8888;
break;
+ case DRM_FORMAT_ARGB8888:
+ plane_ctl |= PLANE_CTL_FORMAT_XRGB_8888;
+ plane_ctl |= PLANE_CTL_ALPHA_SW_PREMULTIPLY;
+ break;
case DRM_FORMAT_XBGR8888:
plane_ctl |= PLANE_CTL_ORDER_RGBX;
plane_ctl |= PLANE_CTL_FORMAT_XRGB_8888;
break;
+ case DRM_FORMAT_ABGR8888:
+ plane_ctl |= PLANE_CTL_ORDER_RGBX;
+ plane_ctl |= PLANE_CTL_FORMAT_XRGB_8888;
+ plane_ctl |= PLANE_CTL_ALPHA_SW_PREMULTIPLY;
+ break;
case DRM_FORMAT_XRGB2101010:
plane_ctl |= PLANE_CTL_FORMAT_XRGB_2101010;
break;
struct drm_framebuffer *fb;
struct intel_framebuffer *intel_fb;
+ val = I915_READ(DSPCNTR(plane));
+ if (!(val & DISPLAY_PLANE_ENABLE))
+ return;
+
intel_fb = kzalloc(sizeof(*intel_fb), GFP_KERNEL);
if (!intel_fb) {
DRM_DEBUG_KMS("failed to alloc fb\n");
fb = &intel_fb->base;
- val = I915_READ(DSPCNTR(plane));
-
if (INTEL_INFO(dev)->gen >= 4)
if (val & DISPPLANE_TILED)
plane_config->tiling = I915_TILING_X;
aligned_height = intel_fb_align_height(dev, fb->height,
plane_config->tiling);
- plane_config->size = PAGE_ALIGN(fb->pitches[0] * aligned_height);
+ plane_config->size = fb->pitches[0] * aligned_height;
DRM_DEBUG_KMS("pipe/plane %c/%d with fb: size=%dx%d@%d, offset=%x, pitch %d, size 0x%x\n",
pipe_name(pipe), plane, fb->width, fb->height,
fb = &intel_fb->base;
val = I915_READ(PLANE_CTL(pipe, 0));
+ if (!(val & PLANE_CTL_ENABLE))
+ goto error;
+
if (val & PLANE_CTL_TILED_MASK)
plane_config->tiling = I915_TILING_X;
aligned_height = intel_fb_align_height(dev, fb->height,
plane_config->tiling);
- plane_config->size = ALIGN(fb->pitches[0] * aligned_height, PAGE_SIZE);
+ plane_config->size = fb->pitches[0] * aligned_height;
DRM_DEBUG_KMS("pipe %c with fb: size=%dx%d@%d, offset=%x, pitch %d, size 0x%x\n",
pipe_name(pipe), fb->width, fb->height,
struct drm_framebuffer *fb;
struct intel_framebuffer *intel_fb;
+ val = I915_READ(DSPCNTR(pipe));
+ if (!(val & DISPLAY_PLANE_ENABLE))
+ return;
+
intel_fb = kzalloc(sizeof(*intel_fb), GFP_KERNEL);
if (!intel_fb) {
DRM_DEBUG_KMS("failed to alloc fb\n");
fb = &intel_fb->base;
- val = I915_READ(DSPCNTR(pipe));
-
if (INTEL_INFO(dev)->gen >= 4)
if (val & DISPPLANE_TILED)
plane_config->tiling = I915_TILING_X;
aligned_height = intel_fb_align_height(dev, fb->height,
plane_config->tiling);
- plane_config->size = PAGE_ALIGN(fb->pitches[0] * aligned_height);
+ plane_config->size = fb->pitches[0] * aligned_height;
DRM_DEBUG_KMS("pipe %c with fb: size=%dx%d@%d, offset=%x, pitch %d, size 0x%x\n",
pipe_name(pipe), fb->width, fb->height,
old->release_fb->funcs->destroy(old->release_fb);
goto fail;
}
+ crtc->primary->crtc = crtc;
/* let the connector get through one full cycle before testing */
intel_wait_for_vblank(dev, intel_crtc->pipe);
struct drm_crtc *crtc = dev_priv->pipe_to_crtc_mapping[pipe];
struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
- WARN_ON(!in_irq());
+ WARN_ON(!in_interrupt());
if (crtc == NULL)
return;
drm_gem_object_reference(&obj->base);
crtc->primary->fb = fb;
+ update_state_fb(crtc->primary);
work->pending_flip_obj = obj;
cleanup_pending:
atomic_dec(&intel_crtc->unpin_work_count);
crtc->primary->fb = old_fb;
+ update_state_fb(crtc->primary);
drm_gem_object_unreference(&work->old_fb_obj->base);
drm_gem_object_unreference(&obj->base);
mutex_unlock(&dev->struct_mutex);
return -ENOMEM;
}
- if (fb == crtc->cursor->fb)
- return 0;
-
/* we only need to pin inside GTT if cursor is non-phy */
mutex_lock(&dev->struct_mutex);
if (!INTEL_INFO(dev)->cursor_needs_physical && obj->tiling_mode) {
/* HP Chromebook 14 (Celeron 2955U) */
{ 0x0a06, 0x103c, 0x21ed, quirk_backlight_present },
+
+ /* Dell Chromebook 11 */
+ { 0x0a06, 0x1028, 0x0a35, quirk_backlight_present },
};
static void intel_init_quirks(struct drm_device *dev)
to_intel_crtc(c)->pipe);
drm_framebuffer_unreference(c->primary->fb);
c->primary->fb = NULL;
+ update_state_fb(c->primary);
}
}
mutex_unlock(&dev->struct_mutex);
return ret;
}
-static bool
-__cpu_fifo_underrun_reporting_enabled(struct drm_i915_private *dev_priv,
- enum pipe pipe)
-{
- struct drm_crtc *crtc = dev_priv->pipe_to_crtc_mapping[pipe];
- struct intel_crtc *intel_crtc = to_intel_crtc(crtc);
-
- return !intel_crtc->cpu_fifo_underrun_disabled;
-}
-
/**
* intel_set_pch_fifo_underrun_reporting - set PCH fifo underrun reporting state
* @dev_priv: i915 device instance
void intel_cpu_fifo_underrun_irq_handler(struct drm_i915_private *dev_priv,
enum pipe pipe)
{
+ struct drm_crtc *crtc = dev_priv->pipe_to_crtc_mapping[pipe];
+
+ /* We may be called too early in init, thanks BIOS! */
+ if (crtc == NULL)
+ return;
+
/* GMCH can't disable fifo underruns, filter them. */
if (HAS_GMCH_DISPLAY(dev_priv->dev) &&
- !__cpu_fifo_underrun_reporting_enabled(dev_priv, pipe))
+ to_intel_crtc(crtc)->cpu_fifo_underrun_disabled)
return;
if (intel_set_cpu_fifo_underrun_reporting(dev_priv, pipe, false))
* If there isn't a request associated with this submission,
* create one as a temporary holder.
*/
- WARN(1, "execlist context submission without request");
request = kzalloc(sizeof(*request), GFP_KERNEL);
if (request == NULL)
return -ENOMEM;
request->ring = ring;
request->ctx = to;
+ kref_init(&request->ref);
+ request->uniq = dev_priv->request_uniq++;
+ i915_gem_context_reference(request->ctx);
} else {
+ i915_gem_request_reference(request);
WARN_ON(to != request->ctx);
}
request->tail = tail;
- i915_gem_request_reference(request);
- i915_gem_context_reference(request->ctx);
intel_runtime_pm_get(dev_priv);
if (ctx_obj && (ctx != ring->default_context))
intel_lr_context_unpin(ring, ctx);
intel_runtime_pm_put(dev_priv);
- i915_gem_context_unreference(ctx);
list_del(&req->execlist_link);
i915_gem_request_unreference(req);
}
drm_modeset_lock_all(dev);
plane = drm_plane_find(dev, set->plane_id);
- if (!plane) {
+ if (!plane || plane->type != DRM_PLANE_TYPE_OVERLAY) {
ret = -ENOENT;
goto out_unlock;
}
drm_modeset_lock_all(dev);
plane = drm_plane_find(dev, get->plane_id);
- if (!plane) {
+ if (!plane || plane->type != DRM_PLANE_TYPE_OVERLAY) {
ret = -ENOENT;
goto out_unlock;
}
/* We need to init first for ECOBUS access and then
* determine later if we want to reinit, in case of MT access is
- * not working
+ * not working. In this stage we don't know which flavour this
+ * ivb is, so it is better to reset also the gen6 fw registers
+ * before the ecobus check.
*/
+
+ __raw_i915_write32(dev_priv, FORCEWAKE, 0);
+ __raw_posting_read(dev_priv, ECOBUS);
+
fw_domain_init(dev_priv, FW_DOMAIN_ID_RENDER,
FORCEWAKE_MT, FORCEWAKE_MT_ACK);
118800000, { 0x091c, 0x091c, 0x06dc },
}, {
216000000, { 0x06dc, 0x0b5c, 0x091c },
- }
+ }, {
+ ~0UL, { 0x0000, 0x0000, 0x0000 },
+ },
};
static const struct dw_hdmi_sym_term imx_sym_term[] = {
.destroy = drm_encoder_cleanup,
};
+static enum drm_mode_status imx6q_hdmi_mode_valid(struct drm_connector *con,
+ struct drm_display_mode *mode)
+{
+ if (mode->clock < 13500)
+ return MODE_CLOCK_LOW;
+ if (mode->clock > 266000)
+ return MODE_CLOCK_HIGH;
+
+ return MODE_OK;
+}
+
+static enum drm_mode_status imx6dl_hdmi_mode_valid(struct drm_connector *con,
+ struct drm_display_mode *mode)
+{
+ if (mode->clock < 13500)
+ return MODE_CLOCK_LOW;
+ if (mode->clock > 270000)
+ return MODE_CLOCK_HIGH;
+
+ return MODE_OK;
+}
+
static struct dw_hdmi_plat_data imx6q_hdmi_drv_data = {
- .mpll_cfg = imx_mpll_cfg,
- .cur_ctr = imx_cur_ctr,
- .sym_term = imx_sym_term,
- .dev_type = IMX6Q_HDMI,
+ .mpll_cfg = imx_mpll_cfg,
+ .cur_ctr = imx_cur_ctr,
+ .sym_term = imx_sym_term,
+ .dev_type = IMX6Q_HDMI,
+ .mode_valid = imx6q_hdmi_mode_valid,
};
static struct dw_hdmi_plat_data imx6dl_hdmi_drv_data = {
.cur_ctr = imx_cur_ctr,
.sym_term = imx_sym_term,
.dev_type = IMX6DL_HDMI,
+ .mode_valid = imx6dl_hdmi_mode_valid,
};
static const struct of_device_id dw_hdmi_imx_dt_ids[] = {
{
struct imx_ldb_channel *imx_ldb_ch = enc_to_imx_ldb_ch(encoder);
struct imx_ldb *ldb = imx_ldb_ch->ldb;
- struct drm_display_mode *mode = &encoder->crtc->hwmode;
u32 pixel_fmt;
- unsigned long serial_clk;
- unsigned long di_clk = mode->clock * 1000;
- int mux = imx_drm_encoder_get_mux_id(imx_ldb_ch->child, encoder);
-
- if (ldb->ldb_ctrl & LDB_SPLIT_MODE_EN) {
- /* dual channel LVDS mode */
- serial_clk = 3500UL * mode->clock;
- imx_ldb_set_clock(ldb, mux, 0, serial_clk, di_clk);
- imx_ldb_set_clock(ldb, mux, 1, serial_clk, di_clk);
- } else {
- serial_clk = 7000UL * mode->clock;
- imx_ldb_set_clock(ldb, mux, imx_ldb_ch->chno, serial_clk,
- di_clk);
- }
switch (imx_ldb_ch->chno) {
case 0:
struct imx_ldb_channel *imx_ldb_ch = enc_to_imx_ldb_ch(encoder);
struct imx_ldb *ldb = imx_ldb_ch->ldb;
int dual = ldb->ldb_ctrl & LDB_SPLIT_MODE_EN;
+ unsigned long serial_clk;
+ unsigned long di_clk = mode->clock * 1000;
+ int mux = imx_drm_encoder_get_mux_id(imx_ldb_ch->child, encoder);
if (mode->clock > 170000) {
dev_warn(ldb->dev,
"%s: mode exceeds 85 MHz pixel clock\n", __func__);
}
+ if (dual) {
+ serial_clk = 3500UL * mode->clock;
+ imx_ldb_set_clock(ldb, mux, 0, serial_clk, di_clk);
+ imx_ldb_set_clock(ldb, mux, 1, serial_clk, di_clk);
+ } else {
+ serial_clk = 7000UL * mode->clock;
+ imx_ldb_set_clock(ldb, mux, imx_ldb_ch->chno, serial_clk,
+ di_clk);
+ }
+
/* FIXME - assumes straight connections DI0 --> CH0, DI1 --> CH1 */
if (imx_ldb_ch == &ldb->channel[0]) {
if (mode->flags & DRM_MODE_FLAG_NVSYNC)
}
panel_node = of_parse_phandle(np, "fsl,panel", 0);
- if (panel_node)
+ if (panel_node) {
imxpd->panel = of_drm_find_panel(panel_node);
+ if (!imxpd->panel)
+ return -EPROBE_DEFER;
+ }
imxpd->dev = dev;
goto vdda_set_fail;
}
- ret = regulator_set_optimum_mode(ctrl->vdda_vreg, VDDA_UA_ON_LOAD);
+ ret = regulator_set_load(ctrl->vdda_vreg, VDDA_UA_ON_LOAD);
if (ret < 0) {
pr_err("%s: vdda_vreg set regulator mode failed.\n", __func__);
goto vdda_set_fail;
lvl_enable_fail:
regulator_disable(ctrl->vdda_vreg);
vdda_enable_fail:
- regulator_set_optimum_mode(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
+ regulator_set_load(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
vdda_set_fail:
return ret;
}
{
regulator_disable(ctrl->lvl_vreg);
regulator_disable(ctrl->vdda_vreg);
- regulator_set_optimum_mode(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
+ regulator_set_load(ctrl->vdda_vreg, VDDA_UA_OFF_LOAD);
}
static int edp_gpio_config(struct edp_ctrl *ctrl)
void mdp4_irq_preinstall(struct msm_kms *kms)
{
struct mdp4_kms *mdp4_kms = to_mdp4_kms(to_mdp_kms(kms));
+ mdp4_enable(mdp4_kms);
mdp4_write(mdp4_kms, REG_MDP4_INTR_CLEAR, 0xffffffff);
+ mdp4_write(mdp4_kms, REG_MDP4_INTR_ENABLE, 0x00000000);
+ mdp4_disable(mdp4_kms);
}
int mdp4_irq_postinstall(struct msm_kms *kms)
void mdp4_irq_uninstall(struct msm_kms *kms)
{
struct mdp4_kms *mdp4_kms = to_mdp4_kms(to_mdp_kms(kms));
+ mdp4_enable(mdp4_kms);
mdp4_write(mdp4_kms, REG_MDP4_INTR_ENABLE, 0x00000000);
+ mdp4_disable(mdp4_kms);
}
irqreturn_t mdp4_irq(struct msm_kms *kms)
git clone https://github.com/freedreno/envytools.git
The rules-ng-ng source files this header was generated from are:
-- /home/robclark/src/freedreno/envytools/rnndb/msm.xml ( 676 bytes, from 2014-12-05 15:34:49)
-- /home/robclark/src/freedreno/envytools/rnndb/freedreno_copyright.xml ( 1453 bytes, from 2013-03-31 16:51:27)
-- /home/robclark/src/freedreno/envytools/rnndb/mdp/mdp4.xml ( 20908 bytes, from 2014-12-08 16:13:00)
-- /home/robclark/src/freedreno/envytools/rnndb/mdp/mdp_common.xml ( 2357 bytes, from 2014-12-08 16:13:00)
-- /home/robclark/src/freedreno/envytools/rnndb/mdp/mdp5.xml ( 27208 bytes, from 2015-01-13 23:56:11)
-- /home/robclark/src/freedreno/envytools/rnndb/dsi/dsi.xml ( 11712 bytes, from 2013-08-17 17:13:43)
-- /home/robclark/src/freedreno/envytools/rnndb/dsi/sfpb.xml ( 344 bytes, from 2013-08-11 19:26:32)
-- /home/robclark/src/freedreno/envytools/rnndb/dsi/mmss_cc.xml ( 1686 bytes, from 2014-10-31 16:48:57)
-- /home/robclark/src/freedreno/envytools/rnndb/hdmi/qfprom.xml ( 600 bytes, from 2013-07-05 19:21:12)
-- /home/robclark/src/freedreno/envytools/rnndb/hdmi/hdmi.xml ( 26848 bytes, from 2015-01-13 23:55:57)
-- /home/robclark/src/freedreno/envytools/rnndb/edp/edp.xml ( 8253 bytes, from 2014-12-08 16:13:00)
+- /local/mnt2/workspace2/sviau/envytools/rnndb/mdp/mdp5.xml ( 27229 bytes, from 2015-02-10 17:00:41)
+- /local/mnt2/workspace2/sviau/envytools/rnndb/freedreno_copyright.xml ( 1453 bytes, from 2014-06-02 18:31:15)
+- /local/mnt2/workspace2/sviau/envytools/rnndb/mdp/mdp_common.xml ( 2357 bytes, from 2015-01-23 16:20:19)
Copyright (C) 2013-2015 by the following authors:
- Rob Clark <robdclark@gmail.com> (robclark)
case 2: return (mdp5_cfg->lm.base[2]);
case 3: return (mdp5_cfg->lm.base[3]);
case 4: return (mdp5_cfg->lm.base[4]);
+ case 5: return (mdp5_cfg->lm.base[5]);
default: return INVALID_IDX(idx);
}
}
/* current cursor being scanned out: */
struct drm_gem_object *scanout_bo;
- uint32_t width;
- uint32_t height;
+ uint32_t width, height;
+ uint32_t x, y;
} cursor;
};
#define to_mdp5_crtc(x) container_of(x, struct mdp5_crtc, base)
struct drm_plane *plane;
uint32_t flush_mask = 0;
- /* we could have already released CTL in the disable path: */
- if (!mdp5_crtc->ctl)
+ /* this should not happen: */
+ if (WARN_ON(!mdp5_crtc->ctl))
return;
drm_atomic_crtc_for_each_plane(plane, crtc) {
drm_atomic_crtc_for_each_plane(plane, crtc) {
mdp5_plane_complete_flip(plane);
}
+
+ if (mdp5_crtc->ctl && !crtc->state->enable) {
+ mdp5_ctl_release(mdp5_crtc->ctl);
+ mdp5_crtc->ctl = NULL;
+ }
}
static void unref_cursor_worker(struct drm_flip_work *work, void *val)
mdp5_crtc->event = crtc->state->event;
spin_unlock_irqrestore(&dev->event_lock, flags);
+ /*
+ * If no CTL has been allocated in mdp5_crtc_atomic_check(),
+ * it means we are trying to flush a CRTC whose state is disabled:
+ * nothing else needs to be done.
+ */
+ if (unlikely(!mdp5_crtc->ctl))
+ return;
+
blend_setup(crtc);
crtc_flush_all(crtc);
request_pending(crtc, PENDING_FLIP);
-
- if (mdp5_crtc->ctl && !crtc->state->enable) {
- mdp5_ctl_release(mdp5_crtc->ctl);
- mdp5_crtc->ctl = NULL;
- }
}
static int mdp5_crtc_set_property(struct drm_crtc *crtc,
return -EINVAL;
}
+static void get_roi(struct drm_crtc *crtc, uint32_t *roi_w, uint32_t *roi_h)
+{
+ struct mdp5_crtc *mdp5_crtc = to_mdp5_crtc(crtc);
+ uint32_t xres = crtc->mode.hdisplay;
+ uint32_t yres = crtc->mode.vdisplay;
+
+ /*
+ * Cursor Region Of Interest (ROI) is a plane read from cursor
+ * buffer to render. The ROI region is determined by the visibility of
+ * the cursor point. In the default Cursor image the cursor point will
+ * be at the top left of the cursor image, unless it is specified
+ * otherwise using hotspot feature.
+ *
+ * If the cursor point reaches the right (xres - x < cursor.width) or
+ * bottom (yres - y < cursor.height) boundary of the screen, then ROI
+ * width and ROI height need to be evaluated to crop the cursor image
+ * accordingly.
+ * (xres-x) will be new cursor width when x > (xres - cursor.width)
+ * (yres-y) will be new cursor height when y > (yres - cursor.height)
+ */
+ *roi_w = min(mdp5_crtc->cursor.width, xres -
+ mdp5_crtc->cursor.x);
+ *roi_h = min(mdp5_crtc->cursor.height, yres -
+ mdp5_crtc->cursor.y);
+}
+
static int mdp5_crtc_cursor_set(struct drm_crtc *crtc,
struct drm_file *file, uint32_t handle,
uint32_t width, uint32_t height)
unsigned int depth;
enum mdp5_cursor_alpha cur_alpha = CURSOR_ALPHA_PER_PIXEL;
uint32_t flush_mask = mdp_ctl_flush_mask_cursor(0);
+ uint32_t roi_w, roi_h;
unsigned long flags;
if ((width > CURSOR_WIDTH) || (height > CURSOR_HEIGHT)) {
spin_lock_irqsave(&mdp5_crtc->cursor.lock, flags);
old_bo = mdp5_crtc->cursor.scanout_bo;
+ mdp5_crtc->cursor.scanout_bo = cursor_bo;
+ mdp5_crtc->cursor.width = width;
+ mdp5_crtc->cursor.height = height;
+
+ get_roi(crtc, &roi_w, &roi_h);
+
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_STRIDE(lm), stride);
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_FORMAT(lm),
MDP5_LM_CURSOR_FORMAT_FORMAT(CURSOR_FMT_ARGB8888));
MDP5_LM_CURSOR_IMG_SIZE_SRC_H(height) |
MDP5_LM_CURSOR_IMG_SIZE_SRC_W(width));
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_SIZE(lm),
- MDP5_LM_CURSOR_SIZE_ROI_H(height) |
- MDP5_LM_CURSOR_SIZE_ROI_W(width));
+ MDP5_LM_CURSOR_SIZE_ROI_H(roi_h) |
+ MDP5_LM_CURSOR_SIZE_ROI_W(roi_w));
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_BASE_ADDR(lm), cursor_addr);
-
blendcfg = MDP5_LM_CURSOR_BLEND_CONFIG_BLEND_EN;
- blendcfg |= MDP5_LM_CURSOR_BLEND_CONFIG_BLEND_TRANSP_EN;
blendcfg |= MDP5_LM_CURSOR_BLEND_CONFIG_BLEND_ALPHA_SEL(cur_alpha);
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_BLEND_CONFIG(lm), blendcfg);
- mdp5_crtc->cursor.scanout_bo = cursor_bo;
- mdp5_crtc->cursor.width = width;
- mdp5_crtc->cursor.height = height;
spin_unlock_irqrestore(&mdp5_crtc->cursor.lock, flags);
ret = mdp5_ctl_set_cursor(mdp5_crtc->ctl, true);
struct mdp5_kms *mdp5_kms = get_kms(crtc);
struct mdp5_crtc *mdp5_crtc = to_mdp5_crtc(crtc);
uint32_t flush_mask = mdp_ctl_flush_mask_cursor(0);
- uint32_t xres = crtc->mode.hdisplay;
- uint32_t yres = crtc->mode.vdisplay;
uint32_t roi_w;
uint32_t roi_h;
unsigned long flags;
- x = (x > 0) ? x : 0;
- y = (y > 0) ? y : 0;
+ /* In case the CRTC is disabled, just drop the cursor update */
+ if (unlikely(!crtc->state->enable))
+ return 0;
- /*
- * Cursor Region Of Interest (ROI) is a plane read from cursor
- * buffer to render. The ROI region is determined by the visiblity of
- * the cursor point. In the default Cursor image the cursor point will
- * be at the top left of the cursor image, unless it is specified
- * otherwise using hotspot feature.
- *
- * If the cursor point reaches the right (xres - x < cursor.width) or
- * bottom (yres - y < cursor.height) boundary of the screen, then ROI
- * width and ROI height need to be evaluated to crop the cursor image
- * accordingly.
- * (xres-x) will be new cursor width when x > (xres - cursor.width)
- * (yres-y) will be new cursor height when y > (yres - cursor.height)
- */
- roi_w = min(mdp5_crtc->cursor.width, xres - x);
- roi_h = min(mdp5_crtc->cursor.height, yres - y);
+ mdp5_crtc->cursor.x = x = max(x, 0);
+ mdp5_crtc->cursor.y = y = max(y, 0);
+
+ get_roi(crtc, &roi_w, &roi_h);
spin_lock_irqsave(&mdp5_crtc->cursor.lock, flags);
mdp5_write(mdp5_kms, REG_MDP5_LM_CURSOR_SIZE(mdp5_crtc->lm),
static const struct drm_crtc_helper_funcs mdp5_crtc_helper_funcs = {
.mode_fixup = mdp5_crtc_mode_fixup,
.mode_set_nofb = mdp5_crtc_mode_set_nofb,
- .prepare = mdp5_crtc_disable,
- .commit = mdp5_crtc_enable,
+ .disable = mdp5_crtc_disable,
+ .enable = mdp5_crtc_enable,
.atomic_check = mdp5_crtc_atomic_check,
.atomic_begin = mdp5_crtc_atomic_begin,
.atomic_flush = mdp5_crtc_atomic_flush,
mdp5_write(mdp5_kms, REG_MDP5_INTF_TIMING_ENGINE_EN(intf), 1);
spin_unlock_irqrestore(&mdp5_encoder->intf_lock, flags);
- mdp5_encoder->enabled = false;
+ mdp5_encoder->enabled = true;
}
static const struct drm_encoder_helper_funcs mdp5_encoder_helper_funcs = {
.mode_fixup = mdp5_encoder_mode_fixup,
.mode_set = mdp5_encoder_mode_set,
- .prepare = mdp5_encoder_disable,
- .commit = mdp5_encoder_enable,
+ .disable = mdp5_encoder_disable,
+ .enable = mdp5_encoder_enable,
};
/* initialize encoder */
void mdp5_irq_preinstall(struct msm_kms *kms)
{
struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(kms));
+ mdp5_enable(mdp5_kms);
mdp5_write(mdp5_kms, REG_MDP5_INTR_CLEAR, 0xffffffff);
+ mdp5_write(mdp5_kms, REG_MDP5_INTR_EN, 0x00000000);
+ mdp5_disable(mdp5_kms);
}
int mdp5_irq_postinstall(struct msm_kms *kms)
void mdp5_irq_uninstall(struct msm_kms *kms)
{
struct mdp5_kms *mdp5_kms = to_mdp5_kms(to_mdp_kms(kms));
+ mdp5_enable(mdp5_kms);
mdp5_write(mdp5_kms, REG_MDP5_INTR_EN, 0x00000000);
+ mdp5_disable(mdp5_kms);
}
static void mdp5_irq_mdp(struct mdp_kms *mdp_kms)
* mark our set of crtc's as busy:
*/
ret = start_atomic(dev->dev_private, c->crtc_mask);
- if (ret)
+ if (ret) {
+ kfree(c);
return ret;
+ }
/*
* This is the point of no return - everything below never fails except
nouveau_fbcon_zfill(dev, fbcon);
/* To allow resizeing without swapping buffers */
- NV_INFO(drm, "allocated %dx%d fb: 0x%lx, bo %p\n",
+ NV_INFO(drm, "allocated %dx%d fb: 0x%llx, bo %p\n",
nouveau_fb->base.width, nouveau_fb->base.height,
nvbo->bo.offset, nvbo);
/* switch mmio to cpu's native endianness */
#ifndef __BIG_ENDIAN
- if (ioread32_native(map + 0x000004) != 0x00000000)
+ if (ioread32_native(map + 0x000004) != 0x00000000) {
#else
- if (ioread32_native(map + 0x000004) == 0x00000000)
+ if (ioread32_native(map + 0x000004) == 0x00000000) {
#endif
iowrite32_native(0x01000001, map + 0x000004);
+ ioread32_native(map);
+ }
/* read boot0 and strapping information */
boot0 = ioread32_native(map + 0x000000);
device->oclass[NVDEV_ENGINE_MSVLD ] = &gk104_msvld_oclass;
device->oclass[NVDEV_ENGINE_MSPDEC ] = &gk104_mspdec_oclass;
device->oclass[NVDEV_ENGINE_MSPPP ] = &gf100_msppp_oclass;
+#endif
+ break;
+ case 0x126:
+ device->cname = "GM206";
+ device->oclass[NVDEV_SUBDEV_VBIOS ] = &nvkm_bios_oclass;
+ device->oclass[NVDEV_SUBDEV_GPIO ] = gk104_gpio_oclass;
+ device->oclass[NVDEV_SUBDEV_I2C ] = gm204_i2c_oclass;
+ device->oclass[NVDEV_SUBDEV_FUSE ] = &gm107_fuse_oclass;
+#if 0
+ /* looks to be some non-trivial changes */
+ device->oclass[NVDEV_SUBDEV_CLK ] = &gk104_clk_oclass;
+ /* priv ring says no to 0x10eb14 writes */
+ device->oclass[NVDEV_SUBDEV_THERM ] = &gm107_therm_oclass;
+#endif
+ device->oclass[NVDEV_SUBDEV_MXM ] = &nv50_mxm_oclass;
+ device->oclass[NVDEV_SUBDEV_DEVINIT] = gm204_devinit_oclass;
+ device->oclass[NVDEV_SUBDEV_MC ] = gk20a_mc_oclass;
+ device->oclass[NVDEV_SUBDEV_BUS ] = gf100_bus_oclass;
+ device->oclass[NVDEV_SUBDEV_TIMER ] = &gk20a_timer_oclass;
+ device->oclass[NVDEV_SUBDEV_FB ] = gm107_fb_oclass;
+ device->oclass[NVDEV_SUBDEV_LTC ] = gm107_ltc_oclass;
+ device->oclass[NVDEV_SUBDEV_IBUS ] = &gk104_ibus_oclass;
+ device->oclass[NVDEV_SUBDEV_INSTMEM] = nv50_instmem_oclass;
+ device->oclass[NVDEV_SUBDEV_MMU ] = &gf100_mmu_oclass;
+ device->oclass[NVDEV_SUBDEV_BAR ] = &gf100_bar_oclass;
+ device->oclass[NVDEV_SUBDEV_PMU ] = gk208_pmu_oclass;
+#if 0
+ device->oclass[NVDEV_SUBDEV_VOLT ] = &nv40_volt_oclass;
+#endif
+ device->oclass[NVDEV_ENGINE_DMAOBJ ] = gf110_dmaeng_oclass;
+#if 0
+ device->oclass[NVDEV_ENGINE_FIFO ] = gk208_fifo_oclass;
+ device->oclass[NVDEV_ENGINE_SW ] = gf100_sw_oclass;
+ device->oclass[NVDEV_ENGINE_GR ] = gm107_gr_oclass;
+#endif
+ device->oclass[NVDEV_ENGINE_DISP ] = gm204_disp_oclass;
+#if 0
+ device->oclass[NVDEV_ENGINE_CE0 ] = &gm204_ce0_oclass;
+ device->oclass[NVDEV_ENGINE_CE1 ] = &gm204_ce1_oclass;
+ device->oclass[NVDEV_ENGINE_CE2 ] = &gm204_ce2_oclass;
+ device->oclass[NVDEV_ENGINE_MSVLD ] = &gk104_msvld_oclass;
+ device->oclass[NVDEV_ENGINE_MSPDEC ] = &gk104_mspdec_oclass;
+ device->oclass[NVDEV_ENGINE_MSPPP ] = &gf100_msppp_oclass;
#endif
break;
default:
{
struct nvkm_device *device = nv_device(subdev);
struct nv04_fifo_priv *priv = (void *)subdev;
- uint32_t status, reassign;
- int cnt = 0;
+ u32 mask = nv_rd32(priv, NV03_PFIFO_INTR_EN_0);
+ u32 stat = nv_rd32(priv, NV03_PFIFO_INTR_0) & mask;
+ u32 reassign, chid, get, sem;
reassign = nv_rd32(priv, NV03_PFIFO_CACHES) & 1;
- while ((status = nv_rd32(priv, NV03_PFIFO_INTR_0)) && (cnt++ < 100)) {
- uint32_t chid, get;
-
- nv_wr32(priv, NV03_PFIFO_CACHES, 0);
-
- chid = nv_rd32(priv, NV03_PFIFO_CACHE1_PUSH1) & priv->base.max;
- get = nv_rd32(priv, NV03_PFIFO_CACHE1_GET);
+ nv_wr32(priv, NV03_PFIFO_CACHES, 0);
- if (status & NV_PFIFO_INTR_CACHE_ERROR) {
- nv04_fifo_cache_error(device, priv, chid, get);
- status &= ~NV_PFIFO_INTR_CACHE_ERROR;
- }
+ chid = nv_rd32(priv, NV03_PFIFO_CACHE1_PUSH1) & priv->base.max;
+ get = nv_rd32(priv, NV03_PFIFO_CACHE1_GET);
- if (status & NV_PFIFO_INTR_DMA_PUSHER) {
- nv04_fifo_dma_pusher(device, priv, chid);
- status &= ~NV_PFIFO_INTR_DMA_PUSHER;
- }
+ if (stat & NV_PFIFO_INTR_CACHE_ERROR) {
+ nv04_fifo_cache_error(device, priv, chid, get);
+ stat &= ~NV_PFIFO_INTR_CACHE_ERROR;
+ }
- if (status & NV_PFIFO_INTR_SEMAPHORE) {
- uint32_t sem;
+ if (stat & NV_PFIFO_INTR_DMA_PUSHER) {
+ nv04_fifo_dma_pusher(device, priv, chid);
+ stat &= ~NV_PFIFO_INTR_DMA_PUSHER;
+ }
- status &= ~NV_PFIFO_INTR_SEMAPHORE;
- nv_wr32(priv, NV03_PFIFO_INTR_0,
- NV_PFIFO_INTR_SEMAPHORE);
+ if (stat & NV_PFIFO_INTR_SEMAPHORE) {
+ stat &= ~NV_PFIFO_INTR_SEMAPHORE;
+ nv_wr32(priv, NV03_PFIFO_INTR_0, NV_PFIFO_INTR_SEMAPHORE);
- sem = nv_rd32(priv, NV10_PFIFO_CACHE1_SEMAPHORE);
- nv_wr32(priv, NV10_PFIFO_CACHE1_SEMAPHORE, sem | 0x1);
+ sem = nv_rd32(priv, NV10_PFIFO_CACHE1_SEMAPHORE);
+ nv_wr32(priv, NV10_PFIFO_CACHE1_SEMAPHORE, sem | 0x1);
- nv_wr32(priv, NV03_PFIFO_CACHE1_GET, get + 4);
- nv_wr32(priv, NV04_PFIFO_CACHE1_PULL0, 1);
- }
+ nv_wr32(priv, NV03_PFIFO_CACHE1_GET, get + 4);
+ nv_wr32(priv, NV04_PFIFO_CACHE1_PULL0, 1);
+ }
- if (device->card_type == NV_50) {
- if (status & 0x00000010) {
- status &= ~0x00000010;
- nv_wr32(priv, 0x002100, 0x00000010);
- }
-
- if (status & 0x40000000) {
- nv_wr32(priv, 0x002100, 0x40000000);
- nvkm_fifo_uevent(&priv->base);
- status &= ~0x40000000;
- }
+ if (device->card_type == NV_50) {
+ if (stat & 0x00000010) {
+ stat &= ~0x00000010;
+ nv_wr32(priv, 0x002100, 0x00000010);
}
- if (status) {
- nv_warn(priv, "unknown intr 0x%08x, ch %d\n",
- status, chid);
- nv_wr32(priv, NV03_PFIFO_INTR_0, status);
- status = 0;
+ if (stat & 0x40000000) {
+ nv_wr32(priv, 0x002100, 0x40000000);
+ nvkm_fifo_uevent(&priv->base);
+ stat &= ~0x40000000;
}
-
- nv_wr32(priv, NV03_PFIFO_CACHES, reassign);
}
- if (status) {
- nv_error(priv, "still angry after %d spins, halt\n", cnt);
- nv_wr32(priv, 0x002140, 0);
- nv_wr32(priv, 0x000140, 0);
+ if (stat) {
+ nv_warn(priv, "unknown intr 0x%08x\n", stat);
+ nv_mask(priv, NV03_PFIFO_INTR_EN_0, stat, 0x00000000);
+ nv_wr32(priv, NV03_PFIFO_INTR_0, stat);
}
- nv_wr32(priv, 0x000100, 0x00000100);
+ nv_wr32(priv, NV03_PFIFO_CACHES, reassign);
}
static int
const int s = 8;
const int b = mmio_vram(info, impl->bundle_size, (1 << s), access);
mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_refn(info, 0x408008, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x408008, 0x80000000 | (impl->bundle_size >> s));
mmio_refn(info, 0x418808, 0x00000000, s, b);
- mmio_refn(info, 0x41880c, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x41880c, 0x80000000 | (impl->bundle_size >> s));
}
void
const int s = 8;
const int b = mmio_vram(info, impl->bundle_size, (1 << s), access);
mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_refn(info, 0x408008, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x408008, 0x80000000 | (impl->bundle_size >> s));
mmio_refn(info, 0x418808, 0x00000000, s, b);
- mmio_refn(info, 0x41880c, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x41880c, 0x80000000 | (impl->bundle_size >> s));
mmio_wr32(info, 0x4064c8, (state_limit << 16) | token_limit);
}
const int s = 8;
const int b = mmio_vram(info, impl->bundle_size, (1 << s), access);
mmio_refn(info, 0x408004, 0x00000000, s, b);
- mmio_refn(info, 0x408008, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x408008, 0x80000000 | (impl->bundle_size >> s));
mmio_refn(info, 0x418e24, 0x00000000, s, b);
- mmio_refn(info, 0x418e28, 0x80000000 | (impl->bundle_size >> s), 0, b);
+ mmio_wr32(info, 0x418e28, 0x80000000 | (impl->bundle_size >> s));
mmio_wr32(info, 0x4064c8, (state_limit << 16) | token_limit);
}
u16 ent = dcb_i2c_entry(bios, idx, &ver, &len);
if (ent) {
if (ver >= 0x41) {
- if (!(nv_ro32(bios, ent) & 0x80000000))
+ u32 ent_value = nv_ro32(bios, ent);
+ u8 i2c_port = (ent_value >> 27) & 0x1f;
+ u8 dpaux_port = (ent_value >> 22) & 0x1f;
+ /* value 0x1f means unused according to DCB 4.x spec */
+ if (i2c_port == 0x1f && dpaux_port == 0x1f)
info->type = DCB_I2C_UNUSED;
else
info->type = DCB_I2C_PMGR;
(x << 16) | y);
viewport_w = crtc->mode.hdisplay;
viewport_h = (crtc->mode.vdisplay + 1) & ~1;
+ if ((rdev->family >= CHIP_BONAIRE) &&
+ (crtc->mode.flags & DRM_MODE_FLAG_INTERLACE))
+ viewport_h *= 2;
WREG32(EVERGREEN_VIEWPORT_SIZE + radeon_crtc->crtc_offset,
(viewport_w << 16) | viewport_h);
switch (msg->request & ~DP_AUX_I2C_MOT) {
case DP_AUX_NATIVE_WRITE:
case DP_AUX_I2C_WRITE:
+ /* The atom implementation only supports writes with a max payload of
+ * 12 bytes since it uses 4 bits for the total count (header + payload)
+ * in the parameter space. The atom interface supports 16 byte
+ * payloads for reads. The hw itself supports up to 16 bytes of payload.
+ */
+ if (WARN_ON_ONCE(msg->size > 12))
+ return -E2BIG;
/* tx_size needs to be 4 even for bare address packets since the atom
* table needs the info in tx_buf[3].
*/
dig_connector = radeon_connector->con_priv;
if ((dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_DISPLAYPORT) ||
(dig_connector->dp_sink_type == CONNECTOR_OBJECT_ID_eDP)) {
- if (radeon_audio != 0 && ASIC_IS_DCE4(rdev) && !ASIC_IS_DCE5(rdev))
+ if (radeon_audio != 0 &&
+ drm_detect_monitor_audio(radeon_connector_edid(connector)) &&
+ ASIC_IS_DCE4(rdev) && !ASIC_IS_DCE5(rdev))
return ATOM_ENCODER_MODE_DP_AUDIO;
return ATOM_ENCODER_MODE_DP;
} else if (radeon_audio != 0) {
}
break;
case DRM_MODE_CONNECTOR_eDP:
- if (radeon_audio != 0 && ASIC_IS_DCE4(rdev) && !ASIC_IS_DCE5(rdev))
+ if (radeon_audio != 0 &&
+ drm_detect_monitor_audio(radeon_connector_edid(connector)) &&
+ ASIC_IS_DCE4(rdev) && !ASIC_IS_DCE5(rdev))
return ATOM_ENCODER_MODE_DP_AUDIO;
return ATOM_ENCODER_MODE_DP;
case DRM_MODE_CONNECTOR_DVIA:
struct radeon_connector *radeon_connector = NULL;
struct radeon_connector_atom_dig *radeon_dig_connector = NULL;
bool travis_quirk = false;
- int encoder_mode;
if (connector) {
radeon_connector = to_radeon_connector(connector);
}
break;
}
-
- encoder_mode = atombios_get_encoder_mode(encoder);
- if (radeon_audio != 0 &&
- (encoder_mode == ATOM_ENCODER_MODE_HDMI || ENCODER_MODE_IS_DP(encoder_mode)))
- radeon_audio_dpms(encoder, mode);
}
static void
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+ struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
+ int encoder_mode = atombios_get_encoder_mode(encoder);
DRM_DEBUG_KMS("encoder dpms %d to mode %d, devices %08x, active_devices %08x\n",
radeon_encoder->encoder_id, mode, radeon_encoder->devices,
radeon_encoder->active_device);
+
+ if (connector && (radeon_audio != 0) &&
+ ((encoder_mode == ATOM_ENCODER_MODE_HDMI) ||
+ (ENCODER_MODE_IS_DP(encoder_mode) &&
+ drm_detect_monitor_audio(radeon_connector_edid(connector)))))
+ radeon_audio_dpms(encoder, mode);
+
switch (radeon_encoder->encoder_id) {
case ENCODER_OBJECT_ID_INTERNAL_TMDS1:
case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_TMDS1:
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
+ struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
int encoder_mode;
radeon_encoder->pixel_clock = adjusted_mode->clock;
case ENCODER_OBJECT_ID_INTERNAL_UNIPHY3:
case ENCODER_OBJECT_ID_INTERNAL_KLDSCP_LVTMA:
/* handled in dpms */
- encoder_mode = atombios_get_encoder_mode(encoder);
- if (radeon_audio != 0 &&
- (encoder_mode == ATOM_ENCODER_MODE_HDMI || ENCODER_MODE_IS_DP(encoder_mode)))
- radeon_audio_mode_set(encoder, adjusted_mode);
break;
case ENCODER_OBJECT_ID_INTERNAL_DDI:
case ENCODER_OBJECT_ID_INTERNAL_DVO1:
}
atombios_apply_encoder_quirks(encoder, adjusted_mode);
+
+ encoder_mode = atombios_get_encoder_mode(encoder);
+ if (connector && (radeon_audio != 0) &&
+ ((encoder_mode == ATOM_ENCODER_MODE_HDMI) ||
+ (ENCODER_MODE_IS_DP(encoder_mode) &&
+ drm_detect_monitor_audio(radeon_connector_edid(connector)))))
+ radeon_audio_mode_set(encoder, adjusted_mode);
}
static bool
}
WREG32(GRBM_CNTL, GRBM_READ_TIMEOUT(0xff));
+ WREG32(SRBM_INT_CNTL, 0x1);
+ WREG32(SRBM_INT_ACK, 0x1);
WREG32(BIF_FB_EN, FB_READ_EN | FB_WRITE_EN);
WREG32(CP_ME2_PIPE3_INT_CNTL, 0);
/* grbm */
WREG32(GRBM_INT_CNTL, 0);
+ /* SRBM */
+ WREG32(SRBM_INT_CNTL, 0);
/* vline/vblank, etc. */
WREG32(LB_INTERRUPT_MASK + EVERGREEN_CRTC0_REGISTER_OFFSET, 0);
WREG32(LB_INTERRUPT_MASK + EVERGREEN_CRTC1_REGISTER_OFFSET, 0);
WREG32(DC_HPD5_INT_CONTROL, hpd5);
WREG32(DC_HPD6_INT_CONTROL, hpd6);
+ /* posting read */
+ RREG32(SRBM_STATUS);
+
return 0;
}
break;
}
break;
+ case 96:
+ DRM_ERROR("SRBM_READ_ERROR: 0x%x\n", RREG32(SRBM_READ_ERROR));
+ WREG32(SRBM_INT_ACK, 0x1);
+ break;
case 124: /* UVD */
DRM_DEBUG("IH: UVD int: 0x%08x\n", src_data);
radeon_fence_process(rdev, R600_RING_TYPE_UVD_INDEX);
#define SOFT_RESET_ORB (1 << 23)
#define SOFT_RESET_VCE (1 << 24)
+#define SRBM_READ_ERROR 0xE98
+#define SRBM_INT_CNTL 0xEA0
+#define SRBM_INT_ACK 0xEA8
+
#define VM_L2_CNTL 0x1400
#define ENABLE_L2_CACHE (1 << 0)
#define ENABLE_L2_FRAGMENT_PROCESSING (1 << 1)
#define VCE_UENC_REG_CLOCK_GATING 0x207c0
#define VCE_SYS_INT_EN 0x21300
# define VCE_SYS_INT_TRAP_INTERRUPT_EN (1 << 3)
+#define VCE_LMI_VCPU_CACHE_40BIT_BAR 0x2145c
#define VCE_LMI_CTRL2 0x21474
#define VCE_LMI_CTRL 0x21498
#define VCE_LMI_VM_CTRL 0x214a0
#include "radeon_audio.h"
#include "sid.h"
+#define DCE8_DCCG_AUDIO_DTO1_PHASE 0x05b8
+#define DCE8_DCCG_AUDIO_DTO1_MODULE 0x05bc
+
u32 dce6_endpoint_rreg(struct radeon_device *rdev,
u32 block_offset, u32 reg)
{
void dce6_hdmi_audio_set_dto(struct radeon_device *rdev,
struct radeon_crtc *crtc, unsigned int clock)
{
- /* Two dtos; generally use dto0 for HDMI */
+ /* Two dtos; generally use dto0 for HDMI */
u32 value = 0;
- if (crtc)
+ if (crtc)
value |= DCCG_AUDIO_DTO0_SOURCE_SEL(crtc->crtc_id);
WREG32(DCCG_AUDIO_DTO_SOURCE, value);
- /* Express [24MHz / target pixel clock] as an exact rational
- * number (coefficient of two integer numbers. DCCG_AUDIO_DTOx_PHASE
- * is the numerator, DCCG_AUDIO_DTOx_MODULE is the denominator
- */
- WREG32(DCCG_AUDIO_DTO0_PHASE, 24000);
- WREG32(DCCG_AUDIO_DTO0_MODULE, clock);
+ /* Express [24MHz / target pixel clock] as an exact rational
+ * number (coefficient of two integer numbers. DCCG_AUDIO_DTOx_PHASE
+ * is the numerator, DCCG_AUDIO_DTOx_MODULE is the denominator
+ */
+ WREG32(DCCG_AUDIO_DTO0_PHASE, 24000);
+ WREG32(DCCG_AUDIO_DTO0_MODULE, clock);
}
void dce6_dp_audio_set_dto(struct radeon_device *rdev,
struct radeon_crtc *crtc, unsigned int clock)
{
- /* Two dtos; generally use dto1 for DP */
+ /* Two dtos; generally use dto1 for DP */
u32 value = 0;
value |= DCCG_AUDIO_DTO_SEL;
- if (crtc)
+ if (crtc)
value |= DCCG_AUDIO_DTO0_SOURCE_SEL(crtc->crtc_id);
WREG32(DCCG_AUDIO_DTO_SOURCE, value);
- /* Express [24MHz / target pixel clock] as an exact rational
- * number (coefficient of two integer numbers. DCCG_AUDIO_DTOx_PHASE
- * is the numerator, DCCG_AUDIO_DTOx_MODULE is the denominator
- */
- WREG32(DCCG_AUDIO_DTO1_PHASE, 24000);
- WREG32(DCCG_AUDIO_DTO1_MODULE, clock);
+ /* Express [24MHz / target pixel clock] as an exact rational
+ * number (coefficient of two integer numbers. DCCG_AUDIO_DTOx_PHASE
+ * is the numerator, DCCG_AUDIO_DTOx_MODULE is the denominator
+ */
+ if (ASIC_IS_DCE8(rdev)) {
+ WREG32(DCE8_DCCG_AUDIO_DTO1_PHASE, 24000);
+ WREG32(DCE8_DCCG_AUDIO_DTO1_MODULE, clock);
+ } else {
+ WREG32(DCCG_AUDIO_DTO1_PHASE, 24000);
+ WREG32(DCCG_AUDIO_DTO1_MODULE, clock);
+ }
}
-void dce6_enable_dp_audio_packets(struct drm_encoder *encoder, bool enable)
+void dce6_dp_enable(struct drm_encoder *encoder, bool enable)
{
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
- uint32_t offset;
if (!dig || !dig->afmt)
return;
- offset = dig->afmt->offset;
-
if (enable) {
- if (dig->afmt->enabled)
- return;
-
- WREG32(EVERGREEN_DP_SEC_TIMESTAMP + offset, EVERGREEN_DP_SEC_TIMESTAMP_MODE(1));
- WREG32(EVERGREEN_DP_SEC_CNTL + offset,
- EVERGREEN_DP_SEC_ASP_ENABLE | /* Audio packet transmission */
- EVERGREEN_DP_SEC_ATP_ENABLE | /* Audio timestamp packet transmission */
- EVERGREEN_DP_SEC_AIP_ENABLE | /* Audio infoframe packet transmission */
- EVERGREEN_DP_SEC_STREAM_ENABLE); /* Master enable for secondary stream engine */
- radeon_audio_enable(rdev, dig->afmt->pin, true);
+ WREG32(EVERGREEN_DP_SEC_TIMESTAMP + dig->afmt->offset,
+ EVERGREEN_DP_SEC_TIMESTAMP_MODE(1));
+ WREG32(EVERGREEN_DP_SEC_CNTL + dig->afmt->offset,
+ EVERGREEN_DP_SEC_ASP_ENABLE | /* Audio packet transmission */
+ EVERGREEN_DP_SEC_ATP_ENABLE | /* Audio timestamp packet transmission */
+ EVERGREEN_DP_SEC_AIP_ENABLE | /* Audio infoframe packet transmission */
+ EVERGREEN_DP_SEC_STREAM_ENABLE); /* Master enable for secondary stream engine */
} else {
- if (!dig->afmt->enabled)
- return;
-
- WREG32(EVERGREEN_DP_SEC_CNTL + offset, 0);
- radeon_audio_enable(rdev, dig->afmt->pin, false);
+ WREG32(EVERGREEN_DP_SEC_CNTL + dig->afmt->offset, 0);
}
dig->afmt->enabled = enable;
}
WREG32(GRBM_CNTL, GRBM_READ_TIMEOUT(0xff));
+ WREG32(SRBM_INT_CNTL, 0x1);
+ WREG32(SRBM_INT_ACK, 0x1);
evergreen_fix_pci_max_read_req_size(rdev);
tmp = RREG32(DMA_CNTL) & ~TRAP_ENABLE;
WREG32(DMA_CNTL, tmp);
WREG32(GRBM_INT_CNTL, 0);
+ WREG32(SRBM_INT_CNTL, 0);
WREG32(INT_MASK + EVERGREEN_CRTC0_REGISTER_OFFSET, 0);
WREG32(INT_MASK + EVERGREEN_CRTC1_REGISTER_OFFSET, 0);
if (rdev->num_crtc >= 4) {
WREG32(AFMT_AUDIO_PACKET_CONTROL + EVERGREEN_CRTC4_REGISTER_OFFSET, afmt5);
WREG32(AFMT_AUDIO_PACKET_CONTROL + EVERGREEN_CRTC5_REGISTER_OFFSET, afmt6);
+ /* posting read */
+ RREG32(SRBM_STATUS);
+
return 0;
}
DRM_ERROR("Unhandled interrupt: %d %d\n", src_id, src_data);
break;
}
+ case 96:
+ DRM_ERROR("SRBM_READ_ERROR: 0x%x\n", RREG32(SRBM_READ_ERROR));
+ WREG32(SRBM_INT_ACK, 0x1);
+ break;
case 124: /* UVD */
DRM_DEBUG("IH: UVD int: 0x%08x\n", src_data);
radeon_fence_process(rdev, R600_RING_TYPE_UVD_INDEX);
}
void dce4_dp_audio_set_dto(struct radeon_device *rdev,
- struct radeon_crtc *crtc, unsigned int clock)
+ struct radeon_crtc *crtc, unsigned int clock)
{
u32 value;
* is the numerator, DCCG_AUDIO_DTOx_MODULE is the denominator
*/
WREG32(DCCG_AUDIO_DTO1_PHASE, 24000);
- WREG32(DCCG_AUDIO_DTO1_MODULE, rdev->clock.max_pixel_clock * 10);
+ WREG32(DCCG_AUDIO_DTO1_MODULE, clock);
}
void dce4_set_vbi_packet(struct drm_encoder *encoder, u32 offset)
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
- WREG32(HDMI_INFOFRAME_CONTROL0 + offset,
- HDMI_AUDIO_INFO_SEND | /* enable audio info frames (frames won't be set until audio is enabled) */
- HDMI_AUDIO_INFO_CONT); /* required for audio info values to be updated */
-
WREG32(AFMT_INFOFRAME_CONTROL0 + offset,
AFMT_AUDIO_INFO_UPDATE); /* required for audio info values to be updated */
- WREG32(HDMI_INFOFRAME_CONTROL1 + offset,
- HDMI_AUDIO_INFO_LINE(2)); /* anything other than 0 */
-
- WREG32(HDMI_AUDIO_PACKET_CONTROL + offset,
- HDMI_AUDIO_DELAY_EN(1) | /* set the default audio delay */
- HDMI_AUDIO_PACKETS_PER_LINE(3)); /* should be suffient for all audio modes and small enough for all hblanks */
-
WREG32(AFMT_60958_0 + offset,
AFMT_60958_CS_CHANNEL_NUMBER_L(1));
if (!dig || !dig->afmt)
return;
- /* Silent, r600_hdmi_enable will raise WARN for us */
- if (enable && dig->afmt->enabled)
- return;
- if (!enable && !dig->afmt->enabled)
- return;
+ if (enable) {
+ WREG32(HDMI_INFOFRAME_CONTROL1 + dig->afmt->offset,
+ HDMI_AUDIO_INFO_LINE(2)); /* anything other than 0 */
+
+ WREG32(HDMI_AUDIO_PACKET_CONTROL + dig->afmt->offset,
+ HDMI_AUDIO_DELAY_EN(1) | /* set the default audio delay */
+ HDMI_AUDIO_PACKETS_PER_LINE(3)); /* should be suffient for all audio modes and small enough for all hblanks */
- if (!enable && dig->afmt->pin) {
- radeon_audio_enable(rdev, dig->afmt->pin, 0);
- dig->afmt->pin = NULL;
+ WREG32(HDMI_INFOFRAME_CONTROL0 + dig->afmt->offset,
+ HDMI_AUDIO_INFO_SEND | /* enable audio info frames (frames won't be set until audio is enabled) */
+ HDMI_AUDIO_INFO_CONT); /* required for audio info values to be updated */
+ } else {
+ WREG32(HDMI_INFOFRAME_CONTROL0 + dig->afmt->offset, 0);
}
dig->afmt->enabled = enable;
enable ? "En" : "Dis", dig->afmt->offset, radeon_encoder->encoder_id);
}
-void evergreen_enable_dp_audio_packets(struct drm_encoder *encoder, bool enable)
+void evergreen_dp_enable(struct drm_encoder *encoder, bool enable)
{
struct drm_device *dev = encoder->dev;
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
- uint32_t offset;
if (!dig || !dig->afmt)
return;
- offset = dig->afmt->offset;
-
if (enable) {
struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
struct radeon_connector *radeon_connector = to_radeon_connector(connector);
struct radeon_connector_atom_dig *dig_connector;
uint32_t val;
- if (dig->afmt->enabled)
- return;
-
- WREG32(EVERGREEN_DP_SEC_TIMESTAMP + offset, EVERGREEN_DP_SEC_TIMESTAMP_MODE(1));
+ WREG32(EVERGREEN_DP_SEC_TIMESTAMP + dig->afmt->offset,
+ EVERGREEN_DP_SEC_TIMESTAMP_MODE(1));
if (radeon_connector->con_priv) {
dig_connector = radeon_connector->con_priv;
- val = RREG32(EVERGREEN_DP_SEC_AUD_N + offset);
+ val = RREG32(EVERGREEN_DP_SEC_AUD_N + dig->afmt->offset);
val &= ~EVERGREEN_DP_SEC_N_BASE_MULTIPLE(0xf);
if (dig_connector->dp_clock == 162000)
else
val |= EVERGREEN_DP_SEC_N_BASE_MULTIPLE(5);
- WREG32(EVERGREEN_DP_SEC_AUD_N + offset, val);
+ WREG32(EVERGREEN_DP_SEC_AUD_N + dig->afmt->offset, val);
}
- WREG32(EVERGREEN_DP_SEC_CNTL + offset,
+ WREG32(EVERGREEN_DP_SEC_CNTL + dig->afmt->offset,
EVERGREEN_DP_SEC_ASP_ENABLE | /* Audio packet transmission */
EVERGREEN_DP_SEC_ATP_ENABLE | /* Audio timestamp packet transmission */
EVERGREEN_DP_SEC_AIP_ENABLE | /* Audio infoframe packet transmission */
EVERGREEN_DP_SEC_STREAM_ENABLE); /* Master enable for secondary stream engine */
- radeon_audio_enable(rdev, dig->afmt->pin, 0xf);
} else {
- if (!dig->afmt->enabled)
- return;
-
- WREG32(EVERGREEN_DP_SEC_CNTL + offset, 0);
- radeon_audio_enable(rdev, dig->afmt->pin, 0);
+ WREG32(EVERGREEN_DP_SEC_CNTL + dig->afmt->offset, 0);
}
dig->afmt->enabled = enable;
#define SOFT_RESET_REGBB (1 << 22)
#define SOFT_RESET_ORB (1 << 23)
+#define SRBM_READ_ERROR 0xE98
+#define SRBM_INT_CNTL 0xEA0
+#define SRBM_INT_ACK 0xEA8
+
/* display watermarks */
#define DC_LB_MEMORY_SPLIT 0x6b0c
#define PRIORITY_A_CNT 0x6b18
}
WREG32(GRBM_CNTL, GRBM_READ_TIMEOUT(0xff));
+ WREG32(SRBM_INT_CNTL, 0x1);
+ WREG32(SRBM_INT_ACK, 0x1);
evergreen_fix_pci_max_read_req_size(rdev);
if ((rdev->config.cayman.max_backends_per_se == 1) &&
(rdev->flags & RADEON_IS_IGP)) {
- if ((disabled_rb_mask & 3) == 1) {
- /* RB0 disabled, RB1 enabled */
- tmp = 0x11111111;
- } else {
+ if ((disabled_rb_mask & 3) == 2) {
/* RB1 disabled, RB0 enabled */
tmp = 0x00000000;
+ } else {
+ /* RB0 disabled, RB1 enabled */
+ tmp = 0x11111111;
}
} else {
tmp = gb_addr_config & NUM_PIPES_MASK;
#define SOFT_RESET_REGBB (1 << 22)
#define SOFT_RESET_ORB (1 << 23)
+#define SRBM_READ_ERROR 0xE98
+#define SRBM_INT_CNTL 0xEA0
+#define SRBM_INT_ACK 0xEA8
+
#define SRBM_STATUS2 0x0EC4
#define DMA_BUSY (1 << 5)
#define DMA1_BUSY (1 << 6)
tmp |= RADEON_FP2_DETECT_MASK;
}
WREG32(RADEON_GEN_INT_CNTL, tmp);
+
+ /* read back to post the write */
+ RREG32(RADEON_GEN_INT_CNTL);
+
return 0;
}
WREG32(RV770_CG_THERMAL_INT, thermal_int);
}
+ /* posting read */
+ RREG32(R_000E50_SRBM_STATUS);
+
return 0;
}
list_for_each_entry(crtc, &dev->mode_config.crtc_list, head) {
radeon_crtc = to_radeon_crtc(crtc);
if (crtc->enabled && radeon_crtc->enabled && radeon_crtc->hw_mode.clock) {
- vrefresh = radeon_crtc->hw_mode.vrefresh;
+ vrefresh = drm_mode_vrefresh(&radeon_crtc->hw_mode);
break;
}
}
if (!dig || !dig->afmt)
return;
- /* Silent, r600_hdmi_enable will raise WARN for us */
- if (enable && dig->afmt->enabled)
- return;
- if (!enable && !dig->afmt->enabled)
- return;
-
- if (!enable && dig->afmt->pin) {
- radeon_audio_enable(rdev, dig->afmt->pin, 0);
- dig->afmt->pin = NULL;
- }
-
/* Older chipsets require setting HDMI and routing manually */
if (!ASIC_IS_DCE3(rdev)) {
if (enable)
int new_active_crtc_count;
u32 current_active_crtcs;
int current_active_crtc_count;
+ bool single_display;
struct radeon_dpm_dynamic_state dyn_state;
struct radeon_dpm_fan fan;
u32 tdp_limit;
struct drm_display_mode *mode);
void r600_hdmi_enable(struct drm_encoder *encoder, bool enable);
void evergreen_hdmi_enable(struct drm_encoder *encoder, bool enable);
-void evergreen_enable_dp_audio_packets(struct drm_encoder *encoder, bool enable);
-void dce6_enable_dp_audio_packets(struct drm_encoder *encoder, bool enable);
+void evergreen_dp_enable(struct drm_encoder *encoder, bool enable);
+void dce6_dp_enable(struct drm_encoder *encoder, bool enable);
static const u32 pin_offsets[7] =
{
.set_avi_packet = evergreen_set_avi_packet,
.set_audio_packet = dce4_set_audio_packet,
.mode_set = radeon_audio_dp_mode_set,
- .dpms = evergreen_enable_dp_audio_packets,
+ .dpms = evergreen_dp_enable,
};
static struct radeon_audio_funcs dce6_hdmi_funcs = {
.set_avi_packet = evergreen_set_avi_packet,
.set_audio_packet = dce4_set_audio_packet,
.mode_set = radeon_audio_dp_mode_set,
- .dpms = dce6_enable_dp_audio_packets,
+ .dpms = dce6_dp_enable,
};
static void radeon_audio_interface_init(struct radeon_device *rdev)
}
void radeon_audio_detect(struct drm_connector *connector,
- enum drm_connector_status status)
+ enum drm_connector_status status)
{
struct radeon_device *rdev;
struct radeon_encoder *radeon_encoder;
else
radeon_encoder->audio = rdev->audio.hdmi_funcs;
- radeon_audio_write_speaker_allocation(connector->encoder);
- radeon_audio_write_sad_regs(connector->encoder);
- if (connector->encoder->crtc)
- radeon_audio_write_latency_fields(connector->encoder,
- &connector->encoder->crtc->mode);
+ dig->afmt->pin = radeon_audio_get_pin(connector->encoder);
radeon_audio_enable(rdev, dig->afmt->pin, 0xf);
} else {
radeon_audio_enable(rdev, dig->afmt->pin, 0);
+ dig->afmt->pin = NULL;
}
}
* update the info frames with the data from the current display mode
*/
static void radeon_audio_hdmi_mode_set(struct drm_encoder *encoder,
- struct drm_display_mode *mode)
+ struct drm_display_mode *mode)
{
- struct radeon_device *rdev = encoder->dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
if (!dig || !dig->afmt)
return;
- /* disable audio prior to setting up hw */
- dig->afmt->pin = radeon_audio_get_pin(encoder);
- radeon_audio_enable(rdev, dig->afmt->pin, 0);
+ radeon_audio_set_mute(encoder, true);
+ radeon_audio_write_speaker_allocation(encoder);
+ radeon_audio_write_sad_regs(encoder);
+ radeon_audio_write_latency_fields(encoder, mode);
radeon_audio_set_dto(encoder, mode->clock);
radeon_audio_set_vbi_packet(encoder);
radeon_hdmi_set_color_depth(encoder);
- radeon_audio_set_mute(encoder, false);
radeon_audio_update_acr(encoder, mode->clock);
radeon_audio_set_audio_packet(encoder);
radeon_audio_select_pin(encoder);
if (radeon_audio_set_avi_packet(encoder, mode) < 0)
return;
- /* enable audio after to setting up hw */
- radeon_audio_enable(rdev, dig->afmt->pin, 0xf);
+ radeon_audio_set_mute(encoder, false);
}
static void radeon_audio_dp_mode_set(struct drm_encoder *encoder,
struct radeon_device *rdev = dev->dev_private;
struct radeon_encoder *radeon_encoder = to_radeon_encoder(encoder);
struct radeon_encoder_atom_dig *dig = radeon_encoder->enc_priv;
+ struct drm_connector *connector = radeon_get_connector_for_encoder(encoder);
+ struct radeon_connector *radeon_connector = to_radeon_connector(connector);
+ struct radeon_connector_atom_dig *dig_connector =
+ radeon_connector->con_priv;
if (!dig || !dig->afmt)
return;
- /* disable audio prior to setting up hw */
- dig->afmt->pin = radeon_audio_get_pin(encoder);
- radeon_audio_enable(rdev, dig->afmt->pin, 0);
-
- radeon_audio_set_dto(encoder, rdev->clock.default_dispclk * 10);
+ radeon_audio_write_speaker_allocation(encoder);
+ radeon_audio_write_sad_regs(encoder);
+ radeon_audio_write_latency_fields(encoder, mode);
+ if (rdev->clock.dp_extclk || ASIC_IS_DCE5(rdev))
+ radeon_audio_set_dto(encoder, rdev->clock.default_dispclk * 10);
+ else
+ radeon_audio_set_dto(encoder, dig_connector->dp_clock);
radeon_audio_set_audio_packet(encoder);
radeon_audio_select_pin(encoder);
if (radeon_audio_set_avi_packet(encoder, mode) < 0)
return;
-
- /* enable audio after to setting up hw */
- radeon_audio_enable(rdev, dig->afmt->pin, 0xf);
}
void radeon_audio_mode_set(struct drm_encoder *encoder,
static bool radeon_read_bios(struct radeon_device *rdev)
{
- uint8_t __iomem *bios;
+ uint8_t __iomem *bios, val1, val2;
size_t size;
rdev->bios = NULL;
return false;
}
- if (size == 0 || bios[0] != 0x55 || bios[1] != 0xaa) {
+ val1 = readb(&bios[0]);
+ val2 = readb(&bios[1]);
+
+ if (size == 0 || val1 != 0x55 || val2 != 0xaa) {
pci_unmap_rom(rdev->pdev, bios);
return false;
}
- rdev->bios = kmemdup(bios, size, GFP_KERNEL);
+ rdev->bios = kzalloc(size, GFP_KERNEL);
if (rdev->bios == NULL) {
pci_unmap_rom(rdev->pdev, bios);
return false;
}
+ memcpy_fromio(rdev->bios, bios, size);
pci_unmap_rom(rdev->pdev, bios);
return true;
}
u32 ring = RADEON_CS_RING_GFX;
s32 priority = 0;
+ INIT_LIST_HEAD(&p->validated);
+
if (!cs->num_chunks) {
return 0;
}
+
/* get chunks */
- INIT_LIST_HEAD(&p->validated);
p->idx = 0;
p->ib.sa_bo = NULL;
p->const_ib.sa_bo = NULL;
struct radeon_cs_chunk *ib_chunk = p->chunk_ib;
struct radeon_device *rdev = p->rdev;
uint32_t header;
+ int ret = 0, i;
if (idx >= ib_chunk->length_dw) {
DRM_ERROR("Can not parse packet at %d after CS end %d !\n",
break;
default:
DRM_ERROR("Unknown packet type %d at %d !\n", pkt->type, idx);
- return -EINVAL;
+ ret = -EINVAL;
+ goto dump_ib;
}
if ((pkt->count + 1 + pkt->idx) >= ib_chunk->length_dw) {
DRM_ERROR("Packet (%d:%d:%d) end after CS buffer (%d) !\n",
pkt->idx, pkt->type, pkt->count, ib_chunk->length_dw);
- return -EINVAL;
+ ret = -EINVAL;
+ goto dump_ib;
}
return 0;
+
+dump_ib:
+ for (i = 0; i < ib_chunk->length_dw; i++) {
+ if (i == idx)
+ printk("\t0x%08x <---\n", radeon_get_ib_value(p, i));
+ else
+ printk("\t0x%08x\n", radeon_get_ib_value(p, i));
+ }
+ return ret;
}
/**
(rdev->pdev->subsystem_vendor == 0x1734) &&
(rdev->pdev->subsystem_device == 0x1107))
use_bl = false;
+/* Older PPC macs use on-GPU backlight controller */
+#ifndef CONFIG_PPC_PMAC
/* disable native backlight control on older asics */
else if (rdev->family < CHIP_R600)
use_bl = false;
+#endif
else
use_bl = true;
}
return test_bit(FENCE_FLAG_SIGNALED_BIT, &fence->base.flags);
}
+struct radeon_wait_cb {
+ struct fence_cb base;
+ struct task_struct *task;
+};
+
+static void
+radeon_fence_wait_cb(struct fence *fence, struct fence_cb *cb)
+{
+ struct radeon_wait_cb *wait =
+ container_of(cb, struct radeon_wait_cb, base);
+
+ wake_up_process(wait->task);
+}
+
static signed long radeon_fence_default_wait(struct fence *f, bool intr,
signed long t)
{
struct radeon_fence *fence = to_radeon_fence(f);
struct radeon_device *rdev = fence->rdev;
- bool signaled;
+ struct radeon_wait_cb cb;
- fence_enable_sw_signaling(&fence->base);
+ cb.task = current;
- /*
- * This function has to return -EDEADLK, but cannot hold
- * exclusive_lock during the wait because some callers
- * may already hold it. This means checking needs_reset without
- * lock, and not fiddling with any gpu internals.
- *
- * The callback installed with fence_enable_sw_signaling will
- * run before our wait_event_*timeout call, so we will see
- * both the signaled fence and the changes to needs_reset.
- */
+ if (fence_add_callback(f, &cb.base, radeon_fence_wait_cb))
+ return t;
+
+ while (t > 0) {
+ if (intr)
+ set_current_state(TASK_INTERRUPTIBLE);
+ else
+ set_current_state(TASK_UNINTERRUPTIBLE);
+
+ /*
+ * radeon_test_signaled must be called after
+ * set_current_state to prevent a race with wake_up_process
+ */
+ if (radeon_test_signaled(fence))
+ break;
+
+ if (rdev->needs_reset) {
+ t = -EDEADLK;
+ break;
+ }
+
+ t = schedule_timeout(t);
+
+ if (t > 0 && intr && signal_pending(current))
+ t = -ERESTARTSYS;
+ }
+
+ __set_current_state(TASK_RUNNING);
+ fence_remove_callback(f, &cb.base);
- if (intr)
- t = wait_event_interruptible_timeout(rdev->fence_queue,
- ((signaled = radeon_test_signaled(fence)) ||
- rdev->needs_reset), t);
- else
- t = wait_event_timeout(rdev->fence_queue,
- ((signaled = radeon_test_signaled(fence)) ||
- rdev->needs_reset), t);
-
- if (t > 0 && !signaled)
- return -EDEADLK;
return t;
}
.compute_vmid_bitmap = 0xFF00,
.first_compute_pipe = 1,
- .compute_pipe_count = 8 - 1,
+ .compute_pipe_count = 4 - 1,
};
radeon_doorbell_get_kfd_info(rdev,
it = interval_tree_iter_first(&rmn->objects, start, end);
while (it) {
struct radeon_bo *bo;
- struct fence *fence;
int r;
bo = container_of(it, struct radeon_bo, mn_it);
continue;
}
- fence = reservation_object_get_excl(bo->tbo.resv);
- if (fence) {
- r = radeon_fence_wait((struct radeon_fence *)fence, false);
- if (r)
- DRM_ERROR("(%d) failed to wait for user bo\n", r);
- }
+ r = reservation_object_wait_timeout_rcu(bo->tbo.resv, true,
+ false, MAX_SCHEDULE_TIMEOUT);
+ if (r)
+ DRM_ERROR("(%d) failed to wait for user bo\n", r);
radeon_ttm_placement_from_domain(bo, RADEON_GEM_DOMAIN_CPU);
r = ttm_bo_validate(&bo->tbo, &bo->placement, false, false);
else
rbo->placements[i].lpfn = 0;
}
-
- /*
- * Use two-ended allocation depending on the buffer size to
- * improve fragmentation quality.
- * 512kb was measured as the most optimal number.
- */
- if (rbo->tbo.mem.size > 512 * 1024) {
- for (i = 0; i < c; i++) {
- rbo->placements[i].flags |= TTM_PL_FLAG_TOPDOWN;
- }
- }
}
int radeon_bo_create(struct radeon_device *rdev,
radeon_pm_compute_clocks(rdev);
}
-static struct radeon_ps *radeon_dpm_pick_power_state(struct radeon_device *rdev,
- enum radeon_pm_state_type dpm_state)
+static bool radeon_dpm_single_display(struct radeon_device *rdev)
{
- int i;
- struct radeon_ps *ps;
- u32 ui_class;
bool single_display = (rdev->pm.dpm.new_active_crtc_count < 2) ?
true : false;
single_display = false;
}
+ /* 120hz tends to be problematic even if they are under the
+ * vblank limit.
+ */
+ if (single_display && (r600_dpm_get_vrefresh(rdev) >= 120))
+ single_display = false;
+
+ return single_display;
+}
+
+static struct radeon_ps *radeon_dpm_pick_power_state(struct radeon_device *rdev,
+ enum radeon_pm_state_type dpm_state)
+{
+ int i;
+ struct radeon_ps *ps;
+ u32 ui_class;
+ bool single_display = radeon_dpm_single_display(rdev);
+
/* certain older asics have a separare 3D performance state,
* so try that first if the user selected performance
*/
struct radeon_ps *ps;
enum radeon_pm_state_type dpm_state;
int ret;
+ bool single_display = radeon_dpm_single_display(rdev);
/* if dpm init failed */
if (!rdev->pm.dpm_enabled)
/* vce just modifies an existing state so force a change */
if (ps->vce_active != rdev->pm.dpm.vce_active)
goto force;
+ /* user has made a display change (such as timing) */
+ if (rdev->pm.dpm.single_display != single_display)
+ goto force;
if ((rdev->family < CHIP_BARTS) || (rdev->flags & RADEON_IS_IGP)) {
/* for pre-BTC and APUs if the num crtcs changed but state is the same,
* all we need to do is update the display configuration.
rdev->pm.dpm.current_active_crtcs = rdev->pm.dpm.new_active_crtcs;
rdev->pm.dpm.current_active_crtc_count = rdev->pm.dpm.new_active_crtc_count;
+ rdev->pm.dpm.single_display = single_display;
/* wait for the rings to drain */
for (i = 0; i < RADEON_NUM_RINGS; i++) {
seq_printf(m, "%u free dwords in ring\n", ring->ring_free_dw);
seq_printf(m, "%u dwords in ring\n", count);
- if (!ring->ready)
+ if (!ring->ring)
return 0;
/* print 8 dw before current rptr as often it's the last executed
enum dma_data_direction direction = write ?
DMA_BIDIRECTIONAL : DMA_TO_DEVICE;
+ /* double check that we don't free the table twice */
+ if (!ttm->sg->sgl)
+ return;
+
/* free the sg table and pages again */
dma_unmap_sg(rdev->dev, ttm->sg->sgl, ttm->sg->nents, direction);
WREG32(R_007D18_DC_HOT_PLUG_DETECT2_INT_CONTROL, hpd2);
if (ASIC_IS_DCE2(rdev))
WREG32(R_007408_HDMI0_AUDIO_PACKET_CONTROL, hdmi0);
+
+ /* posting read */
+ RREG32(R_000040_GEN_INT_CNTL);
+
return 0;
}
}
WREG32(GRBM_CNTL, GRBM_READ_TIMEOUT(0xff));
+ WREG32(SRBM_INT_CNTL, 1);
+ WREG32(SRBM_INT_ACK, 1);
evergreen_fix_pci_max_read_req_size(rdev);
switch (pkt.type) {
case RADEON_PACKET_TYPE0:
dev_err(rdev->dev, "Packet0 not allowed!\n");
- for (i = 0; i < ib->length_dw; i++) {
- if (i == idx)
- printk("\t0x%08x <---\n", ib->ptr[i]);
- else
- printk("\t0x%08x\n", ib->ptr[i]);
- }
ret = -EINVAL;
break;
case RADEON_PACKET_TYPE2:
ret = -EINVAL;
break;
}
- if (ret)
+ if (ret) {
+ for (i = 0; i < ib->length_dw; i++) {
+ if (i == idx)
+ printk("\t0x%08x <---\n", ib->ptr[i]);
+ else
+ printk("\t0x%08x\n", ib->ptr[i]);
+ }
break;
+ }
} while (idx < ib->length_dw);
return ret;
tmp = RREG32(DMA_CNTL + DMA1_REGISTER_OFFSET) & ~TRAP_ENABLE;
WREG32(DMA_CNTL + DMA1_REGISTER_OFFSET, tmp);
WREG32(GRBM_INT_CNTL, 0);
+ WREG32(SRBM_INT_CNTL, 0);
if (rdev->num_crtc >= 2) {
WREG32(INT_MASK + EVERGREEN_CRTC0_REGISTER_OFFSET, 0);
WREG32(INT_MASK + EVERGREEN_CRTC1_REGISTER_OFFSET, 0);
WREG32(CG_THERMAL_INT, thermal_int);
+ /* posting read */
+ RREG32(SRBM_STATUS);
+
return 0;
}
break;
}
break;
+ case 96:
+ DRM_ERROR("SRBM_READ_ERROR: 0x%x\n", RREG32(SRBM_READ_ERROR));
+ WREG32(SRBM_INT_ACK, 0x1);
+ break;
case 124: /* UVD */
DRM_DEBUG("IH: UVD int: 0x%08x\n", src_data);
radeon_fence_process(rdev, R600_RING_TYPE_UVD_INDEX);
WREG32_P(CG_UPLL_FUNC_CNTL, UPLL_BYPASS_EN_MASK, ~UPLL_BYPASS_EN_MASK);
if (!vclk || !dclk) {
- /* keep the Bypass mode, put PLL to sleep */
- WREG32_P(CG_UPLL_FUNC_CNTL, UPLL_SLEEP_MASK, ~UPLL_SLEEP_MASK);
+ /* keep the Bypass mode */
return 0;
}
/* set VCO_MODE to 1 */
WREG32_P(CG_UPLL_FUNC_CNTL, UPLL_VCO_MODE_MASK, ~UPLL_VCO_MODE_MASK);
- /* toggle UPLL_SLEEP to 1 then back to 0 */
- WREG32_P(CG_UPLL_FUNC_CNTL, UPLL_SLEEP_MASK, ~UPLL_SLEEP_MASK);
+ /* disable sleep mode */
WREG32_P(CG_UPLL_FUNC_CNTL, 0, ~UPLL_SLEEP_MASK);
/* deassert UPLL_RESET */
#define CC_SYS_RB_BACKEND_DISABLE 0xe80
#define GC_USER_SYS_RB_BACKEND_DISABLE 0xe84
+#define SRBM_READ_ERROR 0xE98
+#define SRBM_INT_CNTL 0xEA0
+#define SRBM_INT_ACK 0xEA8
+
#define SRBM_STATUS2 0x0EC4
#define DMA_BUSY (1 << 5)
#define DMA1_BUSY (1 << 6)
#define DCCG_AUDIO_DTO0_PHASE 0x05b0
#define DCCG_AUDIO_DTO0_MODULE 0x05b4
-#define DCCG_AUDIO_DTO1_PHASE 0x05b8
-#define DCCG_AUDIO_DTO1_MODULE 0x05bc
+#define DCCG_AUDIO_DTO1_PHASE 0x05c0
+#define DCCG_AUDIO_DTO1_MODULE 0x05c4
#define AFMT_AUDIO_SRC_CONTROL 0x713c
#define AFMT_AUDIO_SRC_SELECT(x) (((x) & 7) << 0)
WREG32(VCE_LMI_SWAP_CNTL1, 0);
WREG32(VCE_LMI_VM_CTRL, 0);
+ WREG32(VCE_LMI_VCPU_CACHE_40BIT_BAR, addr >> 8);
+
+ addr &= 0xff;
size = RADEON_GPU_PAGE_ALIGN(rdev->vce_fw->size);
WREG32(VCE_VCPU_CACHE_OFFSET0, addr & 0x7fffffff);
WREG32(VCE_VCPU_CACHE_SIZE0, size);
crtc->state = NULL;
state = kzalloc(sizeof(*state), GFP_KERNEL);
- if (state)
+ if (state) {
crtc->state = &state->base;
+ crtc->state->crtc = crtc;
+ }
}
static struct drm_crtc_state *
return NULL;
copy->base.mode_changed = false;
+ copy->base.active_changed = false;
copy->base.planes_changed = false;
copy->base.event = NULL;
/* program display mode */
tegra_dc_set_timings(dc, mode);
- if (dc->soc->supports_border_color)
- tegra_dc_writel(dc, 0, DC_DISP_BORDER_COLOR);
-
/* interlacing isn't supported yet, so disable it */
if (dc->soc->supports_interlacing) {
value = tegra_dc_readl(dc, DC_DISP_INTERLACE_CONTROL);
static void tegra_crtc_prepare(struct drm_crtc *crtc)
{
- struct tegra_dc *dc = to_tegra_dc(crtc);
- unsigned int syncpt;
- unsigned long value;
-
drm_crtc_vblank_off(crtc);
-
- if (dc->pipe)
- syncpt = SYNCPT_VBLANK1;
- else
- syncpt = SYNCPT_VBLANK0;
-
- /* initialize display controller */
- tegra_dc_writel(dc, 0x00000100, DC_CMD_GENERAL_INCR_SYNCPT_CNTRL);
- tegra_dc_writel(dc, 0x100 | syncpt, DC_CMD_CONT_SYNCPT_VSYNC);
-
- value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT | WIN_A_OF_INT;
- tegra_dc_writel(dc, value, DC_CMD_INT_TYPE);
-
- value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT |
- WIN_A_OF_INT | WIN_B_OF_INT | WIN_C_OF_INT;
- tegra_dc_writel(dc, value, DC_CMD_INT_POLARITY);
-
- /* initialize timer */
- value = CURSOR_THRESHOLD(0) | WINDOW_A_THRESHOLD(0x20) |
- WINDOW_B_THRESHOLD(0x20) | WINDOW_C_THRESHOLD(0x20);
- tegra_dc_writel(dc, value, DC_DISP_DISP_MEM_HIGH_PRIORITY);
-
- value = CURSOR_THRESHOLD(0) | WINDOW_A_THRESHOLD(1) |
- WINDOW_B_THRESHOLD(1) | WINDOW_C_THRESHOLD(1);
- tegra_dc_writel(dc, value, DC_DISP_DISP_MEM_HIGH_PRIORITY_TIMER);
-
- value = VBLANK_INT | WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT;
- tegra_dc_writel(dc, value, DC_CMD_INT_ENABLE);
-
- value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT;
- tegra_dc_writel(dc, value, DC_CMD_INT_MASK);
}
static void tegra_crtc_commit(struct drm_crtc *crtc)
struct tegra_drm *tegra = drm->dev_private;
struct drm_plane *primary = NULL;
struct drm_plane *cursor = NULL;
+ unsigned int syncpt;
+ u32 value;
int err;
if (tegra->domain) {
goto cleanup;
}
+ /* initialize display controller */
+ if (dc->pipe)
+ syncpt = SYNCPT_VBLANK1;
+ else
+ syncpt = SYNCPT_VBLANK0;
+
+ tegra_dc_writel(dc, 0x00000100, DC_CMD_GENERAL_INCR_SYNCPT_CNTRL);
+ tegra_dc_writel(dc, 0x100 | syncpt, DC_CMD_CONT_SYNCPT_VSYNC);
+
+ value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT | WIN_A_OF_INT;
+ tegra_dc_writel(dc, value, DC_CMD_INT_TYPE);
+
+ value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT |
+ WIN_A_OF_INT | WIN_B_OF_INT | WIN_C_OF_INT;
+ tegra_dc_writel(dc, value, DC_CMD_INT_POLARITY);
+
+ /* initialize timer */
+ value = CURSOR_THRESHOLD(0) | WINDOW_A_THRESHOLD(0x20) |
+ WINDOW_B_THRESHOLD(0x20) | WINDOW_C_THRESHOLD(0x20);
+ tegra_dc_writel(dc, value, DC_DISP_DISP_MEM_HIGH_PRIORITY);
+
+ value = CURSOR_THRESHOLD(0) | WINDOW_A_THRESHOLD(1) |
+ WINDOW_B_THRESHOLD(1) | WINDOW_C_THRESHOLD(1);
+ tegra_dc_writel(dc, value, DC_DISP_DISP_MEM_HIGH_PRIORITY_TIMER);
+
+ value = VBLANK_INT | WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT;
+ tegra_dc_writel(dc, value, DC_CMD_INT_ENABLE);
+
+ value = WIN_A_UF_INT | WIN_B_UF_INT | WIN_C_UF_INT;
+ tegra_dc_writel(dc, value, DC_CMD_INT_MASK);
+
+ if (dc->soc->supports_border_color)
+ tegra_dc_writel(dc, 0, DC_DISP_BORDER_COLOR);
+
return 0;
cleanup:
h_back_porch = mode->htotal - mode->hsync_end;
h_front_porch = mode->hsync_start - mode->hdisplay;
+ err = clk_set_rate(hdmi->clk, pclk);
+ if (err < 0) {
+ dev_err(hdmi->dev, "failed to set HDMI clock frequency: %d\n",
+ err);
+ }
+
+ DRM_DEBUG_KMS("HDMI clock rate: %lu Hz\n", clk_get_rate(hdmi->clk));
+
/* power up sequence */
value = tegra_hdmi_readl(hdmi, HDMI_NV_PDISP_SOR_PLL0);
value &= ~SOR_PLL_PDBG;
pr_err(" has_type: %d\n", man->has_type);
pr_err(" use_type: %d\n", man->use_type);
pr_err(" flags: 0x%08X\n", man->flags);
- pr_err(" gpu_offset: 0x%08lX\n", man->gpu_offset);
+ pr_err(" gpu_offset: 0x%08llX\n", man->gpu_offset);
pr_err(" size: %llu\n", man->size);
pr_err(" available_caching: 0x%08X\n", man->available_caching);
pr_err(" default_caching: 0x%08X\n", man->default_caching);
goto out_err1;
}
- ret = ttm_bo_init_mm(&dev_priv->bdev, TTM_PL_VRAM,
- (dev_priv->vram_size >> PAGE_SHIFT));
- if (unlikely(ret != 0)) {
- DRM_ERROR("Failed initializing memory manager for VRAM.\n");
- goto out_err2;
- }
-
- dev_priv->has_gmr = true;
- if (((dev_priv->capabilities & (SVGA_CAP_GMR | SVGA_CAP_GMR2)) == 0) ||
- refuse_dma || ttm_bo_init_mm(&dev_priv->bdev, VMW_PL_GMR,
- VMW_PL_GMR) != 0) {
- DRM_INFO("No GMR memory available. "
- "Graphics memory resources are very limited.\n");
- dev_priv->has_gmr = false;
- }
-
- if (dev_priv->capabilities & SVGA_CAP_GBOBJECTS) {
- dev_priv->has_mob = true;
- if (ttm_bo_init_mm(&dev_priv->bdev, VMW_PL_MOB,
- VMW_PL_MOB) != 0) {
- DRM_INFO("No MOB memory available. "
- "3D will be disabled.\n");
- dev_priv->has_mob = false;
- }
- }
-
dev_priv->mmio_mtrr = arch_phys_wc_add(dev_priv->mmio_start,
dev_priv->mmio_size);
goto out_no_fman;
}
+
+ ret = ttm_bo_init_mm(&dev_priv->bdev, TTM_PL_VRAM,
+ (dev_priv->vram_size >> PAGE_SHIFT));
+ if (unlikely(ret != 0)) {
+ DRM_ERROR("Failed initializing memory manager for VRAM.\n");
+ goto out_no_vram;
+ }
+
+ dev_priv->has_gmr = true;
+ if (((dev_priv->capabilities & (SVGA_CAP_GMR | SVGA_CAP_GMR2)) == 0) ||
+ refuse_dma || ttm_bo_init_mm(&dev_priv->bdev, VMW_PL_GMR,
+ VMW_PL_GMR) != 0) {
+ DRM_INFO("No GMR memory available. "
+ "Graphics memory resources are very limited.\n");
+ dev_priv->has_gmr = false;
+ }
+
+ if (dev_priv->capabilities & SVGA_CAP_GBOBJECTS) {
+ dev_priv->has_mob = true;
+ if (ttm_bo_init_mm(&dev_priv->bdev, VMW_PL_MOB,
+ VMW_PL_MOB) != 0) {
+ DRM_INFO("No MOB memory available. "
+ "3D will be disabled.\n");
+ dev_priv->has_mob = false;
+ }
+ }
+
vmw_kms_save_vga(dev_priv);
/* Start kms and overlay systems, needs fifo. */
vmw_kms_close(dev_priv);
out_no_kms:
vmw_kms_restore_vga(dev_priv);
+ if (dev_priv->has_mob)
+ (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_MOB);
+ if (dev_priv->has_gmr)
+ (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_GMR);
+ (void)ttm_bo_clean_mm(&dev_priv->bdev, TTM_PL_VRAM);
+out_no_vram:
vmw_fence_manager_takedown(dev_priv->fman);
out_no_fman:
if (dev_priv->capabilities & SVGA_CAP_IRQMASK)
iounmap(dev_priv->mmio_virt);
out_err3:
arch_phys_wc_del(dev_priv->mmio_mtrr);
- if (dev_priv->has_mob)
- (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_MOB);
- if (dev_priv->has_gmr)
- (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_GMR);
- (void)ttm_bo_clean_mm(&dev_priv->bdev, TTM_PL_VRAM);
-out_err2:
(void)ttm_bo_device_release(&dev_priv->bdev);
out_err1:
vmw_ttm_global_release(dev_priv);
}
vmw_kms_close(dev_priv);
vmw_overlay_close(dev_priv);
+
+ if (dev_priv->has_mob)
+ (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_MOB);
+ if (dev_priv->has_gmr)
+ (void)ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_GMR);
+ (void)ttm_bo_clean_mm(&dev_priv->bdev, TTM_PL_VRAM);
+
vmw_fence_manager_takedown(dev_priv->fman);
if (dev_priv->capabilities & SVGA_CAP_IRQMASK)
drm_irq_uninstall(dev_priv->dev);
ttm_object_device_release(&dev_priv->tdev);
iounmap(dev_priv->mmio_virt);
arch_phys_wc_del(dev_priv->mmio_mtrr);
- if (dev_priv->has_mob)
- (void) ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_MOB);
- if (dev_priv->has_gmr)
- (void)ttm_bo_clean_mm(&dev_priv->bdev, VMW_PL_GMR);
- (void)ttm_bo_clean_mm(&dev_priv->bdev, TTM_PL_VRAM);
(void)ttm_bo_device_release(&dev_priv->bdev);
vmw_ttm_global_release(dev_priv);
{
struct drm_device *dev = pci_get_drvdata(pdev);
+ pci_disable_device(pdev);
drm_put_dev(dev);
}
ret = vmw_user_dmabuf_lookup(sw_context->fp->tfile, handle, &vmw_bo);
if (unlikely(ret != 0)) {
DRM_ERROR("Could not find or use MOB buffer.\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto out_no_reloc;
}
bo = &vmw_bo->base;
out_no_reloc:
vmw_dmabuf_unreference(&vmw_bo);
- vmw_bo_p = NULL;
+ *vmw_bo_p = NULL;
return ret;
}
ret = vmw_user_dmabuf_lookup(sw_context->fp->tfile, handle, &vmw_bo);
if (unlikely(ret != 0)) {
DRM_ERROR("Could not find or use GMR region.\n");
- return -EINVAL;
+ ret = -EINVAL;
+ goto out_no_reloc;
}
bo = &vmw_bo->base;
out_no_reloc:
vmw_dmabuf_unreference(&vmw_bo);
- vmw_bo_p = NULL;
+ *vmw_bo_p = NULL;
return ret;
}
NULL, arg->command_size, arg->throttle_us,
(void __user *)(unsigned long)arg->fence_rep,
NULL);
-
+ ttm_read_unlock(&dev_priv->reservation_sem);
if (unlikely(ret != 0))
- goto out_unlock;
+ return ret;
vmw_kms_cursor_post_execbuf(dev_priv);
-out_unlock:
- ttm_read_unlock(&dev_priv->reservation_sem);
- return ret;
+ return 0;
}
int i;
struct drm_mode_config *mode_config = &dev->mode_config;
- ret = ttm_read_lock(&dev_priv->reservation_sem, true);
- if (unlikely(ret != 0))
- return ret;
-
if (!arg->num_outputs) {
struct drm_vmw_rect def_rect = {0, 0, 800, 600};
vmw_du_update_layout(dev_priv, 1, &def_rect);
- goto out_unlock;
+ return 0;
}
rects_size = arg->num_outputs * sizeof(struct drm_vmw_rect);
rects = kcalloc(arg->num_outputs, sizeof(struct drm_vmw_rect),
GFP_KERNEL);
- if (unlikely(!rects)) {
- ret = -ENOMEM;
- goto out_unlock;
- }
+ if (unlikely(!rects))
+ return -ENOMEM;
user_rects = (void __user *)(unsigned long)arg->rects;
ret = copy_from_user(rects, user_rects, rects_size);
out_free:
kfree(rects);
-out_unlock:
- ttm_read_unlock(&dev_priv->reservation_sem);
return ret;
}
clkrate = clk_get_rate(di->clk_ipu);
div = DIV_ROUND_CLOSEST(clkrate, sig->mode.pixelclock);
+ if (div == 0)
+ div = 1;
rate = clkrate / div;
error = rate / (sig->mode.pixelclock / 1000);
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_SIDEWINDER_GV) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_NE4K) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_NE4K_JP) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_NE7K) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_LK6K) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_PRESENTER_8K_USB) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_DIGITAL_MEDIA_3K) },
#endif
#if IS_ENABLED(CONFIG_HID_SAITEK)
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_PS1000) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7_OLD) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7) },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_MMO7) },
{ HID_USB_DEVICE(USB_VENDOR_ID_MADCATZ, USB_DEVICE_ID_MADCATZ_RAT9) },
{ HID_USB_DEVICE(USB_VENDOR_ID_THRUSTMASTER, 0xb65a) },
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE_BT) },
{ HID_USB_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE_PRO) },
{ HID_USB_DEVICE(USB_VENDOR_ID_TOPSEED, USB_DEVICE_ID_TOPSEED_CYBERLINK) },
{ HID_USB_DEVICE(USB_VENDOR_ID_TOPSEED2, USB_DEVICE_ID_TOPSEED2_RF_COMBO) },
{ HID_USB_DEVICE(USB_VENDOR_ID_TWINHAN, USB_DEVICE_ID_TWINHAN_IR_REMOTE) },
#define USB_VENDOR_ID_LOGITECH 0x046d
#define USB_DEVICE_ID_LOGITECH_AUDIOHUB 0x0a0e
#define USB_DEVICE_ID_LOGITECH_T651 0xb00c
+#define USB_DEVICE_ID_LOGITECH_C077 0xc007
#define USB_DEVICE_ID_LOGITECH_RECEIVER 0xc101
#define USB_DEVICE_ID_LOGITECH_HARMONY_FIRST 0xc110
#define USB_DEVICE_ID_LOGITECH_HARMONY_LAST 0xc14f
#define USB_DEVICE_ID_MS_LK6K 0x00f9
#define USB_DEVICE_ID_MS_PRESENTER_8K_BT 0x0701
#define USB_DEVICE_ID_MS_PRESENTER_8K_USB 0x0713
+#define USB_DEVICE_ID_MS_NE7K 0x071d
#define USB_DEVICE_ID_MS_DIGITAL_MEDIA_3K 0x0730
#define USB_DEVICE_ID_MS_COMFORT_MOUSE_4500 0x076c
#define USB_DEVICE_ID_MS_SURFACE_PRO_2 0x0799
#define USB_VENDOR_ID_SAITEK 0x06a3
#define USB_DEVICE_ID_SAITEK_RUMBLEPAD 0xff17
#define USB_DEVICE_ID_SAITEK_PS1000 0x0621
+#define USB_DEVICE_ID_SAITEK_RAT7_OLD 0x0ccb
#define USB_DEVICE_ID_SAITEK_RAT7 0x0cd7
#define USB_DEVICE_ID_SAITEK_MMO7 0x0cd0
#define USB_VENDOR_ID_TIVO 0x150a
#define USB_DEVICE_ID_TIVO_SLIDE_BT 0x1200
#define USB_DEVICE_ID_TIVO_SLIDE 0x1201
+#define USB_DEVICE_ID_TIVO_SLIDE_PRO 0x1203
#define USB_VENDOR_ID_TOPSEED 0x0766
#define USB_DEVICE_ID_TOPSEED_CYBERLINK 0x0204
enum power_supply_property prop,
union power_supply_propval *val)
{
- struct hid_device *dev = container_of(psy, struct hid_device, battery);
+ struct hid_device *dev = power_supply_get_drvdata(psy);
int ret = 0;
__u8 *buf;
static bool hidinput_setup_battery(struct hid_device *dev, unsigned report_type, struct hid_field *field)
{
- struct power_supply *battery = &dev->battery;
- int ret;
+ struct power_supply_desc *psy_desc = NULL;
+ struct power_supply_config psy_cfg = { .drv_data = dev, };
unsigned quirks;
s32 min, max;
if (field->usage->hid != HID_DC_BATTERYSTRENGTH)
return false; /* no match */
- if (battery->name != NULL)
+ if (dev->battery != NULL)
goto out; /* already initialized? */
- battery->name = kasprintf(GFP_KERNEL, "hid-%s-battery", dev->uniq);
- if (battery->name == NULL)
+ psy_desc = kzalloc(sizeof(*psy_desc), GFP_KERNEL);
+ if (psy_desc == NULL)
goto out;
- battery->type = POWER_SUPPLY_TYPE_BATTERY;
- battery->properties = hidinput_battery_props;
- battery->num_properties = ARRAY_SIZE(hidinput_battery_props);
- battery->use_for_apm = 0;
- battery->get_property = hidinput_get_battery_property;
+ psy_desc->name = kasprintf(GFP_KERNEL, "hid-%s-battery", dev->uniq);
+ if (psy_desc->name == NULL) {
+ kfree(psy_desc);
+ goto out;
+ }
+
+ psy_desc->type = POWER_SUPPLY_TYPE_BATTERY;
+ psy_desc->properties = hidinput_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(hidinput_battery_props);
+ psy_desc->use_for_apm = 0;
+ psy_desc->get_property = hidinput_get_battery_property;
quirks = find_battery_quirk(dev);
dev->battery_report_type = report_type;
dev->battery_report_id = field->report->id;
- ret = power_supply_register(&dev->dev, battery);
- if (ret != 0) {
- hid_warn(dev, "can't register power supply: %d\n", ret);
- kfree(battery->name);
- battery->name = NULL;
+ dev->battery = power_supply_register(&dev->dev, psy_desc, &psy_cfg);
+ if (IS_ERR(dev->battery)) {
+ hid_warn(dev, "can't register power supply: %ld\n",
+ PTR_ERR(dev->battery));
+ kfree(psy_desc->name);
+ kfree(psy_desc);
+ dev->battery = NULL;
+ } else {
+ power_supply_powers(dev->battery, &dev->dev);
}
- power_supply_powers(battery, &dev->dev);
-
out:
return true;
}
static void hidinput_cleanup_battery(struct hid_device *dev)
{
- if (!dev->battery.name)
+ if (!dev->battery)
return;
- power_supply_unregister(&dev->battery);
- kfree(dev->battery.name);
- dev->battery.name = NULL;
+ power_supply_unregister(dev->battery);
+ kfree(dev->battery->desc->name);
+ kfree(dev->battery->desc);
+ dev->battery = NULL;
}
#else /* !CONFIG_HID_BATTERY_STRENGTH */
static bool hidinput_setup_battery(struct hid_device *dev, unsigned report_type,
.driver_data = MS_ERGONOMY },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_NE4K_JP),
.driver_data = MS_ERGONOMY },
+ { HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_NE7K),
+ .driver_data = MS_ERGONOMY },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_LK6K),
.driver_data = MS_ERGONOMY | MS_RDESC },
{ HID_USB_DEVICE(USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_PRESENTER_8K_USB),
static const struct hid_device_id saitek_devices[] = {
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_PS1000),
.driver_data = SAITEK_FIX_PS1000 },
+ { HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7_OLD),
+ .driver_data = SAITEK_RELEASE_MODE_RAT7 },
{ HID_USB_DEVICE(USB_VENDOR_ID_SAITEK, USB_DEVICE_ID_SAITEK_RAT7),
.driver_data = SAITEK_RELEASE_MODE_RAT7 },
{ HID_USB_DEVICE(USB_VENDOR_ID_MADCATZ, USB_DEVICE_ID_MADCATZ_RAT9),
{
struct hid_sensor_hub_callbacks_list *callback;
struct sensor_hub_data *pdata = hid_get_drvdata(hdev);
+ unsigned long flags;
- spin_lock(&pdata->dyn_callback_lock);
+ spin_lock_irqsave(&pdata->dyn_callback_lock, flags);
list_for_each_entry(callback, &pdata->dyn_callback_list, list)
if (callback->usage_id == usage_id &&
(collection_index >=
callback->hsdev->end_collection_index)) {
*priv = callback->priv;
*hsdev = callback->hsdev;
- spin_unlock(&pdata->dyn_callback_lock);
+ spin_unlock_irqrestore(&pdata->dyn_callback_lock,
+ flags);
return callback->usage_callback;
}
- spin_unlock(&pdata->dyn_callback_lock);
+ spin_unlock_irqrestore(&pdata->dyn_callback_lock, flags);
return NULL;
}
#define DS4_REPORT_0x81_SIZE 7
#define SIXAXIS_REPORT_0xF2_SIZE 18
-static spinlock_t sony_dev_list_lock;
+static DEFINE_SPINLOCK(sony_dev_list_lock);
static LIST_HEAD(sony_device_list);
static DEFINE_IDA(sony_device_id_allocator);
struct led_classdev *leds[MAX_LEDS];
unsigned long quirks;
struct work_struct state_worker;
- struct power_supply battery;
+ struct power_supply *battery;
+ struct power_supply_desc battery_desc;
int device_id;
__u8 *output_report_dmabuf;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct sony_sc *sc = container_of(psy, struct sony_sc, battery);
+ struct sony_sc *sc = power_supply_get_drvdata(psy);
unsigned long flags;
int ret = 0;
u8 battery_charging, battery_capacity, cable_state;
static int sony_battery_probe(struct sony_sc *sc)
{
+ struct power_supply_config psy_cfg = { .drv_data = sc, };
struct hid_device *hdev = sc->hdev;
int ret;
*/
sc->battery_capacity = 100;
- sc->battery.properties = sony_battery_props;
- sc->battery.num_properties = ARRAY_SIZE(sony_battery_props);
- sc->battery.get_property = sony_battery_get_property;
- sc->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- sc->battery.use_for_apm = 0;
- sc->battery.name = kasprintf(GFP_KERNEL, "sony_controller_battery_%pMR",
- sc->mac_address);
- if (!sc->battery.name)
+ sc->battery_desc.properties = sony_battery_props;
+ sc->battery_desc.num_properties = ARRAY_SIZE(sony_battery_props);
+ sc->battery_desc.get_property = sony_battery_get_property;
+ sc->battery_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ sc->battery_desc.use_for_apm = 0;
+ sc->battery_desc.name = kasprintf(GFP_KERNEL,
+ "sony_controller_battery_%pMR",
+ sc->mac_address);
+ if (!sc->battery_desc.name)
return -ENOMEM;
- ret = power_supply_register(&hdev->dev, &sc->battery);
- if (ret) {
+ sc->battery = power_supply_register(&hdev->dev, &sc->battery_desc,
+ &psy_cfg);
+ if (IS_ERR(sc->battery)) {
+ ret = PTR_ERR(sc->battery);
hid_err(hdev, "Unable to register battery device\n");
goto err_free;
}
- power_supply_powers(&sc->battery, &hdev->dev);
+ power_supply_powers(sc->battery, &hdev->dev);
return 0;
err_free:
- kfree(sc->battery.name);
- sc->battery.name = NULL;
+ kfree(sc->battery_desc.name);
+ sc->battery_desc.name = NULL;
return ret;
}
static void sony_battery_remove(struct sony_sc *sc)
{
- if (!sc->battery.name)
+ if (!sc->battery_desc.name)
return;
- power_supply_unregister(&sc->battery);
- kfree(sc->battery.name);
- sc->battery.name = NULL;
+ power_supply_unregister(sc->battery);
+ kfree(sc->battery_desc.name);
+ sc->battery_desc.name = NULL;
}
/*
return -ENOMEM;
}
+ spin_lock_init(&sc->lock);
+
sc->quirks = quirks;
hid_set_drvdata(hdev, sc);
sc->hdev = hdev;
{
dbg_hid("Sony:%s\n", __func__);
- ida_destroy(&sony_device_id_allocator);
hid_unregister_driver(&sony_driver);
+ ida_destroy(&sony_device_id_allocator);
}
module_init(sony_init);
module_exit(sony_exit);
/* TiVo Slide Bluetooth remote, pairs with a Broadcom dongle */
{ HID_BLUETOOTH_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE_BT) },
{ HID_USB_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE) },
+ { HID_USB_DEVICE(USB_VENDOR_ID_TIVO, USB_DEVICE_ID_TIVO_SLIDE_PRO) },
{ }
};
MODULE_DEVICE_TABLE(hid, tivo_devices);
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wiimote_data *wdata = container_of(psy, struct wiimote_data,
- battery);
+ struct wiimote_data *wdata = power_supply_get_drvdata(psy);
int ret = 0, state;
unsigned long flags;
static int wiimod_battery_probe(const struct wiimod_ops *ops,
struct wiimote_data *wdata)
{
+ struct power_supply_config psy_cfg = { .drv_data = wdata, };
int ret;
- wdata->battery.properties = wiimod_battery_props;
- wdata->battery.num_properties = ARRAY_SIZE(wiimod_battery_props);
- wdata->battery.get_property = wiimod_battery_get_property;
- wdata->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- wdata->battery.use_for_apm = 0;
- wdata->battery.name = kasprintf(GFP_KERNEL, "wiimote_battery_%s",
- wdata->hdev->uniq);
- if (!wdata->battery.name)
+ wdata->battery_desc.properties = wiimod_battery_props;
+ wdata->battery_desc.num_properties = ARRAY_SIZE(wiimod_battery_props);
+ wdata->battery_desc.get_property = wiimod_battery_get_property;
+ wdata->battery_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ wdata->battery_desc.use_for_apm = 0;
+ wdata->battery_desc.name = kasprintf(GFP_KERNEL, "wiimote_battery_%s",
+ wdata->hdev->uniq);
+ if (!wdata->battery_desc.name)
return -ENOMEM;
- ret = power_supply_register(&wdata->hdev->dev, &wdata->battery);
- if (ret) {
+ wdata->battery = power_supply_register(&wdata->hdev->dev,
+ &wdata->battery_desc,
+ &psy_cfg);
+ if (IS_ERR(wdata->battery)) {
hid_err(wdata->hdev, "cannot register battery device\n");
+ ret = PTR_ERR(wdata->battery);
goto err_free;
}
- power_supply_powers(&wdata->battery, &wdata->hdev->dev);
+ power_supply_powers(wdata->battery, &wdata->hdev->dev);
return 0;
err_free:
- kfree(wdata->battery.name);
- wdata->battery.name = NULL;
+ kfree(wdata->battery_desc.name);
+ wdata->battery_desc.name = NULL;
return ret;
}
static void wiimod_battery_remove(const struct wiimod_ops *ops,
struct wiimote_data *wdata)
{
- if (!wdata->battery.name)
+ if (!wdata->battery_desc.name)
return;
- power_supply_unregister(&wdata->battery);
- kfree(wdata->battery.name);
- wdata->battery.name = NULL;
+ power_supply_unregister(wdata->battery);
+ kfree(wdata->battery_desc.name);
+ wdata->battery_desc.name = NULL;
}
static const struct wiimod_ops wiimod_battery = {
struct led_classdev *leds[4];
struct input_dev *accel;
struct input_dev *ir;
- struct power_supply battery;
+ struct power_supply *battery;
+ struct power_supply_desc battery_desc;
struct input_dev *mp;
struct timer_list timer;
struct wiimote_debug *debug;
static void i2c_hid_get_input(struct i2c_hid *ihid)
{
int ret, ret_size;
- int size = ihid->bufsize;
+ int size = le16_to_cpu(ihid->hdesc.wMaxInputLength);
+
+ if (size > ihid->bufsize)
+ size = ihid->bufsize;
ret = i2c_master_recv(ihid->client, ihid->inbuf, size);
if (ret != size) {
dev_dbg(&client->dev, "Requesting IRQ: %d\n", client->irq);
ret = request_threaded_irq(client->irq, NULL, i2c_hid_irq,
- IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
+ IRQF_TRIGGER_LOW | IRQF_ONESHOT,
client->name, ihid);
if (ret < 0) {
dev_warn(&client->dev,
{ USB_VENDOR_ID_ELO, USB_DEVICE_ID_ELO_TS2700, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_FORMOSA, USB_DEVICE_ID_FORMOSA_IR_RECEIVER, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_FREESCALE, USB_DEVICE_ID_FREESCALE_MX28, HID_QUIRK_NOGET },
+ { USB_VENDOR_ID_LOGITECH, USB_DEVICE_ID_LOGITECH_C077, HID_QUIRK_ALWAYS_POLL },
{ USB_VENDOR_ID_MGE, USB_DEVICE_ID_MGE_UPS, HID_QUIRK_NOGET },
{ USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_TYPE_COVER_3, HID_QUIRK_NO_INIT_REPORTS },
{ USB_VENDOR_ID_MICROSOFT, USB_DEVICE_ID_MS_TYPE_COVER_3_JP, HID_QUIRK_NO_INIT_REPORTS },
u8 img_lum; /* OLED matrix display brightness */
} led;
bool led_initialized;
- struct power_supply battery;
- struct power_supply ac;
+ struct power_supply *battery;
+ struct power_supply *ac;
+ struct power_supply_desc battery_desc;
+ struct power_supply_desc ac_desc;
};
static inline void wacom_schedule_work(struct wacom_wac *wacom_wac)
{
struct wacom *wacom = container_of(wacom_wac, struct wacom, wacom_wac);
- power_supply_changed(&wacom->battery);
+ power_supply_changed(wacom->battery);
}
extern const struct hid_device_id wacom_ids[];
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wacom *wacom = container_of(psy, struct wacom, battery);
+ struct wacom *wacom = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wacom *wacom = container_of(psy, struct wacom, ac);
+ struct wacom *wacom = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
static int wacom_initialize_battery(struct wacom *wacom)
{
static atomic_t battery_no = ATOMIC_INIT(0);
- int error;
+ struct power_supply_config psy_cfg = { .drv_data = wacom, };
unsigned long n;
if (wacom->wacom_wac.features.quirks & WACOM_QUIRK_BATTERY) {
+ struct power_supply_desc *bat_desc = &wacom->battery_desc;
+ struct power_supply_desc *ac_desc = &wacom->ac_desc;
n = atomic_inc_return(&battery_no) - 1;
- wacom->battery.properties = wacom_battery_props;
- wacom->battery.num_properties = ARRAY_SIZE(wacom_battery_props);
- wacom->battery.get_property = wacom_battery_get_property;
+ bat_desc->properties = wacom_battery_props;
+ bat_desc->num_properties = ARRAY_SIZE(wacom_battery_props);
+ bat_desc->get_property = wacom_battery_get_property;
sprintf(wacom->wacom_wac.bat_name, "wacom_battery_%ld", n);
- wacom->battery.name = wacom->wacom_wac.bat_name;
- wacom->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- wacom->battery.use_for_apm = 0;
+ bat_desc->name = wacom->wacom_wac.bat_name;
+ bat_desc->type = POWER_SUPPLY_TYPE_BATTERY;
+ bat_desc->use_for_apm = 0;
- wacom->ac.properties = wacom_ac_props;
- wacom->ac.num_properties = ARRAY_SIZE(wacom_ac_props);
- wacom->ac.get_property = wacom_ac_get_property;
+ ac_desc->properties = wacom_ac_props;
+ ac_desc->num_properties = ARRAY_SIZE(wacom_ac_props);
+ ac_desc->get_property = wacom_ac_get_property;
sprintf(wacom->wacom_wac.ac_name, "wacom_ac_%ld", n);
- wacom->ac.name = wacom->wacom_wac.ac_name;
- wacom->ac.type = POWER_SUPPLY_TYPE_MAINS;
- wacom->ac.use_for_apm = 0;
-
- error = power_supply_register(&wacom->hdev->dev,
- &wacom->battery);
- if (error)
- return error;
-
- power_supply_powers(&wacom->battery, &wacom->hdev->dev);
-
- error = power_supply_register(&wacom->hdev->dev, &wacom->ac);
- if (error) {
- power_supply_unregister(&wacom->battery);
- return error;
+ ac_desc->name = wacom->wacom_wac.ac_name;
+ ac_desc->type = POWER_SUPPLY_TYPE_MAINS;
+ ac_desc->use_for_apm = 0;
+
+ wacom->battery = power_supply_register(&wacom->hdev->dev,
+ &wacom->battery_desc, &psy_cfg);
+ if (IS_ERR(wacom->battery))
+ return PTR_ERR(wacom->battery);
+
+ power_supply_powers(wacom->battery, &wacom->hdev->dev);
+
+ wacom->ac = power_supply_register(&wacom->hdev->dev,
+ &wacom->ac_desc,
+ &psy_cfg);
+ if (IS_ERR(wacom->ac)) {
+ power_supply_unregister(wacom->battery);
+ return PTR_ERR(wacom->ac);
}
- power_supply_powers(&wacom->ac, &wacom->hdev->dev);
+ power_supply_powers(wacom->ac, &wacom->hdev->dev);
}
return 0;
static void wacom_destroy_battery(struct wacom *wacom)
{
if ((wacom->wacom_wac.features.quirks & WACOM_QUIRK_BATTERY) &&
- wacom->battery.dev) {
- power_supply_unregister(&wacom->battery);
- wacom->battery.dev = NULL;
- power_supply_unregister(&wacom->ac);
- wacom->ac.dev = NULL;
+ wacom->battery) {
+ power_supply_unregister(wacom->battery);
+ wacom->battery = NULL;
+ power_supply_unregister(wacom->ac);
+ wacom->ac = NULL;
}
}
(features->type == CINTIQ && !(data[1] & 0x40)))
return 1;
- if (features->quirks & WACOM_QUIRK_MULTI_INPUT)
+ if (wacom->shared) {
wacom->shared->stylus_in_proximity = true;
+ if (wacom->shared->touch_down)
+ return 1;
+ }
+
/* in Range while exiting */
if (((data[1] & 0xfe) == 0x20) && wacom->reporting_data) {
input_report_key(input, BTN_TOUCH, 0);
input_report_abs(input, ABS_X, be16_to_cpup((__be16 *)&data[4]));
input_report_abs(input, ABS_Y, be16_to_cpup((__be16 *)&data[6]));
input_report_abs(input, ABS_Z, be16_to_cpup((__be16 *)&data[8]));
+ if ((data[2] & 0x07) | data[4] | data[5] | data[6] | data[7] | data[8] | data[9]) {
+ input_report_abs(input, ABS_MISC, PAD_DEVICE_ID);
+ } else {
+ input_report_abs(input, ABS_MISC, 0);
+ }
} else if (features->type == CINTIQ_HYBRID) {
/*
* Do not send hardware buttons under Android. They
struct input_dev *input = wacom->input;
unsigned char *data = wacom->data;
int i;
- int current_num_contacts = 0;
+ int current_num_contacts = data[61];
int contacts_to_send = 0;
int num_contacts_left = 4; /* maximum contacts per packet */
int byte_per_packet = WACOM_BYTES_PER_24HDT_PACKET;
int y_offset = 2;
+ static int contact_with_no_pen_down_count = 0;
if (wacom->features.type == WACOM_27QHDT) {
current_num_contacts = data[63];
num_contacts_left = 10;
byte_per_packet = WACOM_BYTES_PER_QHDTHID_PACKET;
y_offset = 0;
- } else {
- current_num_contacts = data[61];
}
/*
* First packet resets the counter since only the first
* packet in series will have non-zero current_num_contacts.
*/
- if (current_num_contacts)
+ if (current_num_contacts) {
wacom->num_contacts_left = current_num_contacts;
+ contact_with_no_pen_down_count = 0;
+ }
contacts_to_send = min(num_contacts_left, wacom->num_contacts_left);
input_report_abs(input, ABS_MT_WIDTH_MINOR, min(w, h));
input_report_abs(input, ABS_MT_ORIENTATION, w > h);
}
+ contact_with_no_pen_down_count++;
}
}
input_mt_report_pointer_emulation(input, true);
wacom->num_contacts_left -= contacts_to_send;
- if (wacom->num_contacts_left <= 0)
+ if (wacom->num_contacts_left <= 0) {
wacom->num_contacts_left = 0;
-
- wacom->shared->touch_down = (wacom->num_contacts_left > 0);
+ wacom->shared->touch_down = (contact_with_no_pen_down_count > 0);
+ }
return 1;
}
int current_num_contacts = data[2];
int contacts_to_send = 0;
int x_offset = 0;
+ static int contact_with_no_pen_down_count = 0;
/* MTTPC does not support Height and Width */
if (wacom->features.type == MTTPC || wacom->features.type == MTTPC_B)
* First packet resets the counter since only the first
* packet in series will have non-zero current_num_contacts.
*/
- if (current_num_contacts)
+ if (current_num_contacts) {
wacom->num_contacts_left = current_num_contacts;
+ contact_with_no_pen_down_count = 0;
+ }
/* There are at most 5 contacts per packet */
contacts_to_send = min(5, wacom->num_contacts_left);
int y = get_unaligned_le16(&data[offset + x_offset + 9]);
input_report_abs(input, ABS_MT_POSITION_X, x);
input_report_abs(input, ABS_MT_POSITION_Y, y);
+ contact_with_no_pen_down_count++;
}
}
input_mt_report_pointer_emulation(input, true);
wacom->num_contacts_left -= contacts_to_send;
- if (wacom->num_contacts_left < 0)
+ if (wacom->num_contacts_left <= 0) {
wacom->num_contacts_left = 0;
-
- wacom->shared->touch_down = (wacom->num_contacts_left > 0);
+ wacom->shared->touch_down = (contact_with_no_pen_down_count > 0);
+ }
return 1;
}
{
unsigned char *data = wacom->data;
struct input_dev *input = wacom->input;
- bool prox;
+ bool prox = !wacom->shared->stylus_in_proximity;
int x = 0, y = 0;
if (wacom->features.touch_max > 1 || len > WACOM_PKGLEN_TPC2FG)
return 0;
- if (!wacom->shared->stylus_in_proximity) {
- if (len == WACOM_PKGLEN_TPC1FG) {
- prox = data[0] & 0x01;
- x = get_unaligned_le16(&data[1]);
- y = get_unaligned_le16(&data[3]);
- } else if (len == WACOM_PKGLEN_TPC1FG_B) {
- prox = data[2] & 0x01;
- x = get_unaligned_le16(&data[3]);
- y = get_unaligned_le16(&data[5]);
- } else {
- prox = data[1] & 0x01;
- x = le16_to_cpup((__le16 *)&data[2]);
- y = le16_to_cpup((__le16 *)&data[4]);
- }
- } else
- /* force touch out when pen is in prox */
- prox = 0;
+ if (len == WACOM_PKGLEN_TPC1FG) {
+ prox = prox && (data[0] & 0x01);
+ x = get_unaligned_le16(&data[1]);
+ y = get_unaligned_le16(&data[3]);
+ } else if (len == WACOM_PKGLEN_TPC1FG_B) {
+ prox = prox && (data[2] & 0x01);
+ x = get_unaligned_le16(&data[3]);
+ y = get_unaligned_le16(&data[5]);
+ } else {
+ prox = prox && (data[1] & 0x01);
+ x = le16_to_cpup((__le16 *)&data[2]);
+ y = le16_to_cpup((__le16 *)&data[4]);
+ }
if (prox) {
input_report_abs(input, ABS_X, x);
struct input_dev *pad_input = wacom->pad_input;
unsigned char *data = wacom->data;
int i;
+ int contact_with_no_pen_down_count = 0;
if (data[0] != 0x02)
return 0;
}
input_report_abs(input, ABS_MT_POSITION_X, x);
input_report_abs(input, ABS_MT_POSITION_Y, y);
+ contact_with_no_pen_down_count++;
}
}
input_report_key(pad_input, BTN_FORWARD, (data[1] & 0x04) != 0);
input_report_key(pad_input, BTN_BACK, (data[1] & 0x02) != 0);
input_report_key(pad_input, BTN_RIGHT, (data[1] & 0x01) != 0);
+ wacom->shared->touch_down = (contact_with_no_pen_down_count > 0);
return 1;
}
-static void wacom_bpt3_touch_msg(struct wacom_wac *wacom, unsigned char *data)
+static int wacom_bpt3_touch_msg(struct wacom_wac *wacom, unsigned char *data, int last_touch_count)
{
struct wacom_features *features = &wacom->features;
struct input_dev *input = wacom->input;
int slot = input_mt_get_slot_by_key(input, data[0]);
if (slot < 0)
- return;
+ return 0;
touch = touch && !wacom->shared->stylus_in_proximity;
input_report_abs(input, ABS_MT_POSITION_Y, y);
input_report_abs(input, ABS_MT_TOUCH_MAJOR, width);
input_report_abs(input, ABS_MT_TOUCH_MINOR, height);
+ last_touch_count++;
}
+ return last_touch_count;
}
static void wacom_bpt3_button_msg(struct wacom_wac *wacom, unsigned char *data)
unsigned char *data = wacom->data;
int count = data[1] & 0x07;
int i;
+ int contact_with_no_pen_down_count = 0;
if (data[0] != 0x02)
return 0;
int msg_id = data[offset];
if (msg_id >= 2 && msg_id <= 17)
- wacom_bpt3_touch_msg(wacom, data + offset);
+ contact_with_no_pen_down_count =
+ wacom_bpt3_touch_msg(wacom, data + offset,
+ contact_with_no_pen_down_count);
else if (msg_id == 128)
wacom_bpt3_button_msg(wacom, data + offset);
}
input_mt_report_pointer_emulation(input, true);
+ wacom->shared->touch_down = (contact_with_no_pen_down_count > 0);
return 1;
}
return 0;
}
+ if (wacom->shared->touch_down)
+ return 0;
+
prox = (data[1] & 0x20) == 0x20;
/*
.oVid = USB_VENDOR_ID_WACOM, .oPid = 0xf8, .touch_max = 10,
.check_for_hid_type = true, .hid_type = HID_TYPE_USBNONE };
static const struct wacom_features wacom_features_0x32A =
- { "Wacom Cintiq 27QHD", 119740, 67520, 2047,
- 63, WACOM_27QHD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
- WACOM_27QHD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES };
+ { "Wacom Cintiq 27QHD", 119740, 67520, 2047, 63,
+ WACOM_27QHD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
+ WACOM_CINTIQ_OFFSET, WACOM_CINTIQ_OFFSET };
static const struct wacom_features wacom_features_0x32B =
{ "Wacom Cintiq 27QHD touch", 119740, 67520, 2047, 63,
WACOM_27QHD, WACOM_INTUOS3_RES, WACOM_INTUOS3_RES,
config NOKIA_MODEM
tristate "Nokia Modem"
- depends on HSI && SSI_PROTOCOL
+ depends on HSI && SSI_PROTOCOL && CMT_SPEECH
help
Say Y here if you want to add support for the modem on Nokia
N900 (Nokia RX-51) hardware.
If unsure, say N.
+config CMT_SPEECH
+ tristate "CMT speech"
+ depends on HSI && SSI_PROTOCOL
+ help
+ If you say Y here, you will enable the CMT speech protocol used
+ by Nokia modems. If you say M the protocol will be available as
+ module named cmt_speech.
+
+ If unsure, say N.
+
config SSI_PROTOCOL
tristate "SSI protocol"
depends on HSI && PHONET && OMAP_SSI
obj-$(CONFIG_NOKIA_MODEM) += nokia-modem.o
obj-$(CONFIG_SSI_PROTOCOL) += ssi_protocol.o
+obj-$(CONFIG_CMT_SPEECH) += cmt_speech.o
obj-$(CONFIG_HSI_CHAR) += hsi_char.o
--- /dev/null
+/*
+ * cmt_speech.c - HSI CMT speech driver
+ *
+ * Copyright (C) 2008,2009,2010 Nokia Corporation. All rights reserved.
+ *
+ * Contact: Kai Vehmanen <kai.vehmanen@nokia.com>
+ * Original author: Peter Ujfalusi <peter.ujfalusi@nokia.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+ * 02110-1301 USA
+ */
+
+#include <linux/errno.h>
+#include <linux/module.h>
+#include <linux/types.h>
+#include <linux/init.h>
+#include <linux/device.h>
+#include <linux/miscdevice.h>
+#include <linux/mm.h>
+#include <linux/slab.h>
+#include <linux/fs.h>
+#include <linux/poll.h>
+#include <linux/sched.h>
+#include <linux/ioctl.h>
+#include <linux/uaccess.h>
+#include <linux/pm_qos.h>
+#include <linux/hsi/hsi.h>
+#include <linux/hsi/ssi_protocol.h>
+#include <linux/hsi/cs-protocol.h>
+
+#define CS_MMAP_SIZE PAGE_SIZE
+
+struct char_queue {
+ struct list_head list;
+ u32 msg;
+};
+
+struct cs_char {
+ unsigned int opened;
+ struct hsi_client *cl;
+ struct cs_hsi_iface *hi;
+ struct list_head chardev_queue;
+ struct list_head dataind_queue;
+ int dataind_pending;
+ /* mmap things */
+ unsigned long mmap_base;
+ unsigned long mmap_size;
+ spinlock_t lock;
+ struct fasync_struct *async_queue;
+ wait_queue_head_t wait;
+ /* hsi channel ids */
+ int channel_id_cmd;
+ int channel_id_data;
+};
+
+#define SSI_CHANNEL_STATE_READING 1
+#define SSI_CHANNEL_STATE_WRITING (1 << 1)
+#define SSI_CHANNEL_STATE_POLL (1 << 2)
+#define SSI_CHANNEL_STATE_ERROR (1 << 3)
+
+#define TARGET_MASK 0xf000000
+#define TARGET_REMOTE (1 << CS_DOMAIN_SHIFT)
+#define TARGET_LOCAL 0
+
+/* Number of pre-allocated commands buffers */
+#define CS_MAX_CMDS 4
+
+/*
+ * During data transfers, transactions must be handled
+ * within 20ms (fixed value in cmtspeech HSI protocol)
+ */
+#define CS_QOS_LATENCY_FOR_DATA_USEC 20000
+
+/* Timeout to wait for pending HSI transfers to complete */
+#define CS_HSI_TRANSFER_TIMEOUT_MS 500
+
+
+#define RX_PTR_BOUNDARY_SHIFT 8
+#define RX_PTR_MAX_SHIFT (RX_PTR_BOUNDARY_SHIFT + \
+ CS_MAX_BUFFERS_SHIFT)
+struct cs_hsi_iface {
+ struct hsi_client *cl;
+ struct hsi_client *master;
+
+ unsigned int iface_state;
+ unsigned int wakeline_state;
+ unsigned int control_state;
+ unsigned int data_state;
+
+ /* state exposed to application */
+ struct cs_mmap_config_block *mmap_cfg;
+
+ unsigned long mmap_base;
+ unsigned long mmap_size;
+
+ unsigned int rx_slot;
+ unsigned int tx_slot;
+
+ /* note: for security reasons, we do not trust the contents of
+ * mmap_cfg, but instead duplicate the variables here */
+ unsigned int buf_size;
+ unsigned int rx_bufs;
+ unsigned int tx_bufs;
+ unsigned int rx_ptr_boundary;
+ unsigned int rx_offsets[CS_MAX_BUFFERS];
+ unsigned int tx_offsets[CS_MAX_BUFFERS];
+
+ /* size of aligned memory blocks */
+ unsigned int slot_size;
+ unsigned int flags;
+
+ struct list_head cmdqueue;
+
+ struct hsi_msg *data_rx_msg;
+ struct hsi_msg *data_tx_msg;
+ wait_queue_head_t datawait;
+
+ struct pm_qos_request pm_qos_req;
+
+ spinlock_t lock;
+};
+
+static struct cs_char cs_char_data;
+
+static void cs_hsi_read_on_control(struct cs_hsi_iface *hi);
+static void cs_hsi_read_on_data(struct cs_hsi_iface *hi);
+
+static inline void rx_ptr_shift_too_big(void)
+{
+ BUILD_BUG_ON((1LLU << RX_PTR_MAX_SHIFT) > UINT_MAX);
+}
+
+static void cs_notify(u32 message, struct list_head *head)
+{
+ struct char_queue *entry;
+
+ spin_lock(&cs_char_data.lock);
+
+ if (!cs_char_data.opened) {
+ spin_unlock(&cs_char_data.lock);
+ goto out;
+ }
+
+ entry = kmalloc(sizeof(*entry), GFP_ATOMIC);
+ if (!entry) {
+ dev_err(&cs_char_data.cl->device,
+ "Can't allocate new entry for the queue.\n");
+ spin_unlock(&cs_char_data.lock);
+ goto out;
+ }
+
+ entry->msg = message;
+ list_add_tail(&entry->list, head);
+
+ spin_unlock(&cs_char_data.lock);
+
+ wake_up_interruptible(&cs_char_data.wait);
+ kill_fasync(&cs_char_data.async_queue, SIGIO, POLL_IN);
+
+out:
+ return;
+}
+
+static u32 cs_pop_entry(struct list_head *head)
+{
+ struct char_queue *entry;
+ u32 data;
+
+ entry = list_entry(head->next, struct char_queue, list);
+ data = entry->msg;
+ list_del(&entry->list);
+ kfree(entry);
+
+ return data;
+}
+
+static void cs_notify_control(u32 message)
+{
+ cs_notify(message, &cs_char_data.chardev_queue);
+}
+
+static void cs_notify_data(u32 message, int maxlength)
+{
+ cs_notify(message, &cs_char_data.dataind_queue);
+
+ spin_lock(&cs_char_data.lock);
+ cs_char_data.dataind_pending++;
+ while (cs_char_data.dataind_pending > maxlength &&
+ !list_empty(&cs_char_data.dataind_queue)) {
+ dev_dbg(&cs_char_data.cl->device, "data notification "
+ "queue overrun (%u entries)\n", cs_char_data.dataind_pending);
+
+ cs_pop_entry(&cs_char_data.dataind_queue);
+ cs_char_data.dataind_pending--;
+ }
+ spin_unlock(&cs_char_data.lock);
+}
+
+static inline void cs_set_cmd(struct hsi_msg *msg, u32 cmd)
+{
+ u32 *data = sg_virt(msg->sgt.sgl);
+ *data = cmd;
+}
+
+static inline u32 cs_get_cmd(struct hsi_msg *msg)
+{
+ u32 *data = sg_virt(msg->sgt.sgl);
+ return *data;
+}
+
+static void cs_release_cmd(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+
+ list_add_tail(&msg->link, &hi->cmdqueue);
+}
+
+static void cs_cmd_destructor(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+
+ spin_lock(&hi->lock);
+
+ dev_dbg(&cs_char_data.cl->device, "control cmd destructor\n");
+
+ if (hi->iface_state != CS_STATE_CLOSED)
+ dev_err(&hi->cl->device, "Cmd flushed while driver active\n");
+
+ if (msg->ttype == HSI_MSG_READ)
+ hi->control_state &=
+ ~(SSI_CHANNEL_STATE_POLL | SSI_CHANNEL_STATE_READING);
+ else if (msg->ttype == HSI_MSG_WRITE &&
+ hi->control_state & SSI_CHANNEL_STATE_WRITING)
+ hi->control_state &= ~SSI_CHANNEL_STATE_WRITING;
+
+ cs_release_cmd(msg);
+
+ spin_unlock(&hi->lock);
+}
+
+static struct hsi_msg *cs_claim_cmd(struct cs_hsi_iface* ssi)
+{
+ struct hsi_msg *msg;
+
+ BUG_ON(list_empty(&ssi->cmdqueue));
+
+ msg = list_first_entry(&ssi->cmdqueue, struct hsi_msg, link);
+ list_del(&msg->link);
+ msg->destructor = cs_cmd_destructor;
+
+ return msg;
+}
+
+static void cs_free_cmds(struct cs_hsi_iface *ssi)
+{
+ struct hsi_msg *msg, *tmp;
+
+ list_for_each_entry_safe(msg, tmp, &ssi->cmdqueue, link) {
+ list_del(&msg->link);
+ msg->destructor = NULL;
+ kfree(sg_virt(msg->sgt.sgl));
+ hsi_free_msg(msg);
+ }
+}
+
+static int cs_alloc_cmds(struct cs_hsi_iface *hi)
+{
+ struct hsi_msg *msg;
+ u32 *buf;
+ unsigned int i;
+
+ INIT_LIST_HEAD(&hi->cmdqueue);
+
+ for (i = 0; i < CS_MAX_CMDS; i++) {
+ msg = hsi_alloc_msg(1, GFP_KERNEL);
+ if (!msg)
+ goto out;
+ buf = kmalloc(sizeof(*buf), GFP_KERNEL);
+ if (!buf) {
+ hsi_free_msg(msg);
+ goto out;
+ }
+ sg_init_one(msg->sgt.sgl, buf, sizeof(*buf));
+ msg->channel = cs_char_data.channel_id_cmd;
+ msg->context = hi;
+ list_add_tail(&msg->link, &hi->cmdqueue);
+ }
+
+ return 0;
+
+out:
+ cs_free_cmds(hi);
+ return -ENOMEM;
+}
+
+static void cs_hsi_data_destructor(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+ const char *dir = (msg->ttype == HSI_MSG_READ) ? "TX" : "RX";
+
+ dev_dbg(&cs_char_data.cl->device, "Freeing data %s message\n", dir);
+
+ spin_lock(&hi->lock);
+ if (hi->iface_state != CS_STATE_CLOSED)
+ dev_err(&cs_char_data.cl->device,
+ "Data %s flush while device active\n", dir);
+ if (msg->ttype == HSI_MSG_READ)
+ hi->data_state &=
+ ~(SSI_CHANNEL_STATE_POLL | SSI_CHANNEL_STATE_READING);
+ else
+ hi->data_state &= ~SSI_CHANNEL_STATE_WRITING;
+
+ msg->status = HSI_STATUS_COMPLETED;
+ if (unlikely(waitqueue_active(&hi->datawait)))
+ wake_up_interruptible(&hi->datawait);
+
+ spin_unlock(&hi->lock);
+}
+
+static int cs_hsi_alloc_data(struct cs_hsi_iface *hi)
+{
+ struct hsi_msg *txmsg, *rxmsg;
+ int res = 0;
+
+ rxmsg = hsi_alloc_msg(1, GFP_KERNEL);
+ if (!rxmsg) {
+ res = -ENOMEM;
+ goto out1;
+ }
+ rxmsg->channel = cs_char_data.channel_id_data;
+ rxmsg->destructor = cs_hsi_data_destructor;
+ rxmsg->context = hi;
+
+ txmsg = hsi_alloc_msg(1, GFP_KERNEL);
+ if (!txmsg) {
+ res = -ENOMEM;
+ goto out2;
+ }
+ txmsg->channel = cs_char_data.channel_id_data;
+ txmsg->destructor = cs_hsi_data_destructor;
+ txmsg->context = hi;
+
+ hi->data_rx_msg = rxmsg;
+ hi->data_tx_msg = txmsg;
+
+ return 0;
+
+out2:
+ hsi_free_msg(rxmsg);
+out1:
+ return res;
+}
+
+static void cs_hsi_free_data_msg(struct hsi_msg *msg)
+{
+ WARN_ON(msg->status != HSI_STATUS_COMPLETED &&
+ msg->status != HSI_STATUS_ERROR);
+ hsi_free_msg(msg);
+}
+
+static void cs_hsi_free_data(struct cs_hsi_iface *hi)
+{
+ cs_hsi_free_data_msg(hi->data_rx_msg);
+ cs_hsi_free_data_msg(hi->data_tx_msg);
+}
+
+static inline void __cs_hsi_error_pre(struct cs_hsi_iface *hi,
+ struct hsi_msg *msg, const char *info,
+ unsigned int *state)
+{
+ spin_lock(&hi->lock);
+ dev_err(&hi->cl->device, "HSI %s error, msg %d, state %u\n",
+ info, msg->status, *state);
+}
+
+static inline void __cs_hsi_error_post(struct cs_hsi_iface *hi)
+{
+ spin_unlock(&hi->lock);
+}
+
+static inline void __cs_hsi_error_read_bits(unsigned int *state)
+{
+ *state |= SSI_CHANNEL_STATE_ERROR;
+ *state &= ~(SSI_CHANNEL_STATE_READING | SSI_CHANNEL_STATE_POLL);
+}
+
+static inline void __cs_hsi_error_write_bits(unsigned int *state)
+{
+ *state |= SSI_CHANNEL_STATE_ERROR;
+ *state &= ~SSI_CHANNEL_STATE_WRITING;
+}
+
+static void cs_hsi_control_read_error(struct cs_hsi_iface *hi,
+ struct hsi_msg *msg)
+{
+ __cs_hsi_error_pre(hi, msg, "control read", &hi->control_state);
+ cs_release_cmd(msg);
+ __cs_hsi_error_read_bits(&hi->control_state);
+ __cs_hsi_error_post(hi);
+}
+
+static void cs_hsi_control_write_error(struct cs_hsi_iface *hi,
+ struct hsi_msg *msg)
+{
+ __cs_hsi_error_pre(hi, msg, "control write", &hi->control_state);
+ cs_release_cmd(msg);
+ __cs_hsi_error_write_bits(&hi->control_state);
+ __cs_hsi_error_post(hi);
+
+}
+
+static void cs_hsi_data_read_error(struct cs_hsi_iface *hi, struct hsi_msg *msg)
+{
+ __cs_hsi_error_pre(hi, msg, "data read", &hi->data_state);
+ __cs_hsi_error_read_bits(&hi->data_state);
+ __cs_hsi_error_post(hi);
+}
+
+static void cs_hsi_data_write_error(struct cs_hsi_iface *hi,
+ struct hsi_msg *msg)
+{
+ __cs_hsi_error_pre(hi, msg, "data write", &hi->data_state);
+ __cs_hsi_error_write_bits(&hi->data_state);
+ __cs_hsi_error_post(hi);
+}
+
+static void cs_hsi_read_on_control_complete(struct hsi_msg *msg)
+{
+ u32 cmd = cs_get_cmd(msg);
+ struct cs_hsi_iface *hi = msg->context;
+
+ spin_lock(&hi->lock);
+ hi->control_state &= ~SSI_CHANNEL_STATE_READING;
+ if (msg->status == HSI_STATUS_ERROR) {
+ dev_err(&hi->cl->device, "Control RX error detected\n");
+ cs_hsi_control_read_error(hi, msg);
+ spin_unlock(&hi->lock);
+ goto out;
+ }
+ dev_dbg(&hi->cl->device, "Read on control: %08X\n", cmd);
+ cs_release_cmd(msg);
+ if (hi->flags & CS_FEAT_TSTAMP_RX_CTRL) {
+ struct timespec *tstamp =
+ &hi->mmap_cfg->tstamp_rx_ctrl;
+ do_posix_clock_monotonic_gettime(tstamp);
+ }
+ spin_unlock(&hi->lock);
+
+ cs_notify_control(cmd);
+
+out:
+ cs_hsi_read_on_control(hi);
+}
+
+static void cs_hsi_peek_on_control_complete(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+ int ret;
+
+ if (msg->status == HSI_STATUS_ERROR) {
+ dev_err(&hi->cl->device, "Control peek RX error detected\n");
+ cs_hsi_control_read_error(hi, msg);
+ return;
+ }
+
+ WARN_ON(!(hi->control_state & SSI_CHANNEL_STATE_READING));
+
+ dev_dbg(&hi->cl->device, "Peek on control complete, reading\n");
+ msg->sgt.nents = 1;
+ msg->complete = cs_hsi_read_on_control_complete;
+ ret = hsi_async_read(hi->cl, msg);
+ if (ret)
+ cs_hsi_control_read_error(hi, msg);
+}
+
+static void cs_hsi_read_on_control(struct cs_hsi_iface *hi)
+{
+ struct hsi_msg *msg;
+ int ret;
+
+ spin_lock(&hi->lock);
+ if (hi->control_state & SSI_CHANNEL_STATE_READING) {
+ dev_err(&hi->cl->device, "Control read already pending (%d)\n",
+ hi->control_state);
+ spin_unlock(&hi->lock);
+ return;
+ }
+ if (hi->control_state & SSI_CHANNEL_STATE_ERROR) {
+ dev_err(&hi->cl->device, "Control read error (%d)\n",
+ hi->control_state);
+ spin_unlock(&hi->lock);
+ return;
+ }
+ hi->control_state |= SSI_CHANNEL_STATE_READING;
+ dev_dbg(&hi->cl->device, "Issuing RX on control\n");
+ msg = cs_claim_cmd(hi);
+ spin_unlock(&hi->lock);
+
+ msg->sgt.nents = 0;
+ msg->complete = cs_hsi_peek_on_control_complete;
+ ret = hsi_async_read(hi->cl, msg);
+ if (ret)
+ cs_hsi_control_read_error(hi, msg);
+}
+
+static void cs_hsi_write_on_control_complete(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+ if (msg->status == HSI_STATUS_COMPLETED) {
+ spin_lock(&hi->lock);
+ hi->control_state &= ~SSI_CHANNEL_STATE_WRITING;
+ cs_release_cmd(msg);
+ spin_unlock(&hi->lock);
+ } else if (msg->status == HSI_STATUS_ERROR) {
+ cs_hsi_control_write_error(hi, msg);
+ } else {
+ dev_err(&hi->cl->device,
+ "unexpected status in control write callback %d\n",
+ msg->status);
+ }
+}
+
+static int cs_hsi_write_on_control(struct cs_hsi_iface *hi, u32 message)
+{
+ struct hsi_msg *msg;
+ int ret;
+
+ spin_lock(&hi->lock);
+ if (hi->control_state & SSI_CHANNEL_STATE_ERROR) {
+ spin_unlock(&hi->lock);
+ return -EIO;
+ }
+ if (hi->control_state & SSI_CHANNEL_STATE_WRITING) {
+ dev_err(&hi->cl->device,
+ "Write still pending on control channel.\n");
+ spin_unlock(&hi->lock);
+ return -EBUSY;
+ }
+ hi->control_state |= SSI_CHANNEL_STATE_WRITING;
+ msg = cs_claim_cmd(hi);
+ spin_unlock(&hi->lock);
+
+ cs_set_cmd(msg, message);
+ msg->sgt.nents = 1;
+ msg->complete = cs_hsi_write_on_control_complete;
+ dev_dbg(&hi->cl->device,
+ "Sending control message %08X\n", message);
+ ret = hsi_async_write(hi->cl, msg);
+ if (ret) {
+ dev_err(&hi->cl->device,
+ "async_write failed with %d\n", ret);
+ cs_hsi_control_write_error(hi, msg);
+ }
+
+ /*
+ * Make sure control read is always pending when issuing
+ * new control writes. This is needed as the controller
+ * may flush our messages if e.g. the peer device reboots
+ * unexpectedly (and we cannot directly resubmit a new read from
+ * the message destructor; see cs_cmd_destructor()).
+ */
+ if (!(hi->control_state & SSI_CHANNEL_STATE_READING)) {
+ dev_err(&hi->cl->device, "Restarting control reads\n");
+ cs_hsi_read_on_control(hi);
+ }
+
+ return 0;
+}
+
+static void cs_hsi_read_on_data_complete(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+ u32 payload;
+
+ if (unlikely(msg->status == HSI_STATUS_ERROR)) {
+ cs_hsi_data_read_error(hi, msg);
+ return;
+ }
+
+ spin_lock(&hi->lock);
+ WARN_ON(!(hi->data_state & SSI_CHANNEL_STATE_READING));
+ hi->data_state &= ~SSI_CHANNEL_STATE_READING;
+ payload = CS_RX_DATA_RECEIVED;
+ payload |= hi->rx_slot;
+ hi->rx_slot++;
+ hi->rx_slot %= hi->rx_ptr_boundary;
+ /* expose current rx ptr in mmap area */
+ hi->mmap_cfg->rx_ptr = hi->rx_slot;
+ if (unlikely(waitqueue_active(&hi->datawait)))
+ wake_up_interruptible(&hi->datawait);
+ spin_unlock(&hi->lock);
+
+ cs_notify_data(payload, hi->rx_bufs);
+ cs_hsi_read_on_data(hi);
+}
+
+static void cs_hsi_peek_on_data_complete(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+ u32 *address;
+ int ret;
+
+ if (unlikely(msg->status == HSI_STATUS_ERROR)) {
+ cs_hsi_data_read_error(hi, msg);
+ return;
+ }
+ if (unlikely(hi->iface_state != CS_STATE_CONFIGURED)) {
+ dev_err(&hi->cl->device, "Data received in invalid state\n");
+ cs_hsi_data_read_error(hi, msg);
+ return;
+ }
+
+ spin_lock(&hi->lock);
+ WARN_ON(!(hi->data_state & SSI_CHANNEL_STATE_POLL));
+ hi->data_state &= ~SSI_CHANNEL_STATE_POLL;
+ hi->data_state |= SSI_CHANNEL_STATE_READING;
+ spin_unlock(&hi->lock);
+
+ address = (u32 *)(hi->mmap_base +
+ hi->rx_offsets[hi->rx_slot % hi->rx_bufs]);
+ sg_init_one(msg->sgt.sgl, address, hi->buf_size);
+ msg->sgt.nents = 1;
+ msg->complete = cs_hsi_read_on_data_complete;
+ ret = hsi_async_read(hi->cl, msg);
+ if (ret)
+ cs_hsi_data_read_error(hi, msg);
+}
+
+/*
+ * Read/write transaction is ongoing. Returns false if in
+ * SSI_CHANNEL_STATE_POLL state.
+ */
+static inline int cs_state_xfer_active(unsigned int state)
+{
+ return (state & SSI_CHANNEL_STATE_WRITING) ||
+ (state & SSI_CHANNEL_STATE_READING);
+}
+
+/*
+ * No pending read/writes
+ */
+static inline int cs_state_idle(unsigned int state)
+{
+ return !(state & ~SSI_CHANNEL_STATE_ERROR);
+}
+
+static void cs_hsi_read_on_data(struct cs_hsi_iface *hi)
+{
+ struct hsi_msg *rxmsg;
+ int ret;
+
+ spin_lock(&hi->lock);
+ if (hi->data_state &
+ (SSI_CHANNEL_STATE_READING | SSI_CHANNEL_STATE_POLL)) {
+ dev_dbg(&hi->cl->device, "Data read already pending (%u)\n",
+ hi->data_state);
+ spin_unlock(&hi->lock);
+ return;
+ }
+ hi->data_state |= SSI_CHANNEL_STATE_POLL;
+ spin_unlock(&hi->lock);
+
+ rxmsg = hi->data_rx_msg;
+ sg_init_one(rxmsg->sgt.sgl, (void *)hi->mmap_base, 0);
+ rxmsg->sgt.nents = 0;
+ rxmsg->complete = cs_hsi_peek_on_data_complete;
+
+ ret = hsi_async_read(hi->cl, rxmsg);
+ if (ret)
+ cs_hsi_data_read_error(hi, rxmsg);
+}
+
+static void cs_hsi_write_on_data_complete(struct hsi_msg *msg)
+{
+ struct cs_hsi_iface *hi = msg->context;
+
+ if (msg->status == HSI_STATUS_COMPLETED) {
+ spin_lock(&hi->lock);
+ hi->data_state &= ~SSI_CHANNEL_STATE_WRITING;
+ if (unlikely(waitqueue_active(&hi->datawait)))
+ wake_up_interruptible(&hi->datawait);
+ spin_unlock(&hi->lock);
+ } else {
+ cs_hsi_data_write_error(hi, msg);
+ }
+}
+
+static int cs_hsi_write_on_data(struct cs_hsi_iface *hi, unsigned int slot)
+{
+ u32 *address;
+ struct hsi_msg *txmsg;
+ int ret;
+
+ spin_lock(&hi->lock);
+ if (hi->iface_state != CS_STATE_CONFIGURED) {
+ dev_err(&hi->cl->device, "Not configured, aborting\n");
+ ret = -EINVAL;
+ goto error;
+ }
+ if (hi->data_state & SSI_CHANNEL_STATE_ERROR) {
+ dev_err(&hi->cl->device, "HSI error, aborting\n");
+ ret = -EIO;
+ goto error;
+ }
+ if (hi->data_state & SSI_CHANNEL_STATE_WRITING) {
+ dev_err(&hi->cl->device, "Write pending on data channel.\n");
+ ret = -EBUSY;
+ goto error;
+ }
+ hi->data_state |= SSI_CHANNEL_STATE_WRITING;
+ spin_unlock(&hi->lock);
+
+ hi->tx_slot = slot;
+ address = (u32 *)(hi->mmap_base + hi->tx_offsets[hi->tx_slot]);
+ txmsg = hi->data_tx_msg;
+ sg_init_one(txmsg->sgt.sgl, address, hi->buf_size);
+ txmsg->complete = cs_hsi_write_on_data_complete;
+ ret = hsi_async_write(hi->cl, txmsg);
+ if (ret)
+ cs_hsi_data_write_error(hi, txmsg);
+
+ return ret;
+
+error:
+ spin_unlock(&hi->lock);
+ if (ret == -EIO)
+ cs_hsi_data_write_error(hi, hi->data_tx_msg);
+
+ return ret;
+}
+
+static unsigned int cs_hsi_get_state(struct cs_hsi_iface *hi)
+{
+ return hi->iface_state;
+}
+
+static int cs_hsi_command(struct cs_hsi_iface *hi, u32 cmd)
+{
+ int ret = 0;
+
+ local_bh_disable();
+ switch (cmd & TARGET_MASK) {
+ case TARGET_REMOTE:
+ ret = cs_hsi_write_on_control(hi, cmd);
+ break;
+ case TARGET_LOCAL:
+ if ((cmd & CS_CMD_MASK) == CS_TX_DATA_READY)
+ ret = cs_hsi_write_on_data(hi, cmd & CS_PARAM_MASK);
+ else
+ ret = -EINVAL;
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+ local_bh_enable();
+
+ return ret;
+}
+
+static void cs_hsi_set_wakeline(struct cs_hsi_iface *hi, bool new_state)
+{
+ int change = 0;
+
+ spin_lock_bh(&hi->lock);
+ if (hi->wakeline_state != new_state) {
+ hi->wakeline_state = new_state;
+ change = 1;
+ dev_dbg(&hi->cl->device, "setting wake line to %d (%p)\n",
+ new_state, hi->cl);
+ }
+ spin_unlock_bh(&hi->lock);
+
+ if (change) {
+ if (new_state)
+ ssip_slave_start_tx(hi->master);
+ else
+ ssip_slave_stop_tx(hi->master);
+ }
+
+ dev_dbg(&hi->cl->device, "wake line set to %d (%p)\n",
+ new_state, hi->cl);
+}
+
+static void set_buffer_sizes(struct cs_hsi_iface *hi, int rx_bufs, int tx_bufs)
+{
+ hi->rx_bufs = rx_bufs;
+ hi->tx_bufs = tx_bufs;
+ hi->mmap_cfg->rx_bufs = rx_bufs;
+ hi->mmap_cfg->tx_bufs = tx_bufs;
+
+ if (hi->flags & CS_FEAT_ROLLING_RX_COUNTER) {
+ /*
+ * For more robust overrun detection, let the rx
+ * pointer run in range 0..'boundary-1'. Boundary
+ * is a multiple of rx_bufs, and limited in max size
+ * by RX_PTR_MAX_SHIFT to allow for fast ptr-diff
+ * calculation.
+ */
+ hi->rx_ptr_boundary = (rx_bufs << RX_PTR_BOUNDARY_SHIFT);
+ hi->mmap_cfg->rx_ptr_boundary = hi->rx_ptr_boundary;
+ } else {
+ hi->rx_ptr_boundary = hi->rx_bufs;
+ }
+}
+
+static int check_buf_params(struct cs_hsi_iface *hi,
+ const struct cs_buffer_config *buf_cfg)
+{
+ size_t buf_size_aligned = L1_CACHE_ALIGN(buf_cfg->buf_size) *
+ (buf_cfg->rx_bufs + buf_cfg->tx_bufs);
+ size_t ctrl_size_aligned = L1_CACHE_ALIGN(sizeof(*hi->mmap_cfg));
+ int r = 0;
+
+ if (buf_cfg->rx_bufs > CS_MAX_BUFFERS ||
+ buf_cfg->tx_bufs > CS_MAX_BUFFERS) {
+ r = -EINVAL;
+ } else if ((buf_size_aligned + ctrl_size_aligned) >= hi->mmap_size) {
+ dev_err(&hi->cl->device, "No space for the requested buffer "
+ "configuration\n");
+ r = -ENOBUFS;
+ }
+
+ return r;
+}
+
+/**
+ * Block until pending data transfers have completed.
+ */
+static int cs_hsi_data_sync(struct cs_hsi_iface *hi)
+{
+ int r = 0;
+
+ spin_lock_bh(&hi->lock);
+
+ if (!cs_state_xfer_active(hi->data_state)) {
+ dev_dbg(&hi->cl->device, "hsi_data_sync break, idle\n");
+ goto out;
+ }
+
+ for (;;) {
+ int s;
+ DEFINE_WAIT(wait);
+ if (!cs_state_xfer_active(hi->data_state))
+ goto out;
+ if (signal_pending(current)) {
+ r = -ERESTARTSYS;
+ goto out;
+ }
+ /**
+ * prepare_to_wait must be called with hi->lock held
+ * so that callbacks can check for waitqueue_active()
+ */
+ prepare_to_wait(&hi->datawait, &wait, TASK_INTERRUPTIBLE);
+ spin_unlock_bh(&hi->lock);
+ s = schedule_timeout(
+ msecs_to_jiffies(CS_HSI_TRANSFER_TIMEOUT_MS));
+ spin_lock_bh(&hi->lock);
+ finish_wait(&hi->datawait, &wait);
+ if (!s) {
+ dev_dbg(&hi->cl->device,
+ "hsi_data_sync timeout after %d ms\n",
+ CS_HSI_TRANSFER_TIMEOUT_MS);
+ r = -EIO;
+ goto out;
+ }
+ }
+
+out:
+ spin_unlock_bh(&hi->lock);
+ dev_dbg(&hi->cl->device, "hsi_data_sync done with res %d\n", r);
+
+ return r;
+}
+
+static void cs_hsi_data_enable(struct cs_hsi_iface *hi,
+ struct cs_buffer_config *buf_cfg)
+{
+ unsigned int data_start, i;
+
+ BUG_ON(hi->buf_size == 0);
+
+ set_buffer_sizes(hi, buf_cfg->rx_bufs, buf_cfg->tx_bufs);
+
+ hi->slot_size = L1_CACHE_ALIGN(hi->buf_size);
+ dev_dbg(&hi->cl->device,
+ "setting slot size to %u, buf size %u, align %u\n",
+ hi->slot_size, hi->buf_size, L1_CACHE_BYTES);
+
+ data_start = L1_CACHE_ALIGN(sizeof(*hi->mmap_cfg));
+ dev_dbg(&hi->cl->device,
+ "setting data start at %u, cfg block %u, align %u\n",
+ data_start, sizeof(*hi->mmap_cfg), L1_CACHE_BYTES);
+
+ for (i = 0; i < hi->mmap_cfg->rx_bufs; i++) {
+ hi->rx_offsets[i] = data_start + i * hi->slot_size;
+ hi->mmap_cfg->rx_offsets[i] = hi->rx_offsets[i];
+ dev_dbg(&hi->cl->device, "DL buf #%u at %u\n",
+ i, hi->rx_offsets[i]);
+ }
+ for (i = 0; i < hi->mmap_cfg->tx_bufs; i++) {
+ hi->tx_offsets[i] = data_start +
+ (i + hi->mmap_cfg->rx_bufs) * hi->slot_size;
+ hi->mmap_cfg->tx_offsets[i] = hi->tx_offsets[i];
+ dev_dbg(&hi->cl->device, "UL buf #%u at %u\n",
+ i, hi->rx_offsets[i]);
+ }
+
+ hi->iface_state = CS_STATE_CONFIGURED;
+}
+
+static void cs_hsi_data_disable(struct cs_hsi_iface *hi, int old_state)
+{
+ if (old_state == CS_STATE_CONFIGURED) {
+ dev_dbg(&hi->cl->device,
+ "closing data channel with slot size 0\n");
+ hi->iface_state = CS_STATE_OPENED;
+ }
+}
+
+static int cs_hsi_buf_config(struct cs_hsi_iface *hi,
+ struct cs_buffer_config *buf_cfg)
+{
+ int r = 0;
+ unsigned int old_state = hi->iface_state;
+
+ spin_lock_bh(&hi->lock);
+ /* Prevent new transactions during buffer reconfig */
+ if (old_state == CS_STATE_CONFIGURED)
+ hi->iface_state = CS_STATE_OPENED;
+ spin_unlock_bh(&hi->lock);
+
+ /*
+ * make sure that no non-zero data reads are ongoing before
+ * proceeding to change the buffer layout
+ */
+ r = cs_hsi_data_sync(hi);
+ if (r < 0)
+ return r;
+
+ WARN_ON(cs_state_xfer_active(hi->data_state));
+
+ spin_lock_bh(&hi->lock);
+ r = check_buf_params(hi, buf_cfg);
+ if (r < 0)
+ goto error;
+
+ hi->buf_size = buf_cfg->buf_size;
+ hi->mmap_cfg->buf_size = hi->buf_size;
+ hi->flags = buf_cfg->flags;
+
+ hi->rx_slot = 0;
+ hi->tx_slot = 0;
+ hi->slot_size = 0;
+
+ if (hi->buf_size)
+ cs_hsi_data_enable(hi, buf_cfg);
+ else
+ cs_hsi_data_disable(hi, old_state);
+
+ spin_unlock_bh(&hi->lock);
+
+ if (old_state != hi->iface_state) {
+ if (hi->iface_state == CS_STATE_CONFIGURED) {
+ pm_qos_add_request(&hi->pm_qos_req,
+ PM_QOS_CPU_DMA_LATENCY,
+ CS_QOS_LATENCY_FOR_DATA_USEC);
+ local_bh_disable();
+ cs_hsi_read_on_data(hi);
+ local_bh_enable();
+ } else if (old_state == CS_STATE_CONFIGURED) {
+ pm_qos_remove_request(&hi->pm_qos_req);
+ }
+ }
+ return r;
+
+error:
+ spin_unlock_bh(&hi->lock);
+ return r;
+}
+
+static int cs_hsi_start(struct cs_hsi_iface **hi, struct hsi_client *cl,
+ unsigned long mmap_base, unsigned long mmap_size)
+{
+ int err = 0;
+ struct cs_hsi_iface *hsi_if = kzalloc(sizeof(*hsi_if), GFP_KERNEL);
+
+ dev_dbg(&cl->device, "cs_hsi_start\n");
+
+ if (!hsi_if) {
+ err = -ENOMEM;
+ goto leave0;
+ }
+ spin_lock_init(&hsi_if->lock);
+ hsi_if->cl = cl;
+ hsi_if->iface_state = CS_STATE_CLOSED;
+ hsi_if->mmap_cfg = (struct cs_mmap_config_block *)mmap_base;
+ hsi_if->mmap_base = mmap_base;
+ hsi_if->mmap_size = mmap_size;
+ memset(hsi_if->mmap_cfg, 0, sizeof(*hsi_if->mmap_cfg));
+ init_waitqueue_head(&hsi_if->datawait);
+ err = cs_alloc_cmds(hsi_if);
+ if (err < 0) {
+ dev_err(&cl->device, "Unable to alloc HSI messages\n");
+ goto leave1;
+ }
+ err = cs_hsi_alloc_data(hsi_if);
+ if (err < 0) {
+ dev_err(&cl->device, "Unable to alloc HSI messages for data\n");
+ goto leave2;
+ }
+ err = hsi_claim_port(cl, 1);
+ if (err < 0) {
+ dev_err(&cl->device,
+ "Could not open, HSI port already claimed\n");
+ goto leave3;
+ }
+ hsi_if->master = ssip_slave_get_master(cl);
+ if (IS_ERR(hsi_if->master)) {
+ err = PTR_ERR(hsi_if->master);
+ dev_err(&cl->device, "Could not get HSI master client\n");
+ goto leave4;
+ }
+ if (!ssip_slave_running(hsi_if->master)) {
+ err = -ENODEV;
+ dev_err(&cl->device,
+ "HSI port not initialized\n");
+ goto leave4;
+ }
+
+ hsi_if->iface_state = CS_STATE_OPENED;
+ local_bh_disable();
+ cs_hsi_read_on_control(hsi_if);
+ local_bh_enable();
+
+ dev_dbg(&cl->device, "cs_hsi_start...done\n");
+
+ BUG_ON(!hi);
+ *hi = hsi_if;
+
+ return 0;
+
+leave4:
+ hsi_release_port(cl);
+leave3:
+ cs_hsi_free_data(hsi_if);
+leave2:
+ cs_free_cmds(hsi_if);
+leave1:
+ kfree(hsi_if);
+leave0:
+ dev_dbg(&cl->device, "cs_hsi_start...done/error\n\n");
+
+ return err;
+}
+
+static void cs_hsi_stop(struct cs_hsi_iface *hi)
+{
+ dev_dbg(&hi->cl->device, "cs_hsi_stop\n");
+ cs_hsi_set_wakeline(hi, 0);
+ ssip_slave_put_master(hi->master);
+
+ /* hsi_release_port() needs to be called with CS_STATE_CLOSED */
+ hi->iface_state = CS_STATE_CLOSED;
+ hsi_release_port(hi->cl);
+
+ /*
+ * hsi_release_port() should flush out all the pending
+ * messages, so cs_state_idle() should be true for both
+ * control and data channels.
+ */
+ WARN_ON(!cs_state_idle(hi->control_state));
+ WARN_ON(!cs_state_idle(hi->data_state));
+
+ if (pm_qos_request_active(&hi->pm_qos_req))
+ pm_qos_remove_request(&hi->pm_qos_req);
+
+ spin_lock_bh(&hi->lock);
+ cs_hsi_free_data(hi);
+ cs_free_cmds(hi);
+ spin_unlock_bh(&hi->lock);
+ kfree(hi);
+}
+
+static int cs_char_vma_fault(struct vm_area_struct *vma, struct vm_fault *vmf)
+{
+ struct cs_char *csdata = vma->vm_private_data;
+ struct page *page;
+
+ page = virt_to_page(csdata->mmap_base);
+ get_page(page);
+ vmf->page = page;
+
+ return 0;
+}
+
+static struct vm_operations_struct cs_char_vm_ops = {
+ .fault = cs_char_vma_fault,
+};
+
+static int cs_char_fasync(int fd, struct file *file, int on)
+{
+ struct cs_char *csdata = file->private_data;
+
+ if (fasync_helper(fd, file, on, &csdata->async_queue) < 0)
+ return -EIO;
+
+ return 0;
+}
+
+static unsigned int cs_char_poll(struct file *file, poll_table *wait)
+{
+ struct cs_char *csdata = file->private_data;
+ unsigned int ret = 0;
+
+ poll_wait(file, &cs_char_data.wait, wait);
+ spin_lock_bh(&csdata->lock);
+ if (!list_empty(&csdata->chardev_queue))
+ ret = POLLIN | POLLRDNORM;
+ else if (!list_empty(&csdata->dataind_queue))
+ ret = POLLIN | POLLRDNORM;
+ spin_unlock_bh(&csdata->lock);
+
+ return ret;
+}
+
+static ssize_t cs_char_read(struct file *file, char __user *buf, size_t count,
+ loff_t *unused)
+{
+ struct cs_char *csdata = file->private_data;
+ u32 data;
+ ssize_t retval;
+
+ if (count < sizeof(data))
+ return -EINVAL;
+
+ for (;;) {
+ DEFINE_WAIT(wait);
+
+ spin_lock_bh(&csdata->lock);
+ if (!list_empty(&csdata->chardev_queue)) {
+ data = cs_pop_entry(&csdata->chardev_queue);
+ } else if (!list_empty(&csdata->dataind_queue)) {
+ data = cs_pop_entry(&csdata->dataind_queue);
+ csdata->dataind_pending--;
+ } else {
+ data = 0;
+ }
+ spin_unlock_bh(&csdata->lock);
+
+ if (data)
+ break;
+ if (file->f_flags & O_NONBLOCK) {
+ retval = -EAGAIN;
+ goto out;
+ } else if (signal_pending(current)) {
+ retval = -ERESTARTSYS;
+ goto out;
+ }
+ prepare_to_wait_exclusive(&csdata->wait, &wait,
+ TASK_INTERRUPTIBLE);
+ schedule();
+ finish_wait(&csdata->wait, &wait);
+ }
+
+ retval = put_user(data, (u32 __user *)buf);
+ if (!retval)
+ retval = sizeof(data);
+
+out:
+ return retval;
+}
+
+static ssize_t cs_char_write(struct file *file, const char __user *buf,
+ size_t count, loff_t *unused)
+{
+ struct cs_char *csdata = file->private_data;
+ u32 data;
+ int err;
+ ssize_t retval;
+
+ if (count < sizeof(data))
+ return -EINVAL;
+
+ if (get_user(data, (u32 __user *)buf))
+ retval = -EFAULT;
+ else
+ retval = count;
+
+ err = cs_hsi_command(csdata->hi, data);
+ if (err < 0)
+ retval = err;
+
+ return retval;
+}
+
+static long cs_char_ioctl(struct file *file, unsigned int cmd,
+ unsigned long arg)
+{
+ struct cs_char *csdata = file->private_data;
+ int r = 0;
+
+ switch (cmd) {
+ case CS_GET_STATE: {
+ unsigned int state;
+
+ state = cs_hsi_get_state(csdata->hi);
+ if (copy_to_user((void __user *)arg, &state, sizeof(state)))
+ r = -EFAULT;
+
+ break;
+ }
+ case CS_SET_WAKELINE: {
+ unsigned int state;
+
+ if (copy_from_user(&state, (void __user *)arg, sizeof(state))) {
+ r = -EFAULT;
+ break;
+ }
+
+ if (state > 1) {
+ r = -EINVAL;
+ break;
+ }
+
+ cs_hsi_set_wakeline(csdata->hi, !!state);
+
+ break;
+ }
+ case CS_GET_IF_VERSION: {
+ unsigned int ifver = CS_IF_VERSION;
+
+ if (copy_to_user((void __user *)arg, &ifver, sizeof(ifver)))
+ r = -EFAULT;
+
+ break;
+ }
+ case CS_CONFIG_BUFS: {
+ struct cs_buffer_config buf_cfg;
+
+ if (copy_from_user(&buf_cfg, (void __user *)arg,
+ sizeof(buf_cfg)))
+ r = -EFAULT;
+ else
+ r = cs_hsi_buf_config(csdata->hi, &buf_cfg);
+
+ break;
+ }
+ default:
+ r = -ENOTTY;
+ break;
+ }
+
+ return r;
+}
+
+static int cs_char_mmap(struct file *file, struct vm_area_struct *vma)
+{
+ if (vma->vm_end < vma->vm_start)
+ return -EINVAL;
+
+ if (((vma->vm_end - vma->vm_start) >> PAGE_SHIFT) != 1)
+ return -EINVAL;
+
+ vma->vm_flags |= VM_IO | VM_DONTDUMP | VM_DONTEXPAND;
+ vma->vm_ops = &cs_char_vm_ops;
+ vma->vm_private_data = file->private_data;
+
+ return 0;
+}
+
+static int cs_char_open(struct inode *unused, struct file *file)
+{
+ int ret = 0;
+ unsigned long p;
+
+ spin_lock_bh(&cs_char_data.lock);
+ if (cs_char_data.opened) {
+ ret = -EBUSY;
+ spin_unlock_bh(&cs_char_data.lock);
+ goto out1;
+ }
+ cs_char_data.opened = 1;
+ cs_char_data.dataind_pending = 0;
+ spin_unlock_bh(&cs_char_data.lock);
+
+ p = get_zeroed_page(GFP_KERNEL);
+ if (!p) {
+ ret = -ENOMEM;
+ goto out2;
+ }
+
+ ret = cs_hsi_start(&cs_char_data.hi, cs_char_data.cl, p, CS_MMAP_SIZE);
+ if (ret) {
+ dev_err(&cs_char_data.cl->device, "Unable to initialize HSI\n");
+ goto out3;
+ }
+
+ /* these are only used in release so lock not needed */
+ cs_char_data.mmap_base = p;
+ cs_char_data.mmap_size = CS_MMAP_SIZE;
+
+ file->private_data = &cs_char_data;
+
+ return 0;
+
+out3:
+ free_page(p);
+out2:
+ spin_lock_bh(&cs_char_data.lock);
+ cs_char_data.opened = 0;
+ spin_unlock_bh(&cs_char_data.lock);
+out1:
+ return ret;
+}
+
+static void cs_free_char_queue(struct list_head *head)
+{
+ struct char_queue *entry;
+ struct list_head *cursor, *next;
+
+ if (!list_empty(head)) {
+ list_for_each_safe(cursor, next, head) {
+ entry = list_entry(cursor, struct char_queue, list);
+ list_del(&entry->list);
+ kfree(entry);
+ }
+ }
+
+}
+
+static int cs_char_release(struct inode *unused, struct file *file)
+{
+ struct cs_char *csdata = file->private_data;
+
+ cs_hsi_stop(csdata->hi);
+ spin_lock_bh(&csdata->lock);
+ csdata->hi = NULL;
+ free_page(csdata->mmap_base);
+ cs_free_char_queue(&csdata->chardev_queue);
+ cs_free_char_queue(&csdata->dataind_queue);
+ csdata->opened = 0;
+ spin_unlock_bh(&csdata->lock);
+
+ return 0;
+}
+
+static const struct file_operations cs_char_fops = {
+ .owner = THIS_MODULE,
+ .read = cs_char_read,
+ .write = cs_char_write,
+ .poll = cs_char_poll,
+ .unlocked_ioctl = cs_char_ioctl,
+ .mmap = cs_char_mmap,
+ .open = cs_char_open,
+ .release = cs_char_release,
+ .fasync = cs_char_fasync,
+};
+
+static struct miscdevice cs_char_miscdev = {
+ .minor = MISC_DYNAMIC_MINOR,
+ .name = "cmt_speech",
+ .fops = &cs_char_fops
+};
+
+static int cs_hsi_client_probe(struct device *dev)
+{
+ int err = 0;
+ struct hsi_client *cl = to_hsi_client(dev);
+
+ dev_dbg(dev, "hsi_client_probe\n");
+ init_waitqueue_head(&cs_char_data.wait);
+ spin_lock_init(&cs_char_data.lock);
+ cs_char_data.opened = 0;
+ cs_char_data.cl = cl;
+ cs_char_data.hi = NULL;
+ INIT_LIST_HEAD(&cs_char_data.chardev_queue);
+ INIT_LIST_HEAD(&cs_char_data.dataind_queue);
+
+ cs_char_data.channel_id_cmd = hsi_get_channel_id_by_name(cl,
+ "speech-control");
+ if (cs_char_data.channel_id_cmd < 0) {
+ err = cs_char_data.channel_id_cmd;
+ dev_err(dev, "Could not get cmd channel (%d)\n", err);
+ return err;
+ }
+
+ cs_char_data.channel_id_data = hsi_get_channel_id_by_name(cl,
+ "speech-data");
+ if (cs_char_data.channel_id_data < 0) {
+ err = cs_char_data.channel_id_data;
+ dev_err(dev, "Could not get data channel (%d)\n", err);
+ return err;
+ }
+
+ err = misc_register(&cs_char_miscdev);
+ if (err)
+ dev_err(dev, "Failed to register: %d\n", err);
+
+ return err;
+}
+
+static int cs_hsi_client_remove(struct device *dev)
+{
+ struct cs_hsi_iface *hi;
+
+ dev_dbg(dev, "hsi_client_remove\n");
+ misc_deregister(&cs_char_miscdev);
+ spin_lock_bh(&cs_char_data.lock);
+ hi = cs_char_data.hi;
+ cs_char_data.hi = NULL;
+ spin_unlock_bh(&cs_char_data.lock);
+ if (hi)
+ cs_hsi_stop(hi);
+
+ return 0;
+}
+
+static struct hsi_client_driver cs_hsi_driver = {
+ .driver = {
+ .name = "cmt-speech",
+ .owner = THIS_MODULE,
+ .probe = cs_hsi_client_probe,
+ .remove = cs_hsi_client_remove,
+ },
+};
+
+static int __init cs_char_init(void)
+{
+ pr_info("CMT speech driver added\n");
+ return hsi_register_client_driver(&cs_hsi_driver);
+}
+module_init(cs_char_init);
+
+static void __exit cs_char_exit(void)
+{
+ hsi_unregister_client_driver(&cs_hsi_driver);
+ pr_info("CMT speech driver removed\n");
+}
+module_exit(cs_char_exit);
+
+MODULE_ALIAS("hsi:cmt-speech");
+MODULE_AUTHOR("Kai Vehmanen <kai.vehmanen@nokia.com>");
+MODULE_AUTHOR("Peter Ujfalusi <peter.ujfalusi@nokia.com>");
+MODULE_DESCRIPTION("CMT speech driver");
+MODULE_LICENSE("GPL v2");
struct nokia_modem_gpio *gpios;
int gpio_amount;
struct hsi_client *ssi_protocol;
+ struct hsi_client *cmt_speech;
};
static void do_nokia_modem_rst_ind_tasklet(unsigned long data)
struct hsi_port *port = hsi_get_port(cl);
int irq, pflags, err;
struct hsi_board_info ssip;
+ struct hsi_board_info cmtspeech;
np = dev->of_node;
if (!np) {
modem->ssi_protocol = hsi_new_client(port, &ssip);
if (!modem->ssi_protocol) {
dev_err(dev, "Could not register ssi-protocol device\n");
+ err = -ENOMEM;
goto error2;
}
goto error3;
}
- /* TODO: register cmt-speech hsi client */
+ cmtspeech.name = "cmt-speech";
+ cmtspeech.tx_cfg = cl->tx_cfg;
+ cmtspeech.rx_cfg = cl->rx_cfg;
+ cmtspeech.platform_data = NULL;
+ cmtspeech.archdata = NULL;
+
+ modem->cmt_speech = hsi_new_client(port, &cmtspeech);
+ if (!modem->cmt_speech) {
+ dev_err(dev, "Could not register cmt-speech device\n");
+ err = -ENOMEM;
+ goto error3;
+ }
+
+ err = device_attach(&modem->cmt_speech->device);
+ if (err == 0) {
+ dev_err(dev, "Missing cmt-speech driver\n");
+ err = -EPROBE_DEFER;
+ goto error4;
+ } else if (err < 0) {
+ dev_err(dev, "Could not load cmt-speech driver (%d)\n", err);
+ goto error4;
+ }
dev_info(dev, "Registered Nokia HSI modem\n");
return 0;
+error4:
+ hsi_remove_client(&modem->cmt_speech->device, NULL);
error3:
hsi_remove_client(&modem->ssi_protocol->device, NULL);
error2:
if (!modem)
return 0;
+ if (modem->cmt_speech) {
+ hsi_remove_client(&modem->cmt_speech->device, NULL);
+ modem->cmt_speech = NULL;
+ }
+
if (modem->ssi_protocol) {
hsi_remove_client(&modem->ssi_protocol->device, NULL);
modem->ssi_protocol = NULL;
config SENSORS_GPIO_FAN
tristate "GPIO fan"
depends on GPIOLIB
+ depends on THERMAL || THERMAL=n
help
If you say yes here you get support for fans connected to GPIO lines.
help
If you say yes here you get support for ITE IT8705F, IT8712F,
IT8716F, IT8718F, IT8720F, IT8721F, IT8726F, IT8728F, IT8758E,
- IT8771E, IT8772E, IT8782F, IT8783E/F and IT8603E sensor chips,
- and the SiS950 clone.
+ IT8771E, IT8772E, IT8781F, IT8782F, IT8783E/F, IT8786E, IT8790E,
+ IT8603E, IT8620E, and IT8623E sensor chips, and the SiS950 clone.
This driver can also be built as a module. If so, the module
will be called it87.
mobile devices and servers. Support will include, but not be limited
to, ADT7408, AT30TS00, CAT34TS02, CAT6095, MAX6604, MCP9804, MCP9805,
MCP98242, MCP98243, MCP98244, MCP9843, SE97, SE98, STTS424(E),
- STTS2002, STTS3000, TSE2002B3, TSE2002GB2, TS3000B3, and TS3000GB2.
+ STTS2002, STTS3000, TSE2002, TSE2004, TS3000, and TS3001.
This driver can also be built as a module. If so, the module
will be called jc42.
This driver can also be built as a module. If so, the module
will be called nct7802.
+config SENSORS_NCT7904
+ tristate "Nuvoton NCT7904"
+ depends on I2C
+ help
+ If you say yes here you get support for the Nuvoton NCT7904
+ hardware monitoring chip, including manual fan speed control.
+
+ This driver can also be built as a module. If so, the module
+ will be called nct7904.
+
config SENSORS_PCF8591
tristate "Philips PCF8591 ADC/DAC"
depends on I2C
config SENSORS_PWM_FAN
tristate "PWM fan"
depends on (PWM && OF) || COMPILE_TEST
+ depends on THERMAL || THERMAL=n
help
If you say yes here you get support for fans connected to PWM lines.
The driver uses the generic PWM interface, thus it will work on a
obj-$(CONFIG_SENSORS_NCT6683) += nct6683.o
obj-$(CONFIG_SENSORS_NCT6775) += nct6775.o
obj-$(CONFIG_SENSORS_NCT7802) += nct7802.o
+obj-$(CONFIG_SENSORS_NCT7904) += nct7904.o
obj-$(CONFIG_SENSORS_NTC_THERMISTOR) += ntc_thermistor.o
obj-$(CONFIG_SENSORS_PC87360) += pc87360.o
obj-$(CONFIG_SENSORS_PC87427) += pc87427.o
&ads2830_regmap_config);
}
+ if (IS_ERR(data->regmap))
+ return PTR_ERR(data->regmap);
+
data->cmd_byte = ext_vref ? ADS7828_CMD_PD1 : ADS7828_CMD_PD3;
if (!diff_input)
data->cmd_byte |= ADS7828_CMD_SD_SE;
struct device_attribute *devattr, char *buf) = {
show_label, show_crit_alarm, show_temp, show_tjmax,
show_ttarget };
- static const char *const names[TOTAL_ATTRS] = {
- "temp%d_label", "temp%d_crit_alarm",
- "temp%d_input", "temp%d_crit",
- "temp%d_max" };
+ static const char *const suffixes[TOTAL_ATTRS] = {
+ "label", "crit_alarm", "input", "crit", "max"
+ };
for (i = 0; i < tdata->attr_size; i++) {
- snprintf(tdata->attr_name[i], CORETEMP_NAME_LENGTH, names[i],
- attr_no);
+ snprintf(tdata->attr_name[i], CORETEMP_NAME_LENGTH,
+ "temp%d_%s", attr_no, suffixes[i]);
sysfs_attr_init(&tdata->sd_attrs[i].dev_attr.attr);
tdata->sd_attrs[i].dev_attr.attr.name = tdata->attr_name[i];
tdata->sd_attrs[i].dev_attr.attr.mode = S_IRUGO;
#include <linux/of.h>
#include <linux/of_platform.h>
#include <linux/of_gpio.h>
+#include <linux/thermal.h>
struct gpio_fan_data {
struct platform_device *pdev;
struct device *hwmon_dev;
+ /* Cooling device if any */
+ struct thermal_cooling_device *cdev;
struct mutex lock; /* lock GPIOs operations. */
int num_ctrl;
unsigned *ctrl;
return 0;
}
+static int gpio_fan_get_max_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct gpio_fan_data *fan_data = cdev->devdata;
+
+ if (!fan_data)
+ return -EINVAL;
+
+ *state = fan_data->num_speed - 1;
+ return 0;
+}
+
+static int gpio_fan_get_cur_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct gpio_fan_data *fan_data = cdev->devdata;
+ int r;
+
+ if (!fan_data)
+ return -EINVAL;
+
+ r = get_fan_speed_index(fan_data);
+ if (r < 0)
+ return r;
+
+ *state = r;
+ return 0;
+}
+
+static int gpio_fan_set_cur_state(struct thermal_cooling_device *cdev,
+ unsigned long state)
+{
+ struct gpio_fan_data *fan_data = cdev->devdata;
+
+ if (!fan_data)
+ return -EINVAL;
+
+ set_fan_speed(fan_data, state);
+ return 0;
+}
+
+static const struct thermal_cooling_device_ops gpio_fan_cool_ops = {
+ .get_max_state = gpio_fan_get_max_state,
+ .get_cur_state = gpio_fan_get_cur_state,
+ .set_cur_state = gpio_fan_set_cur_state,
+};
+
#ifdef CONFIG_OF_GPIO
/*
* Translate OpenFirmware node properties into platform_data
node = dev->of_node;
+ /* Alarm GPIO if one exists */
+ if (of_gpio_named_count(node, "alarm-gpios") > 0) {
+ struct gpio_fan_alarm *alarm;
+ int val;
+ enum of_gpio_flags flags;
+
+ alarm = devm_kzalloc(dev, sizeof(struct gpio_fan_alarm),
+ GFP_KERNEL);
+ if (!alarm)
+ return -ENOMEM;
+
+ val = of_get_named_gpio_flags(node, "alarm-gpios", 0, &flags);
+ if (val < 0)
+ return val;
+ alarm->gpio = val;
+ alarm->active_low = flags & OF_GPIO_ACTIVE_LOW;
+
+ pdata->alarm = alarm;
+ }
+
/* Fill GPIO pin array */
pdata->num_ctrl = of_gpio_count(node);
if (pdata->num_ctrl <= 0) {
- dev_err(dev, "gpios DT property empty / missing");
+ if (pdata->alarm)
+ return 0;
+ dev_err(dev, "DT properties empty / missing");
return -ENODEV;
}
ctrl = devm_kzalloc(dev, pdata->num_ctrl * sizeof(unsigned),
}
pdata->speed = speed;
- /* Alarm GPIO if one exists */
- if (of_gpio_named_count(node, "alarm-gpios") > 0) {
- struct gpio_fan_alarm *alarm;
- int val;
- enum of_gpio_flags flags;
-
- alarm = devm_kzalloc(dev, sizeof(struct gpio_fan_alarm),
- GFP_KERNEL);
- if (!alarm)
- return -ENOMEM;
-
- val = of_get_named_gpio_flags(node, "alarm-gpios", 0, &flags);
- if (val < 0)
- return val;
- alarm->gpio = val;
- alarm->active_low = flags & OF_GPIO_ACTIVE_LOW;
-
- pdata->alarm = alarm;
- }
-
return 0;
}
struct gpio_fan_data *fan_data;
struct gpio_fan_platform_data *pdata = dev_get_platdata(&pdev->dev);
+ fan_data = devm_kzalloc(&pdev->dev, sizeof(struct gpio_fan_data),
+ GFP_KERNEL);
+ if (!fan_data)
+ return -ENOMEM;
+
#ifdef CONFIG_OF_GPIO
if (!pdata) {
pdata = devm_kzalloc(&pdev->dev,
return -EINVAL;
#endif /* CONFIG_OF_GPIO */
- fan_data = devm_kzalloc(&pdev->dev, sizeof(struct gpio_fan_data),
- GFP_KERNEL);
- if (!fan_data)
- return -ENOMEM;
-
fan_data->pdev = pdev;
platform_set_drvdata(pdev, fan_data);
mutex_init(&fan_data->lock);
gpio_fan_groups);
if (IS_ERR(fan_data->hwmon_dev))
return PTR_ERR(fan_data->hwmon_dev);
+#ifdef CONFIG_OF_GPIO
+ /* Optional cooling device register for Device tree platforms */
+ fan_data->cdev = thermal_of_cooling_device_register(pdev->dev.of_node,
+ "gpio-fan",
+ fan_data,
+ &gpio_fan_cool_ops);
+#else /* CONFIG_OF_GPIO */
+ /* Optional cooling device register for non Device tree platforms */
+ fan_data->cdev = thermal_cooling_device_register("gpio-fan", fan_data,
+ &gpio_fan_cool_ops);
+#endif /* CONFIG_OF_GPIO */
dev_info(&pdev->dev, "GPIO fan initialized\n");
return 0;
}
-static void gpio_fan_shutdown(struct platform_device *pdev)
+static int gpio_fan_remove(struct platform_device *pdev)
{
- struct gpio_fan_data *fan_data = dev_get_drvdata(&pdev->dev);
+ struct gpio_fan_data *fan_data = platform_get_drvdata(pdev);
+
+ if (!IS_ERR(fan_data->cdev))
+ thermal_cooling_device_unregister(fan_data->cdev);
if (fan_data->ctrl)
set_fan_speed(fan_data, 0);
+
+ return 0;
+}
+
+static void gpio_fan_shutdown(struct platform_device *pdev)
+{
+ gpio_fan_remove(pdev);
}
#ifdef CONFIG_PM_SLEEP
static struct platform_driver gpio_fan_driver = {
.probe = gpio_fan_probe,
+ .remove = gpio_fan_remove,
.shutdown = gpio_fan_shutdown,
.driver = {
.name = "gpio-fan",
static u8 const watt_sensor_sig[] = {0x41, 0x43};
#define PEX_NUM_SENSOR_FUNCS 3
-static char const * const power_sensor_name_templates[] = {
- "%s%d_average",
- "%s%d_average_lowest",
- "%s%d_average_highest"
-};
-static char const * const temp_sensor_name_templates[] = {
- "%s%d_input",
- "%s%d_input_lowest",
- "%s%d_input_highest"
+static const char * const sensor_name_suffixes[] = {
+ "",
+ "_lowest",
+ "_highest"
};
static void ibmpex_msg_handler(struct ipmi_recv_msg *msg, void *user_msg_data);
return -ENOMEM;
if (type == TEMP_SENSOR)
- sprintf(n, temp_sensor_name_templates[func], "temp", counter);
+ sprintf(n, "temp%d_input%s",
+ counter, sensor_name_suffixes[func]);
else if (type == POWER_SENSOR)
- sprintf(n, power_sensor_name_templates[func], "power", counter);
+ sprintf(n, "power%d_average%s",
+ counter, sensor_name_suffixes[func]);
sysfs_attr_init(&data->sensors[sensor].attr[func].dev_attr.attr);
data->sensors[sensor].attr[func].dev_attr.attr.name = n;
#include <linux/platform_device.h>
#include <asm/opal.h>
#include <linux/err.h>
+#include <asm/cputhreads.h>
+#include <asm/smp.h>
#define MAX_ATTR_LEN 32
+#define MAX_LABEL_LEN 64
/* Sensor suffix name from DT */
#define DT_FAULT_ATTR_SUFFIX "faulted"
*/
enum sensors {
FAN,
- AMBIENT_TEMP,
+ TEMP,
POWER_SUPPLY,
POWER_INPUT,
MAX_SENSOR_TYPE,
};
+#define INVALID_INDEX (-1U)
+
static struct sensor_group {
const char *name;
const char *compatible;
struct attribute_group group;
u32 attr_count;
+ u32 hwmon_index;
} sensor_groups[] = {
{"fan", "ibm,opal-sensor-cooling-fan"},
{"temp", "ibm,opal-sensor-amb-temp"},
struct sensor_data {
u32 id; /* An opaque id of the firmware for each sensor */
+ u32 hwmon_index;
+ u32 opal_index;
enum sensors type;
+ char label[MAX_LABEL_LEN];
char name[MAX_ATTR_LEN];
struct device_attribute dev_attr;
};
return ret;
/* Convert temperature to milli-degrees */
- if (sdata->type == AMBIENT_TEMP)
+ if (sdata->type == TEMP)
x *= 1000;
/* Convert power to micro-watts */
else if (sdata->type == POWER_INPUT)
return sprintf(buf, "%u\n", x);
}
-static int get_sensor_index_attr(const char *name, u32 *index,
- char *attr)
+static ssize_t show_label(struct device *dev, struct device_attribute *devattr,
+ char *buf)
+{
+ struct sensor_data *sdata = container_of(devattr, struct sensor_data,
+ dev_attr);
+
+ return sprintf(buf, "%s\n", sdata->label);
+}
+
+static int __init get_logical_cpu(int hwcpu)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu)
+ if (get_hard_smp_processor_id(cpu) == hwcpu)
+ return cpu;
+
+ return -ENOENT;
+}
+
+static void __init make_sensor_label(struct device_node *np,
+ struct sensor_data *sdata,
+ const char *label)
+{
+ u32 id;
+ size_t n;
+
+ n = snprintf(sdata->label, sizeof(sdata->label), "%s", label);
+
+ /*
+ * Core temp pretty print
+ */
+ if (!of_property_read_u32(np, "ibm,pir", &id)) {
+ int cpuid = get_logical_cpu(id);
+
+ if (cpuid >= 0)
+ /*
+ * The digital thermal sensors are associated
+ * with a core. Let's print out the range of
+ * cpu ids corresponding to the hardware
+ * threads of the core.
+ */
+ n += snprintf(sdata->label + n,
+ sizeof(sdata->label) - n, " %d-%d",
+ cpuid, cpuid + threads_per_core - 1);
+ else
+ n += snprintf(sdata->label + n,
+ sizeof(sdata->label) - n, " phy%d", id);
+ }
+
+ /*
+ * Membuffer pretty print
+ */
+ if (!of_property_read_u32(np, "ibm,chip-id", &id))
+ n += snprintf(sdata->label + n, sizeof(sdata->label) - n,
+ " %d", id & 0xffff);
+}
+
+static int get_sensor_index_attr(const char *name, u32 *index, char *attr)
{
char *hash_pos = strchr(name, '#');
char buf[8] = { 0 };
return 0;
}
+static const char *convert_opal_attr_name(enum sensors type,
+ const char *opal_attr)
+{
+ const char *attr_name = NULL;
+
+ if (!strcmp(opal_attr, DT_FAULT_ATTR_SUFFIX)) {
+ attr_name = "fault";
+ } else if (!strcmp(opal_attr, DT_DATA_ATTR_SUFFIX)) {
+ attr_name = "input";
+ } else if (!strcmp(opal_attr, DT_THRESHOLD_ATTR_SUFFIX)) {
+ if (type == TEMP)
+ attr_name = "max";
+ else if (type == FAN)
+ attr_name = "min";
+ }
+
+ return attr_name;
+}
+
/*
* This function translates the DT node name into the 'hwmon' attribute name.
* IBMPOWERNV device node appear like cooling-fan#2-data, amb-temp#1-thrs etc.
* which need to be mapped as fan2_input, temp1_max respectively before
* populating them inside hwmon device class.
*/
-static int create_hwmon_attr_name(struct device *dev, enum sensors type,
- const char *node_name,
- char *hwmon_attr_name)
+static const char *parse_opal_node_name(const char *node_name,
+ enum sensors type, u32 *index)
{
char attr_suffix[MAX_ATTR_LEN];
- char *attr_name;
- u32 index;
+ const char *attr_name;
int err;
- err = get_sensor_index_attr(node_name, &index, attr_suffix);
- if (err) {
- dev_err(dev, "Sensor device node name '%s' is invalid\n",
- node_name);
- return err;
- }
+ err = get_sensor_index_attr(node_name, index, attr_suffix);
+ if (err)
+ return ERR_PTR(err);
- if (!strcmp(attr_suffix, DT_FAULT_ATTR_SUFFIX)) {
- attr_name = "fault";
- } else if (!strcmp(attr_suffix, DT_DATA_ATTR_SUFFIX)) {
- attr_name = "input";
- } else if (!strcmp(attr_suffix, DT_THRESHOLD_ATTR_SUFFIX)) {
- if (type == AMBIENT_TEMP)
- attr_name = "max";
- else if (type == FAN)
- attr_name = "min";
- else
- return -ENOENT;
- } else {
- return -ENOENT;
+ attr_name = convert_opal_attr_name(type, attr_suffix);
+ if (!attr_name)
+ return ERR_PTR(-ENOENT);
+
+ return attr_name;
+}
+
+static int get_sensor_type(struct device_node *np)
+{
+ enum sensors type;
+ const char *str;
+
+ for (type = 0; type < MAX_SENSOR_TYPE; type++) {
+ if (of_device_is_compatible(np, sensor_groups[type].compatible))
+ return type;
}
- snprintf(hwmon_attr_name, MAX_ATTR_LEN, "%s%d_%s",
- sensor_groups[type].name, index, attr_name);
- return 0;
+ /*
+ * Let's check if we have a newer device tree
+ */
+ if (!of_device_is_compatible(np, "ibm,opal-sensor"))
+ return MAX_SENSOR_TYPE;
+
+ if (of_property_read_string(np, "sensor-type", &str))
+ return MAX_SENSOR_TYPE;
+
+ for (type = 0; type < MAX_SENSOR_TYPE; type++)
+ if (!strcmp(str, sensor_groups[type].name))
+ return type;
+
+ return MAX_SENSOR_TYPE;
+}
+
+static u32 get_sensor_hwmon_index(struct sensor_data *sdata,
+ struct sensor_data *sdata_table, int count)
+{
+ int i;
+
+ /*
+ * We don't use the OPAL index on newer device trees
+ */
+ if (sdata->opal_index != INVALID_INDEX) {
+ for (i = 0; i < count; i++)
+ if (sdata_table[i].opal_index == sdata->opal_index &&
+ sdata_table[i].type == sdata->type)
+ return sdata_table[i].hwmon_index;
+ }
+ return ++sensor_groups[sdata->type].hwmon_index;
}
static int populate_attr_groups(struct platform_device *pdev)
opal = of_find_node_by_path("/ibm,opal/sensors");
for_each_child_of_node(opal, np) {
+ const char *label;
+
if (np->name == NULL)
continue;
- for (type = 0; type < MAX_SENSOR_TYPE; type++)
- if (of_device_is_compatible(np,
- sensor_groups[type].compatible)) {
- sensor_groups[type].attr_count++;
- break;
- }
+ type = get_sensor_type(np);
+ if (type == MAX_SENSOR_TYPE)
+ continue;
+
+ sensor_groups[type].attr_count++;
+
+ /*
+ * add a new attribute for labels
+ */
+ if (!of_property_read_string(np, "label", &label))
+ sensor_groups[type].attr_count++;
}
of_node_put(opal);
return 0;
}
+static void create_hwmon_attr(struct sensor_data *sdata, const char *attr_name,
+ ssize_t (*show)(struct device *dev,
+ struct device_attribute *attr,
+ char *buf))
+{
+ snprintf(sdata->name, MAX_ATTR_LEN, "%s%d_%s",
+ sensor_groups[sdata->type].name, sdata->hwmon_index,
+ attr_name);
+
+ sysfs_attr_init(&sdata->dev_attr.attr);
+ sdata->dev_attr.attr.name = sdata->name;
+ sdata->dev_attr.attr.mode = S_IRUGO;
+ sdata->dev_attr.show = show;
+}
+
/*
* Iterate through the device tree for each child of 'sensors' node, create
* a sysfs attribute file, the file is named by translating the DT node name
}
for_each_child_of_node(opal, np) {
+ const char *attr_name;
+ u32 opal_index;
+ const char *label;
+
if (np->name == NULL)
continue;
- for (type = 0; type < MAX_SENSOR_TYPE; type++)
- if (of_device_is_compatible(np,
- sensor_groups[type].compatible))
- break;
-
+ type = get_sensor_type(np);
if (type == MAX_SENSOR_TYPE)
continue;
- if (of_property_read_u32(np, "sensor-id", &sensor_id)) {
+ /*
+ * Newer device trees use a "sensor-data" property
+ * name for input.
+ */
+ if (of_property_read_u32(np, "sensor-id", &sensor_id) &&
+ of_property_read_u32(np, "sensor-data", &sensor_id)) {
dev_info(&pdev->dev,
"'sensor-id' missing in the node '%s'\n",
np->name);
sdata[count].id = sensor_id;
sdata[count].type = type;
- err = create_hwmon_attr_name(&pdev->dev, type, np->name,
- sdata[count].name);
- if (err)
- goto exit_put_node;
- sysfs_attr_init(&sdata[count].dev_attr.attr);
- sdata[count].dev_attr.attr.name = sdata[count].name;
- sdata[count].dev_attr.attr.mode = S_IRUGO;
- sdata[count].dev_attr.show = show_sensor;
+ /*
+ * If we can not parse the node name, it means we are
+ * running on a newer device tree. We can just forget
+ * about the OPAL index and use a defaut value for the
+ * hwmon attribute name
+ */
+ attr_name = parse_opal_node_name(np->name, type, &opal_index);
+ if (IS_ERR(attr_name)) {
+ attr_name = "input";
+ opal_index = INVALID_INDEX;
+ }
+
+ sdata[count].opal_index = opal_index;
+ sdata[count].hwmon_index =
+ get_sensor_hwmon_index(&sdata[count], sdata, count);
+
+ create_hwmon_attr(&sdata[count], attr_name, show_sensor);
pgroups[type]->attrs[sensor_groups[type].attr_count++] =
&sdata[count++].dev_attr.attr;
+
+ if (!of_property_read_string(np, "label", &label)) {
+ /*
+ * For the label attribute, we can reuse the
+ * "properties" of the previous "input"
+ * attribute. They are related to the same
+ * sensor.
+ */
+ sdata[count].type = type;
+ sdata[count].opal_index = sdata[count - 1].opal_index;
+ sdata[count].hwmon_index = sdata[count - 1].hwmon_index;
+
+ make_sensor_label(np, &sdata[count], label);
+
+ create_hwmon_attr(&sdata[count], "label", show_label);
+
+ pgroups[type]->attrs[sensor_groups[type].attr_count++] =
+ &sdata[count++].dev_attr.attr;
+ }
}
exit_put_node:
* similar parts. The other devices are supported by different drivers.
*
* Supports: IT8603E Super I/O chip w/LPC interface
+ * IT8620E Super I/O chip w/LPC interface
* IT8623E Super I/O chip w/LPC interface
* IT8705F Super I/O chip w/LPC interface
* IT8712F Super I/O chip w/LPC interface
* IT8758E Super I/O chip w/LPC interface
* IT8771E Super I/O chip w/LPC interface
* IT8772E Super I/O chip w/LPC interface
+ * IT8781F Super I/O chip w/LPC interface
* IT8782F Super I/O chip w/LPC interface
* IT8783E/F Super I/O chip w/LPC interface
+ * IT8786E Super I/O chip w/LPC interface
+ * IT8790E Super I/O chip w/LPC interface
* Sis950 A clone of the IT8705F
*
* Copyright (C) 2001 Chris Gauthron
#define DRVNAME "it87"
enum chips { it87, it8712, it8716, it8718, it8720, it8721, it8728, it8771,
- it8772, it8782, it8783, it8603 };
+ it8772, it8781, it8782, it8783, it8786, it8790, it8603, it8620 };
static unsigned short force_id;
module_param(force_id, ushort, 0);
#define IT8728F_DEVID 0x8728
#define IT8771E_DEVID 0x8771
#define IT8772E_DEVID 0x8772
+#define IT8781F_DEVID 0x8781
#define IT8782F_DEVID 0x8782
#define IT8783E_DEVID 0x8783
+#define IT8786E_DEVID 0x8786
+#define IT8790E_DEVID 0x8790
#define IT8603E_DEVID 0x8603
+#define IT8620E_DEVID 0x8620
#define IT8623E_DEVID 0x8623
#define IT87_ACT_REG 0x30
#define IT87_BASE_REG 0x60
/* Logical device 7 registers (IT8712F and later) */
#define IT87_SIO_GPIO1_REG 0x25
+#define IT87_SIO_GPIO2_REG 0x26
#define IT87_SIO_GPIO3_REG 0x27
#define IT87_SIO_GPIO5_REG 0x29
#define IT87_SIO_PINX1_REG 0x2a /* Pin selection */
/* Monitors: 9 voltage (0 to 7, battery), 3 temp (1 to 3), 3 fan (1 to 3) */
-static const u8 IT87_REG_FAN[] = { 0x0d, 0x0e, 0x0f, 0x80, 0x82 };
-static const u8 IT87_REG_FAN_MIN[] = { 0x10, 0x11, 0x12, 0x84, 0x86 };
-static const u8 IT87_REG_FANX[] = { 0x18, 0x19, 0x1a, 0x81, 0x83 };
-static const u8 IT87_REG_FANX_MIN[] = { 0x1b, 0x1c, 0x1d, 0x85, 0x87 };
-static const u8 IT87_REG_TEMP_OFFSET[] = { 0x56, 0x57, 0x59 };
+static const u8 IT87_REG_FAN[] = { 0x0d, 0x0e, 0x0f, 0x80, 0x82, 0x4c };
+static const u8 IT87_REG_FAN_MIN[] = { 0x10, 0x11, 0x12, 0x84, 0x86, 0x4e };
+static const u8 IT87_REG_FANX[] = { 0x18, 0x19, 0x1a, 0x81, 0x83, 0x4d };
+static const u8 IT87_REG_FANX_MIN[] = { 0x1b, 0x1c, 0x1d, 0x85, 0x87, 0x4f };
+static const u8 IT87_REG_TEMP_OFFSET[] = { 0x56, 0x57, 0x59 };
#define IT87_REG_FAN_MAIN_CTRL 0x13
#define IT87_REG_FAN_CTL 0x14
struct it87_devices {
const char *name;
+ const char * const suffix;
u16 features;
u8 peci_mask;
u8 old_peci_mask;
#define FEAT_TEMP_OFFSET (1 << 4)
#define FEAT_TEMP_PECI (1 << 5)
#define FEAT_TEMP_OLD_PECI (1 << 6)
+#define FEAT_FAN16_CONFIG (1 << 7) /* Need to enable 16-bit fans */
+#define FEAT_FIVE_FANS (1 << 8) /* Supports five fans */
+#define FEAT_VID (1 << 9) /* Set if chip supports VID */
+#define FEAT_IN7_INTERNAL (1 << 10) /* Set if in7 is internal */
+#define FEAT_SIX_FANS (1 << 11) /* Supports six fans */
static const struct it87_devices it87_devices[] = {
[it87] = {
.name = "it87",
+ .suffix = "F",
.features = FEAT_OLD_AUTOPWM, /* may need to overwrite */
},
[it8712] = {
.name = "it8712",
- .features = FEAT_OLD_AUTOPWM, /* may need to overwrite */
+ .suffix = "F",
+ .features = FEAT_OLD_AUTOPWM | FEAT_VID,
+ /* may need to overwrite */
},
[it8716] = {
.name = "it8716",
- .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET,
+ .suffix = "F",
+ .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET | FEAT_VID
+ | FEAT_FAN16_CONFIG | FEAT_FIVE_FANS,
},
[it8718] = {
.name = "it8718",
- .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET
- | FEAT_TEMP_OLD_PECI,
+ .suffix = "F",
+ .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET | FEAT_VID
+ | FEAT_TEMP_OLD_PECI | FEAT_FAN16_CONFIG | FEAT_FIVE_FANS,
.old_peci_mask = 0x4,
},
[it8720] = {
.name = "it8720",
- .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET
- | FEAT_TEMP_OLD_PECI,
+ .suffix = "F",
+ .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET | FEAT_VID
+ | FEAT_TEMP_OLD_PECI | FEAT_FAN16_CONFIG | FEAT_FIVE_FANS,
.old_peci_mask = 0x4,
},
[it8721] = {
.name = "it8721",
+ .suffix = "F",
.features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
- | FEAT_TEMP_OFFSET | FEAT_TEMP_OLD_PECI | FEAT_TEMP_PECI,
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_OLD_PECI | FEAT_TEMP_PECI
+ | FEAT_FAN16_CONFIG | FEAT_FIVE_FANS | FEAT_IN7_INTERNAL,
.peci_mask = 0x05,
.old_peci_mask = 0x02, /* Actually reports PCH */
},
[it8728] = {
.name = "it8728",
+ .suffix = "F",
.features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
- | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI,
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_FIVE_FANS
+ | FEAT_IN7_INTERNAL,
.peci_mask = 0x07,
},
[it8771] = {
.name = "it8771",
+ .suffix = "E",
.features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
- | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI,
- /* PECI: guesswork */
- /* 12mV ADC (OHM) */
- /* 16 bit fans (OHM) */
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_IN7_INTERNAL,
+ /* PECI: guesswork */
+ /* 12mV ADC (OHM) */
+ /* 16 bit fans (OHM) */
+ /* three fans, always 16 bit (guesswork) */
.peci_mask = 0x07,
},
[it8772] = {
.name = "it8772",
+ .suffix = "E",
.features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
- | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI,
- /* PECI (coreboot) */
- /* 12mV ADC (HWSensors4, OHM) */
- /* 16 bit fans (HWSensors4, OHM) */
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_IN7_INTERNAL,
+ /* PECI (coreboot) */
+ /* 12mV ADC (HWSensors4, OHM) */
+ /* 16 bit fans (HWSensors4, OHM) */
+ /* three fans, always 16 bit (datasheet) */
.peci_mask = 0x07,
},
+ [it8781] = {
+ .name = "it8781",
+ .suffix = "F",
+ .features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET
+ | FEAT_TEMP_OLD_PECI | FEAT_FAN16_CONFIG,
+ .old_peci_mask = 0x4,
+ },
[it8782] = {
.name = "it8782",
+ .suffix = "F",
.features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET
- | FEAT_TEMP_OLD_PECI,
+ | FEAT_TEMP_OLD_PECI | FEAT_FAN16_CONFIG,
.old_peci_mask = 0x4,
},
[it8783] = {
.name = "it8783",
+ .suffix = "E/F",
.features = FEAT_16BIT_FANS | FEAT_TEMP_OFFSET
- | FEAT_TEMP_OLD_PECI,
+ | FEAT_TEMP_OLD_PECI | FEAT_FAN16_CONFIG,
.old_peci_mask = 0x4,
},
+ [it8786] = {
+ .name = "it8786",
+ .suffix = "E",
+ .features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_IN7_INTERNAL,
+ .peci_mask = 0x07,
+ },
+ [it8790] = {
+ .name = "it8790",
+ .suffix = "E",
+ .features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_IN7_INTERNAL,
+ .peci_mask = 0x07,
+ },
[it8603] = {
.name = "it8603",
+ .suffix = "E",
.features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
- | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI,
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_IN7_INTERNAL,
+ .peci_mask = 0x07,
+ },
+ [it8620] = {
+ .name = "it8620",
+ .suffix = "E",
+ .features = FEAT_NEWER_AUTOPWM | FEAT_12MV_ADC | FEAT_16BIT_FANS
+ | FEAT_TEMP_OFFSET | FEAT_TEMP_PECI | FEAT_SIX_FANS
+ | FEAT_IN7_INTERNAL,
.peci_mask = 0x07,
},
};
#define has_temp_old_peci(data, nr) \
(((data)->features & FEAT_TEMP_OLD_PECI) && \
((data)->old_peci_mask & (1 << nr)))
+#define has_fan16_config(data) ((data)->features & FEAT_FAN16_CONFIG)
+#define has_five_fans(data) ((data)->features & (FEAT_FIVE_FANS | \
+ FEAT_SIX_FANS))
+#define has_vid(data) ((data)->features & FEAT_VID)
+#define has_in7_internal(data) ((data)->features & FEAT_IN7_INTERNAL)
+#define has_six_fans(data) ((data)->features & FEAT_SIX_FANS)
struct it87_sio_data {
enum chips type;
u16 in_scaled; /* Internal voltage sensors are scaled */
u8 in[10][3]; /* [nr][0]=in, [1]=min, [2]=max */
u8 has_fan; /* Bitfield, fans enabled */
- u16 fan[5][2]; /* Register values, [nr][0]=fan, [1]=min */
+ u16 fan[6][2]; /* Register values, [nr][0]=fan, [1]=min */
u8 has_temp; /* Bitfield, temp sensors enabled */
s8 temp[3][4]; /* [nr][0]=temp, [1]=min, [2]=max, [3]=offset */
u8 sensor; /* Register value (IT87_REG_TEMP_ENABLE) */
}
#define DIV_FROM_REG(val) (1 << (val))
+/*
+ * PWM base frequencies. The frequency has to be divided by either 128 or 256,
+ * depending on the chip type, to calculate the actual PWM frequency.
+ *
+ * Some of the chip datasheets suggest a base frequency of 51 kHz instead
+ * of 750 kHz for the slowest base frequency, resulting in a PWM frequency
+ * of 200 Hz. Sometimes both PWM frequency select registers are affected,
+ * sometimes just one. It is unknown if this is a datasheet error or real,
+ * so this is ignored for now.
+ */
static const unsigned int pwm_freq[8] = {
- 48000000 / 128,
- 24000000 / 128,
- 12000000 / 128,
- 8000000 / 128,
- 6000000 / 128,
- 3000000 / 128,
- 1500000 / 128,
- 750000 / 128,
+ 48000000,
+ 24000000,
+ 12000000,
+ 8000000,
+ 6000000,
+ 3000000,
+ 1500000,
+ 750000,
};
static int it87_probe(struct platform_device *pdev);
{
struct it87_data *data = it87_update_device(dev);
int index = (data->fan_ctl >> 4) & 0x07;
+ unsigned int freq;
- return sprintf(buf, "%u\n", pwm_freq[index]);
+ freq = pwm_freq[index] / (has_newer_autopwm(data) ? 256 : 128);
+
+ return sprintf(buf, "%u\n", freq);
}
static ssize_t set_fan(struct device *dev, struct device_attribute *attr,
if (kstrtoul(buf, 10, &val) < 0)
return -EINVAL;
+ val = clamp_val(val, 0, 1000000);
+ val *= has_newer_autopwm(data) ? 256 : 128;
+
/* Search for the nearest available frequency */
for (i = 0; i < 7; i++) {
if (val > (pwm_freq[i] + pwm_freq[i+1]) / 2)
static SENSOR_DEVICE_ATTR_2(fan5_min, S_IRUGO | S_IWUSR, show_fan, set_fan,
4, 1);
+static SENSOR_DEVICE_ATTR_2(fan6_input, S_IRUGO, show_fan, NULL, 5, 0);
+static SENSOR_DEVICE_ATTR_2(fan6_min, S_IRUGO | S_IWUSR, show_fan, set_fan,
+ 5, 1);
+
static SENSOR_DEVICE_ATTR(pwm1_enable, S_IRUGO | S_IWUSR,
show_pwm_enable, set_pwm_enable, 0);
static SENSOR_DEVICE_ATTR(pwm1, S_IRUGO | S_IWUSR, show_pwm, set_pwm, 0);
static SENSOR_DEVICE_ATTR(fan3_alarm, S_IRUGO, show_alarm, NULL, 2);
static SENSOR_DEVICE_ATTR(fan4_alarm, S_IRUGO, show_alarm, NULL, 3);
static SENSOR_DEVICE_ATTR(fan5_alarm, S_IRUGO, show_alarm, NULL, 6);
+static SENSOR_DEVICE_ATTR(fan6_alarm, S_IRUGO, show_alarm, NULL, 7);
static SENSOR_DEVICE_ATTR(temp1_alarm, S_IRUGO, show_alarm, NULL, 16);
static SENSOR_DEVICE_ATTR(temp2_alarm, S_IRUGO, show_alarm, NULL, 17);
static SENSOR_DEVICE_ATTR(temp3_alarm, S_IRUGO, show_alarm, NULL, 18);
static SENSOR_DEVICE_ATTR(fan3_beep, S_IRUGO, show_beep, set_beep, 0);
static SENSOR_DEVICE_ATTR(fan4_beep, S_IRUGO, show_beep, set_beep, 0);
static SENSOR_DEVICE_ATTR(fan5_beep, S_IRUGO, show_beep, set_beep, 0);
+static SENSOR_DEVICE_ATTR(fan6_beep, S_IRUGO, show_beep, set_beep, 0);
static SENSOR_DEVICE_ATTR(temp1_beep, S_IRUGO | S_IWUSR,
show_beep, set_beep, 2);
static SENSOR_DEVICE_ATTR(temp2_beep, S_IRUGO, show_beep, NULL, 2);
&sensor_dev_attr_temp3_beep.dev_attr.attr,
};
-static struct attribute *it87_attributes_fan[5][3+1] = { {
+static struct attribute *it87_attributes_fan[6][3+1] = { {
&sensor_dev_attr_fan1_input.dev_attr.attr,
&sensor_dev_attr_fan1_min.dev_attr.attr,
&sensor_dev_attr_fan1_alarm.dev_attr.attr,
&sensor_dev_attr_fan5_min.dev_attr.attr,
&sensor_dev_attr_fan5_alarm.dev_attr.attr,
NULL
+}, {
+ &sensor_dev_attr_fan6_input.dev_attr.attr,
+ &sensor_dev_attr_fan6_min.dev_attr.attr,
+ &sensor_dev_attr_fan6_alarm.dev_attr.attr,
+ NULL
} };
-static const struct attribute_group it87_group_fan[5] = {
+static const struct attribute_group it87_group_fan[6] = {
{ .attrs = it87_attributes_fan[0] },
{ .attrs = it87_attributes_fan[1] },
{ .attrs = it87_attributes_fan[2] },
{ .attrs = it87_attributes_fan[3] },
{ .attrs = it87_attributes_fan[4] },
+ { .attrs = it87_attributes_fan[5] },
};
static const struct attribute *it87_attributes_fan_div[] = {
&sensor_dev_attr_fan3_beep.dev_attr.attr,
&sensor_dev_attr_fan4_beep.dev_attr.attr,
&sensor_dev_attr_fan5_beep.dev_attr.attr,
+ &sensor_dev_attr_fan6_beep.dev_attr.attr,
};
static struct attribute *it87_attributes_vid[] = {
int err;
u16 chip_type;
const char *board_vendor, *board_name;
+ const struct it87_devices *config;
err = superio_enter();
if (err)
case IT8772E_DEVID:
sio_data->type = it8772;
break;
+ case IT8781F_DEVID:
+ sio_data->type = it8781;
+ break;
case IT8782F_DEVID:
sio_data->type = it8782;
break;
case IT8783E_DEVID:
sio_data->type = it8783;
break;
+ case IT8786E_DEVID:
+ sio_data->type = it8786;
+ break;
+ case IT8790E_DEVID:
+ sio_data->type = it8790;
+ break;
case IT8603E_DEVID:
case IT8623E_DEVID:
sio_data->type = it8603;
break;
+ case IT8620E_DEVID:
+ sio_data->type = it8620;
+ break;
case 0xffff: /* No device at all */
goto exit;
default:
err = 0;
sio_data->revision = superio_inb(DEVREV) & 0x0f;
- pr_info("Found IT%04x%c chip at 0x%x, revision %d\n", chip_type,
- chip_type == 0x8771 || chip_type == 0x8772 ||
- chip_type == 0x8603 ? 'E' : 'F', *address,
- sio_data->revision);
+ pr_info("Found IT%04x%s chip at 0x%x, revision %d\n", chip_type,
+ it87_devices[sio_data->type].suffix,
+ *address, sio_data->revision);
+
+ config = &it87_devices[sio_data->type];
+
+ /* in7 (VSB or VCCH5V) is always internal on some chips */
+ if (has_in7_internal(config))
+ sio_data->internal |= (1 << 1);
/* in8 (Vbat) is always internal */
- sio_data->internal = (1 << 2);
+ sio_data->internal |= (1 << 2);
+
/* Only the IT8603E has in9 */
if (sio_data->type != it8603)
sio_data->skip_in |= (1 << 9);
- /* Read GPIO config and VID value from LDN 7 (GPIO) */
- if (sio_data->type == it87) {
- /* The IT8705F doesn't have VID pins at all */
+ if (!has_vid(config))
sio_data->skip_vid = 1;
+ /* Read GPIO config and VID value from LDN 7 (GPIO) */
+ if (sio_data->type == it87) {
/* The IT8705F has a different LD number for GPIO */
superio_select(5);
sio_data->beep_pin = superio_inb(IT87_SIO_BEEP_PIN_REG) & 0x3f;
} else if (sio_data->type == it8783) {
int reg25, reg27, reg2a, reg2c, regef;
- sio_data->skip_vid = 1; /* No VID */
-
superio_select(GPIO);
reg25 = superio_inb(IT87_SIO_GPIO1_REG);
} else if (sio_data->type == it8603) {
int reg27, reg29;
- sio_data->skip_vid = 1; /* No VID */
superio_select(GPIO);
reg27 = superio_inb(IT87_SIO_GPIO3_REG);
sio_data->skip_in |= (1 << 5); /* No VIN5 */
sio_data->skip_in |= (1 << 6); /* No VIN6 */
- /* no fan4 */
- sio_data->skip_pwm |= (1 << 3);
- sio_data->skip_fan |= (1 << 3);
-
- sio_data->internal |= (1 << 1); /* in7 is VSB */
sio_data->internal |= (1 << 3); /* in9 is AVCC */
+ sio_data->beep_pin = superio_inb(IT87_SIO_BEEP_PIN_REG) & 0x3f;
+ } else if (sio_data->type == it8620) {
+ int reg;
+
+ superio_select(GPIO);
+
+ /* Check for fan4, fan5 */
+ reg = superio_inb(IT87_SIO_GPIO2_REG);
+ if (!(reg & (1 << 5)))
+ sio_data->skip_fan |= (1 << 3);
+ if (!(reg & (1 << 4)))
+ sio_data->skip_fan |= (1 << 4);
+
+ /* Check for pwm3, fan3 */
+ reg = superio_inb(IT87_SIO_GPIO3_REG);
+ if (reg & (1 << 6))
+ sio_data->skip_pwm |= (1 << 2);
+ if (reg & (1 << 7))
+ sio_data->skip_fan |= (1 << 2);
+
+ /* Check for pwm2, fan2 */
+ reg = superio_inb(IT87_SIO_GPIO5_REG);
+ if (reg & (1 << 1))
+ sio_data->skip_pwm |= (1 << 1);
+ if (reg & (1 << 2))
+ sio_data->skip_fan |= (1 << 1);
+
sio_data->beep_pin = superio_inb(IT87_SIO_BEEP_PIN_REG) & 0x3f;
} else {
int reg;
superio_select(GPIO);
reg = superio_inb(IT87_SIO_GPIO3_REG);
- if (sio_data->type == it8721 || sio_data->type == it8728 ||
- sio_data->type == it8771 || sio_data->type == it8772 ||
- sio_data->type == it8782) {
- /*
- * IT8721F/IT8758E, and IT8782F don't have VID pins
- * at all, not sure about the IT8728F and compatibles.
- */
- sio_data->skip_vid = 1;
- } else {
+ if (!sio_data->skip_vid) {
/* We need at least 4 VID pins */
if (reg & 0x0f) {
pr_info("VID is disabled (pins used for GPIO)\n");
}
if (reg & (1 << 0))
sio_data->internal |= (1 << 0);
- if ((reg & (1 << 1)) || sio_data->type == it8721 ||
- sio_data->type == it8728 ||
- sio_data->type == it8771 ||
- sio_data->type == it8772)
+ if (reg & (1 << 1))
sio_data->internal |= (1 << 1);
/*
sysfs_remove_file(&dev->kobj,
it87_attributes_temp_beep[i]);
}
- for (i = 0; i < 5; i++) {
+ for (i = 0; i < 6; i++) {
if (!(data->has_fan & (1 << i)))
continue;
sysfs_remove_group(&dev->kobj, &it87_group_fan[i]);
it87_attributes_fan_div[i]);
}
for (i = 0; i < 3; i++) {
- if (sio_data->skip_pwm & (1 << 0))
+ if (sio_data->skip_pwm & (1 << i))
continue;
sysfs_remove_group(&dev->kobj, &it87_group_pwm[i]);
if (has_old_autopwm(data))
case it87:
if (sio_data->revision >= 0x03) {
data->features &= ~FEAT_OLD_AUTOPWM;
- data->features |= FEAT_16BIT_FANS;
+ data->features |= FEAT_FAN16_CONFIG | FEAT_16BIT_FANS;
}
break;
case it8712:
if (sio_data->revision >= 0x08) {
data->features &= ~FEAT_OLD_AUTOPWM;
- data->features |= FEAT_16BIT_FANS;
+ data->features |= FEAT_FAN16_CONFIG | FEAT_16BIT_FANS |
+ FEAT_FIVE_FANS;
}
break;
default:
data->in_scaled |= (1 << 8); /* in8 is Vbat */
if (sio_data->internal & (1 << 3))
data->in_scaled |= (1 << 9); /* in9 is AVCC */
- } else if (sio_data->type == it8782 || sio_data->type == it8783) {
+ } else if (sio_data->type == it8781 || sio_data->type == it8782 ||
+ sio_data->type == it8783) {
if (sio_data->internal & (1 << 0))
data->in_scaled |= (1 << 3); /* in3 is VCC5V */
if (sio_data->internal & (1 << 1))
/* Do not create fan files for disabled fans */
fan_beep_need_rw = 1;
- for (i = 0; i < 5; i++) {
+ for (i = 0; i < 6; i++) {
if (!(data->has_fan & (1 << i)))
continue;
err = sysfs_create_group(&dev->kobj, &it87_group_fan[i]);
}
data->has_fan = (data->fan_main_ctrl >> 4) & 0x07;
- /* Set tachometers to 16-bit mode if needed, IT8603E (and IT8728F?)
- * has it by default */
- if (has_16bit_fans(data) && data->type != it8603) {
- tmp = it87_read_value(data, IT87_REG_FAN_16BIT);
+ tmp = it87_read_value(data, IT87_REG_FAN_16BIT);
+
+ /* Set tachometers to 16-bit mode if needed */
+ if (has_fan16_config(data)) {
if (~tmp & 0x07 & data->has_fan) {
dev_dbg(&pdev->dev,
"Setting fan1-3 to 16-bit mode\n");
it87_write_value(data, IT87_REG_FAN_16BIT,
tmp | 0x07);
}
- /* IT8705F, IT8782F, and IT8783E/F only support three fans. */
- if (data->type != it87 && data->type != it8782 &&
- data->type != it8783) {
- if (tmp & (1 << 4))
- data->has_fan |= (1 << 3); /* fan4 enabled */
- if (tmp & (1 << 5))
- data->has_fan |= (1 << 4); /* fan5 enabled */
- }
+ }
+
+ /* Check for additional fans */
+ if (has_five_fans(data)) {
+ if (tmp & (1 << 4))
+ data->has_fan |= (1 << 3); /* fan4 enabled */
+ if (tmp & (1 << 5))
+ data->has_fan |= (1 << 4); /* fan5 enabled */
+ if (has_six_fans(data) && (tmp & (1 << 2)))
+ data->has_fan |= (1 << 5); /* fan6 enabled */
}
/* Fan input pins may be used for alternative functions */
if (data->type == it8603)
data->in[9][0] = it87_read_value(data, 0x2f);
- for (i = 0; i < 5; i++) {
+ for (i = 0; i < 6; i++) {
/* Skip disabled fans */
if (!(data->has_fan & (1 << i)))
continue;
#define AT30TSE004_DEVID_MASK 0xffff
/* IDT */
-#define TS3000B3_DEVID 0x2903 /* Also matches TSE2002B3 */
-#define TS3000B3_DEVID_MASK 0xffff
+#define TSE2004_DEVID 0x2200
+#define TSE2004_DEVID_MASK 0xff00
-#define TS3000GB2_DEVID 0x2912 /* Also matches TSE2002GB2 */
-#define TS3000GB2_DEVID_MASK 0xffff
+#define TS3000_DEVID 0x2900 /* Also matches TSE2002 */
+#define TS3000_DEVID_MASK 0xff00
+
+#define TS3001_DEVID 0x3000
+#define TS3001_DEVID_MASK 0xff00
/* Maxim */
#define MAX6604_DEVID 0x3e00
{ ADT_MANID, ADT7408_DEVID, ADT7408_DEVID_MASK },
{ ATMEL_MANID, AT30TS00_DEVID, AT30TS00_DEVID_MASK },
{ ATMEL_MANID2, AT30TSE004_DEVID, AT30TSE004_DEVID_MASK },
- { IDT_MANID, TS3000B3_DEVID, TS3000B3_DEVID_MASK },
- { IDT_MANID, TS3000GB2_DEVID, TS3000GB2_DEVID_MASK },
+ { IDT_MANID, TSE2004_DEVID, TSE2004_DEVID_MASK },
+ { IDT_MANID, TS3000_DEVID, TS3000_DEVID_MASK },
+ { IDT_MANID, TS3001_DEVID, TS3001_DEVID_MASK },
{ MAX_MANID, MAX6604_DEVID, MAX6604_DEVID_MASK },
{ MCP_MANID, MCP9804_DEVID, MCP9804_DEVID_MASK },
{ MCP_MANID, MCP98242_DEVID, MCP98242_DEVID_MASK },
#include <linux/err.h>
#include <linux/mutex.h>
#include <linux/acpi.h>
+#include <linux/dmi.h>
#include <linux/io.h>
#include "lm75.h"
u16 have_temp;
u16 have_temp_fixed;
u16 have_in;
-#ifdef CONFIG_PM
+
/* Remember extra register values over suspend/resume */
u8 vbat;
u8 fandiv1;
u8 fandiv2;
-#endif
+ u8 sio_reg_enable;
};
struct nct6775_sio_data {
int sioreg = data->sioreg;
int regval;
+ /* Store SIO_REG_ENABLE for use during resume */
+ superio_select(sioreg, NCT6775_LD_HWM);
+ data->sio_reg_enable = superio_inb(sioreg, SIO_REG_ENABLE);
+
/* fan4 and fan5 share some pins with the GPIO and serial flash */
if (data->kind == nct6775) {
regval = superio_inb(sioreg, 0x2c);
pwm6pin = false;
} else if (data->kind == nct6776) {
bool gpok = superio_inb(sioreg, 0x27) & 0x80;
+ const char *board_vendor, *board_name;
- superio_select(sioreg, NCT6775_LD_HWM);
- regval = superio_inb(sioreg, SIO_REG_ENABLE);
+ board_vendor = dmi_get_system_info(DMI_BOARD_VENDOR);
+ board_name = dmi_get_system_info(DMI_BOARD_NAME);
+
+ if (board_name && board_vendor &&
+ !strcmp(board_vendor, "ASRock")) {
+ /*
+ * Auxiliary fan monitoring is not enabled on ASRock
+ * Z77 Pro4-M if booted in UEFI Ultra-FastBoot mode.
+ * Observed with BIOS version 2.00.
+ */
+ if (!strcmp(board_name, "Z77 Pro4-M")) {
+ if ((data->sio_reg_enable & 0xe0) != 0xe0) {
+ data->sio_reg_enable |= 0xe0;
+ superio_outb(sioreg, SIO_REG_ENABLE,
+ data->sio_reg_enable);
+ }
+ }
+ }
- if (regval & 0x80)
+ if (data->sio_reg_enable & 0x80)
fan3pin = gpok;
else
fan3pin = !(superio_inb(sioreg, 0x24) & 0x40);
- if (regval & 0x40)
+ if (data->sio_reg_enable & 0x40)
fan4pin = gpok;
else
fan4pin = superio_inb(sioreg, 0x1C) & 0x01;
- if (regval & 0x20)
+ if (data->sio_reg_enable & 0x20)
fan5pin = gpok;
else
fan5pin = superio_inb(sioreg, 0x1C) & 0x02;
}
}
-#ifdef CONFIG_PM
-static int nct6775_suspend(struct device *dev)
+static int __maybe_unused nct6775_suspend(struct device *dev)
{
struct nct6775_data *data = nct6775_update_device(dev);
return 0;
}
-static int nct6775_resume(struct device *dev)
+static int __maybe_unused nct6775_resume(struct device *dev)
{
struct nct6775_data *data = dev_get_drvdata(dev);
+ int sioreg = data->sioreg;
int i, j, err = 0;
+ u8 reg;
mutex_lock(&data->update_lock);
data->bank = 0xff; /* Force initial bank selection */
- if (data->kind == nct6791 || data->kind == nct6792) {
- err = superio_enter(data->sioreg);
- if (err)
- goto abort;
+ err = superio_enter(sioreg);
+ if (err)
+ goto abort;
- nct6791_enable_io_mapping(data->sioreg);
- superio_exit(data->sioreg);
- }
+ superio_select(sioreg, NCT6775_LD_HWM);
+ reg = superio_inb(sioreg, SIO_REG_ENABLE);
+ if (reg != data->sio_reg_enable)
+ superio_outb(sioreg, SIO_REG_ENABLE, data->sio_reg_enable);
+
+ if (data->kind == nct6791 || data->kind == nct6792)
+ nct6791_enable_io_mapping(sioreg);
+
+ superio_exit(sioreg);
/* Restore limits */
for (i = 0; i < data->in_num; i++) {
return err;
}
-static const struct dev_pm_ops nct6775_dev_pm_ops = {
- .suspend = nct6775_suspend,
- .resume = nct6775_resume,
- .freeze = nct6775_suspend,
- .restore = nct6775_resume,
-};
-
-#define NCT6775_DEV_PM_OPS (&nct6775_dev_pm_ops)
-#else
-#define NCT6775_DEV_PM_OPS NULL
-#endif /* CONFIG_PM */
+static SIMPLE_DEV_PM_OPS(nct6775_dev_pm_ops, nct6775_suspend, nct6775_resume);
static struct platform_driver nct6775_driver = {
.driver = {
.name = DRVNAME,
- .pm = NCT6775_DEV_PM_OPS,
+ .pm = &nct6775_dev_pm_ops,
},
.probe = nct6775_probe,
};
--- /dev/null
+/*
+ * nct7904.c - driver for Nuvoton NCT7904D.
+ *
+ * Copyright (c) 2015 Kontron
+ * Author: Vadim V. Vlasov <vvlasov@dev.rtsoft.ru>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/init.h>
+#include <linux/i2c.h>
+#include <linux/mutex.h>
+#include <linux/hwmon.h>
+#include <linux/hwmon-sysfs.h>
+
+#define VENDOR_ID_REG 0x7A /* Any bank */
+#define NUVOTON_ID 0x50
+#define CHIP_ID_REG 0x7B /* Any bank */
+#define NCT7904_ID 0xC5
+#define DEVICE_ID_REG 0x7C /* Any bank */
+
+#define BANK_SEL_REG 0xFF
+#define BANK_0 0x00
+#define BANK_1 0x01
+#define BANK_2 0x02
+#define BANK_3 0x03
+#define BANK_4 0x04
+#define BANK_MAX 0x04
+
+#define FANIN_MAX 12 /* Counted from 1 */
+#define VSEN_MAX 21 /* VSEN1..14, 3VDD, VBAT, V3VSB,
+ LTD (not a voltage), VSEN17..19 */
+#define FANCTL_MAX 4 /* Counted from 1 */
+#define TCPU_MAX 8 /* Counted from 1 */
+#define TEMP_MAX 4 /* Counted from 1 */
+
+#define VT_ADC_CTRL0_REG 0x20 /* Bank 0 */
+#define VT_ADC_CTRL1_REG 0x21 /* Bank 0 */
+#define VT_ADC_CTRL2_REG 0x22 /* Bank 0 */
+#define FANIN_CTRL0_REG 0x24
+#define FANIN_CTRL1_REG 0x25
+#define DTS_T_CTRL0_REG 0x26
+#define DTS_T_CTRL1_REG 0x27
+#define VT_ADC_MD_REG 0x2E
+
+#define VSEN1_HV_REG 0x40 /* Bank 0; 2 regs (HV/LV) per sensor */
+#define TEMP_CH1_HV_REG 0x42 /* Bank 0; same as VSEN2_HV */
+#define LTD_HV_REG 0x62 /* Bank 0; 2 regs in VSEN range */
+#define FANIN1_HV_REG 0x80 /* Bank 0; 2 regs (HV/LV) per sensor */
+#define T_CPU1_HV_REG 0xA0 /* Bank 0; 2 regs (HV/LV) per sensor */
+
+#define PRTS_REG 0x03 /* Bank 2 */
+#define FANCTL1_FMR_REG 0x00 /* Bank 3; 1 reg per channel */
+#define FANCTL1_OUT_REG 0x10 /* Bank 3; 1 reg per channel */
+
+static const unsigned short normal_i2c[] = {
+ 0x2d, 0x2e, I2C_CLIENT_END
+};
+
+struct nct7904_data {
+ struct i2c_client *client;
+ struct mutex bank_lock;
+ int bank_sel;
+ u32 fanin_mask;
+ u32 vsen_mask;
+ u32 tcpu_mask;
+ u8 fan_mode[FANCTL_MAX];
+};
+
+/* Access functions */
+static int nct7904_bank_lock(struct nct7904_data *data, unsigned bank)
+{
+ int ret;
+
+ mutex_lock(&data->bank_lock);
+ if (data->bank_sel == bank)
+ return 0;
+ ret = i2c_smbus_write_byte_data(data->client, BANK_SEL_REG, bank);
+ if (ret == 0)
+ data->bank_sel = bank;
+ else
+ data->bank_sel = -1;
+ return ret;
+}
+
+static inline void nct7904_bank_release(struct nct7904_data *data)
+{
+ mutex_unlock(&data->bank_lock);
+}
+
+/* Read 1-byte register. Returns unsigned reg or -ERRNO on error. */
+static int nct7904_read_reg(struct nct7904_data *data,
+ unsigned bank, unsigned reg)
+{
+ struct i2c_client *client = data->client;
+ int ret;
+
+ ret = nct7904_bank_lock(data, bank);
+ if (ret == 0)
+ ret = i2c_smbus_read_byte_data(client, reg);
+
+ nct7904_bank_release(data);
+ return ret;
+}
+
+/*
+ * Read 2-byte register. Returns register in big-endian format or
+ * -ERRNO on error.
+ */
+static int nct7904_read_reg16(struct nct7904_data *data,
+ unsigned bank, unsigned reg)
+{
+ struct i2c_client *client = data->client;
+ int ret, hi;
+
+ ret = nct7904_bank_lock(data, bank);
+ if (ret == 0) {
+ ret = i2c_smbus_read_byte_data(client, reg);
+ if (ret >= 0) {
+ hi = ret;
+ ret = i2c_smbus_read_byte_data(client, reg + 1);
+ if (ret >= 0)
+ ret |= hi << 8;
+ }
+ }
+
+ nct7904_bank_release(data);
+ return ret;
+}
+
+/* Write 1-byte register. Returns 0 or -ERRNO on error. */
+static int nct7904_write_reg(struct nct7904_data *data,
+ unsigned bank, unsigned reg, u8 val)
+{
+ struct i2c_client *client = data->client;
+ int ret;
+
+ ret = nct7904_bank_lock(data, bank);
+ if (ret == 0)
+ ret = i2c_smbus_write_byte_data(client, reg, val);
+
+ nct7904_bank_release(data);
+ return ret;
+}
+
+/* FANIN ATTR */
+static ssize_t show_fan(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int ret;
+ unsigned cnt, rpm;
+
+ ret = nct7904_read_reg16(data, BANK_0, FANIN1_HV_REG + index * 2);
+ if (ret < 0)
+ return ret;
+ cnt = ((ret & 0xff00) >> 3) | (ret & 0x1f);
+ if (cnt == 0x1fff)
+ rpm = 0;
+ else
+ rpm = 1350000 / cnt;
+ return sprintf(buf, "%u\n", rpm);
+}
+
+static umode_t nct7904_fanin_is_visible(struct kobject *kobj,
+ struct attribute *a, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct nct7904_data *data = dev_get_drvdata(dev);
+
+ if (data->fanin_mask & (1 << n))
+ return a->mode;
+ return 0;
+}
+
+static SENSOR_DEVICE_ATTR(fan1_input, S_IRUGO, show_fan, NULL, 0);
+static SENSOR_DEVICE_ATTR(fan2_input, S_IRUGO, show_fan, NULL, 1);
+static SENSOR_DEVICE_ATTR(fan3_input, S_IRUGO, show_fan, NULL, 2);
+static SENSOR_DEVICE_ATTR(fan4_input, S_IRUGO, show_fan, NULL, 3);
+static SENSOR_DEVICE_ATTR(fan5_input, S_IRUGO, show_fan, NULL, 4);
+static SENSOR_DEVICE_ATTR(fan6_input, S_IRUGO, show_fan, NULL, 5);
+static SENSOR_DEVICE_ATTR(fan7_input, S_IRUGO, show_fan, NULL, 6);
+static SENSOR_DEVICE_ATTR(fan8_input, S_IRUGO, show_fan, NULL, 7);
+static SENSOR_DEVICE_ATTR(fan9_input, S_IRUGO, show_fan, NULL, 8);
+static SENSOR_DEVICE_ATTR(fan10_input, S_IRUGO, show_fan, NULL, 9);
+static SENSOR_DEVICE_ATTR(fan11_input, S_IRUGO, show_fan, NULL, 10);
+static SENSOR_DEVICE_ATTR(fan12_input, S_IRUGO, show_fan, NULL, 11);
+
+static struct attribute *nct7904_fanin_attrs[] = {
+ &sensor_dev_attr_fan1_input.dev_attr.attr,
+ &sensor_dev_attr_fan2_input.dev_attr.attr,
+ &sensor_dev_attr_fan3_input.dev_attr.attr,
+ &sensor_dev_attr_fan4_input.dev_attr.attr,
+ &sensor_dev_attr_fan5_input.dev_attr.attr,
+ &sensor_dev_attr_fan6_input.dev_attr.attr,
+ &sensor_dev_attr_fan7_input.dev_attr.attr,
+ &sensor_dev_attr_fan8_input.dev_attr.attr,
+ &sensor_dev_attr_fan9_input.dev_attr.attr,
+ &sensor_dev_attr_fan10_input.dev_attr.attr,
+ &sensor_dev_attr_fan11_input.dev_attr.attr,
+ &sensor_dev_attr_fan12_input.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group nct7904_fanin_group = {
+ .attrs = nct7904_fanin_attrs,
+ .is_visible = nct7904_fanin_is_visible,
+};
+
+/* VSEN ATTR */
+static ssize_t show_voltage(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int ret;
+ int volt;
+
+ ret = nct7904_read_reg16(data, BANK_0, VSEN1_HV_REG + index * 2);
+ if (ret < 0)
+ return ret;
+ volt = ((ret & 0xff00) >> 5) | (ret & 0x7);
+ if (index < 14)
+ volt *= 2; /* 0.002V scale */
+ else
+ volt *= 6; /* 0.006V scale */
+
+ return sprintf(buf, "%d\n", volt);
+}
+
+static ssize_t show_ltemp(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int ret;
+ int temp;
+
+ ret = nct7904_read_reg16(data, BANK_0, LTD_HV_REG);
+ if (ret < 0)
+ return ret;
+ temp = ((ret & 0xff00) >> 5) | (ret & 0x7);
+ temp = sign_extend32(temp, 10) * 125;
+
+ return sprintf(buf, "%d\n", temp);
+}
+
+static umode_t nct7904_vsen_is_visible(struct kobject *kobj,
+ struct attribute *a, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct nct7904_data *data = dev_get_drvdata(dev);
+
+ if (data->vsen_mask & (1 << n))
+ return a->mode;
+ return 0;
+}
+
+static SENSOR_DEVICE_ATTR(in1_input, S_IRUGO, show_voltage, NULL, 0);
+static SENSOR_DEVICE_ATTR(in2_input, S_IRUGO, show_voltage, NULL, 1);
+static SENSOR_DEVICE_ATTR(in3_input, S_IRUGO, show_voltage, NULL, 2);
+static SENSOR_DEVICE_ATTR(in4_input, S_IRUGO, show_voltage, NULL, 3);
+static SENSOR_DEVICE_ATTR(in5_input, S_IRUGO, show_voltage, NULL, 4);
+static SENSOR_DEVICE_ATTR(in6_input, S_IRUGO, show_voltage, NULL, 5);
+static SENSOR_DEVICE_ATTR(in7_input, S_IRUGO, show_voltage, NULL, 6);
+static SENSOR_DEVICE_ATTR(in8_input, S_IRUGO, show_voltage, NULL, 7);
+static SENSOR_DEVICE_ATTR(in9_input, S_IRUGO, show_voltage, NULL, 8);
+static SENSOR_DEVICE_ATTR(in10_input, S_IRUGO, show_voltage, NULL, 9);
+static SENSOR_DEVICE_ATTR(in11_input, S_IRUGO, show_voltage, NULL, 10);
+static SENSOR_DEVICE_ATTR(in12_input, S_IRUGO, show_voltage, NULL, 11);
+static SENSOR_DEVICE_ATTR(in13_input, S_IRUGO, show_voltage, NULL, 12);
+static SENSOR_DEVICE_ATTR(in14_input, S_IRUGO, show_voltage, NULL, 13);
+/*
+ * Next 3 voltage sensors have specific names in the Nuvoton doc
+ * (3VDD, VBAT, 3VSB) but we use vacant numbers for them.
+ */
+static SENSOR_DEVICE_ATTR(in15_input, S_IRUGO, show_voltage, NULL, 14);
+static SENSOR_DEVICE_ATTR(in16_input, S_IRUGO, show_voltage, NULL, 15);
+static SENSOR_DEVICE_ATTR(in20_input, S_IRUGO, show_voltage, NULL, 16);
+/* This is not a voltage, but a local temperature sensor. */
+static SENSOR_DEVICE_ATTR(temp1_input, S_IRUGO, show_ltemp, NULL, 0);
+static SENSOR_DEVICE_ATTR(in17_input, S_IRUGO, show_voltage, NULL, 18);
+static SENSOR_DEVICE_ATTR(in18_input, S_IRUGO, show_voltage, NULL, 19);
+static SENSOR_DEVICE_ATTR(in19_input, S_IRUGO, show_voltage, NULL, 20);
+
+static struct attribute *nct7904_vsen_attrs[] = {
+ &sensor_dev_attr_in1_input.dev_attr.attr,
+ &sensor_dev_attr_in2_input.dev_attr.attr,
+ &sensor_dev_attr_in3_input.dev_attr.attr,
+ &sensor_dev_attr_in4_input.dev_attr.attr,
+ &sensor_dev_attr_in5_input.dev_attr.attr,
+ &sensor_dev_attr_in6_input.dev_attr.attr,
+ &sensor_dev_attr_in7_input.dev_attr.attr,
+ &sensor_dev_attr_in8_input.dev_attr.attr,
+ &sensor_dev_attr_in9_input.dev_attr.attr,
+ &sensor_dev_attr_in10_input.dev_attr.attr,
+ &sensor_dev_attr_in11_input.dev_attr.attr,
+ &sensor_dev_attr_in12_input.dev_attr.attr,
+ &sensor_dev_attr_in13_input.dev_attr.attr,
+ &sensor_dev_attr_in14_input.dev_attr.attr,
+ &sensor_dev_attr_in15_input.dev_attr.attr,
+ &sensor_dev_attr_in16_input.dev_attr.attr,
+ &sensor_dev_attr_in20_input.dev_attr.attr,
+ &sensor_dev_attr_temp1_input.dev_attr.attr,
+ &sensor_dev_attr_in17_input.dev_attr.attr,
+ &sensor_dev_attr_in18_input.dev_attr.attr,
+ &sensor_dev_attr_in19_input.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group nct7904_vsen_group = {
+ .attrs = nct7904_vsen_attrs,
+ .is_visible = nct7904_vsen_is_visible,
+};
+
+/* CPU_TEMP ATTR */
+static ssize_t show_tcpu(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int ret;
+ int temp;
+
+ ret = nct7904_read_reg16(data, BANK_0, T_CPU1_HV_REG + index * 2);
+ if (ret < 0)
+ return ret;
+
+ temp = ((ret & 0xff00) >> 5) | (ret & 0x7);
+ temp = sign_extend32(temp, 10) * 125;
+ return sprintf(buf, "%d\n", temp);
+}
+
+static umode_t nct7904_tcpu_is_visible(struct kobject *kobj,
+ struct attribute *a, int n)
+{
+ struct device *dev = container_of(kobj, struct device, kobj);
+ struct nct7904_data *data = dev_get_drvdata(dev);
+
+ if (data->tcpu_mask & (1 << n))
+ return a->mode;
+ return 0;
+}
+
+/* "temp1_input" reserved for local temp */
+static SENSOR_DEVICE_ATTR(temp2_input, S_IRUGO, show_tcpu, NULL, 0);
+static SENSOR_DEVICE_ATTR(temp3_input, S_IRUGO, show_tcpu, NULL, 1);
+static SENSOR_DEVICE_ATTR(temp4_input, S_IRUGO, show_tcpu, NULL, 2);
+static SENSOR_DEVICE_ATTR(temp5_input, S_IRUGO, show_tcpu, NULL, 3);
+static SENSOR_DEVICE_ATTR(temp6_input, S_IRUGO, show_tcpu, NULL, 4);
+static SENSOR_DEVICE_ATTR(temp7_input, S_IRUGO, show_tcpu, NULL, 5);
+static SENSOR_DEVICE_ATTR(temp8_input, S_IRUGO, show_tcpu, NULL, 6);
+static SENSOR_DEVICE_ATTR(temp9_input, S_IRUGO, show_tcpu, NULL, 7);
+
+static struct attribute *nct7904_tcpu_attrs[] = {
+ &sensor_dev_attr_temp2_input.dev_attr.attr,
+ &sensor_dev_attr_temp3_input.dev_attr.attr,
+ &sensor_dev_attr_temp4_input.dev_attr.attr,
+ &sensor_dev_attr_temp5_input.dev_attr.attr,
+ &sensor_dev_attr_temp6_input.dev_attr.attr,
+ &sensor_dev_attr_temp7_input.dev_attr.attr,
+ &sensor_dev_attr_temp8_input.dev_attr.attr,
+ &sensor_dev_attr_temp9_input.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group nct7904_tcpu_group = {
+ .attrs = nct7904_tcpu_attrs,
+ .is_visible = nct7904_tcpu_is_visible,
+};
+
+/* PWM ATTR */
+static ssize_t store_pwm(struct device *dev, struct device_attribute *devattr,
+ const char *buf, size_t count)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ unsigned long val;
+ int ret;
+
+ if (kstrtoul(buf, 10, &val) < 0)
+ return -EINVAL;
+ if (val > 255)
+ return -EINVAL;
+
+ ret = nct7904_write_reg(data, BANK_3, FANCTL1_OUT_REG + index, val);
+
+ return ret ? ret : count;
+}
+
+static ssize_t show_pwm(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int val;
+
+ val = nct7904_read_reg(data, BANK_3, FANCTL1_OUT_REG + index);
+ if (val < 0)
+ return val;
+
+ return sprintf(buf, "%d\n", val);
+}
+
+static ssize_t store_mode(struct device *dev, struct device_attribute *devattr,
+ const char *buf, size_t count)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ unsigned long val;
+ int ret;
+
+ if (kstrtoul(buf, 10, &val) < 0)
+ return -EINVAL;
+ if (val > 1 || (val && !data->fan_mode[index]))
+ return -EINVAL;
+
+ ret = nct7904_write_reg(data, BANK_3, FANCTL1_FMR_REG + index,
+ val ? data->fan_mode[index] : 0);
+
+ return ret ? ret : count;
+}
+
+/* Return 0 for manual mode or 1 for SmartFan mode */
+static ssize_t show_mode(struct device *dev,
+ struct device_attribute *devattr, char *buf)
+{
+ int index = to_sensor_dev_attr(devattr)->index;
+ struct nct7904_data *data = dev_get_drvdata(dev);
+ int val;
+
+ val = nct7904_read_reg(data, BANK_3, FANCTL1_FMR_REG + index);
+ if (val < 0)
+ return val;
+
+ return sprintf(buf, "%d\n", val ? 1 : 0);
+}
+
+/* 2 attributes per channel: pwm and mode */
+static SENSOR_DEVICE_ATTR(fan1_pwm, S_IRUGO | S_IWUSR,
+ show_pwm, store_pwm, 0);
+static SENSOR_DEVICE_ATTR(fan1_mode, S_IRUGO | S_IWUSR,
+ show_mode, store_mode, 0);
+static SENSOR_DEVICE_ATTR(fan2_pwm, S_IRUGO | S_IWUSR,
+ show_pwm, store_pwm, 1);
+static SENSOR_DEVICE_ATTR(fan2_mode, S_IRUGO | S_IWUSR,
+ show_mode, store_mode, 1);
+static SENSOR_DEVICE_ATTR(fan3_pwm, S_IRUGO | S_IWUSR,
+ show_pwm, store_pwm, 2);
+static SENSOR_DEVICE_ATTR(fan3_mode, S_IRUGO | S_IWUSR,
+ show_mode, store_mode, 2);
+static SENSOR_DEVICE_ATTR(fan4_pwm, S_IRUGO | S_IWUSR,
+ show_pwm, store_pwm, 3);
+static SENSOR_DEVICE_ATTR(fan4_mode, S_IRUGO | S_IWUSR,
+ show_mode, store_mode, 3);
+
+static struct attribute *nct7904_fanctl_attrs[] = {
+ &sensor_dev_attr_fan1_pwm.dev_attr.attr,
+ &sensor_dev_attr_fan1_mode.dev_attr.attr,
+ &sensor_dev_attr_fan2_pwm.dev_attr.attr,
+ &sensor_dev_attr_fan2_mode.dev_attr.attr,
+ &sensor_dev_attr_fan3_pwm.dev_attr.attr,
+ &sensor_dev_attr_fan3_mode.dev_attr.attr,
+ &sensor_dev_attr_fan4_pwm.dev_attr.attr,
+ &sensor_dev_attr_fan4_mode.dev_attr.attr,
+ NULL
+};
+
+static const struct attribute_group nct7904_fanctl_group = {
+ .attrs = nct7904_fanctl_attrs,
+};
+
+static const struct attribute_group *nct7904_groups[] = {
+ &nct7904_fanin_group,
+ &nct7904_vsen_group,
+ &nct7904_tcpu_group,
+ &nct7904_fanctl_group,
+ NULL
+};
+
+/* Return 0 if detection is successful, -ENODEV otherwise */
+static int nct7904_detect(struct i2c_client *client,
+ struct i2c_board_info *info)
+{
+ struct i2c_adapter *adapter = client->adapter;
+
+ if (!i2c_check_functionality(adapter,
+ I2C_FUNC_SMBUS_READ_BYTE |
+ I2C_FUNC_SMBUS_WRITE_BYTE_DATA))
+ return -ENODEV;
+
+ /* Determine the chip type. */
+ if (i2c_smbus_read_byte_data(client, VENDOR_ID_REG) != NUVOTON_ID ||
+ i2c_smbus_read_byte_data(client, CHIP_ID_REG) != NCT7904_ID ||
+ (i2c_smbus_read_byte_data(client, DEVICE_ID_REG) & 0xf0) != 0x50 ||
+ (i2c_smbus_read_byte_data(client, BANK_SEL_REG) & 0xf8) != 0x00)
+ return -ENODEV;
+
+ strlcpy(info->type, "nct7904", I2C_NAME_SIZE);
+
+ return 0;
+}
+
+static int nct7904_probe(struct i2c_client *client,
+ const struct i2c_device_id *id)
+{
+ struct nct7904_data *data;
+ struct device *hwmon_dev;
+ struct device *dev = &client->dev;
+ int ret, i;
+ u32 mask;
+
+ data = devm_kzalloc(dev, sizeof(struct nct7904_data), GFP_KERNEL);
+ if (!data)
+ return -ENOMEM;
+
+ data->client = client;
+ mutex_init(&data->bank_lock);
+ data->bank_sel = -1;
+
+ /* Setup sensor groups. */
+ /* FANIN attributes */
+ ret = nct7904_read_reg16(data, BANK_0, FANIN_CTRL0_REG);
+ if (ret < 0)
+ return ret;
+ data->fanin_mask = (ret >> 8) | ((ret & 0xff) << 8);
+
+ /*
+ * VSEN attributes
+ *
+ * Note: voltage sensors overlap with external temperature
+ * sensors. So, if we ever decide to support the latter
+ * we will have to adjust 'vsen_mask' accordingly.
+ */
+ mask = 0;
+ ret = nct7904_read_reg16(data, BANK_0, VT_ADC_CTRL0_REG);
+ if (ret >= 0)
+ mask = (ret >> 8) | ((ret & 0xff) << 8);
+ ret = nct7904_read_reg(data, BANK_0, VT_ADC_CTRL2_REG);
+ if (ret >= 0)
+ mask |= (ret << 16);
+ data->vsen_mask = mask;
+
+ /* CPU_TEMP attributes */
+ ret = nct7904_read_reg16(data, BANK_0, DTS_T_CTRL0_REG);
+ if (ret < 0)
+ return ret;
+ data->tcpu_mask = ((ret >> 8) & 0xf) | ((ret & 0xf) << 4);
+
+ for (i = 0; i < FANCTL_MAX; i++) {
+ ret = nct7904_read_reg(data, BANK_3, FANCTL1_FMR_REG + i);
+ if (ret < 0)
+ return ret;
+ data->fan_mode[i] = ret;
+ }
+
+ hwmon_dev =
+ devm_hwmon_device_register_with_groups(dev, client->name, data,
+ nct7904_groups);
+ return PTR_ERR_OR_ZERO(hwmon_dev);
+}
+
+static const struct i2c_device_id nct7904_id[] = {
+ {"nct7904", 0},
+ {}
+};
+
+static struct i2c_driver nct7904_driver = {
+ .class = I2C_CLASS_HWMON,
+ .driver = {
+ .name = "nct7904",
+ },
+ .probe = nct7904_probe,
+ .id_table = nct7904_id,
+ .detect = nct7904_detect,
+ .address_list = normal_i2c,
+};
+
+module_i2c_driver(nct7904_driver);
+
+MODULE_AUTHOR("Vadim V. Vlasov <vvlasov@dev.rtsoft.ru>");
+MODULE_DESCRIPTION("Hwmon driver for NUVOTON NCT7904");
+MODULE_LICENSE("GPL");
#include <linux/platform_device.h>
#include <linux/pwm.h>
#include <linux/sysfs.h>
+#include <linux/thermal.h>
#define MAX_PWM 255
struct pwm_fan_ctx {
struct mutex lock;
struct pwm_device *pwm;
- unsigned char pwm_value;
+ unsigned int pwm_value;
+ unsigned int pwm_fan_state;
+ unsigned int pwm_fan_max_state;
+ unsigned int *pwm_fan_cooling_levels;
+ struct thermal_cooling_device *cdev;
};
-static ssize_t set_pwm(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
+static int __set_pwm(struct pwm_fan_ctx *ctx, unsigned long pwm)
{
- struct pwm_fan_ctx *ctx = dev_get_drvdata(dev);
- unsigned long pwm, duty;
- ssize_t ret;
-
- if (kstrtoul(buf, 10, &pwm) || pwm > MAX_PWM)
- return -EINVAL;
+ unsigned long duty;
+ int ret = 0;
mutex_lock(&ctx->lock);
-
if (ctx->pwm_value == pwm)
- goto exit_set_pwm_no_change;
-
- if (pwm == 0) {
- pwm_disable(ctx->pwm);
- goto exit_set_pwm;
- }
+ goto exit_set_pwm_err;
duty = DIV_ROUND_UP(pwm * (ctx->pwm->period - 1), MAX_PWM);
ret = pwm_config(ctx->pwm, duty, ctx->pwm->period);
if (ret)
goto exit_set_pwm_err;
+ if (pwm == 0)
+ pwm_disable(ctx->pwm);
+
if (ctx->pwm_value == 0) {
ret = pwm_enable(ctx->pwm);
if (ret)
goto exit_set_pwm_err;
}
-exit_set_pwm:
ctx->pwm_value = pwm;
-exit_set_pwm_no_change:
- ret = count;
exit_set_pwm_err:
mutex_unlock(&ctx->lock);
return ret;
}
+static void pwm_fan_update_state(struct pwm_fan_ctx *ctx, unsigned long pwm)
+{
+ int i;
+
+ for (i = 0; i < ctx->pwm_fan_max_state; ++i)
+ if (pwm < ctx->pwm_fan_cooling_levels[i + 1])
+ break;
+
+ ctx->pwm_fan_state = i;
+}
+
+static ssize_t set_pwm(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct pwm_fan_ctx *ctx = dev_get_drvdata(dev);
+ unsigned long pwm;
+ int ret;
+
+ if (kstrtoul(buf, 10, &pwm) || pwm > MAX_PWM)
+ return -EINVAL;
+
+ ret = __set_pwm(ctx, pwm);
+ if (ret)
+ return ret;
+
+ pwm_fan_update_state(ctx, pwm);
+ return count;
+}
+
static ssize_t show_pwm(struct device *dev,
struct device_attribute *attr, char *buf)
{
ATTRIBUTE_GROUPS(pwm_fan);
+/* thermal cooling device callbacks */
+static int pwm_fan_get_max_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct pwm_fan_ctx *ctx = cdev->devdata;
+
+ if (!ctx)
+ return -EINVAL;
+
+ *state = ctx->pwm_fan_max_state;
+
+ return 0;
+}
+
+static int pwm_fan_get_cur_state(struct thermal_cooling_device *cdev,
+ unsigned long *state)
+{
+ struct pwm_fan_ctx *ctx = cdev->devdata;
+
+ if (!ctx)
+ return -EINVAL;
+
+ *state = ctx->pwm_fan_state;
+
+ return 0;
+}
+
+static int
+pwm_fan_set_cur_state(struct thermal_cooling_device *cdev, unsigned long state)
+{
+ struct pwm_fan_ctx *ctx = cdev->devdata;
+ int ret;
+
+ if (!ctx || (state > ctx->pwm_fan_max_state))
+ return -EINVAL;
+
+ if (state == ctx->pwm_fan_state)
+ return 0;
+
+ ret = __set_pwm(ctx, ctx->pwm_fan_cooling_levels[state]);
+ if (ret) {
+ dev_err(&cdev->device, "Cannot set pwm!\n");
+ return ret;
+ }
+
+ ctx->pwm_fan_state = state;
+
+ return ret;
+}
+
+static const struct thermal_cooling_device_ops pwm_fan_cooling_ops = {
+ .get_max_state = pwm_fan_get_max_state,
+ .get_cur_state = pwm_fan_get_cur_state,
+ .set_cur_state = pwm_fan_set_cur_state,
+};
+
+static int pwm_fan_of_get_cooling_data(struct device *dev,
+ struct pwm_fan_ctx *ctx)
+{
+ struct device_node *np = dev->of_node;
+ int num, i, ret;
+
+ if (!of_find_property(np, "cooling-levels", NULL))
+ return 0;
+
+ ret = of_property_count_u32_elems(np, "cooling-levels");
+ if (ret <= 0) {
+ dev_err(dev, "Wrong data!\n");
+ return ret ? : -EINVAL;
+ }
+
+ num = ret;
+ ctx->pwm_fan_cooling_levels = devm_kzalloc(dev, num * sizeof(u32),
+ GFP_KERNEL);
+ if (!ctx->pwm_fan_cooling_levels)
+ return -ENOMEM;
+
+ ret = of_property_read_u32_array(np, "cooling-levels",
+ ctx->pwm_fan_cooling_levels, num);
+ if (ret) {
+ dev_err(dev, "Property 'cooling-levels' cannot be read!\n");
+ return ret;
+ }
+
+ for (i = 0; i < num; i++) {
+ if (ctx->pwm_fan_cooling_levels[i] > MAX_PWM) {
+ dev_err(dev, "PWM fan state[%d]:%d > %d\n", i,
+ ctx->pwm_fan_cooling_levels[i], MAX_PWM);
+ return -EINVAL;
+ }
+ }
+
+ ctx->pwm_fan_max_state = num - 1;
+
+ return 0;
+}
+
static int pwm_fan_probe(struct platform_device *pdev)
{
- struct device *hwmon;
+ struct thermal_cooling_device *cdev;
struct pwm_fan_ctx *ctx;
+ struct device *hwmon;
int duty_cycle;
int ret;
pwm_disable(ctx->pwm);
return PTR_ERR(hwmon);
}
+
+ ret = pwm_fan_of_get_cooling_data(&pdev->dev, ctx);
+ if (ret)
+ return ret;
+
+ ctx->pwm_fan_state = ctx->pwm_fan_max_state;
+ if (IS_ENABLED(CONFIG_THERMAL)) {
+ cdev = thermal_of_cooling_device_register(pdev->dev.of_node,
+ "pwm-fan", ctx,
+ &pwm_fan_cooling_ops);
+ if (IS_ERR(cdev)) {
+ dev_err(&pdev->dev,
+ "Failed to register pwm-fan as cooling device");
+ pwm_disable(ctx->pwm);
+ return PTR_ERR(cdev);
+ }
+ ctx->cdev = cdev;
+ thermal_cdev_update(cdev);
+ }
+
return 0;
}
{
struct pwm_fan_ctx *ctx = platform_get_drvdata(pdev);
+ thermal_cooling_device_unregister(ctx->cdev);
if (ctx->pwm_value)
pwm_disable(ctx->pwm);
return 0;
static SIMPLE_DEV_PM_OPS(pwm_fan_pm, pwm_fan_suspend, pwm_fan_resume);
-static struct of_device_id of_pwm_fan_match[] = {
+static const struct of_device_id of_pwm_fan_match[] = {
{ .compatible = "pwm-fan", },
{},
};
},
};
-static struct of_device_id vexpress_hwmon_of_match[] = {
+static const struct of_device_id vexpress_hwmon_of_match[] = {
#if !defined(CONFIG_REGULATOR_VEXPRESS)
{
.compatible = "arm,vexpress-volt",
#include <linux/acpi.h>
#include <linux/i2c.h>
#include <linux/interrupt.h>
+
#include <asm/iosf_mbi.h>
+
#include "i2c-designware-core.h"
#define SEMAPHORE_TIMEOUT 100
#define PUNIT_SEMAPHORE 0x7
+#define PUNIT_SEMAPHORE_BIT BIT(0)
+#define PUNIT_SEMAPHORE_ACQUIRE BIT(1)
static unsigned long acquired;
static int get_sem(struct device *dev, u32 *sem)
{
- u32 reg_val;
+ u32 data;
int ret;
ret = iosf_mbi_read(BT_MBI_UNIT_PMC, BT_MBI_BUNIT_READ, PUNIT_SEMAPHORE,
- ®_val);
+ &data);
if (ret) {
dev_err(dev, "iosf failed to read punit semaphore\n");
return ret;
}
- *sem = reg_val & 0x1;
+ *sem = data & PUNIT_SEMAPHORE_BIT;
return 0;
}
return;
}
- data = data & 0xfffffffe;
+ data &= ~PUNIT_SEMAPHORE_BIT;
if (iosf_mbi_write(BT_MBI_UNIT_PMC, BT_MBI_BUNIT_WRITE,
- PUNIT_SEMAPHORE, data))
+ PUNIT_SEMAPHORE, data))
dev_err(dev, "iosf failed to reset punit semaphore during write\n");
}
-int baytrail_i2c_acquire(struct dw_i2c_dev *dev)
+static int baytrail_i2c_acquire(struct dw_i2c_dev *dev)
{
- u32 sem = 0;
+ u32 sem;
int ret;
unsigned long start, end;
+ might_sleep();
+
if (!dev || !dev->dev)
return -ENODEV;
- if (!dev->acquire_lock)
+ if (!dev->release_lock)
return 0;
- /* host driver writes 0x2 to side band semaphore register */
+ /* host driver writes to side band semaphore register */
ret = iosf_mbi_write(BT_MBI_UNIT_PMC, BT_MBI_BUNIT_WRITE,
- PUNIT_SEMAPHORE, 0x2);
+ PUNIT_SEMAPHORE, PUNIT_SEMAPHORE_ACQUIRE);
if (ret) {
dev_err(dev->dev, "iosf punit semaphore request failed\n");
return ret;
/* host driver waits for bit 0 to be set in semaphore register */
start = jiffies;
end = start + msecs_to_jiffies(SEMAPHORE_TIMEOUT);
- while (!time_after(jiffies, end)) {
+ do {
ret = get_sem(dev->dev, &sem);
if (!ret && sem) {
acquired = jiffies;
}
usleep_range(1000, 2000);
- }
+ } while (time_before(jiffies, end));
dev_err(dev->dev, "punit semaphore timed out, resetting\n");
reset_semaphore(dev->dev);
ret = iosf_mbi_read(BT_MBI_UNIT_PMC, BT_MBI_BUNIT_READ,
- PUNIT_SEMAPHORE, &sem);
- if (!ret)
+ PUNIT_SEMAPHORE, &sem);
+ if (ret)
dev_err(dev->dev, "iosf failed to read punit semaphore\n");
else
dev_err(dev->dev, "PUNIT SEM: %d\n", sem);
return -ETIMEDOUT;
}
-EXPORT_SYMBOL(baytrail_i2c_acquire);
-void baytrail_i2c_release(struct dw_i2c_dev *dev)
+static void baytrail_i2c_release(struct dw_i2c_dev *dev)
{
if (!dev || !dev->dev)
return;
dev_dbg(dev->dev, "punit semaphore held for %ums\n",
jiffies_to_msecs(jiffies - acquired));
}
-EXPORT_SYMBOL(baytrail_i2c_release);
int i2c_dw_eval_lock_support(struct dw_i2c_dev *dev)
{
return 0;
status = acpi_evaluate_integer(handle, "_SEM", NULL, &shared_host);
-
if (ACPI_FAILURE(status))
return 0;
return 0;
}
-EXPORT_SYMBOL(i2c_dw_eval_lock_support);
MODULE_AUTHOR("David E. Box <david.e.box@linux.intel.com>");
MODULE_DESCRIPTION("Baytrail I2C Semaphore driver");
status = driver->remove(client);
}
- if (dev->of_node)
- irq_dispose_mapping(client->irq);
-
dev_pm_domain_detach(&client->dev, true);
return status;
}
printk(KERN_CONT "DataRequest ");
if (stat & ATA_CORR)
printk(KERN_CONT "CorrectedError ");
- if (stat & ATA_IDX)
- printk(KERN_CONT "Index ");
+ if (stat & ATA_SENSE)
+ printk(KERN_CONT "Sense ");
if (stat & ATA_ERR)
printk(KERN_CONT "Error ");
}
(hwif->host_flags & IDE_HFLAG_BROKEN_ALTSTATUS) == 0) {
a = tp_ops->read_altstatus(hwif);
s = tp_ops->read_status(hwif);
- if ((a ^ s) & ~ATA_IDX)
+ if ((a ^ s) & ~ATA_SENSE)
/* ancient Seagate drives, broken interfaces */
printk(KERN_INFO "%s: probing with STATUS(0x%02x) "
"instead of ALTSTATUS(0x%02x)\n",
tape->best_dsc_rw_freq = clamp_t(unsigned long, t, IDETAPE_DSC_RW_MIN,
IDETAPE_DSC_RW_MAX);
printk(KERN_INFO "ide-tape: %s <-> %s: %dKBps, %d*%dkB buffer, "
- "%lums tDSC%s\n",
+ "%ums tDSC%s\n",
drive->name, tape->name, *(u16 *)&tape->caps[14],
(*(u16 *)&tape->caps[16] * 512) / tape->buffer_size,
tape->buffer_size / 1024,
- tape->best_dsc_rw_freq * 1000 / HZ,
+ jiffies_to_msecs(tape->best_dsc_rw_freq),
(drive->dev_flags & IDE_DFLAG_USING_DMA) ? ", DMA" : "");
ide_proc_register_driver(drive, tape->driver);
#include <linux/kernel.h>
#include <linux/cpuidle.h>
-#include <linux/clockchips.h>
+#include <linux/tick.h>
#include <trace/events/power.h>
#include <linux/sched.h>
#include <linux/notifier.h>
leave_mm(cpu);
if (!(lapic_timer_reliable_states & (1 << (cstate))))
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &cpu);
+ tick_broadcast_enter();
mwait_idle_with_hints(eax, ecx);
if (!(lapic_timer_reliable_states & (1 << (cstate))))
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &cpu);
+ tick_broadcast_exit();
return index;
}
static void __setup_broadcast_timer(void *arg)
{
- unsigned long reason = (unsigned long)arg;
- int cpu = smp_processor_id();
-
- reason = reason ?
- CLOCK_EVT_NOTIFY_BROADCAST_ON : CLOCK_EVT_NOTIFY_BROADCAST_OFF;
+ unsigned long on = (unsigned long)arg;
- clockevents_notify(reason, &cpu);
+ if (on)
+ tick_broadcast_enable();
+ else
+ tick_broadcast_disable();
}
static int cpu_hotplug_notify(struct notifier_block *n,
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = bma180_get_data_reg(data, bit);
if (ret < 0) {
int val;
int val2;
u8 bw_bits;
-} bmc150_accel_samp_freq_table[] = { {7, 810000, 0x08},
- {15, 630000, 0x09},
- {31, 250000, 0x0A},
- {62, 500000, 0x0B},
- {125, 0, 0x0C},
- {250, 0, 0x0D},
- {500, 0, 0x0E},
- {1000, 0, 0x0F} };
+} bmc150_accel_samp_freq_table[] = { {15, 620000, 0x08},
+ {31, 260000, 0x09},
+ {62, 500000, 0x0A},
+ {125, 0, 0x0B},
+ {250, 0, 0x0C},
+ {500, 0, 0x0D},
+ {1000, 0, 0x0E},
+ {2000, 0, 0x0F} };
static const struct {
int bw_bits;
}
static IIO_CONST_ATTR_SAMP_FREQ_AVAIL(
- "7.810000 15.630000 31.250000 62.500000 125 250 500 1000");
+ "15.620000 31.260000 62.50000 125 250 500 1000 2000");
static struct attribute *bmc150_accel_attributes[] = {
&iio_const_attr_sampling_frequency_available.dev_attr.attr,
int bit, ret, i = 0;
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = i2c_smbus_read_word_data(data->client,
BMC150_ACCEL_AXIS_TO_REG(bit));
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = kxcjk1013_get_acc_reg(data, bit);
if (ret < 0) {
device. Depending on platform configuration, this general purpose ADC can
be used for sampling sensors such as thermal resistors.
+config DA9150_GPADC
+ tristate "Dialog DA9150 GPADC driver support"
+ depends on MFD_DA9150
+ help
+ Say yes here to build support for Dialog DA9150 GPADC.
+
+ This driver can also be built as a module. If chosen, the module name
+ will be da9150-gpadc.
+
config CC10001_ADC
tristate "Cosmic Circuits 10001 ADC driver"
- depends on HAS_IOMEM || HAVE_CLK || REGULATOR
+ depends on HAVE_CLK || REGULATOR
+ depends on HAS_IOMEM
select IIO_BUFFER
select IIO_TRIGGERED_BUFFER
help
obj-$(CONFIG_AD799X) += ad799x.o
obj-$(CONFIG_AT91_ADC) += at91_adc.o
obj-$(CONFIG_AXP288_ADC) += axp288_adc.o
+obj-$(CONFIG_DA9150_GPADC) += da9150-gpadc.o
obj-$(CONFIG_CC10001_ADC) += cc10001_adc.o
obj-$(CONFIG_EXYNOS_ADC) += exynos_adc.o
obj-$(CONFIG_LP8788_ADC) += lp8788_adc.o
{
struct iio_dev *idev = iio_trigger_get_drvdata(trig);
struct at91_adc_state *st = iio_priv(idev);
- struct iio_buffer *buffer = idev->buffer;
struct at91_adc_reg_desc *reg = st->registers;
u32 status = at91_adc_readl(st, reg->trigger_register);
int value;
at91_adc_writel(st, reg->trigger_register,
status | value);
- for_each_set_bit(bit, buffer->scan_mask,
+ for_each_set_bit(bit, idev->active_scan_mask,
st->num_channels) {
struct iio_chan_spec const *chan = idev->channels + bit;
at91_adc_writel(st, AT91_ADC_CHER,
at91_adc_writel(st, reg->trigger_register,
status & ~value);
- for_each_set_bit(bit, buffer->scan_mask,
+ for_each_set_bit(bit, idev->active_scan_mask,
st->num_channels) {
struct iio_chan_spec const *chan = idev->channels + bit;
at91_adc_writel(st, AT91_ADC_CHDR,
--- /dev/null
+/*
+ * DA9150 GPADC Driver
+ *
+ * Copyright (c) 2014 Dialog Semiconductor
+ *
+ * Author: Adam Thomson <Adam.Thomson.Opensource@diasemi.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/interrupt.h>
+#include <linux/mutex.h>
+#include <linux/completion.h>
+#include <linux/iio/iio.h>
+#include <linux/iio/machine.h>
+#include <linux/iio/driver.h>
+#include <linux/mfd/da9150/core.h>
+#include <linux/mfd/da9150/registers.h>
+
+/* Channels */
+enum da9150_gpadc_hw_channel {
+ DA9150_GPADC_HW_CHAN_GPIOA_2V = 0,
+ DA9150_GPADC_HW_CHAN_GPIOA_2V_,
+ DA9150_GPADC_HW_CHAN_GPIOB_2V,
+ DA9150_GPADC_HW_CHAN_GPIOB_2V_,
+ DA9150_GPADC_HW_CHAN_GPIOC_2V,
+ DA9150_GPADC_HW_CHAN_GPIOC_2V_,
+ DA9150_GPADC_HW_CHAN_GPIOD_2V,
+ DA9150_GPADC_HW_CHAN_GPIOD_2V_,
+ DA9150_GPADC_HW_CHAN_IBUS_SENSE,
+ DA9150_GPADC_HW_CHAN_IBUS_SENSE_,
+ DA9150_GPADC_HW_CHAN_VBUS_DIV,
+ DA9150_GPADC_HW_CHAN_VBUS_DIV_,
+ DA9150_GPADC_HW_CHAN_ID,
+ DA9150_GPADC_HW_CHAN_ID_,
+ DA9150_GPADC_HW_CHAN_VSYS,
+ DA9150_GPADC_HW_CHAN_VSYS_,
+ DA9150_GPADC_HW_CHAN_GPIOA_6V,
+ DA9150_GPADC_HW_CHAN_GPIOA_6V_,
+ DA9150_GPADC_HW_CHAN_GPIOB_6V,
+ DA9150_GPADC_HW_CHAN_GPIOB_6V_,
+ DA9150_GPADC_HW_CHAN_GPIOC_6V,
+ DA9150_GPADC_HW_CHAN_GPIOC_6V_,
+ DA9150_GPADC_HW_CHAN_GPIOD_6V,
+ DA9150_GPADC_HW_CHAN_GPIOD_6V_,
+ DA9150_GPADC_HW_CHAN_VBAT,
+ DA9150_GPADC_HW_CHAN_VBAT_,
+ DA9150_GPADC_HW_CHAN_TBAT,
+ DA9150_GPADC_HW_CHAN_TBAT_,
+ DA9150_GPADC_HW_CHAN_TJUNC_CORE,
+ DA9150_GPADC_HW_CHAN_TJUNC_CORE_,
+ DA9150_GPADC_HW_CHAN_TJUNC_OVP,
+ DA9150_GPADC_HW_CHAN_TJUNC_OVP_,
+};
+
+enum da9150_gpadc_channel {
+ DA9150_GPADC_CHAN_GPIOA = 0,
+ DA9150_GPADC_CHAN_GPIOB,
+ DA9150_GPADC_CHAN_GPIOC,
+ DA9150_GPADC_CHAN_GPIOD,
+ DA9150_GPADC_CHAN_IBUS,
+ DA9150_GPADC_CHAN_VBUS,
+ DA9150_GPADC_CHAN_VSYS,
+ DA9150_GPADC_CHAN_VBAT,
+ DA9150_GPADC_CHAN_TBAT,
+ DA9150_GPADC_CHAN_TJUNC_CORE,
+ DA9150_GPADC_CHAN_TJUNC_OVP,
+};
+
+/* Private data */
+struct da9150_gpadc {
+ struct da9150 *da9150;
+ struct device *dev;
+
+ struct mutex lock;
+ struct completion complete;
+};
+
+
+static irqreturn_t da9150_gpadc_irq(int irq, void *data)
+{
+
+ struct da9150_gpadc *gpadc = data;
+
+ complete(&gpadc->complete);
+
+ return IRQ_HANDLED;
+}
+
+static int da9150_gpadc_read_adc(struct da9150_gpadc *gpadc, int hw_chan)
+{
+ u8 result_regs[2];
+ int result;
+
+ mutex_lock(&gpadc->lock);
+
+ /* Set channel & enable measurement */
+ da9150_reg_write(gpadc->da9150, DA9150_GPADC_MAN,
+ (DA9150_GPADC_EN_MASK |
+ hw_chan << DA9150_GPADC_MUX_SHIFT));
+
+ /* Consume left-over completion from a previous timeout */
+ try_wait_for_completion(&gpadc->complete);
+
+ /* Check for actual completion */
+ wait_for_completion_timeout(&gpadc->complete, msecs_to_jiffies(5));
+
+ /* Read result and status from device */
+ da9150_bulk_read(gpadc->da9150, DA9150_GPADC_RES_A, 2, result_regs);
+
+ mutex_unlock(&gpadc->lock);
+
+ /* Check to make sure device really has completed reading */
+ if (result_regs[1] & DA9150_GPADC_RUN_MASK) {
+ dev_err(gpadc->dev, "Timeout on channel %d of GPADC\n",
+ hw_chan);
+ return -ETIMEDOUT;
+ }
+
+ /* LSBs - 2 bits */
+ result = (result_regs[1] & DA9150_GPADC_RES_L_MASK) >>
+ DA9150_GPADC_RES_L_SHIFT;
+ /* MSBs - 8 bits */
+ result |= result_regs[0] << DA9150_GPADC_RES_L_BITS;
+
+ return result;
+}
+
+static inline int da9150_gpadc_gpio_6v_voltage_now(int raw_val)
+{
+ /* Convert to mV */
+ return (6 * ((raw_val * 1000) + 500)) / 1024;
+}
+
+static inline int da9150_gpadc_ibus_current_avg(int raw_val)
+{
+ /* Convert to mA */
+ return (4 * ((raw_val * 1000) + 500)) / 2048;
+}
+
+static inline int da9150_gpadc_vbus_21v_voltage_now(int raw_val)
+{
+ /* Convert to mV */
+ return (21 * ((raw_val * 1000) + 500)) / 1024;
+}
+
+static inline int da9150_gpadc_vsys_6v_voltage_now(int raw_val)
+{
+ /* Convert to mV */
+ return (3 * ((raw_val * 1000) + 500)) / 512;
+}
+
+static int da9150_gpadc_read_processed(struct da9150_gpadc *gpadc, int channel,
+ int hw_chan, int *val)
+{
+ int raw_val;
+
+ raw_val = da9150_gpadc_read_adc(gpadc, hw_chan);
+ if (raw_val < 0)
+ return raw_val;
+
+ switch (channel) {
+ case DA9150_GPADC_CHAN_GPIOA:
+ case DA9150_GPADC_CHAN_GPIOB:
+ case DA9150_GPADC_CHAN_GPIOC:
+ case DA9150_GPADC_CHAN_GPIOD:
+ *val = da9150_gpadc_gpio_6v_voltage_now(raw_val);
+ break;
+ case DA9150_GPADC_CHAN_IBUS:
+ *val = da9150_gpadc_ibus_current_avg(raw_val);
+ break;
+ case DA9150_GPADC_CHAN_VBUS:
+ *val = da9150_gpadc_vbus_21v_voltage_now(raw_val);
+ break;
+ case DA9150_GPADC_CHAN_VSYS:
+ *val = da9150_gpadc_vsys_6v_voltage_now(raw_val);
+ break;
+ default:
+ /* No processing for other channels so return raw value */
+ *val = raw_val;
+ break;
+ }
+
+ return IIO_VAL_INT;
+}
+
+static int da9150_gpadc_read_scale(int channel, int *val, int *val2)
+{
+ switch (channel) {
+ case DA9150_GPADC_CHAN_VBAT:
+ *val = 2932;
+ *val2 = 1000;
+ return IIO_VAL_FRACTIONAL;
+ case DA9150_GPADC_CHAN_TJUNC_CORE:
+ case DA9150_GPADC_CHAN_TJUNC_OVP:
+ *val = 1000000;
+ *val2 = 4420;
+ return IIO_VAL_FRACTIONAL;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int da9150_gpadc_read_offset(int channel, int *val)
+{
+ switch (channel) {
+ case DA9150_GPADC_CHAN_VBAT:
+ *val = 1500000 / 2932;
+ return IIO_VAL_INT;
+ case DA9150_GPADC_CHAN_TJUNC_CORE:
+ case DA9150_GPADC_CHAN_TJUNC_OVP:
+ *val = -144;
+ return IIO_VAL_INT;
+ default:
+ return -EINVAL;
+ }
+}
+
+static int da9150_gpadc_read_raw(struct iio_dev *indio_dev,
+ struct iio_chan_spec const *chan,
+ int *val, int *val2, long mask)
+{
+ struct da9150_gpadc *gpadc = iio_priv(indio_dev);
+
+ if ((chan->channel < DA9150_GPADC_CHAN_GPIOA) ||
+ (chan->channel > DA9150_GPADC_CHAN_TJUNC_OVP))
+ return -EINVAL;
+
+ switch (mask) {
+ case IIO_CHAN_INFO_RAW:
+ case IIO_CHAN_INFO_PROCESSED:
+ return da9150_gpadc_read_processed(gpadc, chan->channel,
+ chan->address, val);
+ case IIO_CHAN_INFO_SCALE:
+ return da9150_gpadc_read_scale(chan->channel, val, val2);
+ case IIO_CHAN_INFO_OFFSET:
+ return da9150_gpadc_read_offset(chan->channel, val);
+ default:
+ return -EINVAL;
+ }
+}
+
+static const struct iio_info da9150_gpadc_info = {
+ .read_raw = &da9150_gpadc_read_raw,
+ .driver_module = THIS_MODULE,
+};
+
+#define DA9150_GPADC_CHANNEL(_id, _hw_id, _type, chan_info, \
+ _ext_name) { \
+ .type = _type, \
+ .indexed = 1, \
+ .channel = DA9150_GPADC_CHAN_##_id, \
+ .address = DA9150_GPADC_HW_CHAN_##_hw_id, \
+ .info_mask_separate = chan_info, \
+ .extend_name = _ext_name, \
+ .datasheet_name = #_id, \
+}
+
+#define DA9150_GPADC_CHANNEL_RAW(_id, _hw_id, _type, _ext_name) \
+ DA9150_GPADC_CHANNEL(_id, _hw_id, _type, \
+ BIT(IIO_CHAN_INFO_RAW), _ext_name)
+
+#define DA9150_GPADC_CHANNEL_SCALED(_id, _hw_id, _type, _ext_name) \
+ DA9150_GPADC_CHANNEL(_id, _hw_id, _type, \
+ BIT(IIO_CHAN_INFO_RAW) | \
+ BIT(IIO_CHAN_INFO_SCALE) | \
+ BIT(IIO_CHAN_INFO_OFFSET), \
+ _ext_name)
+
+#define DA9150_GPADC_CHANNEL_PROCESSED(_id, _hw_id, _type, _ext_name) \
+ DA9150_GPADC_CHANNEL(_id, _hw_id, _type, \
+ BIT(IIO_CHAN_INFO_PROCESSED), _ext_name)
+
+/* Supported channels */
+static const struct iio_chan_spec da9150_gpadc_channels[] = {
+ DA9150_GPADC_CHANNEL_PROCESSED(GPIOA, GPIOA_6V, IIO_VOLTAGE, NULL),
+ DA9150_GPADC_CHANNEL_PROCESSED(GPIOB, GPIOB_6V, IIO_VOLTAGE, NULL),
+ DA9150_GPADC_CHANNEL_PROCESSED(GPIOC, GPIOC_6V, IIO_VOLTAGE, NULL),
+ DA9150_GPADC_CHANNEL_PROCESSED(GPIOD, GPIOD_6V, IIO_VOLTAGE, NULL),
+ DA9150_GPADC_CHANNEL_PROCESSED(IBUS, IBUS_SENSE, IIO_CURRENT, "ibus"),
+ DA9150_GPADC_CHANNEL_PROCESSED(VBUS, VBUS_DIV_, IIO_VOLTAGE, "vbus"),
+ DA9150_GPADC_CHANNEL_PROCESSED(VSYS, VSYS, IIO_VOLTAGE, "vsys"),
+ DA9150_GPADC_CHANNEL_SCALED(VBAT, VBAT, IIO_VOLTAGE, "vbat"),
+ DA9150_GPADC_CHANNEL_RAW(TBAT, TBAT, IIO_VOLTAGE, "tbat"),
+ DA9150_GPADC_CHANNEL_SCALED(TJUNC_CORE, TJUNC_CORE, IIO_TEMP,
+ "tjunc_core"),
+ DA9150_GPADC_CHANNEL_SCALED(TJUNC_OVP, TJUNC_OVP, IIO_TEMP,
+ "tjunc_ovp"),
+};
+
+/* Default maps used by da9150-charger */
+static struct iio_map da9150_gpadc_default_maps[] = {
+ {
+ .consumer_dev_name = "da9150-charger",
+ .consumer_channel = "CHAN_IBUS",
+ .adc_channel_label = "IBUS",
+ },
+ {
+ .consumer_dev_name = "da9150-charger",
+ .consumer_channel = "CHAN_VBUS",
+ .adc_channel_label = "VBUS",
+ },
+ {
+ .consumer_dev_name = "da9150-charger",
+ .consumer_channel = "CHAN_TJUNC",
+ .adc_channel_label = "TJUNC_CORE",
+ },
+ {
+ .consumer_dev_name = "da9150-charger",
+ .consumer_channel = "CHAN_VBAT",
+ .adc_channel_label = "VBAT",
+ },
+ {},
+};
+
+static int da9150_gpadc_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct da9150 *da9150 = dev_get_drvdata(dev->parent);
+ struct da9150_gpadc *gpadc;
+ struct iio_dev *indio_dev;
+ int irq, ret;
+
+ indio_dev = devm_iio_device_alloc(dev, sizeof(*gpadc));
+ if (!indio_dev) {
+ dev_err(&pdev->dev, "Failed to allocate IIO device\n");
+ return -ENOMEM;
+ }
+ gpadc = iio_priv(indio_dev);
+
+ platform_set_drvdata(pdev, indio_dev);
+ gpadc->da9150 = da9150;
+ gpadc->dev = dev;
+ mutex_init(&gpadc->lock);
+ init_completion(&gpadc->complete);
+
+ irq = platform_get_irq_byname(pdev, "GPADC");
+ if (irq < 0) {
+ dev_err(dev, "Failed to get IRQ: %d\n", irq);
+ return irq;
+ }
+
+ ret = devm_request_threaded_irq(dev, irq, NULL, da9150_gpadc_irq,
+ IRQF_ONESHOT, "GPADC", gpadc);
+ if (ret) {
+ dev_err(dev, "Failed to request IRQ %d: %d\n", irq, ret);
+ return ret;
+ }
+
+ ret = iio_map_array_register(indio_dev, da9150_gpadc_default_maps);
+ if (ret) {
+ dev_err(dev, "Failed to register IIO maps: %d\n", ret);
+ return ret;
+ }
+
+ indio_dev->name = dev_name(dev);
+ indio_dev->dev.parent = dev;
+ indio_dev->dev.of_node = pdev->dev.of_node;
+ indio_dev->info = &da9150_gpadc_info;
+ indio_dev->modes = INDIO_DIRECT_MODE;
+ indio_dev->channels = da9150_gpadc_channels;
+ indio_dev->num_channels = ARRAY_SIZE(da9150_gpadc_channels);
+
+ ret = iio_device_register(indio_dev);
+ if (ret) {
+ dev_err(dev, "Failed to register IIO device: %d\n", ret);
+ goto iio_map_unreg;
+ }
+
+ return 0;
+
+iio_map_unreg:
+ iio_map_array_unregister(indio_dev);
+
+ return ret;
+}
+
+static int da9150_gpadc_remove(struct platform_device *pdev)
+{
+ struct iio_dev *indio_dev = platform_get_drvdata(pdev);
+
+ iio_device_unregister(indio_dev);
+ iio_map_array_unregister(indio_dev);
+
+ return 0;
+}
+
+static struct platform_driver da9150_gpadc_driver = {
+ .driver = {
+ .name = "da9150-gpadc",
+ },
+ .probe = da9150_gpadc_probe,
+ .remove = da9150_gpadc_remove,
+};
+
+module_platform_driver(da9150_gpadc_driver);
+
+MODULE_DESCRIPTION("GPADC Driver for DA9150");
+MODULE_AUTHOR("Adam Thomson <Adam.Thomson.Opensource@diasemi.com>");
+MODULE_LICENSE("GPL");
.info_mask_shared_by_type = BIT(IIO_CHAN_INFO_SAMP_FREQ), \
}
-/* LSB is in nV to eliminate floating point */
-static const u32 rates_to_lsb[] = {1000000, 250000, 62500, 15625};
-
-/*
- * scales calculated as:
- * rates_to_lsb[sample_rate] / (1 << pga);
- * pga is 1 for 0, 2
- */
-
static const int mcp3422_scales[4][4] = {
- { 1000000, 250000, 62500, 15625 },
- { 500000 , 125000, 31250, 7812 },
- { 250000 , 62500 , 15625, 3906 },
- { 125000 , 31250 , 7812 , 1953 } };
+ { 1000000, 500000, 250000, 125000 },
+ { 250000 , 125000, 62500 , 31250 },
+ { 62500 , 31250 , 15625 , 7812 },
+ { 15625 , 7812 , 3906 , 1953 } };
/* Constant msleep times for data acquisitions */
static const int mcp3422_read_times[4] = {
if (iadc->poll_eoc) {
ret = iadc_poll_wait_eoc(iadc, wait);
} else {
- ret = wait_for_completion_timeout(&iadc->complete, wait);
+ ret = wait_for_completion_timeout(&iadc->complete,
+ usecs_to_jiffies(wait));
if (!ret)
ret = -ETIMEDOUT;
else
static int tiadc_buffer_postenable(struct iio_dev *indio_dev)
{
struct tiadc_device *adc_dev = iio_priv(indio_dev);
- struct iio_buffer *buffer = indio_dev->buffer;
unsigned int enb = 0;
u8 bit;
tiadc_step_config(indio_dev);
- for_each_set_bit(bit, buffer->scan_mask, adc_dev->channels)
+ for_each_set_bit(bit, indio_dev->active_scan_mask, adc_dev->channels)
enb |= (get_adc_step_bit(adc_dev, bit) << 1);
adc_dev->buffer_en_ch_steps = enb;
struct regulator *vref;
struct vf610_adc_feature adc_feature;
+ u32 sample_freq_avail[5];
+
struct completion completion;
};
+static const u32 vf610_hw_avgs[] = { 1, 4, 8, 16, 32 };
+
#define VF610_ADC_CHAN(_idx, _chan_type) { \
.type = (_chan_type), \
.indexed = 1, \
/* sentinel */
};
-/*
- * ADC sample frequency, unit is ADCK cycles.
- * ADC clk source is ipg clock, which is the same as bus clock.
- *
- * ADC conversion time = SFCAdder + AverageNum x (BCT + LSTAdder)
- * SFCAdder: fixed to 6 ADCK cycles
- * AverageNum: 1, 4, 8, 16, 32 samples for hardware average.
- * BCT (Base Conversion Time): fixed to 25 ADCK cycles for 12 bit mode
- * LSTAdder(Long Sample Time): fixed to 3 ADCK cycles
- *
- * By default, enable 12 bit resolution mode, clock source
- * set to ipg clock, So get below frequency group:
- */
-static const u32 vf610_sample_freq_avail[5] =
-{1941176, 559332, 286957, 145374, 73171};
+static inline void vf610_adc_calculate_rates(struct vf610_adc *info)
+{
+ unsigned long adck_rate, ipg_rate = clk_get_rate(info->clk);
+ int i;
+
+ /*
+ * Calculate ADC sample frequencies
+ * Sample time unit is ADCK cycles. ADCK clk source is ipg clock,
+ * which is the same as bus clock.
+ *
+ * ADC conversion time = SFCAdder + AverageNum x (BCT + LSTAdder)
+ * SFCAdder: fixed to 6 ADCK cycles
+ * AverageNum: 1, 4, 8, 16, 32 samples for hardware average.
+ * BCT (Base Conversion Time): fixed to 25 ADCK cycles for 12 bit mode
+ * LSTAdder(Long Sample Time): fixed to 3 ADCK cycles
+ */
+ adck_rate = ipg_rate / info->adc_feature.clk_div;
+ for (i = 0; i < ARRAY_SIZE(vf610_hw_avgs); i++)
+ info->sample_freq_avail[i] =
+ adck_rate / (6 + vf610_hw_avgs[i] * (25 + 3));
+}
static inline void vf610_adc_cfg_init(struct vf610_adc *info)
{
+ struct vf610_adc_feature *adc_feature = &info->adc_feature;
+
/* set default Configuration for ADC controller */
- info->adc_feature.clk_sel = VF610_ADCIOC_BUSCLK_SET;
- info->adc_feature.vol_ref = VF610_ADCIOC_VR_VREF_SET;
+ adc_feature->clk_sel = VF610_ADCIOC_BUSCLK_SET;
+ adc_feature->vol_ref = VF610_ADCIOC_VR_VREF_SET;
+
+ adc_feature->calibration = true;
+ adc_feature->ovwren = true;
+
+ adc_feature->res_mode = 12;
+ adc_feature->sample_rate = 1;
+ adc_feature->lpm = true;
- info->adc_feature.calibration = true;
- info->adc_feature.ovwren = true;
+ /* Use a save ADCK which is below 20MHz on all devices */
+ adc_feature->clk_div = 8;
- info->adc_feature.clk_div = 1;
- info->adc_feature.res_mode = 12;
- info->adc_feature.sample_rate = 1;
- info->adc_feature.lpm = true;
+ vf610_adc_calculate_rates(info);
}
static void vf610_adc_cfg_post_set(struct vf610_adc *info)
cfg_data = readl(info->regs + VF610_REG_ADC_CFG);
- /* low power configuration */
cfg_data &= ~VF610_ADC_ADLPC_EN;
if (adc_feature->lpm)
cfg_data |= VF610_ADC_ADLPC_EN;
- /* disable high speed */
cfg_data &= ~VF610_ADC_ADHSC_EN;
writel(cfg_data, info->regs + VF610_REG_ADC_CFG);
return IRQ_HANDLED;
}
-static IIO_CONST_ATTR_SAMP_FREQ_AVAIL("1941176, 559332, 286957, 145374, 73171");
+static ssize_t vf610_show_samp_freq_avail(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct vf610_adc *info = iio_priv(dev_to_iio_dev(dev));
+ size_t len = 0;
+ int i;
+
+ for (i = 0; i < ARRAY_SIZE(info->sample_freq_avail); i++)
+ len += scnprintf(buf + len, PAGE_SIZE - len,
+ "%u ", info->sample_freq_avail[i]);
+
+ /* replace trailing space by newline */
+ buf[len - 1] = '\n';
+
+ return len;
+}
+
+static IIO_DEV_ATTR_SAMP_FREQ_AVAIL(vf610_show_samp_freq_avail);
static struct attribute *vf610_attributes[] = {
- &iio_const_attr_sampling_frequency_available.dev_attr.attr,
+ &iio_dev_attr_sampling_frequency_available.dev_attr.attr,
NULL
};
return IIO_VAL_FRACTIONAL_LOG2;
case IIO_CHAN_INFO_SAMP_FREQ:
- *val = vf610_sample_freq_avail[info->adc_feature.sample_rate];
+ *val = info->sample_freq_avail[info->adc_feature.sample_rate];
*val2 = 0;
return IIO_VAL_INT;
switch (mask) {
case IIO_CHAN_INFO_SAMP_FREQ:
for (i = 0;
- i < ARRAY_SIZE(vf610_sample_freq_avail);
+ i < ARRAY_SIZE(info->sample_freq_avail);
i++)
- if (val == vf610_sample_freq_avail[i]) {
+ if (val == info->sample_freq_avail[i]) {
info->adc_feature.sample_rate = i;
vf610_adc_sample_set(info);
return 0;
return 0;
}
+#ifdef CONFIG_PM_SLEEP
static int ssp_suspend(struct device *dev)
{
int ret;
return 0;
}
+#endif /* CONFIG_PM_SLEEP */
static const struct dev_pm_ops ssp_pm_ops = {
SET_SYSTEM_SLEEP_PM_OPS(ssp_suspend, ssp_resume)
st = iio_priv(indio_dev);
spi_set_drvdata(spi, indio_dev);
- st->reg = devm_regulator_get(&spi->dev, "vcc");
+ st->reg = devm_regulator_get_optional(&spi->dev, "vcc");
if (!IS_ERR(st->reg)) {
ret = regulator_enable(st->reg);
if (ret)
int bit, ret, i = 0;
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = i2c_smbus_read_word_data(data->client,
BMG160_AXIS_TO_REG(bit));
#include <linux/wait.h>
#include <linux/bitops.h>
#include <linux/completion.h>
+#include <linux/mutex.h>
#include <linux/delay.h>
#include <linux/gpio.h>
#include <linux/of_gpio.h>
#define DHT11_DATA_VALID_TIME 2000000000 /* 2s in ns */
-#define DHT11_EDGES_PREAMBLE 4
+#define DHT11_EDGES_PREAMBLE 2
#define DHT11_BITS_PER_READ 40
+/*
+ * Note that when reading the sensor actually 84 edges are detected, but
+ * since the last edge is not significant, we only store 83:
+ */
#define DHT11_EDGES_PER_READ (2*DHT11_BITS_PER_READ + DHT11_EDGES_PREAMBLE + 1)
/* Data transmission timing (nano seconds) */
int irq;
struct completion completion;
+ struct mutex lock;
s64 timestamp;
int temperature;
unsigned char temp_int, temp_dec, hum_int, hum_dec, checksum;
/* Calculate timestamp resolution */
- for (i = 0; i < dht11->num_edges; ++i) {
+ for (i = 1; i < dht11->num_edges; ++i) {
t = dht11->edges[i].ts - dht11->edges[i-1].ts;
if (t > 0 && t < timeres)
timeres = t;
return 0;
}
+/*
+ * IRQ handler called on GPIO edges
+ */
+static irqreturn_t dht11_handle_irq(int irq, void *data)
+{
+ struct iio_dev *iio = data;
+ struct dht11 *dht11 = iio_priv(iio);
+
+ /* TODO: Consider making the handler safe for IRQ sharing */
+ if (dht11->num_edges < DHT11_EDGES_PER_READ && dht11->num_edges >= 0) {
+ dht11->edges[dht11->num_edges].ts = iio_get_time_ns();
+ dht11->edges[dht11->num_edges++].value =
+ gpio_get_value(dht11->gpio);
+
+ if (dht11->num_edges >= DHT11_EDGES_PER_READ)
+ complete(&dht11->completion);
+ }
+
+ return IRQ_HANDLED;
+}
+
static int dht11_read_raw(struct iio_dev *iio_dev,
const struct iio_chan_spec *chan,
int *val, int *val2, long m)
struct dht11 *dht11 = iio_priv(iio_dev);
int ret;
+ mutex_lock(&dht11->lock);
if (dht11->timestamp + DHT11_DATA_VALID_TIME < iio_get_time_ns()) {
reinit_completion(&dht11->completion);
if (ret)
goto err;
+ ret = request_irq(dht11->irq, dht11_handle_irq,
+ IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING,
+ iio_dev->name, iio_dev);
+ if (ret)
+ goto err;
+
ret = wait_for_completion_killable_timeout(&dht11->completion,
HZ);
+
+ free_irq(dht11->irq, iio_dev);
+
if (ret == 0 && dht11->num_edges < DHT11_EDGES_PER_READ - 1) {
dev_err(&iio_dev->dev,
"Only %d signal edges detected\n",
ret = -EINVAL;
err:
dht11->num_edges = -1;
+ mutex_unlock(&dht11->lock);
return ret;
}
.read_raw = dht11_read_raw,
};
-/*
- * IRQ handler called on GPIO edges
-*/
-static irqreturn_t dht11_handle_irq(int irq, void *data)
-{
- struct iio_dev *iio = data;
- struct dht11 *dht11 = iio_priv(iio);
-
- /* TODO: Consider making the handler safe for IRQ sharing */
- if (dht11->num_edges < DHT11_EDGES_PER_READ && dht11->num_edges >= 0) {
- dht11->edges[dht11->num_edges].ts = iio_get_time_ns();
- dht11->edges[dht11->num_edges++].value =
- gpio_get_value(dht11->gpio);
-
- if (dht11->num_edges >= DHT11_EDGES_PER_READ)
- complete(&dht11->completion);
- }
-
- return IRQ_HANDLED;
-}
-
static const struct iio_chan_spec dht11_chan_spec[] = {
{ .type = IIO_TEMP,
.info_mask_separate = BIT(IIO_CHAN_INFO_PROCESSED), },
dev_err(dev, "GPIO %d has no interrupt\n", dht11->gpio);
return -EINVAL;
}
- ret = devm_request_irq(dev, dht11->irq, dht11_handle_irq,
- IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING,
- pdev->name, iio);
- if (ret)
- return ret;
dht11->timestamp = iio_get_time_ns() - DHT11_DATA_VALID_TIME - 1;
dht11->num_edges = -1;
platform_set_drvdata(pdev, iio);
init_completion(&dht11->completion);
+ mutex_init(&dht11->lock);
iio->name = pdev->name;
iio->dev.parent = &pdev->dev;
iio->info = &dht11_iio_info;
struct iio_chan_spec const *chan, int *val,
int *val2, long mask)
{
- struct i2c_client *client = iio_priv(indio_dev);
+ struct i2c_client **client = iio_priv(indio_dev);
int ret;
switch (mask) {
case IIO_CHAN_INFO_RAW:
- ret = i2c_smbus_read_word_data(client,
+ ret = i2c_smbus_read_word_data(*client,
chan->type == IIO_TEMP ?
SI7020CMD_TEMP_HOLD :
SI7020CMD_RH_HOLD);
/* Wait the maximum power-up time after software reset. */
msleep(15);
- indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*client));
+ indio_dev = devm_iio_device_alloc(&client->dev, sizeof(*data));
if (!indio_dev)
return -ENOMEM;
#include <linux/list.h>
#include <linux/module.h>
#include <linux/debugfs.h>
+#include <linux/bitops.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
mutex_unlock(&indio_dev->mlock);
if (ret)
return ret;
- val16 = ((val16 & 0xFFF) << 4) >> 4;
+ val16 = sign_extend32(val16, 11);
*val = val16;
return IIO_VAL_INT;
case IIO_CHAN_INFO_OFFSET:
iio_trigger_set_drvdata(adis->trig, adis);
ret = iio_trigger_register(adis->trig);
- indio_dev->trig = adis->trig;
+ indio_dev->trig = iio_trigger_get(adis->trig);
if (ret)
goto error_free_irq;
}
}
-static int inv_mpu6050_write_fsr(struct inv_mpu6050_state *st, int fsr)
+static int inv_mpu6050_write_gyro_scale(struct inv_mpu6050_state *st, int val)
{
- int result;
+ int result, i;
u8 d;
- if (fsr < 0 || fsr > INV_MPU6050_MAX_GYRO_FS_PARAM)
- return -EINVAL;
- if (fsr == st->chip_config.fsr)
- return 0;
+ for (i = 0; i < ARRAY_SIZE(gyro_scale_6050); ++i) {
+ if (gyro_scale_6050[i] == val) {
+ d = (i << INV_MPU6050_GYRO_CONFIG_FSR_SHIFT);
+ result = inv_mpu6050_write_reg(st,
+ st->reg->gyro_config, d);
+ if (result)
+ return result;
- d = (fsr << INV_MPU6050_GYRO_CONFIG_FSR_SHIFT);
- result = inv_mpu6050_write_reg(st, st->reg->gyro_config, d);
- if (result)
- return result;
- st->chip_config.fsr = fsr;
+ st->chip_config.fsr = i;
+ return 0;
+ }
+ }
- return 0;
+ return -EINVAL;
}
-static int inv_mpu6050_write_accel_fs(struct inv_mpu6050_state *st, int fs)
+static int inv_mpu6050_write_accel_scale(struct inv_mpu6050_state *st, int val)
{
- int result;
+ int result, i;
u8 d;
- if (fs < 0 || fs > INV_MPU6050_MAX_ACCL_FS_PARAM)
- return -EINVAL;
- if (fs == st->chip_config.accl_fs)
- return 0;
+ for (i = 0; i < ARRAY_SIZE(accel_scale); ++i) {
+ if (accel_scale[i] == val) {
+ d = (i << INV_MPU6050_ACCL_CONFIG_FSR_SHIFT);
+ result = inv_mpu6050_write_reg(st,
+ st->reg->accl_config, d);
+ if (result)
+ return result;
- d = (fs << INV_MPU6050_ACCL_CONFIG_FSR_SHIFT);
- result = inv_mpu6050_write_reg(st, st->reg->accl_config, d);
- if (result)
- return result;
- st->chip_config.accl_fs = fs;
+ st->chip_config.accl_fs = i;
+ return 0;
+ }
+ }
- return 0;
+ return -EINVAL;
}
static int inv_mpu6050_write_raw(struct iio_dev *indio_dev,
case IIO_CHAN_INFO_SCALE:
switch (chan->type) {
case IIO_ANGL_VEL:
- result = inv_mpu6050_write_fsr(st, val);
+ result = inv_mpu6050_write_gyro_scale(st, val2);
break;
case IIO_ACCEL:
- result = inv_mpu6050_write_accel_fs(st, val);
+ result = inv_mpu6050_write_accel_scale(st, val2);
break;
default:
result = -EINVAL;
i2c_set_clientdata(client, indio_dev);
indio_dev->dev.parent = &client->dev;
- indio_dev->name = id->name;
+ /* id will be NULL when enumerated via ACPI */
+ if (id)
+ indio_dev->name = (char *)id->name;
+ else
+ indio_dev->name = (char *)dev_name(&client->dev);
indio_dev->channels = inv_mpu_channels;
indio_dev->num_channels = ARRAY_SIZE(inv_mpu_channels);
#include <linux/poll.h>
#include "inv_mpu_iio.h"
+static void inv_clear_kfifo(struct inv_mpu6050_state *st)
+{
+ unsigned long flags;
+
+ /* take the spin lock sem to avoid interrupt kick in */
+ spin_lock_irqsave(&st->time_stamp_lock, flags);
+ kfifo_reset(&st->timestamps);
+ spin_unlock_irqrestore(&st->time_stamp_lock, flags);
+}
+
int inv_reset_fifo(struct iio_dev *indio_dev)
{
int result;
INV_MPU6050_BIT_FIFO_RST);
if (result)
goto reset_fifo_fail;
+
+ /* clear timestamps fifo */
+ inv_clear_kfifo(st);
+
/* enable interrupt */
if (st->chip_config.accl_fifo_enable ||
st->chip_config.gyro_fifo_enable) {
return result;
}
-static void inv_clear_kfifo(struct inv_mpu6050_state *st)
-{
- unsigned long flags;
-
- /* take the spin lock sem to avoid interrupt kick in */
- spin_lock_irqsave(&st->time_stamp_lock, flags);
- kfifo_reset(&st->timestamps);
- spin_unlock_irqrestore(&st->time_stamp_lock, flags);
-}
-
/**
* inv_mpu6050_irq_handler() - Cache a timestamp at each data ready interrupt.
*/
flush_fifo:
/* Flush HW and SW FIFOs. */
inv_reset_fifo(indio_dev);
- inv_clear_kfifo(st);
mutex_unlock(&indio_dev->mlock);
iio_trigger_notify_done(indio_dev->trig);
base = KMX61_MAG_XOUT_L;
mutex_lock(&data->lock);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = kmx61_read_measurement(data, base, bit);
if (ret < 0) {
* @attr_list: List of IIO device attributes
*
* This function frees the memory allocated for each of the IIO device
- * attributes in the list. Note: if you want to reuse the list after calling
- * this function you have to reinitialize it using INIT_LIST_HEAD().
+ * attributes in the list.
*/
void iio_free_chan_devattr_list(struct list_head *attr_list)
{
list_for_each_entry_safe(p, n, attr_list, l) {
kfree(p->dev_attr.attr.name);
+ list_del(&p->l);
kfree(p);
}
}
iio_free_chan_devattr_list(&indio_dev->channel_attr_list);
kfree(indio_dev->chan_attr_group.attrs);
+ indio_dev->chan_attr_group.attrs = NULL;
}
static void iio_dev_release(struct device *device)
error_free_setup_event_lines:
iio_free_chan_devattr_list(&indio_dev->event_interface->dev_attr_list);
kfree(indio_dev->event_interface);
+ indio_dev->event_interface = NULL;
return ret;
}
config GP2AP020A00F
tristate "Sharp GP2AP020A00F Proximity/ALS sensor"
depends on I2C
+ select REGMAP_I2C
select IIO_BUFFER
select IIO_TRIGGERED_BUFFER
select IRQ_WORK
config JSA1212
tristate "JSA1212 ALS and proximity sensor driver"
depends on I2C
+ select REGMAP_I2C
help
Say Y here if you want to build a IIO driver for JSA1212
proximity & ALS sensor device.
config AK09911
tristate "Asahi Kasei AK09911 3-axis Compass"
+ depends on I2C
+ depends on GPIOLIB
select AK8975
help
Deprecated: AK09911 is now supported by AK8975 driver.
mutex_lock(&data->mutex);
- for_each_set_bit(bit, indio_dev->buffer->scan_mask,
+ for_each_set_bit(bit, indio_dev->active_scan_mask,
indio_dev->masklength) {
ret = sx9500_read_proximity(data, &indio_dev->channels[bit],
&val);
if (dmasync)
dma_set_attr(DMA_ATTR_WRITE_BARRIER, &attrs);
+ /*
+ * If the combination of the addr and size requested for this memory
+ * region causes an integer overflow, return error.
+ */
+ if ((PAGE_ALIGN(addr + size) <= size) ||
+ (PAGE_ALIGN(addr + size) <= addr))
+ return ERR_PTR(-EINVAL);
+
if (!can_do_mlock())
return ERR_PTR(-EPERM);
#define GUID_TBL_BLK_NUM_ENTRIES 8
#define GUID_TBL_BLK_SIZE (GUID_TBL_ENTRY_SIZE * GUID_TBL_BLK_NUM_ENTRIES)
+/* Counters should be saturate once they reach their maximum value */
+#define ASSIGN_32BIT_COUNTER(counter, value) do {\
+ if ((value) > U32_MAX) \
+ counter = cpu_to_be32(U32_MAX); \
+ else \
+ counter = cpu_to_be32(value); \
+} while (0)
+
struct mlx4_mad_rcv_buf {
struct ib_grh grh;
u8 payload[256];
static void edit_counter(struct mlx4_counter *cnt,
struct ib_pma_portcounters *pma_cnt)
{
- pma_cnt->port_xmit_data = cpu_to_be32((be64_to_cpu(cnt->tx_bytes)>>2));
- pma_cnt->port_rcv_data = cpu_to_be32((be64_to_cpu(cnt->rx_bytes)>>2));
- pma_cnt->port_xmit_packets = cpu_to_be32(be64_to_cpu(cnt->tx_frames));
- pma_cnt->port_rcv_packets = cpu_to_be32(be64_to_cpu(cnt->rx_frames));
+ ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_data,
+ (be64_to_cpu(cnt->tx_bytes) >> 2));
+ ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_data,
+ (be64_to_cpu(cnt->rx_bytes) >> 2));
+ ASSIGN_32BIT_COUNTER(pma_cnt->port_xmit_packets,
+ be64_to_cpu(cnt->tx_frames));
+ ASSIGN_32BIT_COUNTER(pma_cnt->port_rcv_packets,
+ be64_to_cpu(cnt->rx_frames));
}
static int iboe_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
spin_lock_bh(&ibdev->iboe.lock);
for (i = 0; i < MLX4_MAX_PORTS; ++i) {
struct net_device *curr_netdev = ibdev->iboe.netdevs[i];
+ enum ib_port_state curr_port_state;
- enum ib_port_state curr_port_state =
+ if (!curr_netdev)
+ continue;
+
+ curr_port_state =
(netif_running(curr_netdev) &&
netif_carrier_ok(curr_netdev)) ?
IB_PORT_ACTIVE : IB_PORT_DOWN;
input_set_drvdata(input, keypad);
- error = request_threaded_irq(irq, NULL,
- tc3589x_keypad_irq, plat->irqtype,
- "tc3589x-keypad", keypad);
+ error = request_threaded_irq(irq, NULL, tc3589x_keypad_irq,
+ plat->irqtype | IRQF_ONESHOT,
+ "tc3589x-keypad", keypad);
if (error < 0) {
dev_err(&pdev->dev,
"Could not allocate irq %d,error %d\n",
idev->private = m;
idev->input->name = MMA8450_DRV_NAME;
idev->input->id.bustype = BUS_I2C;
+ idev->input->dev.parent = &c->dev;
idev->poll = mma8450_poll;
idev->poll_interval = POLL_INTERVAL;
idev->poll_interval_max = POLL_INTERVAL_MAX;
mutex_unlock(&alps_mutex);
}
-static void alps_report_bare_ps2_packet(struct input_dev *dev,
+static void alps_report_bare_ps2_packet(struct psmouse *psmouse,
unsigned char packet[],
bool report_buttons)
{
+ struct alps_data *priv = psmouse->private;
+ struct input_dev *dev;
+
+ /* Figure out which device to use to report the bare packet */
+ if (priv->proto_version == ALPS_PROTO_V2 &&
+ (priv->flags & ALPS_DUALPOINT)) {
+ /* On V2 devices the DualPoint Stick reports bare packets */
+ dev = priv->dev2;
+ } else if (unlikely(IS_ERR_OR_NULL(priv->dev3))) {
+ /* Register dev3 mouse if we received PS/2 packet first time */
+ if (!IS_ERR(priv->dev3))
+ psmouse_queue_work(psmouse, &priv->dev3_register_work,
+ 0);
+ return;
+ } else {
+ dev = priv->dev3;
+ }
+
if (report_buttons)
alps_report_buttons(dev, NULL,
packet[0] & 1, packet[0] & 2, packet[0] & 4);
* de-synchronization.
*/
- alps_report_bare_ps2_packet(priv->dev2,
- &psmouse->packet[3], false);
+ alps_report_bare_ps2_packet(psmouse, &psmouse->packet[3],
+ false);
/*
* Continue with the standard ALPS protocol handling,
* properly we only do this if the device is fully synchronized.
*/
if (!psmouse->out_of_sync_cnt && (psmouse->packet[0] & 0xc8) == 0x08) {
-
- /* Register dev3 mouse if we received PS/2 packet first time */
- if (unlikely(!priv->dev3))
- psmouse_queue_work(psmouse,
- &priv->dev3_register_work, 0);
-
if (psmouse->pktcnt == 3) {
- /* Once dev3 mouse device is registered report data */
- if (likely(!IS_ERR_OR_NULL(priv->dev3)))
- alps_report_bare_ps2_packet(priv->dev3,
- psmouse->packet,
- true);
+ alps_report_bare_ps2_packet(psmouse, psmouse->packet,
+ true);
return PSMOUSE_FULL_PACKET;
}
return PSMOUSE_GOOD_DATA;
priv->set_abs_params = alps_set_abs_params_mt;
priv->nibble_commands = alps_v3_nibble_commands;
priv->addr_command = PSMOUSE_CMD_RESET_WRAP;
- priv->x_max = 1360;
- priv->y_max = 660;
priv->x_bits = 23;
priv->y_bits = 12;
+
+ if (alps_dolphin_get_device_area(psmouse, priv))
+ return -EIO;
+
break;
case ALPS_PROTO_V6:
priv->set_abs_params = alps_set_abs_params_mt;
priv->nibble_commands = alps_v3_nibble_commands;
priv->addr_command = PSMOUSE_CMD_RESET_WRAP;
-
- if (alps_dolphin_get_device_area(psmouse, priv))
- return -EIO;
+ priv->x_max = 0xfff;
+ priv->y_max = 0x7ff;
if (priv->fw_ver[1] != 0xba)
priv->flags |= ALPS_BUTTONPAD;
return -ENOMEM;
error = alps_identify(psmouse, priv);
- if (error)
+ if (error) {
+ kfree(priv);
return error;
+ }
if (set_properties) {
psmouse->vendor = "ALPS";
#include <linux/input/mt.h>
#include <linux/module.h>
#include <linux/slab.h>
-#include <linux/unaligned/access_ok.h>
+#include <asm/unaligned.h>
#include "cyapa.h"
#include <linux/mutex.h>
#include <linux/completion.h>
#include <linux/slab.h>
-#include <linux/unaligned/access_ok.h>
+#include <asm/unaligned.h>
#include <linux/crc-itu-t.h>
#include "cyapa.h"
electrodes_tx = cyapa->electrodes_x;
max_element_cnt = ((cyapa->aligned_electrodes_rx + 7) &
~7u) * electrodes_tx;
- } else if (idac_data_type == GEN5_RETRIEVE_SELF_CAP_PWC_DATA) {
+ } else {
offset = 2;
max_element_cnt = cyapa->electrodes_x +
cyapa->electrodes_y;
#define FOC_MAX_FINGERS 5
-#define FOC_MAX_X 2431
-#define FOC_MAX_Y 1663
-
/*
* Current state of a single finger on the touchpad.
*/
input_mt_slot(dev, i);
input_mt_report_slot_state(dev, MT_TOOL_FINGER, active);
if (active) {
- input_report_abs(dev, ABS_MT_POSITION_X, finger->x);
+ unsigned int clamped_x, clamped_y;
+ /*
+ * The touchpad might report invalid data, so we clamp
+ * the resulting values so that we do not confuse
+ * userspace.
+ */
+ clamped_x = clamp(finger->x, 0U, priv->x_max);
+ clamped_y = clamp(finger->y, 0U, priv->y_max);
+ input_report_abs(dev, ABS_MT_POSITION_X, clamped_x);
input_report_abs(dev, ABS_MT_POSITION_Y,
- FOC_MAX_Y - finger->y);
+ priv->y_max - clamped_y);
}
}
input_mt_report_pointer_emulation(dev, true);
state->pressed = (packet[0] >> 4) & 1;
- /*
- * packet[5] contains some kind of tool size in the most
- * significant nibble. 0xff is a special value (latching) that
- * signals a large contact area.
- */
- if (packet[5] == 0xff) {
- state->fingers[finger].valid = false;
- return;
- }
-
state->fingers[finger].x = ((packet[1] & 0xf) << 8) | packet[2];
state->fingers[finger].y = (packet[3] << 8) | packet[4];
state->fingers[finger].valid = true;
return 0;
}
+
+void focaltech_set_resolution(struct psmouse *psmouse, unsigned int resolution)
+{
+ /* not supported yet */
+}
+
+static void focaltech_set_rate(struct psmouse *psmouse, unsigned int rate)
+{
+ /* not supported yet */
+}
+
+static void focaltech_set_scale(struct psmouse *psmouse,
+ enum psmouse_scale scale)
+{
+ /* not supported yet */
+}
+
int focaltech_init(struct psmouse *psmouse)
{
struct focaltech_data *priv;
psmouse->cleanup = focaltech_reset;
/* resync is not supported yet */
psmouse->resync_time = 0;
+ /*
+ * rate/resolution/scale changes are not supported yet, and
+ * the generic implementations of these functions seem to
+ * confuse some touchpads
+ */
+ psmouse->set_resolution = focaltech_set_resolution;
+ psmouse->set_rate = focaltech_set_rate;
+ psmouse->set_scale = focaltech_set_scale;
return 0;
psmouse->rate = r;
}
+/*
+ * Here we set the mouse scaling.
+ */
+
+static void psmouse_set_scale(struct psmouse *psmouse, enum psmouse_scale scale)
+{
+ ps2_command(&psmouse->ps2dev, NULL,
+ scale == PSMOUSE_SCALE21 ? PSMOUSE_CMD_SETSCALE21 :
+ PSMOUSE_CMD_SETSCALE11);
+}
+
/*
* psmouse_poll() - default poll handler. Everyone except for ALPS uses it.
*/
psmouse->set_rate = psmouse_set_rate;
psmouse->set_resolution = psmouse_set_resolution;
+ psmouse->set_scale = psmouse_set_scale;
psmouse->poll = psmouse_poll;
psmouse->protocol_handler = psmouse_process_byte;
psmouse->pktsize = 3;
if (psmouse_max_proto != PSMOUSE_PS2) {
psmouse->set_rate(psmouse, psmouse->rate);
psmouse->set_resolution(psmouse, psmouse->resolution);
- ps2_command(&psmouse->ps2dev, NULL, PSMOUSE_CMD_SETSCALE11);
+ psmouse->set_scale(psmouse, PSMOUSE_SCALE11);
}
}
PSMOUSE_FULL_PACKET
} psmouse_ret_t;
+enum psmouse_scale {
+ PSMOUSE_SCALE11,
+ PSMOUSE_SCALE21
+};
+
struct psmouse {
void *private;
struct input_dev *dev;
psmouse_ret_t (*protocol_handler)(struct psmouse *psmouse);
void (*set_rate)(struct psmouse *psmouse, unsigned int rate);
void (*set_resolution)(struct psmouse *psmouse, unsigned int resolution);
+ void (*set_scale)(struct psmouse *psmouse, enum psmouse_scale scale);
int (*reconnect)(struct psmouse *psmouse);
void (*disconnect)(struct psmouse *psmouse);
#define X_MAX_POSITIVE 8176
#define Y_MAX_POSITIVE 8176
-/* maximum ABS_MT_POSITION displacement (in mm) */
-#define DMAX 10
-
/*****************************************************************************
* Stuff we need even when we do not want native Synaptics support
****************************************************************************/
static bool cr48_profile_sensor;
+#define ANY_BOARD_ID 0
struct min_max_quirk {
const char * const *pnp_ids;
+ struct {
+ unsigned long int min, max;
+ } board_id;
int x_min, x_max, y_min, y_max;
};
static const struct min_max_quirk min_max_pnpid_table[] = {
{
(const char * const []){"LEN0033", NULL},
+ {ANY_BOARD_ID, ANY_BOARD_ID},
1024, 5052, 2258, 4832
},
{
- (const char * const []){"LEN0035", "LEN0042", NULL},
+ (const char * const []){"LEN0042", NULL},
+ {ANY_BOARD_ID, ANY_BOARD_ID},
1232, 5710, 1156, 4696
},
{
(const char * const []){"LEN0034", "LEN0036", "LEN0037",
"LEN0039", "LEN2002", "LEN2004",
NULL},
+ {ANY_BOARD_ID, 2961},
1024, 5112, 2024, 4832
},
{
(const char * const []){"LEN2001", NULL},
+ {ANY_BOARD_ID, ANY_BOARD_ID},
1024, 5022, 2508, 4832
},
{
(const char * const []){"LEN2006", NULL},
+ {2691, 2691},
+ 1024, 5045, 2457, 4832
+ },
+ {
+ (const char * const []){"LEN2006", NULL},
+ {ANY_BOARD_ID, ANY_BOARD_ID},
1264, 5675, 1171, 4688
},
{ }
"LEN0041",
"LEN0042", /* Yoga */
"LEN0045",
- "LEN0046",
"LEN0047",
- "LEN0048",
"LEN0049",
"LEN2000",
"LEN2001", /* Edge E431 */
"LEN2003",
"LEN2004", /* L440 */
"LEN2005",
- "LEN2006",
+ "LEN2006", /* Edge E440/E540 */
"LEN2007",
"LEN2008",
"LEN2009",
return 0;
}
+static int synaptics_more_extended_queries(struct psmouse *psmouse)
+{
+ struct synaptics_data *priv = psmouse->private;
+ unsigned char buf[3];
+
+ if (synaptics_send_cmd(psmouse, SYN_QUE_MEXT_CAPAB_10, buf))
+ return -1;
+
+ priv->ext_cap_10 = (buf[0]<<16) | (buf[1]<<8) | buf[2];
+
+ return 0;
+}
+
/*
- * Read the board id from the touchpad
+ * Read the board id and the "More Extended Queries" from the touchpad
* The board id is encoded in the "QUERY MODES" response
*/
-static int synaptics_board_id(struct psmouse *psmouse)
+static int synaptics_query_modes(struct psmouse *psmouse)
{
struct synaptics_data *priv = psmouse->private;
unsigned char bid[3];
+ /* firmwares prior 7.5 have no board_id encoded */
+ if (SYN_ID_FULL(priv->identity) < 0x705)
+ return 0;
+
if (synaptics_send_cmd(psmouse, SYN_QUE_MODES, bid))
return -1;
priv->board_id = ((bid[0] & 0xfc) << 6) | bid[1];
+
+ if (SYN_MEXT_CAP_BIT(bid[0]))
+ return synaptics_more_extended_queries(psmouse);
+
return 0;
}
{
struct synaptics_data *priv = psmouse->private;
unsigned char resp[3];
- int i;
if (SYN_ID_MAJOR(priv->identity) < 4)
return 0;
}
}
- for (i = 0; min_max_pnpid_table[i].pnp_ids; i++) {
- if (psmouse_matches_pnp_id(psmouse,
- min_max_pnpid_table[i].pnp_ids)) {
- priv->x_min = min_max_pnpid_table[i].x_min;
- priv->x_max = min_max_pnpid_table[i].x_max;
- priv->y_min = min_max_pnpid_table[i].y_min;
- priv->y_max = min_max_pnpid_table[i].y_max;
- return 0;
- }
- }
-
if (SYN_EXT_CAP_REQUESTS(priv->capabilities) >= 5 &&
SYN_CAP_MAX_DIMENSIONS(priv->ext_cap_0c)) {
if (synaptics_send_cmd(psmouse, SYN_QUE_EXT_MAX_COORDS, resp)) {
} else {
priv->x_max = (resp[0] << 5) | ((resp[1] & 0x0f) << 1);
priv->y_max = (resp[2] << 5) | ((resp[1] & 0xf0) >> 3);
+ psmouse_info(psmouse,
+ "queried max coordinates: x [..%d], y [..%d]\n",
+ priv->x_max, priv->y_max);
}
}
- if (SYN_EXT_CAP_REQUESTS(priv->capabilities) >= 7 &&
- SYN_CAP_MIN_DIMENSIONS(priv->ext_cap_0c)) {
+ if (SYN_CAP_MIN_DIMENSIONS(priv->ext_cap_0c) &&
+ (SYN_EXT_CAP_REQUESTS(priv->capabilities) >= 7 ||
+ /*
+ * Firmware v8.1 does not report proper number of extended
+ * capabilities, but has been proven to report correct min
+ * coordinates.
+ */
+ SYN_ID_FULL(priv->identity) == 0x801)) {
if (synaptics_send_cmd(psmouse, SYN_QUE_EXT_MIN_COORDS, resp)) {
psmouse_warn(psmouse,
"device claims to have min coordinates query, but I'm not able to read it.\n");
} else {
priv->x_min = (resp[0] << 5) | ((resp[1] & 0x0f) << 1);
priv->y_min = (resp[2] << 5) | ((resp[1] & 0xf0) >> 3);
+ psmouse_info(psmouse,
+ "queried min coordinates: x [%d..], y [%d..]\n",
+ priv->x_min, priv->y_min);
}
}
return 0;
}
+/*
+ * Apply quirk(s) if the hardware matches
+ */
+
+static void synaptics_apply_quirks(struct psmouse *psmouse)
+{
+ struct synaptics_data *priv = psmouse->private;
+ int i;
+
+ for (i = 0; min_max_pnpid_table[i].pnp_ids; i++) {
+ if (!psmouse_matches_pnp_id(psmouse,
+ min_max_pnpid_table[i].pnp_ids))
+ continue;
+
+ if (min_max_pnpid_table[i].board_id.min != ANY_BOARD_ID &&
+ priv->board_id < min_max_pnpid_table[i].board_id.min)
+ continue;
+
+ if (min_max_pnpid_table[i].board_id.max != ANY_BOARD_ID &&
+ priv->board_id > min_max_pnpid_table[i].board_id.max)
+ continue;
+
+ priv->x_min = min_max_pnpid_table[i].x_min;
+ priv->x_max = min_max_pnpid_table[i].x_max;
+ priv->y_min = min_max_pnpid_table[i].y_min;
+ priv->y_max = min_max_pnpid_table[i].y_max;
+ psmouse_info(psmouse,
+ "quirked min/max coordinates: x [%d..%d], y [%d..%d]\n",
+ priv->x_min, priv->x_max,
+ priv->y_min, priv->y_max);
+ break;
+ }
+}
+
static int synaptics_query_hardware(struct psmouse *psmouse)
{
if (synaptics_identify(psmouse))
return -1;
if (synaptics_firmware_id(psmouse))
return -1;
- if (synaptics_board_id(psmouse))
+ if (synaptics_query_modes(psmouse))
return -1;
if (synaptics_capability(psmouse))
return -1;
if (synaptics_resolution(psmouse))
return -1;
+ synaptics_apply_quirks(psmouse);
+
return 0;
}
return (buf[0] & 0xFC) == 0x84 && (buf[3] & 0xCC) == 0xC4;
}
-static void synaptics_pass_pt_packet(struct serio *ptport, unsigned char *packet)
+static void synaptics_pass_pt_packet(struct psmouse *psmouse,
+ struct serio *ptport,
+ unsigned char *packet)
{
+ struct synaptics_data *priv = psmouse->private;
struct psmouse *child = serio_get_drvdata(ptport);
if (child && child->state == PSMOUSE_ACTIVATED) {
- serio_interrupt(ptport, packet[1], 0);
+ serio_interrupt(ptport, packet[1] | priv->pt_buttons, 0);
serio_interrupt(ptport, packet[4], 0);
serio_interrupt(ptport, packet[5], 0);
if (child->pktsize == 4)
serio_interrupt(ptport, packet[2], 0);
- } else
+ } else {
serio_interrupt(ptport, packet[1], 0);
+ }
}
static void synaptics_pt_activate(struct psmouse *psmouse)
}
}
+static void synaptics_parse_ext_buttons(const unsigned char buf[],
+ struct synaptics_data *priv,
+ struct synaptics_hw_state *hw)
+{
+ unsigned int ext_bits =
+ (SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap) + 1) >> 1;
+ unsigned int ext_mask = GENMASK(ext_bits - 1, 0);
+
+ hw->ext_buttons = buf[4] & ext_mask;
+ hw->ext_buttons |= (buf[5] & ext_mask) << ext_bits;
+}
+
static bool is_forcepad;
static int synaptics_parse_hw_state(const unsigned char buf[],
hw->down = ((buf[0] ^ buf[3]) & 0x02) ? 1 : 0;
}
- if (SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap) &&
+ if (SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap) > 0 &&
((buf[0] ^ buf[3]) & 0x02)) {
- switch (SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap) & ~0x01) {
- default:
- /*
- * if nExtBtn is greater than 8 it should be
- * considered invalid and treated as 0
- */
- break;
- case 8:
- hw->ext_buttons |= ((buf[5] & 0x08)) ? 0x80 : 0;
- hw->ext_buttons |= ((buf[4] & 0x08)) ? 0x40 : 0;
- case 6:
- hw->ext_buttons |= ((buf[5] & 0x04)) ? 0x20 : 0;
- hw->ext_buttons |= ((buf[4] & 0x04)) ? 0x10 : 0;
- case 4:
- hw->ext_buttons |= ((buf[5] & 0x02)) ? 0x08 : 0;
- hw->ext_buttons |= ((buf[4] & 0x02)) ? 0x04 : 0;
- case 2:
- hw->ext_buttons |= ((buf[5] & 0x01)) ? 0x02 : 0;
- hw->ext_buttons |= ((buf[4] & 0x01)) ? 0x01 : 0;
- }
+ synaptics_parse_ext_buttons(buf, priv, hw);
}
} else {
hw->x = (((buf[1] & 0x1f) << 8) | buf[2]);
}
}
+static void synaptics_report_ext_buttons(struct psmouse *psmouse,
+ const struct synaptics_hw_state *hw)
+{
+ struct input_dev *dev = psmouse->dev;
+ struct synaptics_data *priv = psmouse->private;
+ int ext_bits = (SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap) + 1) >> 1;
+ char buf[6] = { 0x00, 0x00, 0x00, 0x00, 0x00, 0x00};
+ int i;
+
+ if (!SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap))
+ return;
+
+ /* Bug in FW 8.1, buttons are reported only when ExtBit is 1 */
+ if (SYN_ID_FULL(priv->identity) == 0x801 &&
+ !((psmouse->packet[0] ^ psmouse->packet[3]) & 0x02))
+ return;
+
+ if (!SYN_CAP_EXT_BUTTONS_STICK(priv->ext_cap_10)) {
+ for (i = 0; i < ext_bits; i++) {
+ input_report_key(dev, BTN_0 + 2 * i,
+ hw->ext_buttons & (1 << i));
+ input_report_key(dev, BTN_1 + 2 * i,
+ hw->ext_buttons & (1 << (i + ext_bits)));
+ }
+ return;
+ }
+
+ /*
+ * This generation of touchpads has the trackstick buttons
+ * physically wired to the touchpad. Re-route them through
+ * the pass-through interface.
+ */
+ if (!priv->pt_port)
+ return;
+
+ /* The trackstick expects at most 3 buttons */
+ priv->pt_buttons = SYN_CAP_EXT_BUTTON_STICK_L(hw->ext_buttons) |
+ SYN_CAP_EXT_BUTTON_STICK_R(hw->ext_buttons) << 1 |
+ SYN_CAP_EXT_BUTTON_STICK_M(hw->ext_buttons) << 2;
+
+ synaptics_pass_pt_packet(psmouse, priv->pt_port, buf);
+}
+
static void synaptics_report_buttons(struct psmouse *psmouse,
const struct synaptics_hw_state *hw)
{
struct input_dev *dev = psmouse->dev;
struct synaptics_data *priv = psmouse->private;
- int i;
input_report_key(dev, BTN_LEFT, hw->left);
input_report_key(dev, BTN_RIGHT, hw->right);
input_report_key(dev, BTN_BACK, hw->down);
}
- for (i = 0; i < SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap); i++)
- input_report_key(dev, BTN_0 + i, hw->ext_buttons & (1 << i));
+ synaptics_report_ext_buttons(psmouse, hw);
}
static void synaptics_report_mt_data(struct psmouse *psmouse,
pos[i].y = synaptics_invert_y(hw[i]->y);
}
- input_mt_assign_slots(dev, slot, pos, nsemi, DMAX * priv->x_res);
+ input_mt_assign_slots(dev, slot, pos, nsemi, 0);
for (i = 0; i < nsemi; i++) {
input_mt_slot(dev, slot[i]);
if (SYN_CAP_PASS_THROUGH(priv->capabilities) &&
synaptics_is_pt_packet(psmouse->packet)) {
if (priv->pt_port)
- synaptics_pass_pt_packet(priv->pt_port, psmouse->packet);
+ synaptics_pass_pt_packet(psmouse, priv->pt_port,
+ psmouse->packet);
} else
synaptics_process_packet(psmouse);
__set_bit(BTN_BACK, dev->keybit);
}
- for (i = 0; i < SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap); i++)
- __set_bit(BTN_0 + i, dev->keybit);
+ if (!SYN_CAP_EXT_BUTTONS_STICK(priv->ext_cap_10))
+ for (i = 0; i < SYN_CAP_MULTI_BUTTON_NO(priv->ext_cap); i++)
+ __set_bit(BTN_0 + i, dev->keybit);
__clear_bit(EV_REL, dev->evbit);
__clear_bit(REL_X, dev->relbit);
if (SYN_CAP_CLICKPAD(priv->ext_cap_0c)) {
__set_bit(INPUT_PROP_BUTTONPAD, dev->propbit);
- if (psmouse_matches_pnp_id(psmouse, topbuttonpad_pnp_ids))
+ if (psmouse_matches_pnp_id(psmouse, topbuttonpad_pnp_ids) &&
+ !SYN_CAP_EXT_BUTTONS_STICK(priv->ext_cap_10))
__set_bit(INPUT_PROP_TOPBUTTONPAD, dev->propbit);
/* Clickpads report only left button */
__clear_bit(BTN_RIGHT, dev->keybit);
#define SYN_QUE_EXT_CAPAB_0C 0x0c
#define SYN_QUE_EXT_MAX_COORDS 0x0d
#define SYN_QUE_EXT_MIN_COORDS 0x0f
+#define SYN_QUE_MEXT_CAPAB_10 0x10
/* synatics modes */
#define SYN_BIT_ABSOLUTE_MODE (1 << 7)
#define SYN_EXT_CAP_REQUESTS(c) (((c) & 0x700000) >> 20)
#define SYN_CAP_MULTI_BUTTON_NO(ec) (((ec) & 0x00f000) >> 12)
#define SYN_CAP_PRODUCT_ID(ec) (((ec) & 0xff0000) >> 16)
+#define SYN_MEXT_CAP_BIT(m) ((m) & (1 << 1))
/*
* The following describes response for the 0x0c query.
#define SYN_CAP_REDUCED_FILTERING(ex0c) ((ex0c) & 0x000400)
#define SYN_CAP_IMAGE_SENSOR(ex0c) ((ex0c) & 0x000800)
+/*
+ * The following descibes response for the 0x10 query.
+ *
+ * byte mask name meaning
+ * ---- ---- ------- ------------
+ * 1 0x01 ext buttons are stick buttons exported in the extended
+ * capability are actually meant to be used
+ * by the tracktick (pass-through).
+ * 1 0x02 SecurePad the touchpad is a SecurePad, so it
+ * contains a built-in fingerprint reader.
+ * 1 0xe0 more ext count how many more extented queries are
+ * available after this one.
+ * 2 0xff SecurePad width the width of the SecurePad fingerprint
+ * reader.
+ * 3 0xff SecurePad height the height of the SecurePad fingerprint
+ * reader.
+ */
+#define SYN_CAP_EXT_BUTTONS_STICK(ex10) ((ex10) & 0x010000)
+#define SYN_CAP_SECUREPAD(ex10) ((ex10) & 0x020000)
+
+#define SYN_CAP_EXT_BUTTON_STICK_L(eb) (!!((eb) & 0x01))
+#define SYN_CAP_EXT_BUTTON_STICK_M(eb) (!!((eb) & 0x02))
+#define SYN_CAP_EXT_BUTTON_STICK_R(eb) (!!((eb) & 0x04))
+
/* synaptics modes query bits */
#define SYN_MODE_ABSOLUTE(m) ((m) & (1 << 7))
#define SYN_MODE_RATE(m) ((m) & (1 << 6))
unsigned long int capabilities; /* Capabilities */
unsigned long int ext_cap; /* Extended Capabilities */
unsigned long int ext_cap_0c; /* Ext Caps from 0x0c query */
+ unsigned long int ext_cap_10; /* Ext Caps from 0x10 query */
unsigned long int identity; /* Identification */
unsigned int x_res, y_res; /* X/Y resolution in units/mm */
unsigned int x_max, y_max; /* Max coordinates (from FW) */
bool disable_gesture; /* disable gestures */
struct serio *pt_port; /* Pass-through serio port */
+ unsigned char pt_buttons; /* Pass-through buttons */
/*
* Last received Advanced Gesture Mode (AGM) packet. An AGM packet
tristate "Allwinner sun4i resistive touchscreen controller support"
depends on ARCH_SUNXI || COMPILE_TEST
depends on HWMON
+ depends on THERMAL || !THERMAL_OF
help
This selects support for the resistive touchscreen controller
found on Allwinner sunxi SoCs.
config IOMMU_IO_PGTABLE_LPAE
bool "ARMv7/v8 Long Descriptor Format"
select IOMMU_IO_PGTABLE
+ depends on ARM || ARM64 || COMPILE_TEST
help
Enable support for the ARM long descriptor pagetable format.
This allocator supports 4K/2M/1G, 16K/32M and 64K/512M page
bool "MSM IOMMU Support"
depends on ARM
depends on ARCH_MSM8X60 || ARCH_MSM8960 || COMPILE_TEST
+ depends on BROKEN
select IOMMU_API
help
Support for the IOMMUs found on certain Qualcomm SOCs.
return 0;
spin_lock_irqsave(&smmu_domain->pgtbl_lock, flags);
- if (smmu_domain->smmu->features & ARM_SMMU_FEAT_TRANS_OPS)
+ if (smmu_domain->smmu->features & ARM_SMMU_FEAT_TRANS_OPS &&
+ smmu_domain->stage == ARM_SMMU_DOMAIN_S1) {
ret = arm_smmu_iova_to_phys_hard(domain, iova);
- else
+ } else {
ret = ops->iova_to_phys(ops, iova);
+ }
+
spin_unlock_irqrestore(&smmu_domain->pgtbl_lock, flags);
return ret;
return -ENODEV;
}
- if (smmu->version == 1 || (!(id & ID0_ATOSNS) && (id & ID0_S1TS))) {
+ if ((id & ID0_S1TS) && ((smmu->version == 1) || (id & ID0_ATOSNS))) {
smmu->features |= ARM_SMMU_FEAT_TRANS_OPS;
dev_notice(smmu->dev, "\taddress translation ops\n");
}
static int __init exynos_iommu_init(void)
{
+ struct device_node *np;
int ret;
+ np = of_find_matching_node(NULL, sysmmu_of_match);
+ if (!np)
+ return 0;
+
+ of_node_put(np);
+
lv2table_kmem_cache = kmem_cache_create("exynos-iommu-lv2table",
LV2TABLE_SIZE, LV2TABLE_SIZE, 0, NULL);
if (!lv2table_kmem_cache) {
static void domain_exit(struct dmar_domain *domain)
{
- struct dmar_drhd_unit *drhd;
- struct intel_iommu *iommu;
struct page *freelist = NULL;
+ int i;
/* Domain 0 is reserved, so dont process it */
if (!domain)
/* clear attached or cached domains */
rcu_read_lock();
- for_each_active_iommu(iommu, drhd)
- iommu_detach_domain(domain, iommu);
+ for_each_set_bit(i, domain->iommu_bmp, g_num_of_iommus)
+ iommu_detach_domain(domain, g_iommus[i]);
rcu_read_unlock();
dma_free_pagelist(freelist);
((((d)->levels - ((l) - ARM_LPAE_START_LVL(d) + 1)) \
* (d)->bits_per_level) + (d)->pg_shift)
-#define ARM_LPAE_PAGES_PER_PGD(d) ((d)->pgd_size >> (d)->pg_shift)
+#define ARM_LPAE_PAGES_PER_PGD(d) \
+ DIV_ROUND_UP((d)->pgd_size, 1UL << (d)->pg_shift)
/*
* Calculate the index at level l used to map virtual address a using the
((l) == ARM_LPAE_START_LVL(d) ? ilog2(ARM_LPAE_PAGES_PER_PGD(d)) : 0)
#define ARM_LPAE_LVL_IDX(a,l,d) \
- (((a) >> ARM_LPAE_LVL_SHIFT(l,d)) & \
+ (((u64)(a) >> ARM_LPAE_LVL_SHIFT(l,d)) & \
((1 << ((d)->bits_per_level + ARM_LPAE_PGD_IDX(l,d))) - 1))
/* Calculate the block/page mapping size at level l for pagetable in d. */
static const struct of_device_id ipmmu_of_ids[] = {
{ .compatible = "renesas,ipmmu-vmsa", },
+ { }
};
static struct platform_driver ipmmu_driver = {
return ops;
}
-struct iommu_ops *of_iommu_configure(struct device *dev)
+struct iommu_ops *of_iommu_configure(struct device *dev,
+ struct device_node *master_np)
{
struct of_phandle_args iommu_spec;
struct device_node *np;
struct iommu_ops *ops = NULL;
int idx = 0;
+ if (dev_is_pci(dev)) {
+ dev_err(dev, "IOMMU is currently not supported for PCI\n");
+ return NULL;
+ }
+
/*
* We don't currently walk up the tree looking for a parent IOMMU.
* See the `Notes:' section of
* Documentation/devicetree/bindings/iommu/iommu.txt
*/
- while (!of_parse_phandle_with_args(dev->of_node, "iommus",
+ while (!of_parse_phandle_with_args(master_np, "iommus",
"#iommu-cells", idx,
&iommu_spec)) {
np = iommu_spec.np;
struct kmem_cache *p;
const unsigned long flags = SLAB_HWCACHE_ALIGN;
size_t align = 1 << 10; /* L2 pagetable alignement */
+ struct device_node *np;
+
+ np = of_find_matching_node(NULL, omap_iommu_of_match);
+ if (!np)
+ return 0;
+
+ of_node_put(np);
p = kmem_cache_create("iopte_cache", IOPTE_TABLE_SIZE, align, flags,
iopte_cachep_ctor);
static int __init rk_iommu_init(void)
{
+ struct device_node *np;
int ret;
+ np = of_find_matching_node(NULL, rk_iommu_dt_ids);
+ if (!np)
+ return 0;
+
+ of_node_put(np);
+
ret = bus_set_iommu(&platform_bus_type, &rk_iommu_ops);
if (ret)
return ret;
bool
select IRQ_DOMAIN
+config ST_IRQCHIP
+ bool
+ select REGMAP
+ select MFD_SYSCON
+ help
+ Enables SysCfg Controlled IRQs on STi based platforms.
+
config TB10X_IRQC
bool
select IRQ_DOMAIN
obj-$(CONFIG_ARCH_MMP) += irq-mmp.o
obj-$(CONFIG_ARCH_MVEBU) += irq-armada-370-xp.o
obj-$(CONFIG_ARCH_MXS) += irq-mxs.o
+obj-$(CONFIG_ARCH_TEGRA) += irq-tegra.o
obj-$(CONFIG_ARCH_S3C24XX) += irq-s3c24xx.o
obj-$(CONFIG_DW_APB_ICTL) += irq-dw-apb-ictl.o
obj-$(CONFIG_METAG) += irq-metag-ext.o
obj-$(CONFIG_VERSATILE_FPGA_IRQ) += irq-versatile-fpga.o
obj-$(CONFIG_ARCH_NSPIRE) += irq-zevio.o
obj-$(CONFIG_ARCH_VT8500) += irq-vt8500.o
+obj-$(CONFIG_ST_IRQCHIP) += irq-st.o
obj-$(CONFIG_TB10X_IRQC) += irq-tb10x.o
obj-$(CONFIG_XTENSA) += irq-xtensa-pic.o
obj-$(CONFIG_XTENSA_MX) += irq-xtensa-mx.o
obj-$(CONFIG_IRQ_CROSSBAR) += irq-crossbar.o
+obj-$(CONFIG_SOC_VF610) += irq-vf610-mscm-ir.o
obj-$(CONFIG_BCM7120_L2_IRQ) += irq-bcm7120-l2.o
obj-$(CONFIG_BRCMSTB_L2_IRQ) += irq-brcmstb-l2.o
obj-$(CONFIG_KEYSTONE_IRQ) += irq-keystone.o
/* Interrupt Controller Registers Map */
#define ARMADA_370_XP_INT_SET_MASK_OFFS (0x48)
#define ARMADA_370_XP_INT_CLEAR_MASK_OFFS (0x4C)
+#define ARMADA_370_XP_INT_FABRIC_MASK_OFFS (0x54)
+#define ARMADA_370_XP_INT_CAUSE_PERF(cpu) (1 << cpu)
#define ARMADA_370_XP_INT_CONTROL (0x00)
#define ARMADA_370_XP_INT_SET_ENABLE_OFFS (0x30)
#define ARMADA_370_XP_MAX_PER_CPU_IRQS (28)
#define ARMADA_370_XP_TIMER0_PER_CPU_IRQ (5)
+#define ARMADA_370_XP_FABRIC_IRQ (3)
#define IPI_DOORBELL_START (0)
#define IPI_DOORBELL_END (8)
static void __iomem *main_int_base;
static struct irq_domain *armada_370_xp_mpic_domain;
static u32 doorbell_mask_reg;
+static int parent_irq;
#ifdef CONFIG_PCI_MSI
static struct irq_domain *armada_370_xp_msi_domain;
static DECLARE_BITMAP(msi_used, PCI_MSI_DOORBELL_NR);
static phys_addr_t msi_doorbell_addr;
#endif
+static inline bool is_percpu_irq(irq_hw_number_t irq)
+{
+ switch (irq) {
+ case ARMADA_370_XP_TIMER0_PER_CPU_IRQ:
+ case ARMADA_370_XP_FABRIC_IRQ:
+ return true;
+ default:
+ return false;
+ }
+}
+
/*
* In SMP mode:
* For shared global interrupts, mask/unmask global enable bit
{
irq_hw_number_t hwirq = irqd_to_hwirq(d);
- if (hwirq != ARMADA_370_XP_TIMER0_PER_CPU_IRQ)
+ if (!is_percpu_irq(hwirq))
writel(hwirq, main_int_base +
ARMADA_370_XP_INT_CLEAR_ENABLE_OFFS);
else
{
irq_hw_number_t hwirq = irqd_to_hwirq(d);
- if (hwirq != ARMADA_370_XP_TIMER0_PER_CPU_IRQ)
+ if (!is_percpu_irq(hwirq))
writel(hwirq, main_int_base +
ARMADA_370_XP_INT_SET_ENABLE_OFFS);
else
#ifdef CONFIG_SMP
.irq_set_affinity = armada_xp_set_affinity,
#endif
+ .flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND,
};
static int armada_370_xp_mpic_irq_map(struct irq_domain *h,
unsigned int virq, irq_hw_number_t hw)
{
armada_370_xp_irq_mask(irq_get_irq_data(virq));
- if (hw != ARMADA_370_XP_TIMER0_PER_CPU_IRQ)
+ if (!is_percpu_irq(hw))
writel(hw, per_cpu_int_base +
ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
else
writel(hw, main_int_base + ARMADA_370_XP_INT_SET_ENABLE_OFFS);
irq_set_status_flags(virq, IRQ_LEVEL);
- if (hw == ARMADA_370_XP_TIMER0_PER_CPU_IRQ) {
+ if (is_percpu_irq(hw)) {
irq_set_percpu_devid(virq);
irq_set_chip_and_handler(virq, &armada_370_xp_irq_chip,
handle_percpu_devid_irq);
return 0;
}
-#ifdef CONFIG_SMP
-static void armada_mpic_send_doorbell(const struct cpumask *mask,
- unsigned int irq)
-{
- int cpu;
- unsigned long map = 0;
-
- /* Convert our logical CPU mask into a physical one. */
- for_each_cpu(cpu, mask)
- map |= 1 << cpu_logical_map(cpu);
-
- /*
- * Ensure that stores to Normal memory are visible to the
- * other CPUs before issuing the IPI.
- */
- dsb();
-
- /* submit softirq */
- writel((map << 8) | irq, main_int_base +
- ARMADA_370_XP_SW_TRIG_INT_OFFS);
-}
-
static void armada_xp_mpic_smp_cpu_init(void)
{
u32 control;
writel(0, per_cpu_int_base + ARMADA_370_XP_INT_CLEAR_MASK_OFFS);
}
+static void armada_xp_mpic_perf_init(void)
+{
+ unsigned long cpuid = cpu_logical_map(smp_processor_id());
+
+ /* Enable Performance Counter Overflow interrupts */
+ writel(ARMADA_370_XP_INT_CAUSE_PERF(cpuid),
+ per_cpu_int_base + ARMADA_370_XP_INT_FABRIC_MASK_OFFS);
+}
+
+#ifdef CONFIG_SMP
+static void armada_mpic_send_doorbell(const struct cpumask *mask,
+ unsigned int irq)
+{
+ int cpu;
+ unsigned long map = 0;
+
+ /* Convert our logical CPU mask into a physical one. */
+ for_each_cpu(cpu, mask)
+ map |= 1 << cpu_logical_map(cpu);
+
+ /*
+ * Ensure that stores to Normal memory are visible to the
+ * other CPUs before issuing the IPI.
+ */
+ dsb();
+
+ /* submit softirq */
+ writel((map << 8) | irq, main_int_base +
+ ARMADA_370_XP_SW_TRIG_INT_OFFS);
+}
+
static int armada_xp_mpic_secondary_init(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
- if (action == CPU_STARTING || action == CPU_STARTING_FROZEN)
+ if (action == CPU_STARTING || action == CPU_STARTING_FROZEN) {
+ armada_xp_mpic_perf_init();
armada_xp_mpic_smp_cpu_init();
+ }
+
return NOTIFY_OK;
}
.priority = 100,
};
+static int mpic_cascaded_secondary_init(struct notifier_block *nfb,
+ unsigned long action, void *hcpu)
+{
+ if (action == CPU_STARTING || action == CPU_STARTING_FROZEN) {
+ armada_xp_mpic_perf_init();
+ enable_percpu_irq(parent_irq, IRQ_TYPE_NONE);
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block mpic_cascaded_cpu_notifier = {
+ .notifier_call = mpic_cascaded_secondary_init,
+ .priority = 100,
+};
#endif /* CONFIG_SMP */
static struct irq_domain_ops armada_370_xp_mpic_irq_ops = {
struct device_node *parent)
{
struct resource main_int_res, per_cpu_int_res;
- int parent_irq, nr_irqs, i;
+ int nr_irqs, i;
u32 control;
BUG_ON(of_address_to_resource(node, 0, &main_int_res));
BUG_ON(!armada_370_xp_mpic_domain);
-#ifdef CONFIG_SMP
+ /* Setup for the boot CPU */
+ armada_xp_mpic_perf_init();
armada_xp_mpic_smp_cpu_init();
-#endif
armada_370_xp_msi_init(node, main_int_res.start);
register_cpu_notifier(&armada_370_xp_mpic_cpu_notifier);
#endif
} else {
+#ifdef CONFIG_SMP
+ register_cpu_notifier(&mpic_cascaded_cpu_notifier);
+#endif
irq_set_chained_handler(parent_irq,
armada_370_xp_mpic_handle_cascade_irq);
}
*/
#include <linux/err.h>
#include <linux/io.h>
+#include <linux/irqdomain.h>
#include <linux/of_address.h>
#include <linux/of_irq.h>
#include <linux/slab.h>
-#include <linux/irqchip/arm-gic.h>
-#include <linux/irqchip/irq-crossbar.h>
+
+#include "irqchip.h"
#define IRQ_FREE -1
#define IRQ_RESERVED -2
/**
* struct crossbar_device - crossbar device description
+ * @lock: spinlock serializing access to @irq_map
* @int_max: maximum number of supported interrupts
* @safe_map: safe default value to initialize the crossbar
* @max_crossbar_sources: Maximum number of crossbar sources
* @write: register write function pointer
*/
struct crossbar_device {
+ raw_spinlock_t lock;
uint int_max;
uint safe_map;
uint max_crossbar_sources;
static struct crossbar_device *cb;
-static inline void crossbar_writel(int irq_no, int cb_no)
+static void crossbar_writel(int irq_no, int cb_no)
{
writel(cb_no, cb->crossbar_base + cb->register_offsets[irq_no]);
}
-static inline void crossbar_writew(int irq_no, int cb_no)
+static void crossbar_writew(int irq_no, int cb_no)
{
writew(cb_no, cb->crossbar_base + cb->register_offsets[irq_no]);
}
-static inline void crossbar_writeb(int irq_no, int cb_no)
+static void crossbar_writeb(int irq_no, int cb_no)
{
writeb(cb_no, cb->crossbar_base + cb->register_offsets[irq_no]);
}
-static inline int get_prev_map_irq(int cb_no)
-{
- int i;
-
- for (i = cb->int_max - 1; i >= 0; i--)
- if (cb->irq_map[i] == cb_no)
- return i;
-
- return -ENODEV;
-}
+static struct irq_chip crossbar_chip = {
+ .name = "CBAR",
+ .irq_eoi = irq_chip_eoi_parent,
+ .irq_mask = irq_chip_mask_parent,
+ .irq_unmask = irq_chip_unmask_parent,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .irq_set_wake = irq_chip_set_wake_parent,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+#endif
+};
-static inline int allocate_free_irq(int cb_no)
+static int allocate_gic_irq(struct irq_domain *domain, unsigned virq,
+ irq_hw_number_t hwirq)
{
+ struct of_phandle_args args;
int i;
+ int err;
+ raw_spin_lock(&cb->lock);
for (i = cb->int_max - 1; i >= 0; i--) {
if (cb->irq_map[i] == IRQ_FREE) {
- cb->irq_map[i] = cb_no;
- return i;
+ cb->irq_map[i] = hwirq;
+ break;
}
}
+ raw_spin_unlock(&cb->lock);
- return -ENODEV;
-}
+ if (i < 0)
+ return -ENODEV;
-static inline bool needs_crossbar_write(irq_hw_number_t hw)
-{
- int cb_no;
+ args.np = domain->parent->of_node;
+ args.args_count = 3;
+ args.args[0] = 0; /* SPI */
+ args.args[1] = i;
+ args.args[2] = IRQ_TYPE_LEVEL_HIGH;
- if (hw > GIC_IRQ_START) {
- cb_no = cb->irq_map[hw - GIC_IRQ_START];
- if (cb_no != IRQ_RESERVED && cb_no != IRQ_SKIP)
- return true;
- }
+ err = irq_domain_alloc_irqs_parent(domain, virq, 1, &args);
+ if (err)
+ cb->irq_map[i] = IRQ_FREE;
+ else
+ cb->write(i, hwirq);
- return false;
+ return err;
}
-static int crossbar_domain_map(struct irq_domain *d, unsigned int irq,
- irq_hw_number_t hw)
+static int crossbar_domain_alloc(struct irq_domain *d, unsigned int virq,
+ unsigned int nr_irqs, void *data)
{
- if (needs_crossbar_write(hw))
- cb->write(hw - GIC_IRQ_START, cb->irq_map[hw - GIC_IRQ_START]);
+ struct of_phandle_args *args = data;
+ irq_hw_number_t hwirq;
+ int i;
+
+ if (args->args_count != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (args->args[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ hwirq = args->args[1];
+ if ((hwirq + nr_irqs) > cb->max_crossbar_sources)
+ return -EINVAL; /* Can't deal with this */
+
+ for (i = 0; i < nr_irqs; i++) {
+ int err = allocate_gic_irq(d, virq + i, hwirq + i);
+
+ if (err)
+ return err;
+
+ irq_domain_set_hwirq_and_chip(d, virq + i, hwirq + i,
+ &crossbar_chip, NULL);
+ }
return 0;
}
/**
- * crossbar_domain_unmap - unmap a crossbar<->irq connection
- * @d: domain of irq to unmap
- * @irq: virq number
+ * crossbar_domain_free - unmap/free a crossbar<->irq connection
+ * @domain: domain of irq to unmap
+ * @virq: virq number
+ * @nr_irqs: number of irqs to free
*
* We do not maintain a use count of total number of map/unmap
* calls for a particular irq to find out if a irq can be really
* after which irq is anyways unusable. So an explicit map has to be called
* after that.
*/
-static void crossbar_domain_unmap(struct irq_domain *d, unsigned int irq)
+static void crossbar_domain_free(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs)
{
- irq_hw_number_t hw = irq_get_irq_data(irq)->hwirq;
+ int i;
- if (needs_crossbar_write(hw)) {
- cb->irq_map[hw - GIC_IRQ_START] = IRQ_FREE;
- cb->write(hw - GIC_IRQ_START, cb->safe_map);
+ raw_spin_lock(&cb->lock);
+ for (i = 0; i < nr_irqs; i++) {
+ struct irq_data *d = irq_domain_get_irq_data(domain, virq + i);
+
+ irq_domain_reset_irq_data(d);
+ cb->irq_map[d->hwirq] = IRQ_FREE;
+ cb->write(d->hwirq, cb->safe_map);
}
+ raw_spin_unlock(&cb->lock);
}
static int crossbar_domain_xlate(struct irq_domain *d,
unsigned long *out_hwirq,
unsigned int *out_type)
{
- int ret;
- int req_num = intspec[1];
- int direct_map_num;
-
- if (req_num >= cb->max_crossbar_sources) {
- direct_map_num = req_num - cb->max_crossbar_sources;
- if (direct_map_num < cb->int_max) {
- ret = cb->irq_map[direct_map_num];
- if (ret == IRQ_RESERVED || ret == IRQ_SKIP) {
- /* We use the interrupt num as h/w irq num */
- ret = direct_map_num;
- goto found;
- }
- }
-
- pr_err("%s: requested crossbar number %d > max %d\n",
- __func__, req_num, cb->max_crossbar_sources);
- return -EINVAL;
- }
-
- ret = get_prev_map_irq(req_num);
- if (ret >= 0)
- goto found;
-
- ret = allocate_free_irq(req_num);
-
- if (ret < 0)
- return ret;
-
-found:
- *out_hwirq = ret + GIC_IRQ_START;
+ if (d->of_node != controller)
+ return -EINVAL; /* Shouldn't happen, really... */
+ if (intsize != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (intspec[0] != 0)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ *out_hwirq = intspec[1];
+ *out_type = intspec[2];
return 0;
}
-static const struct irq_domain_ops routable_irq_domain_ops = {
- .map = crossbar_domain_map,
- .unmap = crossbar_domain_unmap,
- .xlate = crossbar_domain_xlate
+static const struct irq_domain_ops crossbar_domain_ops = {
+ .alloc = crossbar_domain_alloc,
+ .free = crossbar_domain_free,
+ .xlate = crossbar_domain_xlate,
};
static int __init crossbar_of_init(struct device_node *node)
cb->write(i, cb->safe_map);
}
- register_routable_domain_ops(&routable_irq_domain_ops);
+ raw_spin_lock_init(&cb->lock);
+
return 0;
err_reg_offset:
return ret;
}
-static const struct of_device_id crossbar_match[] __initconst = {
- { .compatible = "ti,irq-crossbar" },
- {}
-};
-
-int __init irqcrossbar_init(void)
+static int __init irqcrossbar_init(struct device_node *node,
+ struct device_node *parent)
{
- struct device_node *np;
- np = of_find_matching_node(NULL, crossbar_match);
- if (!np)
+ struct irq_domain *parent_domain, *domain;
+ int err;
+
+ if (!parent) {
+ pr_err("%s: no parent, giving up\n", node->full_name);
return -ENODEV;
+ }
+
+ parent_domain = irq_find_host(parent);
+ if (!parent_domain) {
+ pr_err("%s: unable to obtain parent domain\n", node->full_name);
+ return -ENXIO;
+ }
+
+ err = crossbar_of_init(node);
+ if (err)
+ return err;
+
+ domain = irq_domain_add_hierarchy(parent_domain, 0,
+ cb->max_crossbar_sources,
+ node, &crossbar_domain_ops,
+ NULL);
+ if (!domain) {
+ pr_err("%s: failed to allocated domain\n", node->full_name);
+ return -ENOMEM;
+ }
- crossbar_of_init(np);
return 0;
}
+
+IRQCHIP_DECLARE(ti_irqcrossbar, "ti,irq-crossbar", irqcrossbar_init);
} while (1);
}
-static void digicolor_set_gc(void __iomem *reg_base, unsigned irq_base,
- unsigned en_reg, unsigned ack_reg)
+static void __init digicolor_set_gc(void __iomem *reg_base, unsigned irq_base,
+ unsigned en_reg, unsigned ack_reg)
{
struct irq_chip_generic *gc;
static void its_encode_devid(struct its_cmd_block *cmd, u32 devid)
{
- cmd->raw_cmd[0] &= ~(0xffffUL << 32);
+ cmd->raw_cmd[0] &= BIT_ULL(32) - 1;
cmd->raw_cmd[0] |= ((u64)devid) << 32;
}
{
struct its_cmd_block *cmd, *sync_cmd, *next_cmd;
struct its_collection *sync_col;
+ unsigned long flags;
- raw_spin_lock(&its->lock);
+ raw_spin_lock_irqsave(&its->lock, flags);
cmd = its_allocate_entry(its);
if (!cmd) { /* We're soooooo screewed... */
pr_err_ratelimited("ITS can't allocate, dropping command\n");
- raw_spin_unlock(&its->lock);
+ raw_spin_unlock_irqrestore(&its->lock, flags);
return;
}
sync_col = builder(cmd, desc);
post:
next_cmd = its_post_commands(its);
- raw_spin_unlock(&its->lock);
+ raw_spin_unlock_irqrestore(&its->lock, flags);
its_wait_for_range_completion(its, cmd, next_cmd);
}
{
int err;
int i;
- int psz = PAGE_SIZE;
+ int psz = SZ_64K;
u64 shr = GITS_BASER_InnerShareable;
+ u64 cache = GITS_BASER_WaWb;
for (i = 0; i < GITS_BASER_NR_REGS; i++) {
u64 val = readq_relaxed(its->base + GITS_BASER + i * 8);
u64 type = GITS_BASER_TYPE(val);
u64 entry_size = GITS_BASER_ENTRY_SIZE(val);
+ int order = get_order(psz);
+ int alloc_size;
u64 tmp;
void *base;
if (type == GITS_BASER_TYPE_NONE)
continue;
- /* We're lazy and only allocate a single page for now */
- base = (void *)get_zeroed_page(GFP_KERNEL);
+ /*
+ * Allocate as many entries as required to fit the
+ * range of device IDs that the ITS can grok... The ID
+ * space being incredibly sparse, this results in a
+ * massive waste of memory.
+ *
+ * For other tables, only allocate a single page.
+ */
+ if (type == GITS_BASER_TYPE_DEVICE) {
+ u64 typer = readq_relaxed(its->base + GITS_TYPER);
+ u32 ids = GITS_TYPER_DEVBITS(typer);
+
+ order = get_order((1UL << ids) * entry_size);
+ if (order >= MAX_ORDER) {
+ order = MAX_ORDER - 1;
+ pr_warn("%s: Device Table too large, reduce its page order to %u\n",
+ its->msi_chip.of_node->full_name, order);
+ }
+ }
+
+ alloc_size = (1 << order) * PAGE_SIZE;
+ base = (void *)__get_free_pages(GFP_KERNEL | __GFP_ZERO, order);
if (!base) {
err = -ENOMEM;
goto out_free;
val = (virt_to_phys(base) |
(type << GITS_BASER_TYPE_SHIFT) |
((entry_size - 1) << GITS_BASER_ENTRY_SIZE_SHIFT) |
- GITS_BASER_WaWb |
+ cache |
shr |
GITS_BASER_VALID);
break;
}
- val |= (PAGE_SIZE / psz) - 1;
+ val |= (alloc_size / psz) - 1;
writeq_relaxed(val, its->base + GITS_BASER + i * 8);
tmp = readq_relaxed(its->base + GITS_BASER + i * 8);
* Shareability didn't stick. Just use
* whatever the read reported, which is likely
* to be the only thing this redistributor
- * supports.
+ * supports. If that's zero, make it
+ * non-cacheable as well.
*/
shr = tmp & GITS_BASER_SHAREABILITY_MASK;
+ if (!shr)
+ cache = GITS_BASER_nC;
goto retry_baser;
}
}
pr_info("ITS: allocated %d %s @%lx (psz %dK, shr %d)\n",
- (int)(PAGE_SIZE / entry_size),
+ (int)(alloc_size / entry_size),
its_base_type_string[type],
(unsigned long)virt_to_phys(base),
psz / SZ_1K, (int)shr >> GITS_BASER_SHAREABILITY_SHIFT);
tmp = readq_relaxed(rbase + GICR_PROPBASER);
if ((tmp ^ val) & GICR_PROPBASER_SHAREABILITY_MASK) {
+ if (!(tmp & GICR_PROPBASER_SHAREABILITY_MASK)) {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as
+ * well.
+ */
+ val &= ~(GICR_PROPBASER_SHAREABILITY_MASK |
+ GICR_PROPBASER_CACHEABILITY_MASK);
+ val |= GICR_PROPBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PROPBASER);
+ }
pr_info_once("GIC: using cache flushing for LPI property table\n");
gic_rdists->flags |= RDIST_FLAGS_PROPBASE_NEEDS_FLUSHING;
}
/* set PENDBASE */
val = (page_to_phys(pend_page) |
- GICR_PROPBASER_InnerShareable |
- GICR_PROPBASER_WaWb);
+ GICR_PENDBASER_InnerShareable |
+ GICR_PENDBASER_WaWb);
writeq_relaxed(val, rbase + GICR_PENDBASER);
+ tmp = readq_relaxed(rbase + GICR_PENDBASER);
+
+ if (!(tmp & GICR_PENDBASER_SHAREABILITY_MASK)) {
+ /*
+ * The HW reports non-shareable, we must remove the
+ * cacheability attributes as well.
+ */
+ val &= ~(GICR_PENDBASER_SHAREABILITY_MASK |
+ GICR_PENDBASER_CACHEABILITY_MASK);
+ val |= GICR_PENDBASER_nC;
+ writeq_relaxed(val, rbase + GICR_PENDBASER);
+ }
/* Enable LPIs */
val = readl_relaxed(rbase + GICR_CTLR);
* This ITS wants a linear CPU number.
*/
target = readq_relaxed(gic_data_rdist_rd_base() + GICR_TYPER);
- target = GICR_TYPER_CPU_NUMBER(target);
+ target = GICR_TYPER_CPU_NUMBER(target) << 16;
}
/* Perform collection mapping */
static struct its_device *its_find_device(struct its_node *its, u32 dev_id)
{
struct its_device *its_dev = NULL, *tmp;
+ unsigned long flags;
- raw_spin_lock(&its->lock);
+ raw_spin_lock_irqsave(&its->lock, flags);
list_for_each_entry(tmp, &its->its_device_list, entry) {
if (tmp->device_id == dev_id) {
}
}
- raw_spin_unlock(&its->lock);
+ raw_spin_unlock_irqrestore(&its->lock, flags);
return its_dev;
}
{
struct its_device *dev;
unsigned long *lpi_map;
+ unsigned long flags;
void *itt;
int lpi_base;
int nr_lpis;
nr_ites = max(2UL, roundup_pow_of_two(nvecs));
sz = nr_ites * its->ite_size;
sz = max(sz, ITS_ITT_ALIGN) + ITS_ITT_ALIGN - 1;
- itt = kmalloc(sz, GFP_KERNEL);
+ itt = kzalloc(sz, GFP_KERNEL);
lpi_map = its_lpi_alloc_chunks(nvecs, &lpi_base, &nr_lpis);
if (!dev || !itt || !lpi_map) {
dev->device_id = dev_id;
INIT_LIST_HEAD(&dev->entry);
- raw_spin_lock(&its->lock);
+ raw_spin_lock_irqsave(&its->lock, flags);
list_add(&dev->entry, &its->its_device_list);
- raw_spin_unlock(&its->lock);
+ raw_spin_unlock_irqrestore(&its->lock, flags);
/* Bind the device to the first possible CPU */
cpu = cpumask_first(cpu_online_mask);
static void its_free_device(struct its_device *its_dev)
{
- raw_spin_lock(&its_dev->its->lock);
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&its_dev->its->lock, flags);
list_del(&its_dev->entry);
- raw_spin_unlock(&its_dev->its->lock);
+ raw_spin_unlock_irqrestore(&its_dev->its->lock, flags);
kfree(its_dev->itt);
kfree(its_dev);
}
return 0;
}
+struct its_pci_alias {
+ struct pci_dev *pdev;
+ u32 dev_id;
+ u32 count;
+};
+
+static int its_pci_msi_vec_count(struct pci_dev *pdev)
+{
+ int msi, msix;
+
+ msi = max(pci_msi_vec_count(pdev), 0);
+ msix = max(pci_msix_vec_count(pdev), 0);
+
+ return max(msi, msix);
+}
+
+static int its_get_pci_alias(struct pci_dev *pdev, u16 alias, void *data)
+{
+ struct its_pci_alias *dev_alias = data;
+
+ dev_alias->dev_id = alias;
+ if (pdev != dev_alias->pdev)
+ dev_alias->count += its_pci_msi_vec_count(dev_alias->pdev);
+
+ return 0;
+}
+
static int its_msi_prepare(struct irq_domain *domain, struct device *dev,
int nvec, msi_alloc_info_t *info)
{
struct pci_dev *pdev;
struct its_node *its;
- u32 dev_id;
struct its_device *its_dev;
+ struct its_pci_alias dev_alias;
if (!dev_is_pci(dev))
return -EINVAL;
pdev = to_pci_dev(dev);
- dev_id = PCI_DEVID(pdev->bus->number, pdev->devfn);
+ dev_alias.pdev = pdev;
+ dev_alias.count = nvec;
+
+ pci_for_each_dma_alias(pdev, its_get_pci_alias, &dev_alias);
its = domain->parent->host_data;
- its_dev = its_find_device(its, dev_id);
- if (WARN_ON(its_dev))
- return -EINVAL;
+ its_dev = its_find_device(its, dev_alias.dev_id);
+ if (its_dev) {
+ /*
+ * We already have seen this ID, probably through
+ * another alias (PCI bridge of some sort). No need to
+ * create the device.
+ */
+ dev_dbg(dev, "Reusing ITT for devID %x\n", dev_alias.dev_id);
+ goto out;
+ }
- its_dev = its_create_device(its, dev_id, nvec);
+ its_dev = its_create_device(its, dev_alias.dev_id, dev_alias.count);
if (!its_dev)
return -ENOMEM;
- dev_dbg(&pdev->dev, "ITT %d entries, %d bits\n", nvec, ilog2(nvec));
-
+ dev_dbg(&pdev->dev, "ITT %d entries, %d bits\n",
+ dev_alias.count, ilog2(dev_alias.count));
+out:
info->scratchpad[0].ptr = its_dev;
info->scratchpad[1].ptr = dev;
return 0;
.deactivate = its_irq_domain_deactivate,
};
+static int its_force_quiescent(void __iomem *base)
+{
+ u32 count = 1000000; /* 1s */
+ u32 val;
+
+ val = readl_relaxed(base + GITS_CTLR);
+ if (val & GITS_CTLR_QUIESCENT)
+ return 0;
+
+ /* Disable the generation of all interrupts to this ITS */
+ val &= ~GITS_CTLR_ENABLE;
+ writel_relaxed(val, base + GITS_CTLR);
+
+ /* Poll GITS_CTLR and wait until ITS becomes quiescent */
+ while (1) {
+ val = readl_relaxed(base + GITS_CTLR);
+ if (val & GITS_CTLR_QUIESCENT)
+ return 0;
+
+ count--;
+ if (!count)
+ return -EBUSY;
+
+ cpu_relax();
+ udelay(1);
+ }
+}
+
static int its_probe(struct device_node *node, struct irq_domain *parent)
{
struct resource res;
goto out_unmap;
}
+ err = its_force_quiescent(its_base);
+ if (err) {
+ pr_warn("%s: failed to quiesce, giving up\n",
+ node->full_name);
+ goto out_unmap;
+ }
+
pr_info("ITS: %s\n", node->full_name);
its = kzalloc(sizeof(*its), GFP_KERNEL);
writeq_relaxed(baser, its->base + GITS_CBASER);
tmp = readq_relaxed(its->base + GITS_CBASER);
- writeq_relaxed(0, its->base + GITS_CWRITER);
- writel_relaxed(1, its->base + GITS_CTLR);
- if ((tmp ^ baser) & GITS_BASER_SHAREABILITY_MASK) {
+ if ((tmp ^ baser) & GITS_CBASER_SHAREABILITY_MASK) {
+ if (!(tmp & GITS_CBASER_SHAREABILITY_MASK)) {
+ /*
+ * The HW reports non-shareable, we must
+ * remove the cacheability attributes as
+ * well.
+ */
+ baser &= ~(GITS_CBASER_SHAREABILITY_MASK |
+ GITS_CBASER_CACHEABILITY_MASK);
+ baser |= GITS_CBASER_nC;
+ writeq_relaxed(baser, its->base + GITS_CBASER);
+ }
pr_info("ITS: using cache flushing for cmd queue\n");
its->flags |= ITS_FLAGS_CMDQ_NEEDS_FLUSHING;
}
+ writeq_relaxed(0, its->base + GITS_CWRITER);
+ writel_relaxed(GITS_CTLR_ENABLE, its->base + GITS_CTLR);
+
if (of_property_read_bool(its->msi_chip.of_node, "msi-controller")) {
its->domain = irq_domain_add_tree(NULL, &its_domain_ops, its);
if (!its->domain) {
int its_cpu_init(void)
{
- if (!gic_rdists_supports_plpis()) {
- pr_info("CPU%d: LPIs not supported\n", smp_processor_id());
- return -ENXIO;
- }
-
if (!list_empty(&its_nodes)) {
+ if (!gic_rdists_supports_plpis()) {
+ pr_info("CPU%d: LPIs not supported\n", smp_processor_id());
+ return -ENXIO;
+ }
its_cpu_init_lpis();
its_cpu_init_collection();
}
/*
* Routines to disable, enable, EOI and route interrupts
*/
+static int gic_peek_irq(struct irq_data *d, u32 offset)
+{
+ u32 mask = 1 << (gic_irq(d) % 32);
+ void __iomem *base;
+
+ if (gic_irq_in_rdist(d))
+ base = gic_data_rdist_sgi_base();
+ else
+ base = gic_data.dist_base;
+
+ return !!(readl_relaxed(base + offset + (gic_irq(d) / 32) * 4) & mask);
+}
+
static void gic_poke_irq(struct irq_data *d, u32 offset)
{
u32 mask = 1 << (gic_irq(d) % 32);
gic_poke_irq(d, GICD_ISENABLER);
}
+static int gic_irq_set_irqchip_state(struct irq_data *d,
+ enum irqchip_irq_state which, bool val)
+{
+ u32 reg;
+
+ if (d->hwirq >= gic_data.irq_nr) /* PPI/SPI only */
+ return -EINVAL;
+
+ switch (which) {
+ case IRQCHIP_STATE_PENDING:
+ reg = val ? GICD_ISPENDR : GICD_ICPENDR;
+ break;
+
+ case IRQCHIP_STATE_ACTIVE:
+ reg = val ? GICD_ISACTIVER : GICD_ICACTIVER;
+ break;
+
+ case IRQCHIP_STATE_MASKED:
+ reg = val ? GICD_ICENABLER : GICD_ISENABLER;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ gic_poke_irq(d, reg);
+ return 0;
+}
+
+static int gic_irq_get_irqchip_state(struct irq_data *d,
+ enum irqchip_irq_state which, bool *val)
+{
+ if (d->hwirq >= gic_data.irq_nr) /* PPI/SPI only */
+ return -EINVAL;
+
+ switch (which) {
+ case IRQCHIP_STATE_PENDING:
+ *val = gic_peek_irq(d, GICD_ISPENDR);
+ break;
+
+ case IRQCHIP_STATE_ACTIVE:
+ *val = gic_peek_irq(d, GICD_ISACTIVER);
+ break;
+
+ case IRQCHIP_STATE_MASKED:
+ *val = !gic_peek_irq(d, GICD_ISENABLER);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static void gic_eoi_irq(struct irq_data *d)
{
gic_write_eoir(gic_irq(d));
}
#ifdef CONFIG_SMP
-static int gic_peek_irq(struct irq_data *d, u32 offset)
-{
- u32 mask = 1 << (gic_irq(d) % 32);
- void __iomem *base;
-
- if (gic_irq_in_rdist(d))
- base = gic_data_rdist_sgi_base();
- else
- base = gic_data.dist_base;
-
- return !!(readl_relaxed(base + offset + (gic_irq(d) / 32) * 4) & mask);
-}
-
static int gic_secondary_init(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
tlist |= 1 << (mpidr & 0xf);
cpu = cpumask_next(cpu, mask);
- if (cpu == nr_cpu_ids)
+ if (cpu >= nr_cpu_ids)
goto out;
mpidr = cpu_logical_map(cpu);
.irq_eoi = gic_eoi_irq,
.irq_set_type = gic_set_type,
.irq_set_affinity = gic_set_affinity,
+ .irq_get_irqchip_state = gic_irq_get_irqchip_state,
+ .irq_set_irqchip_state = gic_irq_set_irqchip_state,
};
#define GIC_ID_NR (1U << gic_data.rdists.id_bits)
/*
* Routines to acknowledge, disable and enable interrupts
*/
-static void gic_mask_irq(struct irq_data *d)
+static void gic_poke_irq(struct irq_data *d, u32 offset)
{
u32 mask = 1 << (gic_irq(d) % 32);
+ writel_relaxed(mask, gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4);
+}
- raw_spin_lock(&irq_controller_lock);
- writel_relaxed(mask, gic_dist_base(d) + GIC_DIST_ENABLE_CLEAR + (gic_irq(d) / 32) * 4);
+static int gic_peek_irq(struct irq_data *d, u32 offset)
+{
+ u32 mask = 1 << (gic_irq(d) % 32);
+ return !!(readl_relaxed(gic_dist_base(d) + offset + (gic_irq(d) / 32) * 4) & mask);
+}
+
+static void gic_mask_irq(struct irq_data *d)
+{
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&irq_controller_lock, flags);
+ gic_poke_irq(d, GIC_DIST_ENABLE_CLEAR);
if (gic_arch_extn.irq_mask)
gic_arch_extn.irq_mask(d);
- raw_spin_unlock(&irq_controller_lock);
+ raw_spin_unlock_irqrestore(&irq_controller_lock, flags);
}
static void gic_unmask_irq(struct irq_data *d)
{
- u32 mask = 1 << (gic_irq(d) % 32);
+ unsigned long flags;
- raw_spin_lock(&irq_controller_lock);
+ raw_spin_lock_irqsave(&irq_controller_lock, flags);
if (gic_arch_extn.irq_unmask)
gic_arch_extn.irq_unmask(d);
- writel_relaxed(mask, gic_dist_base(d) + GIC_DIST_ENABLE_SET + (gic_irq(d) / 32) * 4);
- raw_spin_unlock(&irq_controller_lock);
+ gic_poke_irq(d, GIC_DIST_ENABLE_SET);
+ raw_spin_unlock_irqrestore(&irq_controller_lock, flags);
}
static void gic_eoi_irq(struct irq_data *d)
writel_relaxed(gic_irq(d), gic_cpu_base(d) + GIC_CPU_EOI);
}
+static int gic_irq_set_irqchip_state(struct irq_data *d,
+ enum irqchip_irq_state which, bool val)
+{
+ u32 reg;
+
+ switch (which) {
+ case IRQCHIP_STATE_PENDING:
+ reg = val ? GIC_DIST_PENDING_SET : GIC_DIST_PENDING_CLEAR;
+ break;
+
+ case IRQCHIP_STATE_ACTIVE:
+ reg = val ? GIC_DIST_ACTIVE_SET : GIC_DIST_ACTIVE_CLEAR;
+ break;
+
+ case IRQCHIP_STATE_MASKED:
+ reg = val ? GIC_DIST_ENABLE_CLEAR : GIC_DIST_ENABLE_SET;
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ gic_poke_irq(d, reg);
+ return 0;
+}
+
+static int gic_irq_get_irqchip_state(struct irq_data *d,
+ enum irqchip_irq_state which, bool *val)
+{
+ switch (which) {
+ case IRQCHIP_STATE_PENDING:
+ *val = gic_peek_irq(d, GIC_DIST_PENDING_SET);
+ break;
+
+ case IRQCHIP_STATE_ACTIVE:
+ *val = gic_peek_irq(d, GIC_DIST_ACTIVE_SET);
+ break;
+
+ case IRQCHIP_STATE_MASKED:
+ *val = !gic_peek_irq(d, GIC_DIST_ENABLE_SET);
+ break;
+
+ default:
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int gic_set_type(struct irq_data *d, unsigned int type)
{
void __iomem *base = gic_dist_base(d);
unsigned int gicirq = gic_irq(d);
+ unsigned long flags;
int ret;
/* Interrupt configuration for SGIs can't be changed */
type != IRQ_TYPE_EDGE_RISING)
return -EINVAL;
- raw_spin_lock(&irq_controller_lock);
+ raw_spin_lock_irqsave(&irq_controller_lock, flags);
if (gic_arch_extn.irq_set_type)
gic_arch_extn.irq_set_type(d, type);
ret = gic_configure_irq(gicirq, type, base, NULL);
- raw_spin_unlock(&irq_controller_lock);
+ raw_spin_unlock_irqrestore(&irq_controller_lock, flags);
return ret;
}
void __iomem *reg = gic_dist_base(d) + GIC_DIST_TARGET + (gic_irq(d) & ~3);
unsigned int cpu, shift = (gic_irq(d) % 4) * 8;
u32 val, mask, bit;
+ unsigned long flags;
if (!force)
cpu = cpumask_any_and(mask_val, cpu_online_mask);
if (cpu >= NR_GIC_CPU_IF || cpu >= nr_cpu_ids)
return -EINVAL;
- raw_spin_lock(&irq_controller_lock);
+ raw_spin_lock_irqsave(&irq_controller_lock, flags);
mask = 0xff << shift;
bit = gic_cpu_map[cpu] << shift;
val = readl_relaxed(reg) & ~mask;
writel_relaxed(val | bit, reg);
- raw_spin_unlock(&irq_controller_lock);
+ raw_spin_unlock_irqrestore(&irq_controller_lock, flags);
return IRQ_SET_MASK_OK;
}
.irq_set_affinity = gic_set_affinity,
#endif
.irq_set_wake = gic_set_wake,
+ .irq_get_irqchip_state = gic_irq_get_irqchip_state,
+ .irq_set_irqchip_state = gic_irq_set_irqchip_state,
};
void __init gic_cascade_irq(unsigned int gic_nr, unsigned int irq)
break;
}
- if (!mask)
+ if (!mask && num_possible_cpus() > 1)
pr_crit("GIC CPU mask not found - kernel will fail to boot.\n");
return mask;
irq_domain_set_info(d, irq, hw, &gic_chip, d->host_data,
handle_fasteoi_irq, NULL, NULL);
set_irq_flags(irq, IRQF_VALID | IRQF_PROBE);
-
- gic_routable_irq_domain_ops->map(d, irq, hw);
}
return 0;
}
static void gic_irq_domain_unmap(struct irq_domain *d, unsigned int irq)
{
- gic_routable_irq_domain_ops->unmap(d, irq);
}
static int gic_irq_domain_xlate(struct irq_domain *d,
*out_hwirq = intspec[1] + 16;
/* For SPIs, we need to add 16 more to get the GIC irq ID number */
- if (!intspec[0]) {
- ret = gic_routable_irq_domain_ops->xlate(d, controller,
- intspec,
- intsize,
- out_hwirq,
- out_type);
-
- if (IS_ERR_VALUE(ret))
- return ret;
- }
+ if (!intspec[0])
+ *out_hwirq += 16;
*out_type = intspec[2] & IRQ_TYPE_SENSE_MASK;
.xlate = gic_irq_domain_xlate,
};
-/* Default functions for routable irq domain */
-static int gic_routable_irq_domain_map(struct irq_domain *d, unsigned int irq,
- irq_hw_number_t hw)
-{
- return 0;
-}
-
-static void gic_routable_irq_domain_unmap(struct irq_domain *d,
- unsigned int irq)
+void gic_set_irqchip_flags(unsigned long flags)
{
+ gic_chip.flags |= flags;
}
-static int gic_routable_irq_domain_xlate(struct irq_domain *d,
- struct device_node *controller,
- const u32 *intspec, unsigned int intsize,
- unsigned long *out_hwirq,
- unsigned int *out_type)
-{
- *out_hwirq += 16;
- return 0;
-}
-
-static const struct irq_domain_ops gic_default_routable_irq_domain_ops = {
- .map = gic_routable_irq_domain_map,
- .unmap = gic_routable_irq_domain_unmap,
- .xlate = gic_routable_irq_domain_xlate,
-};
-
-const struct irq_domain_ops *gic_routable_irq_domain_ops =
- &gic_default_routable_irq_domain_ops;
-
void __init gic_init_bases(unsigned int gic_nr, int irq_start,
void __iomem *dist_base, void __iomem *cpu_base,
u32 percpu_offset, struct device_node *node)
irq_hw_number_t hwirq_base;
struct gic_chip_data *gic;
int gic_irqs, irq_base, i;
- int nr_routable_irqs;
BUG_ON(gic_nr >= MAX_GIC_NR);
gic->gic_irqs = gic_irqs;
if (node) { /* DT case */
- const struct irq_domain_ops *ops = &gic_irq_domain_hierarchy_ops;
-
- if (!of_property_read_u32(node, "arm,routable-irqs",
- &nr_routable_irqs)) {
- ops = &gic_irq_domain_ops;
- gic_irqs = nr_routable_irqs;
- }
-
- gic->domain = irq_domain_add_linear(node, gic_irqs, ops, gic);
+ gic->domain = irq_domain_add_linear(node, gic_irqs,
+ &gic_irq_domain_hierarchy_ops,
+ gic);
} else { /* Non-DT case */
/*
* For primary GICs, skip over SGIs.
return (((cycle_t) hi) << 32) + lo;
}
+
+void gic_start_count(void)
+{
+ u32 gicconfig;
+
+ /* Start the counter */
+ gicconfig = gic_read(GIC_REG(SHARED, GIC_SH_CONFIG));
+ gicconfig &= ~(1 << GIC_SH_CONFIG_COUNTSTOP_SHF);
+ gic_write(GIC_REG(SHARED, GIC_SH_CONFIG), gicconfig);
+}
+
+void gic_stop_count(void)
+{
+ u32 gicconfig;
+
+ /* Stop the counter */
+ gicconfig = gic_read(GIC_REG(SHARED, GIC_SH_CONFIG));
+ gicconfig |= 1 << GIC_SH_CONFIG_COUNTSTOP_SHF;
+ gic_write(GIC_REG(SHARED, GIC_SH_CONFIG), gicconfig);
+}
+
#endif
static bool gic_local_irq_is_routable(int intr)
* Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
*/
+#include <linux/clk.h>
#include <linux/init.h>
#include <linux/platform_device.h>
#include <linux/spinlock.h>
#include <linux/slab.h>
#include <linux/module.h>
#include <linux/platform_data/irq-renesas-irqc.h>
+#include <linux/pm_runtime.h>
-#define IRQC_IRQ_MAX 32 /* maximum 32 interrupts per driver instance */
+#define IRQC_IRQ_MAX 32 /* maximum 32 interrupts per driver instance */
-#define IRQC_REQ_STS 0x00
-#define IRQC_EN_STS 0x04
-#define IRQC_EN_SET 0x08
+#define IRQC_REQ_STS 0x00 /* Interrupt Request Status Register */
+#define IRQC_EN_STS 0x04 /* Interrupt Enable Status Register */
+#define IRQC_EN_SET 0x08 /* Interrupt Enable Set Register */
#define IRQC_INT_CPU_BASE(n) (0x000 + ((n) * 0x10))
-#define DETECT_STATUS 0x100
+ /* SYS-CPU vs. RT-CPU */
+#define DETECT_STATUS 0x100 /* IRQn Detect Status Register */
+#define MONITOR 0x104 /* IRQn Signal Level Monitor Register */
+#define HLVL_STS 0x108 /* IRQn High Level Detect Status Register */
+#define LLVL_STS 0x10c /* IRQn Low Level Detect Status Register */
+#define S_R_EDGE_STS 0x110 /* IRQn Sync Rising Edge Detect Status Reg. */
+#define S_F_EDGE_STS 0x114 /* IRQn Sync Falling Edge Detect Status Reg. */
+#define A_R_EDGE_STS 0x118 /* IRQn Async Rising Edge Detect Status Reg. */
+#define A_F_EDGE_STS 0x11c /* IRQn Async Falling Edge Detect Status Reg. */
+#define CHTEN_STS 0x120 /* Chattering Reduction Status Register */
#define IRQC_CONFIG(n) (0x180 + ((n) * 0x04))
+ /* IRQn Configuration Register */
struct irqc_irq {
int hw_irq;
struct platform_device *pdev;
struct irq_chip irq_chip;
struct irq_domain *irq_domain;
+ struct clk *clk;
};
static void irqc_dbg(struct irqc_irq *i, char *str)
struct irqc_priv *p = irq_data_get_irq_chip_data(d);
int hw_irq = irqd_to_hwirq(d);
unsigned char value = irqc_sense[type & IRQ_TYPE_SENSE_MASK];
- unsigned long tmp;
+ u32 tmp;
irqc_dbg(&p->irq[hw_irq], "sense");
return 0;
}
+static int irqc_irq_set_wake(struct irq_data *d, unsigned int on)
+{
+ struct irqc_priv *p = irq_data_get_irq_chip_data(d);
+
+ if (!p->clk)
+ return 0;
+
+ if (on)
+ clk_enable(p->clk);
+ else
+ clk_disable(p->clk);
+
+ return 0;
+}
+
static irqreturn_t irqc_irq_handler(int irq, void *dev_id)
{
struct irqc_irq *i = dev_id;
struct irqc_priv *p = i->p;
- unsigned long bit = BIT(i->hw_irq);
+ u32 bit = BIT(i->hw_irq);
irqc_dbg(i, "demux1");
p->pdev = pdev;
platform_set_drvdata(pdev, p);
+ p->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(p->clk)) {
+ dev_warn(&pdev->dev, "unable to get clock\n");
+ p->clk = NULL;
+ }
+
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_get_sync(&pdev->dev);
+
/* get hold of manadatory IOMEM */
io = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (!io) {
irq_chip->irq_mask = irqc_irq_disable;
irq_chip->irq_unmask = irqc_irq_enable;
irq_chip->irq_set_type = irqc_irq_set_type;
- irq_chip->flags = IRQCHIP_SKIP_SET_WAKE | IRQCHIP_MASK_ON_SUSPEND;
+ irq_chip->irq_set_wake = irqc_irq_set_wake;
+ irq_chip->flags = IRQCHIP_MASK_ON_SUSPEND;
p->irq_domain = irq_domain_add_simple(pdev->dev.of_node,
p->number_of_irqs,
err2:
iounmap(p->iomem);
err1:
+ pm_runtime_put(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
kfree(p);
err0:
return ret;
irq_domain_remove(p->irq_domain);
iounmap(p->iomem);
+ pm_runtime_put(&pdev->dev);
+ pm_runtime_disable(&pdev->dev);
kfree(p);
return 0;
}
--- /dev/null
+/*
+ * Copyright (C) 2014 STMicroelectronics – All Rights Reserved
+ *
+ * Author: Lee Jones <lee.jones@linaro.org>
+ *
+ * This is a re-write of Christophe Kerello's PMU driver.
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#include <dt-bindings/interrupt-controller/irq-st.h>
+#include <linux/err.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/regmap.h>
+#include <linux/slab.h>
+
+#define STIH415_SYSCFG_642 0x0a8
+#define STIH416_SYSCFG_7543 0x87c
+#define STIH407_SYSCFG_5102 0x198
+#define STID127_SYSCFG_734 0x088
+
+#define ST_A9_IRQ_MASK 0x001FFFFF
+#define ST_A9_IRQ_MAX_CHANS 2
+
+#define ST_A9_IRQ_EN_CTI_0 BIT(0)
+#define ST_A9_IRQ_EN_CTI_1 BIT(1)
+#define ST_A9_IRQ_EN_PMU_0 BIT(2)
+#define ST_A9_IRQ_EN_PMU_1 BIT(3)
+#define ST_A9_IRQ_EN_PL310_L2 BIT(4)
+#define ST_A9_IRQ_EN_EXT_0 BIT(5)
+#define ST_A9_IRQ_EN_EXT_1 BIT(6)
+#define ST_A9_IRQ_EN_EXT_2 BIT(7)
+
+#define ST_A9_FIQ_N_SEL(dev, chan) (dev << (8 + (chan * 3)))
+#define ST_A9_IRQ_N_SEL(dev, chan) (dev << (14 + (chan * 3)))
+#define ST_A9_EXTIRQ_INV_SEL(dev) (dev << 20)
+
+struct st_irq_syscfg {
+ struct regmap *regmap;
+ unsigned int syscfg;
+ unsigned int config;
+ bool ext_inverted;
+};
+
+static const struct of_device_id st_irq_syscfg_match[] = {
+ {
+ .compatible = "st,stih415-irq-syscfg",
+ .data = (void *)STIH415_SYSCFG_642,
+ },
+ {
+ .compatible = "st,stih416-irq-syscfg",
+ .data = (void *)STIH416_SYSCFG_7543,
+ },
+ {
+ .compatible = "st,stih407-irq-syscfg",
+ .data = (void *)STIH407_SYSCFG_5102,
+ },
+ {
+ .compatible = "st,stid127-irq-syscfg",
+ .data = (void *)STID127_SYSCFG_734,
+ },
+ {}
+};
+
+static int st_irq_xlate(struct platform_device *pdev,
+ int device, int channel, bool irq)
+{
+ struct st_irq_syscfg *ddata = dev_get_drvdata(&pdev->dev);
+
+ /* Set the device enable bit. */
+ switch (device) {
+ case ST_IRQ_SYSCFG_EXT_0:
+ ddata->config |= ST_A9_IRQ_EN_EXT_0;
+ break;
+ case ST_IRQ_SYSCFG_EXT_1:
+ ddata->config |= ST_A9_IRQ_EN_EXT_1;
+ break;
+ case ST_IRQ_SYSCFG_EXT_2:
+ ddata->config |= ST_A9_IRQ_EN_EXT_2;
+ break;
+ case ST_IRQ_SYSCFG_CTI_0:
+ ddata->config |= ST_A9_IRQ_EN_CTI_0;
+ break;
+ case ST_IRQ_SYSCFG_CTI_1:
+ ddata->config |= ST_A9_IRQ_EN_CTI_1;
+ break;
+ case ST_IRQ_SYSCFG_PMU_0:
+ ddata->config |= ST_A9_IRQ_EN_PMU_0;
+ break;
+ case ST_IRQ_SYSCFG_PMU_1:
+ ddata->config |= ST_A9_IRQ_EN_PMU_1;
+ break;
+ case ST_IRQ_SYSCFG_pl310_L2:
+ ddata->config |= ST_A9_IRQ_EN_PL310_L2;
+ break;
+ case ST_IRQ_SYSCFG_DISABLED:
+ return 0;
+ default:
+ dev_err(&pdev->dev, "Unrecognised device %d\n", device);
+ return -EINVAL;
+ }
+
+ /* Select IRQ/FIQ channel for device. */
+ ddata->config |= irq ?
+ ST_A9_IRQ_N_SEL(device, channel) :
+ ST_A9_FIQ_N_SEL(device, channel);
+
+ return 0;
+}
+
+static int st_irq_syscfg_enable(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ struct st_irq_syscfg *ddata = dev_get_drvdata(&pdev->dev);
+ int channels, ret, i;
+ u32 device, invert;
+
+ channels = of_property_count_u32_elems(np, "st,irq-device");
+ if (channels != ST_A9_IRQ_MAX_CHANS) {
+ dev_err(&pdev->dev, "st,enable-irq-device must have 2 elems\n");
+ return -EINVAL;
+ }
+
+ channels = of_property_count_u32_elems(np, "st,fiq-device");
+ if (channels != ST_A9_IRQ_MAX_CHANS) {
+ dev_err(&pdev->dev, "st,enable-fiq-device must have 2 elems\n");
+ return -EINVAL;
+ }
+
+ for (i = 0; i < ST_A9_IRQ_MAX_CHANS; i++) {
+ of_property_read_u32_index(np, "st,irq-device", i, &device);
+
+ ret = st_irq_xlate(pdev, device, i, true);
+ if (ret)
+ return ret;
+
+ of_property_read_u32_index(np, "st,fiq-device", i, &device);
+
+ ret = st_irq_xlate(pdev, device, i, false);
+ if (ret)
+ return ret;
+ }
+
+ /* External IRQs may be inverted. */
+ of_property_read_u32(np, "st,invert-ext", &invert);
+ ddata->config |= ST_A9_EXTIRQ_INV_SEL(invert);
+
+ return regmap_update_bits(ddata->regmap, ddata->syscfg,
+ ST_A9_IRQ_MASK, ddata->config);
+}
+
+static int st_irq_syscfg_probe(struct platform_device *pdev)
+{
+ struct device_node *np = pdev->dev.of_node;
+ const struct of_device_id *match;
+ struct st_irq_syscfg *ddata;
+
+ ddata = devm_kzalloc(&pdev->dev, sizeof(*ddata), GFP_KERNEL);
+ if (!ddata)
+ return -ENOMEM;
+
+ match = of_match_device(st_irq_syscfg_match, &pdev->dev);
+ if (!match)
+ return -ENODEV;
+
+ ddata->syscfg = (unsigned int)match->data;
+
+ ddata->regmap = syscon_regmap_lookup_by_phandle(np, "st,syscfg");
+ if (IS_ERR(ddata->regmap)) {
+ dev_err(&pdev->dev, "syscfg phandle missing\n");
+ return PTR_ERR(ddata->regmap);
+ }
+
+ dev_set_drvdata(&pdev->dev, ddata);
+
+ return st_irq_syscfg_enable(pdev);
+}
+
+static int st_irq_syscfg_resume(struct device *dev)
+{
+ struct st_irq_syscfg *ddata = dev_get_drvdata(dev);
+
+ return regmap_update_bits(ddata->regmap, ddata->syscfg,
+ ST_A9_IRQ_MASK, ddata->config);
+}
+
+static SIMPLE_DEV_PM_OPS(st_irq_syscfg_pm_ops, NULL, st_irq_syscfg_resume);
+
+static struct platform_driver st_irq_syscfg_driver = {
+ .driver = {
+ .name = "st_irq_syscfg",
+ .pm = &st_irq_syscfg_pm_ops,
+ .of_match_table = st_irq_syscfg_match,
+ },
+ .probe = st_irq_syscfg_probe,
+};
+
+static int __init st_irq_syscfg_init(void)
+{
+ return platform_driver_register(&st_irq_syscfg_driver);
+}
+core_initcall(st_irq_syscfg_init);
--- /dev/null
+/*
+ * Driver code for Tegra's Legacy Interrupt Controller
+ *
+ * Author: Marc Zyngier <marc.zyngier@arm.com>
+ *
+ * Heavily based on the original arch/arm/mach-tegra/irq.c code:
+ * Copyright (C) 2011 Google, Inc.
+ *
+ * Author:
+ * Colin Cross <ccross@android.com>
+ *
+ * Copyright (C) 2010,2013, NVIDIA Corporation
+ *
+ * This software is licensed under the terms of the GNU General Public
+ * License version 2, as published by the Free Software Foundation, and
+ * may be copied, distributed, and modified under those terms.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ *
+ */
+
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/syscore_ops.h>
+
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+
+#include "irqchip.h"
+
+#define ICTLR_CPU_IEP_VFIQ 0x08
+#define ICTLR_CPU_IEP_FIR 0x14
+#define ICTLR_CPU_IEP_FIR_SET 0x18
+#define ICTLR_CPU_IEP_FIR_CLR 0x1c
+
+#define ICTLR_CPU_IER 0x20
+#define ICTLR_CPU_IER_SET 0x24
+#define ICTLR_CPU_IER_CLR 0x28
+#define ICTLR_CPU_IEP_CLASS 0x2C
+
+#define ICTLR_COP_IER 0x30
+#define ICTLR_COP_IER_SET 0x34
+#define ICTLR_COP_IER_CLR 0x38
+#define ICTLR_COP_IEP_CLASS 0x3c
+
+#define TEGRA_MAX_NUM_ICTLRS 6
+
+static unsigned int num_ictlrs;
+
+struct tegra_ictlr_soc {
+ unsigned int num_ictlrs;
+};
+
+static const struct tegra_ictlr_soc tegra20_ictlr_soc = {
+ .num_ictlrs = 4,
+};
+
+static const struct tegra_ictlr_soc tegra30_ictlr_soc = {
+ .num_ictlrs = 5,
+};
+
+static const struct tegra_ictlr_soc tegra210_ictlr_soc = {
+ .num_ictlrs = 6,
+};
+
+static const struct of_device_id ictlr_matches[] = {
+ { .compatible = "nvidia,tegra210-ictlr", .data = &tegra210_ictlr_soc },
+ { .compatible = "nvidia,tegra30-ictlr", .data = &tegra30_ictlr_soc },
+ { .compatible = "nvidia,tegra20-ictlr", .data = &tegra20_ictlr_soc },
+ { }
+};
+
+struct tegra_ictlr_info {
+ void __iomem *base[TEGRA_MAX_NUM_ICTLRS];
+#ifdef CONFIG_PM_SLEEP
+ u32 cop_ier[TEGRA_MAX_NUM_ICTLRS];
+ u32 cop_iep[TEGRA_MAX_NUM_ICTLRS];
+ u32 cpu_ier[TEGRA_MAX_NUM_ICTLRS];
+ u32 cpu_iep[TEGRA_MAX_NUM_ICTLRS];
+
+ u32 ictlr_wake_mask[TEGRA_MAX_NUM_ICTLRS];
+#endif
+};
+
+static struct tegra_ictlr_info *lic;
+
+static inline void tegra_ictlr_write_mask(struct irq_data *d, unsigned long reg)
+{
+ void __iomem *base = d->chip_data;
+ u32 mask;
+
+ mask = BIT(d->hwirq % 32);
+ writel_relaxed(mask, base + reg);
+}
+
+static void tegra_mask(struct irq_data *d)
+{
+ tegra_ictlr_write_mask(d, ICTLR_CPU_IER_CLR);
+ irq_chip_mask_parent(d);
+}
+
+static void tegra_unmask(struct irq_data *d)
+{
+ tegra_ictlr_write_mask(d, ICTLR_CPU_IER_SET);
+ irq_chip_unmask_parent(d);
+}
+
+static void tegra_eoi(struct irq_data *d)
+{
+ tegra_ictlr_write_mask(d, ICTLR_CPU_IEP_FIR_CLR);
+ irq_chip_eoi_parent(d);
+}
+
+static int tegra_retrigger(struct irq_data *d)
+{
+ tegra_ictlr_write_mask(d, ICTLR_CPU_IEP_FIR_SET);
+ return irq_chip_retrigger_hierarchy(d);
+}
+
+#ifdef CONFIG_PM_SLEEP
+static int tegra_set_wake(struct irq_data *d, unsigned int enable)
+{
+ u32 irq = d->hwirq;
+ u32 index, mask;
+
+ index = (irq / 32);
+ mask = BIT(irq % 32);
+ if (enable)
+ lic->ictlr_wake_mask[index] |= mask;
+ else
+ lic->ictlr_wake_mask[index] &= ~mask;
+
+ /*
+ * Do *not* call into the parent, as the GIC doesn't have any
+ * wake-up facility...
+ */
+ return 0;
+}
+
+static int tegra_ictlr_suspend(void)
+{
+ unsigned long flags;
+ unsigned int i;
+
+ local_irq_save(flags);
+ for (i = 0; i < num_ictlrs; i++) {
+ void __iomem *ictlr = lic->base[i];
+
+ /* Save interrupt state */
+ lic->cpu_ier[i] = readl_relaxed(ictlr + ICTLR_CPU_IER);
+ lic->cpu_iep[i] = readl_relaxed(ictlr + ICTLR_CPU_IEP_CLASS);
+ lic->cop_ier[i] = readl_relaxed(ictlr + ICTLR_COP_IER);
+ lic->cop_iep[i] = readl_relaxed(ictlr + ICTLR_COP_IEP_CLASS);
+
+ /* Disable COP interrupts */
+ writel_relaxed(~0ul, ictlr + ICTLR_COP_IER_CLR);
+
+ /* Disable CPU interrupts */
+ writel_relaxed(~0ul, ictlr + ICTLR_CPU_IER_CLR);
+
+ /* Enable the wakeup sources of ictlr */
+ writel_relaxed(lic->ictlr_wake_mask[i], ictlr + ICTLR_CPU_IER_SET);
+ }
+ local_irq_restore(flags);
+
+ return 0;
+}
+
+static void tegra_ictlr_resume(void)
+{
+ unsigned long flags;
+ unsigned int i;
+
+ local_irq_save(flags);
+ for (i = 0; i < num_ictlrs; i++) {
+ void __iomem *ictlr = lic->base[i];
+
+ writel_relaxed(lic->cpu_iep[i],
+ ictlr + ICTLR_CPU_IEP_CLASS);
+ writel_relaxed(~0ul, ictlr + ICTLR_CPU_IER_CLR);
+ writel_relaxed(lic->cpu_ier[i],
+ ictlr + ICTLR_CPU_IER_SET);
+ writel_relaxed(lic->cop_iep[i],
+ ictlr + ICTLR_COP_IEP_CLASS);
+ writel_relaxed(~0ul, ictlr + ICTLR_COP_IER_CLR);
+ writel_relaxed(lic->cop_ier[i],
+ ictlr + ICTLR_COP_IER_SET);
+ }
+ local_irq_restore(flags);
+}
+
+static struct syscore_ops tegra_ictlr_syscore_ops = {
+ .suspend = tegra_ictlr_suspend,
+ .resume = tegra_ictlr_resume,
+};
+
+static void tegra_ictlr_syscore_init(void)
+{
+ register_syscore_ops(&tegra_ictlr_syscore_ops);
+}
+#else
+#define tegra_set_wake NULL
+static inline void tegra_ictlr_syscore_init(void) {}
+#endif
+
+static struct irq_chip tegra_ictlr_chip = {
+ .name = "LIC",
+ .irq_eoi = tegra_eoi,
+ .irq_mask = tegra_mask,
+ .irq_unmask = tegra_unmask,
+ .irq_retrigger = tegra_retrigger,
+ .irq_set_wake = tegra_set_wake,
+ .flags = IRQCHIP_MASK_ON_SUSPEND,
+#ifdef CONFIG_SMP
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+#endif
+};
+
+static int tegra_ictlr_domain_xlate(struct irq_domain *domain,
+ struct device_node *controller,
+ const u32 *intspec,
+ unsigned int intsize,
+ unsigned long *out_hwirq,
+ unsigned int *out_type)
+{
+ if (domain->of_node != controller)
+ return -EINVAL; /* Shouldn't happen, really... */
+ if (intsize != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (intspec[0] != GIC_SPI)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ *out_hwirq = intspec[1];
+ *out_type = intspec[2];
+ return 0;
+}
+
+static int tegra_ictlr_domain_alloc(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs, void *data)
+{
+ struct of_phandle_args *args = data;
+ struct of_phandle_args parent_args;
+ struct tegra_ictlr_info *info = domain->host_data;
+ irq_hw_number_t hwirq;
+ unsigned int i;
+
+ if (args->args_count != 3)
+ return -EINVAL; /* Not GIC compliant */
+ if (args->args[0] != GIC_SPI)
+ return -EINVAL; /* No PPI should point to this domain */
+
+ hwirq = args->args[1];
+ if (hwirq >= (num_ictlrs * 32))
+ return -EINVAL;
+
+ for (i = 0; i < nr_irqs; i++) {
+ int ictlr = (hwirq + i) / 32;
+
+ irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+ &tegra_ictlr_chip,
+ &info->base[ictlr]);
+ }
+
+ parent_args = *args;
+ parent_args.np = domain->parent->of_node;
+ return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, &parent_args);
+}
+
+static void tegra_ictlr_domain_free(struct irq_domain *domain,
+ unsigned int virq,
+ unsigned int nr_irqs)
+{
+ unsigned int i;
+
+ for (i = 0; i < nr_irqs; i++) {
+ struct irq_data *d = irq_domain_get_irq_data(domain, virq + i);
+ irq_domain_reset_irq_data(d);
+ }
+}
+
+static const struct irq_domain_ops tegra_ictlr_domain_ops = {
+ .xlate = tegra_ictlr_domain_xlate,
+ .alloc = tegra_ictlr_domain_alloc,
+ .free = tegra_ictlr_domain_free,
+};
+
+static int __init tegra_ictlr_init(struct device_node *node,
+ struct device_node *parent)
+{
+ struct irq_domain *parent_domain, *domain;
+ const struct of_device_id *match;
+ const struct tegra_ictlr_soc *soc;
+ unsigned int i;
+ int err;
+
+ if (!parent) {
+ pr_err("%s: no parent, giving up\n", node->full_name);
+ return -ENODEV;
+ }
+
+ parent_domain = irq_find_host(parent);
+ if (!parent_domain) {
+ pr_err("%s: unable to obtain parent domain\n", node->full_name);
+ return -ENXIO;
+ }
+
+ match = of_match_node(ictlr_matches, node);
+ if (!match) /* Should never happen... */
+ return -ENODEV;
+
+ soc = match->data;
+
+ lic = kzalloc(sizeof(*lic), GFP_KERNEL);
+ if (!lic)
+ return -ENOMEM;
+
+ for (i = 0; i < TEGRA_MAX_NUM_ICTLRS; i++) {
+ void __iomem *base;
+
+ base = of_iomap(node, i);
+ if (!base)
+ break;
+
+ lic->base[i] = base;
+
+ /* Disable all interrupts */
+ writel_relaxed(~0UL, base + ICTLR_CPU_IER_CLR);
+ /* All interrupts target IRQ */
+ writel_relaxed(0, base + ICTLR_CPU_IEP_CLASS);
+
+ num_ictlrs++;
+ }
+
+ if (!num_ictlrs) {
+ pr_err("%s: no valid regions, giving up\n", node->full_name);
+ err = -ENOMEM;
+ goto out_free;
+ }
+
+ WARN(num_ictlrs != soc->num_ictlrs,
+ "%s: Found %u interrupt controllers in DT; expected %u.\n",
+ node->full_name, num_ictlrs, soc->num_ictlrs);
+
+
+ domain = irq_domain_add_hierarchy(parent_domain, 0, num_ictlrs * 32,
+ node, &tegra_ictlr_domain_ops,
+ lic);
+ if (!domain) {
+ pr_err("%s: failed to allocated domain\n", node->full_name);
+ err = -ENOMEM;
+ goto out_unmap;
+ }
+
+ tegra_ictlr_syscore_init();
+
+ pr_info("%s: %d interrupts forwarded to %s\n",
+ node->full_name, num_ictlrs * 32, parent->full_name);
+
+ return 0;
+
+out_unmap:
+ for (i = 0; i < num_ictlrs; i++)
+ iounmap(lic->base[i]);
+out_free:
+ kfree(lic);
+ return err;
+}
+
+IRQCHIP_DECLARE(tegra20_ictlr, "nvidia,tegra20-ictlr", tegra_ictlr_init);
+IRQCHIP_DECLARE(tegra30_ictlr, "nvidia,tegra30-ictlr", tegra_ictlr_init);
+IRQCHIP_DECLARE(tegra210_ictlr, "nvidia,tegra210-ictlr", tegra_ictlr_init);
--- /dev/null
+/*
+ * Copyright (C) 2014-2015 Toradex AG
+ * Author: Stefan Agner <stefan@agner.ch>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ *
+ *
+ * IRQ chip driver for MSCM interrupt router available on Vybrid SoC's.
+ * The interrupt router is between the CPU's interrupt controller and the
+ * peripheral. The router allows to route the peripheral interrupts to
+ * one of the two available CPU's on Vybrid VF6xx SoC's (Cortex-A5 or
+ * Cortex-M4). The router will be configured transparently on a IRQ
+ * request.
+ *
+ * o All peripheral interrupts of the Vybrid SoC can be routed to
+ * CPU 0, CPU 1 or both. The routing is useful for dual-core
+ * variants of Vybrid SoC such as VF6xx. This driver routes the
+ * requested interrupt to the CPU currently running on.
+ *
+ * o It is required to setup the interrupt router even on single-core
+ * variants of Vybrid.
+ */
+
+#include <linux/cpu_pm.h>
+#include <linux/io.h>
+#include <linux/irq.h>
+#include <linux/irqdomain.h>
+#include <linux/mfd/syscon.h>
+#include <dt-bindings/interrupt-controller/arm-gic.h>
+#include <linux/of.h>
+#include <linux/of_address.h>
+#include <linux/slab.h>
+#include <linux/regmap.h>
+
+#include "irqchip.h"
+
+#define MSCM_CPxNUM 0x4
+
+#define MSCM_IRSPRC(n) (0x80 + 2 * (n))
+#define MSCM_IRSPRC_CPEN_MASK 0x3
+
+#define MSCM_IRSPRC_NUM 112
+
+struct vf610_mscm_ir_chip_data {
+ void __iomem *mscm_ir_base;
+ u16 cpu_mask;
+ u16 saved_irsprc[MSCM_IRSPRC_NUM];
+};
+
+static struct vf610_mscm_ir_chip_data *mscm_ir_data;
+
+static inline void vf610_mscm_ir_save(struct vf610_mscm_ir_chip_data *data)
+{
+ int i;
+
+ for (i = 0; i < MSCM_IRSPRC_NUM; i++)
+ data->saved_irsprc[i] = readw_relaxed(data->mscm_ir_base + MSCM_IRSPRC(i));
+}
+
+static inline void vf610_mscm_ir_restore(struct vf610_mscm_ir_chip_data *data)
+{
+ int i;
+
+ for (i = 0; i < MSCM_IRSPRC_NUM; i++)
+ writew_relaxed(data->saved_irsprc[i], data->mscm_ir_base + MSCM_IRSPRC(i));
+}
+
+static int vf610_mscm_ir_notifier(struct notifier_block *self,
+ unsigned long cmd, void *v)
+{
+ switch (cmd) {
+ case CPU_CLUSTER_PM_ENTER:
+ vf610_mscm_ir_save(mscm_ir_data);
+ break;
+ case CPU_CLUSTER_PM_ENTER_FAILED:
+ case CPU_CLUSTER_PM_EXIT:
+ vf610_mscm_ir_restore(mscm_ir_data);
+ break;
+ }
+
+ return NOTIFY_OK;
+}
+
+static struct notifier_block mscm_ir_notifier_block = {
+ .notifier_call = vf610_mscm_ir_notifier,
+};
+
+static void vf610_mscm_ir_enable(struct irq_data *data)
+{
+ irq_hw_number_t hwirq = data->hwirq;
+ struct vf610_mscm_ir_chip_data *chip_data = data->chip_data;
+ u16 irsprc;
+
+ irsprc = readw_relaxed(chip_data->mscm_ir_base + MSCM_IRSPRC(hwirq));
+ irsprc &= MSCM_IRSPRC_CPEN_MASK;
+
+ WARN_ON(irsprc & ~chip_data->cpu_mask);
+
+ writew_relaxed(chip_data->cpu_mask,
+ chip_data->mscm_ir_base + MSCM_IRSPRC(hwirq));
+
+ irq_chip_unmask_parent(data);
+}
+
+static void vf610_mscm_ir_disable(struct irq_data *data)
+{
+ irq_hw_number_t hwirq = data->hwirq;
+ struct vf610_mscm_ir_chip_data *chip_data = data->chip_data;
+
+ writew_relaxed(0x0, chip_data->mscm_ir_base + MSCM_IRSPRC(hwirq));
+
+ irq_chip_mask_parent(data);
+}
+
+static struct irq_chip vf610_mscm_ir_irq_chip = {
+ .name = "mscm-ir",
+ .irq_mask = irq_chip_mask_parent,
+ .irq_unmask = irq_chip_unmask_parent,
+ .irq_eoi = irq_chip_eoi_parent,
+ .irq_enable = vf610_mscm_ir_enable,
+ .irq_disable = vf610_mscm_ir_disable,
+ .irq_retrigger = irq_chip_retrigger_hierarchy,
+ .irq_set_affinity = irq_chip_set_affinity_parent,
+};
+
+static int vf610_mscm_ir_domain_alloc(struct irq_domain *domain, unsigned int virq,
+ unsigned int nr_irqs, void *arg)
+{
+ int i;
+ irq_hw_number_t hwirq;
+ struct of_phandle_args *irq_data = arg;
+ struct of_phandle_args gic_data;
+
+ if (irq_data->args_count != 2)
+ return -EINVAL;
+
+ hwirq = irq_data->args[0];
+ for (i = 0; i < nr_irqs; i++)
+ irq_domain_set_hwirq_and_chip(domain, virq + i, hwirq + i,
+ &vf610_mscm_ir_irq_chip,
+ domain->host_data);
+
+ gic_data.np = domain->parent->of_node;
+ gic_data.args_count = 3;
+ gic_data.args[0] = GIC_SPI;
+ gic_data.args[1] = irq_data->args[0];
+ gic_data.args[2] = irq_data->args[1];
+ return irq_domain_alloc_irqs_parent(domain, virq, nr_irqs, &gic_data);
+}
+
+static const struct irq_domain_ops mscm_irq_domain_ops = {
+ .xlate = irq_domain_xlate_twocell,
+ .alloc = vf610_mscm_ir_domain_alloc,
+ .free = irq_domain_free_irqs_common,
+};
+
+static int __init vf610_mscm_ir_of_init(struct device_node *node,
+ struct device_node *parent)
+{
+ struct irq_domain *domain, *domain_parent;
+ struct regmap *mscm_cp_regmap;
+ int ret, cpuid;
+
+ domain_parent = irq_find_host(parent);
+ if (!domain_parent) {
+ pr_err("vf610_mscm_ir: interrupt-parent not found\n");
+ return -EINVAL;
+ }
+
+ mscm_ir_data = kzalloc(sizeof(*mscm_ir_data), GFP_KERNEL);
+ if (!mscm_ir_data)
+ return -ENOMEM;
+
+ mscm_ir_data->mscm_ir_base = of_io_request_and_map(node, 0, "mscm-ir");
+
+ if (!mscm_ir_data->mscm_ir_base) {
+ pr_err("vf610_mscm_ir: unable to map mscm register\n");
+ ret = -ENOMEM;
+ goto out_free;
+ }
+
+ mscm_cp_regmap = syscon_regmap_lookup_by_phandle(node, "fsl,cpucfg");
+ if (IS_ERR(mscm_cp_regmap)) {
+ ret = PTR_ERR(mscm_cp_regmap);
+ pr_err("vf610_mscm_ir: regmap lookup for cpucfg failed\n");
+ goto out_unmap;
+ }
+
+ regmap_read(mscm_cp_regmap, MSCM_CPxNUM, &cpuid);
+ mscm_ir_data->cpu_mask = 0x1 << cpuid;
+
+ domain = irq_domain_add_hierarchy(domain_parent, 0,
+ MSCM_IRSPRC_NUM, node,
+ &mscm_irq_domain_ops, mscm_ir_data);
+ if (!domain) {
+ ret = -ENOMEM;
+ goto out_unmap;
+ }
+
+ cpu_pm_register_notifier(&mscm_ir_notifier_block);
+
+ return 0;
+
+out_unmap:
+ iounmap(mscm_ir_data->mscm_ir_base);
+out_free:
+ kfree(mscm_ir_data);
+ return ret;
+}
+IRQCHIP_DECLARE(vf610_mscm_ir, "fsl,vf610-mscm-ir", vf610_mscm_ir_of_init);
enable_hwirq(hc);
spin_unlock_irqrestore(&hc->lock, flags);
/* Timeout 80ms */
- current->state = TASK_UNINTERRUPTIBLE;
+ set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout((80 * HZ) / 1000);
printk(KERN_INFO "HFC PCI: IRQ %d count %d\n",
hc->irq, hc->irqcnt);
if (ints[0] > 1)
membase = (unsigned long)ints[2];
if (str && *str) {
- strcpy(sid, str);
+ strlcpy(sid, str, sizeof(sid));
icn_id = sid;
if ((p = strchr(sid, ','))) {
*p++ = 0;
config LGUEST
tristate "Linux hypervisor example code"
- depends on X86_32 && EVENTFD && TTY
+ depends on X86_32 && EVENTFD && TTY && PCI_DIRECT
select HVC_DRIVER
---help---
This is a very simple module which allows you to run
struct request_queue *q = bdev_get_queue(where->bdev);
unsigned short logical_block_size = queue_logical_block_size(q);
sector_t num_sectors;
+ unsigned int uninitialized_var(special_cmd_max_sectors);
- /* Reject unsupported discard requests */
- if ((rw & REQ_DISCARD) && !blk_queue_discard(q)) {
+ /*
+ * Reject unsupported discard and write same requests.
+ */
+ if (rw & REQ_DISCARD)
+ special_cmd_max_sectors = q->limits.max_discard_sectors;
+ else if (rw & REQ_WRITE_SAME)
+ special_cmd_max_sectors = q->limits.max_write_same_sectors;
+ if ((rw & (REQ_DISCARD | REQ_WRITE_SAME)) && special_cmd_max_sectors == 0) {
dec_count(io, region, -EOPNOTSUPP);
return;
}
store_io_and_region_in_bio(bio, io, region);
if (rw & REQ_DISCARD) {
- num_sectors = min_t(sector_t, q->limits.max_discard_sectors, remaining);
+ num_sectors = min_t(sector_t, special_cmd_max_sectors, remaining);
bio->bi_iter.bi_size = num_sectors << SECTOR_SHIFT;
remaining -= num_sectors;
} else if (rw & REQ_WRITE_SAME) {
*/
dp->get_page(dp, &page, &len, &offset);
bio_add_page(bio, page, logical_block_size, offset);
- num_sectors = min_t(sector_t, q->limits.max_write_same_sectors, remaining);
+ num_sectors = min_t(sector_t, special_cmd_max_sectors, remaining);
bio->bi_iter.bi_size = num_sectors << SECTOR_SHIFT;
offset = 0;
#include <linux/log2.h>
#include <linux/dm-kcopyd.h>
+#include "dm.h"
+
#include "dm-exception-store.h"
#define DM_MSG_PREFIX "snapshots"
struct list_head snapshots;
};
+/*
+ * This structure is allocated for each origin target
+ */
+struct dm_origin {
+ struct dm_dev *dev;
+ struct dm_target *ti;
+ unsigned split_boundary;
+ struct list_head hash_list;
+};
+
/*
* Size of the hash table for origin volumes. If we make this
* the size of the minors list then it should be nearly perfect
#define ORIGIN_HASH_SIZE 256
#define ORIGIN_MASK 0xFF
static struct list_head *_origins;
+static struct list_head *_dm_origins;
static struct rw_semaphore _origins_lock;
static DECLARE_WAIT_QUEUE_HEAD(_pending_exceptions_done);
_origins = kmalloc(ORIGIN_HASH_SIZE * sizeof(struct list_head),
GFP_KERNEL);
if (!_origins) {
- DMERR("unable to allocate memory");
+ DMERR("unable to allocate memory for _origins");
return -ENOMEM;
}
-
for (i = 0; i < ORIGIN_HASH_SIZE; i++)
INIT_LIST_HEAD(_origins + i);
+
+ _dm_origins = kmalloc(ORIGIN_HASH_SIZE * sizeof(struct list_head),
+ GFP_KERNEL);
+ if (!_dm_origins) {
+ DMERR("unable to allocate memory for _dm_origins");
+ kfree(_origins);
+ return -ENOMEM;
+ }
+ for (i = 0; i < ORIGIN_HASH_SIZE; i++)
+ INIT_LIST_HEAD(_dm_origins + i);
+
init_rwsem(&_origins_lock);
return 0;
static void exit_origin_hash(void)
{
kfree(_origins);
+ kfree(_dm_origins);
}
static unsigned origin_hash(struct block_device *bdev)
list_add_tail(&o->hash_list, sl);
}
+static struct dm_origin *__lookup_dm_origin(struct block_device *origin)
+{
+ struct list_head *ol;
+ struct dm_origin *o;
+
+ ol = &_dm_origins[origin_hash(origin)];
+ list_for_each_entry (o, ol, hash_list)
+ if (bdev_equal(o->dev->bdev, origin))
+ return o;
+
+ return NULL;
+}
+
+static void __insert_dm_origin(struct dm_origin *o)
+{
+ struct list_head *sl = &_dm_origins[origin_hash(o->dev->bdev)];
+ list_add_tail(&o->hash_list, sl);
+}
+
+static void __remove_dm_origin(struct dm_origin *o)
+{
+ list_del(&o->hash_list);
+}
+
/*
* _origins_lock must be held when calling this function.
* Returns number of snapshots registered using the supplied cow device, plus:
static void snapshot_resume(struct dm_target *ti)
{
struct dm_snapshot *s = ti->private;
- struct dm_snapshot *snap_src = NULL, *snap_dest = NULL;
+ struct dm_snapshot *snap_src = NULL, *snap_dest = NULL, *snap_merging = NULL;
+ struct dm_origin *o;
+ struct mapped_device *origin_md = NULL;
+ bool must_restart_merging = false;
down_read(&_origins_lock);
+
+ o = __lookup_dm_origin(s->origin->bdev);
+ if (o)
+ origin_md = dm_table_get_md(o->ti->table);
+ if (!origin_md) {
+ (void) __find_snapshots_sharing_cow(s, NULL, NULL, &snap_merging);
+ if (snap_merging)
+ origin_md = dm_table_get_md(snap_merging->ti->table);
+ }
+ if (origin_md == dm_table_get_md(ti->table))
+ origin_md = NULL;
+ if (origin_md) {
+ if (dm_hold(origin_md))
+ origin_md = NULL;
+ }
+
+ up_read(&_origins_lock);
+
+ if (origin_md) {
+ dm_internal_suspend_fast(origin_md);
+ if (snap_merging && test_bit(RUNNING_MERGE, &snap_merging->state_bits)) {
+ must_restart_merging = true;
+ stop_merge(snap_merging);
+ }
+ }
+
+ down_read(&_origins_lock);
+
(void) __find_snapshots_sharing_cow(s, &snap_src, &snap_dest, NULL);
if (snap_src && snap_dest) {
down_write(&snap_src->lock);
up_write(&snap_dest->lock);
up_write(&snap_src->lock);
}
+
up_read(&_origins_lock);
+ if (origin_md) {
+ if (must_restart_merging)
+ start_merge(snap_merging);
+ dm_internal_resume_fast(origin_md);
+ dm_put(origin_md);
+ }
+
/* Now we have correct chunk size, reregister */
reregister_snapshot(s);
* Origin: maps a linear range of a device, with hooks for snapshotting.
*/
-struct dm_origin {
- struct dm_dev *dev;
- unsigned split_boundary;
-};
-
/*
* Construct an origin mapping: <dev_path>
* The context for an origin is merely a 'struct dm_dev *'
goto bad_open;
}
+ o->ti = ti;
ti->private = o;
ti->num_flush_bios = 1;
static void origin_dtr(struct dm_target *ti)
{
struct dm_origin *o = ti->private;
+
dm_put_device(ti, o->dev);
kfree(o);
}
struct dm_origin *o = ti->private;
o->split_boundary = get_origin_minimum_chunksize(o->dev->bdev);
+
+ down_write(&_origins_lock);
+ __insert_dm_origin(o);
+ up_write(&_origins_lock);
+}
+
+static void origin_postsuspend(struct dm_target *ti)
+{
+ struct dm_origin *o = ti->private;
+
+ down_write(&_origins_lock);
+ __remove_dm_origin(o);
+ up_write(&_origins_lock);
}
static void origin_status(struct dm_target *ti, status_type_t type,
static struct target_type origin_target = {
.name = "snapshot-origin",
- .version = {1, 8, 1},
+ .version = {1, 9, 0},
.module = THIS_MODULE,
.ctr = origin_ctr,
.dtr = origin_dtr,
.map = origin_map,
.resume = origin_resume,
+ .postsuspend = origin_postsuspend,
.status = origin_status,
.merge = origin_merge,
.iterate_devices = origin_iterate_devices,
static struct target_type snapshot_target = {
.name = "snapshot",
- .version = {1, 12, 0},
+ .version = {1, 13, 0},
.module = THIS_MODULE,
.ctr = snapshot_ctr,
.dtr = snapshot_dtr,
static struct target_type merge_target = {
.name = dm_snapshot_merge_target_name,
- .version = {1, 2, 0},
+ .version = {1, 3, 0},
.module = THIS_MODULE,
.ctr = snapshot_ctr,
.dtr = snapshot_dtr,
return DM_MAPIO_REMAPPED;
case -ENODATA:
- if (get_pool_mode(tc->pool) == PM_READ_ONLY) {
- /*
- * This block isn't provisioned, and we have no way
- * of doing so.
- */
- handle_unserviceable_bio(tc->pool, bio);
- cell_defer_no_holder(tc, virt_cell);
- return DM_MAPIO_SUBMITTED;
- }
- /* fall through */
-
case -EWOULDBLOCK:
thin_defer_cell(tc, virt_cell);
return DM_MAPIO_SUBMITTED;
dm_get(md);
atomic_inc(&md->open_count);
-
out:
spin_unlock(&_minor_lock);
static void dm_blk_close(struct gendisk *disk, fmode_t mode)
{
- struct mapped_device *md = disk->private_data;
+ struct mapped_device *md;
spin_lock(&_minor_lock);
+ md = disk->private_data;
+ if (WARN_ON(!md))
+ goto out;
+
if (atomic_dec_and_test(&md->open_count) &&
(test_bit(DMF_DEFERRED_REMOVE, &md->flags)))
queue_work(deferred_remove_workqueue, &deferred_remove_work);
dm_put(md);
-
+out:
spin_unlock(&_minor_lock);
}
int minor = MINOR(disk_devt(md->disk));
unlock_fs(md);
- bdput(md->bdev);
destroy_workqueue(md->wq);
if (md->kworker_task)
mempool_destroy(md->rq_pool);
if (md->bs)
bioset_free(md->bs);
- blk_integrity_unregister(md->disk);
- del_gendisk(md->disk);
+
cleanup_srcu_struct(&md->io_barrier);
free_table_devices(&md->table_devices);
- free_minor(minor);
+ dm_stats_cleanup(&md->stats);
spin_lock(&_minor_lock);
md->disk->private_data = NULL;
spin_unlock(&_minor_lock);
-
+ if (blk_get_integrity(md->disk))
+ blk_integrity_unregister(md->disk);
+ del_gendisk(md->disk);
put_disk(md->disk);
blk_cleanup_queue(md->queue);
- dm_stats_cleanup(&md->stats);
+ bdput(md->bdev);
+ free_minor(minor);
+
module_put(THIS_MODULE);
kfree(md);
}
BUG_ON(test_bit(DMF_FREEING, &md->flags));
}
+int dm_hold(struct mapped_device *md)
+{
+ spin_lock(&_minor_lock);
+ if (test_bit(DMF_FREEING, &md->flags)) {
+ spin_unlock(&_minor_lock);
+ return -EBUSY;
+ }
+ dm_get(md);
+ spin_unlock(&_minor_lock);
+ return 0;
+}
+EXPORT_SYMBOL_GPL(dm_hold);
+
const char *dm_device_name(struct mapped_device *md)
{
return md->name;
might_sleep();
- spin_lock(&_minor_lock);
map = dm_get_live_table(md, &srcu_idx);
+
+ spin_lock(&_minor_lock);
idr_replace(&_minor_idr, MINOR_ALLOCED, MINOR(disk_devt(dm_disk(md))));
set_bit(DMF_FREEING, &md->flags);
spin_unlock(&_minor_lock);
if (dm_request_based(md))
flush_kthread_worker(&md->kworker);
+ /*
+ * Take suspend_lock so that presuspend and postsuspend methods
+ * do not race with internal suspend.
+ */
+ mutex_lock(&md->suspend_lock);
if (!dm_suspended_md(md)) {
dm_table_presuspend_targets(map);
dm_table_postsuspend_targets(map);
}
+ mutex_unlock(&md->suspend_lock);
/* dm_put_live_table must be before msleep, otherwise deadlock is possible */
dm_put_live_table(md, srcu_idx);
flush_workqueue(md->wq);
dm_wait_for_completion(md, TASK_UNINTERRUPTIBLE);
}
+EXPORT_SYMBOL_GPL(dm_internal_suspend_fast);
void dm_internal_resume_fast(struct mapped_device *md)
{
done:
mutex_unlock(&md->suspend_lock);
}
+EXPORT_SYMBOL_GPL(dm_internal_resume_fast);
/*-----------------------------------------------------------------
* Event notification.
const int rw = bio_data_dir(bio);
struct mddev *mddev = q->queuedata;
unsigned int sectors;
+ int cpu;
if (mddev == NULL || mddev->pers == NULL
|| !mddev->ready) {
sectors = bio_sectors(bio);
mddev->pers->make_request(mddev, bio);
- generic_start_io_acct(rw, sectors, &mddev->gendisk->part0);
+ cpu = part_stat_lock();
+ part_stat_inc(cpu, &mddev->gendisk->part0, ios[rw]);
+ part_stat_add(cpu, &mddev->gendisk->part0, sectors[rw], sectors);
+ part_stat_unlock();
if (atomic_dec_and_test(&mddev->active_io) && mddev->suspended)
wake_up(&mddev->sb_wait);
return err ? err : len;
}
static struct rdev_sysfs_entry rdev_state =
-__ATTR(state, S_IRUGO|S_IWUSR, state_show, state_store);
+__ATTR_PREALLOC(state, S_IRUGO|S_IWUSR, state_show, state_store);
static ssize_t
errors_show(struct md_rdev *rdev, char *page)
return err ?: len;
}
static struct md_sysfs_entry md_resync_start =
-__ATTR(resync_start, S_IRUGO|S_IWUSR, resync_start_show, resync_start_store);
+__ATTR_PREALLOC(resync_start, S_IRUGO|S_IWUSR,
+ resync_start_show, resync_start_store);
/*
* The array state can be:
return err ?: len;
}
static struct md_sysfs_entry md_array_state =
-__ATTR(array_state, S_IRUGO|S_IWUSR, array_state_show, array_state_store);
+__ATTR_PREALLOC(array_state, S_IRUGO|S_IWUSR, array_state_show, array_state_store);
static ssize_t
max_corrected_read_errors_show(struct mddev *mddev, char *page) {
}
static struct md_sysfs_entry md_metadata =
-__ATTR(metadata_version, S_IRUGO|S_IWUSR, metadata_show, metadata_store);
+__ATTR_PREALLOC(metadata_version, S_IRUGO|S_IWUSR, metadata_show, metadata_store);
static ssize_t
action_show(struct mddev *mddev, char *page)
}
static struct md_sysfs_entry md_scan_mode =
-__ATTR(sync_action, S_IRUGO|S_IWUSR, action_show, action_store);
+__ATTR_PREALLOC(sync_action, S_IRUGO|S_IWUSR, action_show, action_store);
static ssize_t
last_sync_action_show(struct mddev *mddev, char *page)
return sprintf(page, "%llu / %llu\n", resync, max_sectors);
}
-static struct md_sysfs_entry md_sync_completed = __ATTR_RO(sync_completed);
+static struct md_sysfs_entry md_sync_completed =
+ __ATTR_PREALLOC(sync_completed, S_IRUGO, sync_completed_show, NULL);
static ssize_t
min_sync_show(struct mddev *mddev, char *page)
}
if (err) {
mddev_detach(mddev);
- pers->free(mddev, mddev->private);
+ if (mddev->private)
+ pers->free(mddev, mddev->private);
module_put(pers->owner);
bitmap_destroy(mddev);
return err;
/*
* remaps the bio to the target device. we separate two flows.
- * power 2 flow and a general flow for the sake of perfromance
+ * power 2 flow and a general flow for the sake of performance
*/
static struct md_rdev *map_sector(struct mddev *mddev, struct strip_zone *zone,
sector_t sector, sector_t *sector_offset)
dump_zones(mddev);
ret = md_integrity_register(mddev);
- if (ret)
- raid0_free(mddev, conf);
return ret;
}
split = bio;
}
+ sector = bio->bi_iter.bi_sector;
zone = find_zone(mddev->private, §or);
tmp_dev = map_sector(mddev, zone, sector, §or);
split->bi_bdev = tmp_dev->bdev;
if (test_bit(WriteMostly, &rdev->flags)) {
/* Don't balance among write-mostly, just
* use the first as a last resort */
- if (best_disk < 0) {
+ if (best_dist_disk < 0) {
if (is_badblock(rdev, this_sector, sectors,
&first_bad, &bad_sectors)) {
if (first_bad < this_sector)
best_good_sectors = first_bad - this_sector;
} else
best_good_sectors = sectors;
- best_disk = disk;
+ best_dist_disk = disk;
+ best_pending_disk = disk;
}
continue;
}
schedule_timeout_uninterruptible(1);
}
/* Need to check if array will still be degraded after recovery/resync
- * We don't need to check the 'failed' flag as when that gets set,
- * recovery aborts.
+ * Note in case of > 1 drive failures it's possible we're rebuilding
+ * one drive while leaving another faulty drive in array.
*/
- for (i = 0; i < conf->raid_disks; i++)
- if (conf->disks[i].rdev == NULL)
+ rcu_read_lock();
+ for (i = 0; i < conf->raid_disks; i++) {
+ struct md_rdev *rdev = ACCESS_ONCE(conf->disks[i].rdev);
+
+ if (rdev == NULL || test_bit(Faulty, &rdev->flags))
still_degraded = 1;
+ }
+ rcu_read_unlock();
bitmap_start_sync(mddev->bitmap, sector_nr, &sync_blocks, still_degraded);
dev->regmap_config.max_register = 5 * 0x100,
dev->regmap_config.ranges = regmap_range_cfg,
dev->regmap_config.num_ranges = ARRAY_SIZE(regmap_range_cfg),
- dev->regmap_config.cache_type = REGCACHE_RBTREE,
+ dev->regmap_config.cache_type = REGCACHE_NONE,
dev->regmap = regmap_init(&client->dev, ®map_bus, client,
&dev->regmap_config);
if (IS_ERR(dev->regmap)) {
strlcpy(cap->driver, dev->name, sizeof(cap->driver));
strlcpy(cap->card, cx23885_boards[tsport->dev->board].name,
sizeof(cap->card));
- sprintf(cap->bus_info, "PCI:%s", pci_name(dev->pci));
- cap->capabilities =
- V4L2_CAP_VIDEO_CAPTURE |
- V4L2_CAP_READWRITE |
- V4L2_CAP_STREAMING |
- 0;
+ sprintf(cap->bus_info, "PCIe:%s", pci_name(dev->pci));
+ cap->device_caps = V4L2_CAP_VIDEO_CAPTURE | V4L2_CAP_READWRITE |
+ V4L2_CAP_STREAMING;
if (dev->tuner_type != TUNER_ABSENT)
- cap->capabilities |= V4L2_CAP_TUNER;
+ cap->device_caps |= V4L2_CAP_TUNER;
+ cap->capabilities = cap->device_caps | V4L2_CAP_VBI_CAPTURE |
+ V4L2_CAP_AUDIO | V4L2_CAP_DEVICE_CAPS;
return 0;
}
struct s5p_jpeg_addr jpeg_addr;
u32 pix_size, padding_bytes = 0;
+ jpeg_addr.cb = 0;
+ jpeg_addr.cr = 0;
+
pix_size = ctx->cap_q.w * ctx->cap_q.h;
if (ctx->mode == S5P_JPEG_ENCODE) {
void exynos3250_jpeg_reset(void __iomem *regs)
{
- u32 reg = 0;
+ u32 reg = 1;
int count = 1000;
writel(1, regs + EXYNOS3250_SW_RESET);
q->type = V4L2_BUF_TYPE_VIDEO_OUTPUT_MPLANE;
q->io_modes = VB2_MMAP;
q->drv_priv = &ctx->fh;
+ q->lock = &dev->mfc_mutex;
if (vdev == dev->vfd_dec) {
q->io_modes = VB2_MMAP;
q->ops = get_dec_queue_ops();
/* Offset base used to differentiate between CAPTURE and OUTPUT
* while mmaping */
-#define DST_QUEUE_OFF_BASE (TASK_SIZE / 2)
+#define DST_QUEUE_OFF_BASE (1 << 30)
#define MFC_BANK1_ALLOC_CTX 0
#define MFC_BANK2_ALLOC_CTX 1
void (*write_info)(struct s5p_mfc_ctx *ctx, unsigned int data,
unsigned int ofs);
unsigned int (*read_info)(struct s5p_mfc_ctx *ctx,
- unsigned int ofs);
+ unsigned long ofs);
int (*get_dspl_y_adr)(struct s5p_mfc_dev *dev);
int (*get_dec_y_adr)(struct s5p_mfc_dev *dev);
int (*get_dspl_status)(struct s5p_mfc_dev *dev);
static void s5p_mfc_write_info_v5(struct s5p_mfc_ctx *ctx, unsigned int data,
unsigned int ofs)
{
- writel(data, (volatile void __iomem *)(ctx->shm.virt + ofs));
+ writel(data, (void *)(ctx->shm.virt + ofs));
wmb();
}
static unsigned int s5p_mfc_read_info_v5(struct s5p_mfc_ctx *ctx,
- unsigned int ofs)
+ unsigned long ofs)
{
rmb();
- return readl((volatile void __iomem *)(ctx->shm.virt + ofs));
+ return readl((void *)(ctx->shm.virt + ofs));
}
static void s5p_mfc_dec_calc_dpb_size_v5(struct s5p_mfc_ctx *ctx)
unsigned int ofs)
{
s5p_mfc_clock_on();
- writel(data, (volatile void __iomem *)((unsigned long)ofs));
+ writel(data, (void *)((unsigned long)ofs));
s5p_mfc_clock_off();
}
static unsigned int
-s5p_mfc_read_info_v6(struct s5p_mfc_ctx *ctx, unsigned int ofs)
+s5p_mfc_read_info_v6(struct s5p_mfc_ctx *ctx, unsigned long ofs)
{
int ret;
s5p_mfc_clock_on();
- ret = readl((volatile void __iomem *)((unsigned long)ofs));
+ ret = readl((void *)ofs);
s5p_mfc_clock_off();
return ret;
config VIDEO_SAMSUNG_S5P_HDMI
tristate "Samsung HDMI Driver"
depends on VIDEO_V4L2
+ depends on I2C
depends on VIDEO_SAMSUNG_S5P_TV
select VIDEO_SAMSUNG_S5P_HDMIPHY
help
}
*vdev = sh_veu_videodev;
+ vdev->v4l2_dev = &veu->v4l2_dev;
spin_lock_init(&veu->lock);
mutex_init(&veu->fop_lock);
vdev->lock = &veu->fop_lock;
if (isi->pdata.full_mode)
cfg1 |= ISI_CFG1_FULL_MODE;
+ cfg1 |= ISI_CFG1_THMASK_BEATS_16;
+
isi_writel(isi, ISI_CTRL, ISI_CTRL_DIS);
isi_writel(isi, ISI_CFG1, cfg1);
eaddpdev:
platform_device_put(sasc->pdev);
eallocpdev:
- devm_kfree(ici->v4l2_dev.dev, sasc);
+ devm_kfree(ici->v4l2_dev.dev, info);
dev_err(ici->v4l2_dev.dev, "group probe failed: %d\n", ret);
return ret;
case TUNER_RTL2832_TUA9001:
return rtl2832u_tua9001_tuner_callback(d, cmd, arg);
}
- default:
- return -EINVAL;
}
return 0;
menuconfig USB_GSPCA
tristate "GSPCA based webcams"
depends on VIDEO_V4L2
+ depends on INPUT || INPUT=n
default m
---help---
Say Y here if you want to enable selecting webcams based
if (threadio == NULL)
return 0;
- call_void_qop(q, wait_finish, q);
threadio->stop = true;
- vb2_internal_streamoff(q, q->type);
- call_void_qop(q, wait_prepare, q);
+ /* Wake up all pending sleeps in the thread */
+ vb2_queue_error(q);
err = kthread_stop(threadio->thread);
- q->fileio = NULL;
- fileio->req.count = 0;
- vb2_reqbufs(q, &fileio->req);
- kfree(fileio);
+ __vb2_cleanup_fileio(q);
threadio->thread = NULL;
kfree(threadio);
- q->fileio = NULL;
q->threadio = NULL;
return err;
}
}
/* extract page list from userspace mapping */
- ret = vb2_dc_get_user_pages(start, pages, n_pages, vma,
- dma_dir == DMA_FROM_DEVICE);
+ ret = vb2_dc_get_user_pages(start, pages, n_pages, vma, dma_dir);
if (ret) {
unsigned long pfn;
if (vb2_dc_get_user_pfn(start, n_pages, vma, &pfn) == 0) {
if (!psy)
continue;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_ONLINE,
+ &val);
+ power_supply_put(psy);
if (!ret && val.intval) {
charger_present = true;
/* Check if battery is known */
psy = power_supply_get_by_name("ab8500_btemp");
if (psy) {
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_TECHNOLOGY,
- &val);
+ ret = power_supply_get_property(psy,
+ POWER_SUPPLY_PROP_TECHNOLOGY, &val);
if (!ret && val.intval != POWER_SUPPLY_TECHNOLOGY_UNKNOWN) {
printk(KERN_INFO
"Charger \"%s\" is connected with known battery."
pss[i]);
machine_restart("charging");
}
+ power_supply_put(psy);
}
shutdown:
},
};
-static struct resource axp288_battery_resources[] = {
+static struct resource axp288_fuel_gauge_resources[] = {
{
.start = AXP288_IRQ_QWBTU,
.end = AXP288_IRQ_QWBTU,
.resources = axp288_charger_resources,
},
{
- .name = "axp288_battery",
- .num_resources = ARRAY_SIZE(axp288_battery_resources),
- .resources = axp288_battery_resources,
+ .name = "axp288_fuel_gauge",
+ .num_resources = ARRAY_SIZE(axp288_fuel_gauge_resources),
+ .resources = axp288_fuel_gauge_resources,
},
{
.name = "axp288_pmic_acpi",
for (id = kempld_dmi_table;
id->matches[0].slot != DMI_NONE; id++)
if (strstr(id->ident, force_device_id))
- if (id->callback && id->callback(id))
+ if (id->callback && !id->callback(id))
break;
if (id->matches[0].slot == DMI_NONE)
return -ENODEV;
int rtsx_usb_ep0_read_register(struct rtsx_ucr *ucr, u16 addr, u8 *data)
{
u16 value;
+ u8 *buf;
+ int ret;
if (!data)
return -EINVAL;
- *data = 0;
+
+ buf = kzalloc(sizeof(u8), GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
addr |= EP0_READ_REG_CMD << EP0_OP_SHIFT;
value = swab16(addr);
- return usb_control_msg(ucr->pusb_dev,
+ ret = usb_control_msg(ucr->pusb_dev,
usb_rcvctrlpipe(ucr->pusb_dev, 0), RTSX_USB_REQ_REG_OP,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- value, 0, data, 1, 100);
+ value, 0, buf, 1, 100);
+ *data = *buf;
+
+ kfree(buf);
+ return ret;
}
EXPORT_SYMBOL_GPL(rtsx_usb_ep0_read_register);
int rtsx_usb_get_card_status(struct rtsx_ucr *ucr, u16 *status)
{
int ret;
+ u16 *buf;
if (!status)
return -EINVAL;
- if (polling_pipe == 0)
+ if (polling_pipe == 0) {
+ buf = kzalloc(sizeof(u16), GFP_KERNEL);
+ if (!buf)
+ return -ENOMEM;
+
ret = usb_control_msg(ucr->pusb_dev,
usb_rcvctrlpipe(ucr->pusb_dev, 0),
RTSX_USB_REQ_POLL,
USB_DIR_IN | USB_TYPE_VENDOR | USB_RECIP_DEVICE,
- 0, 0, status, 2, 100);
- else
+ 0, 0, buf, 2, 100);
+ *status = *buf;
+
+ kfree(buf);
+ } else {
ret = rtsx_usb_get_status_with_bulk(ucr, status);
+ }
/* usb_control_msg may return positive when success */
if (ret < 0)
{
char name[ENCLOSURE_NAME_SIZE];
+ enclosure_link_name(cdev, name);
+
/*
* In odd circumstances, like multipath devices, something else may
* already have removed the links, so check for this condition first.
*/
- if (!cdev->dev->kobj.sd)
- return;
+ if (cdev->dev->kobj.sd)
+ sysfs_remove_link(&cdev->dev->kobj, name);
- enclosure_link_name(cdev, name);
- sysfs_remove_link(&cdev->dev->kobj, name);
- sysfs_remove_link(&cdev->cdev.kobj, "device");
+ if (cdev->cdev.kobj.sd)
+ sysfs_remove_link(&cdev->cdev.kobj, "device");
}
static int enclosure_add_links(struct enclosure_component *cdev)
dev->dev_state = MEI_DEV_POWER_DOWN;
mei_reset(dev);
+ /* move device to disabled state unconditionally */
+ dev->dev_state = MEI_DEV_DISABLED;
mutex_unlock(&dev->device_lock);
if (((die_args->trapnr == X86_TRAP_MF) ||
(die_args->trapnr == X86_TRAP_XF)) &&
- !user_mode_vm(die_args->regs))
+ !user_mode(die_args->regs))
xpc_die_deactivate();
break;
DECLARE_WAITQUEUE(wait, current);
unsigned long flags;
int stop;
+ bool pm = false;
might_sleep();
host->claimed = 1;
host->claimer = current;
host->claim_cnt += 1;
+ if (host->claim_cnt == 1)
+ pm = true;
} else
wake_up(&host->wq);
spin_unlock_irqrestore(&host->lock, flags);
remove_wait_queue(&host->wq, &wait);
- if (host->ops->enable && !stop && host->claim_cnt == 1)
- host->ops->enable(host);
+
+ if (pm)
+ pm_runtime_get_sync(mmc_dev(host));
+
return stop;
}
-
EXPORT_SYMBOL(__mmc_claim_host);
/**
WARN_ON(!host->claimed);
- if (host->ops->disable && host->claim_cnt == 1)
- host->ops->disable(host);
-
spin_lock_irqsave(&host->lock, flags);
if (--host->claim_cnt) {
/* Release for nested claim */
host->claimer = NULL;
spin_unlock_irqrestore(&host->lock, flags);
wake_up(&host->wq);
+ pm_runtime_mark_last_busy(mmc_dev(host));
+ pm_runtime_put_autosuspend(mmc_dev(host));
}
}
EXPORT_SYMBOL(mmc_release_host);
*/
#include <linux/err.h>
+#include <linux/of.h>
#include <linux/slab.h>
#include <linux/stat.h>
#include <linux/pm_runtime.h>
{
int err = 0, idx;
unsigned int part_size;
+ struct device_node *np;
+ bool broken_hpi = false;
/* Version is coded in the CSD_STRUCTURE byte in the EXT_CSD register */
card->ext_csd.raw_ext_csd_structure = ext_csd[EXT_CSD_STRUCTURE];
}
}
+ np = mmc_of_find_child_device(card->host, 0);
+ if (np && of_device_is_compatible(np, "mmc-card"))
+ broken_hpi = of_property_read_bool(np, "broken-hpi");
+ of_node_put(np);
+
/*
* The EXT_CSD format is meant to be forward compatible. As long
* as CSD_STRUCTURE does not change, all values for EXT_CSD_REV
}
/* check whether the eMMC card supports HPI */
- if (ext_csd[EXT_CSD_HPI_FEATURES] & 0x1) {
+ if (!broken_hpi && (ext_csd[EXT_CSD_HPI_FEATURES] & 0x1)) {
card->ext_csd.hpi = 1;
if (ext_csd[EXT_CSD_HPI_FEATURES] & 0x2)
card->ext_csd.hpi_cmd = MMC_STOP_TRANSMISSION;
struct mmc_pwrseq_match {
const char *compatible;
- int (*alloc)(struct mmc_host *host, struct device *dev);
+ struct mmc_pwrseq *(*alloc)(struct mmc_host *host, struct device *dev);
};
static struct mmc_pwrseq_match pwrseq_match[] = {
struct platform_device *pdev;
struct device_node *np;
struct mmc_pwrseq_match *match;
+ struct mmc_pwrseq *pwrseq;
int ret = 0;
np = of_parse_phandle(host->parent->of_node, "mmc-pwrseq", 0);
goto err;
}
- ret = match->alloc(host, &pdev->dev);
- if (!ret)
- dev_info(host->parent, "allocated mmc-pwrseq\n");
+ pwrseq = match->alloc(host, &pdev->dev);
+ if (IS_ERR(pwrseq)) {
+ ret = PTR_ERR(host->pwrseq);
+ goto err;
+ }
+
+ host->pwrseq = pwrseq;
+ dev_info(host->parent, "allocated mmc-pwrseq\n");
err:
of_node_put(np);
if (pwrseq && pwrseq->ops && pwrseq->ops->free)
pwrseq->ops->free(host);
+
+ host->pwrseq = NULL;
}
void mmc_pwrseq_power_off(struct mmc_host *host);
void mmc_pwrseq_free(struct mmc_host *host);
-int mmc_pwrseq_simple_alloc(struct mmc_host *host, struct device *dev);
-int mmc_pwrseq_emmc_alloc(struct mmc_host *host, struct device *dev);
+struct mmc_pwrseq *mmc_pwrseq_simple_alloc(struct mmc_host *host,
+ struct device *dev);
+struct mmc_pwrseq *mmc_pwrseq_emmc_alloc(struct mmc_host *host,
+ struct device *dev);
#else
unregister_restart_handler(&pwrseq->reset_nb);
gpiod_put(pwrseq->reset_gpio);
kfree(pwrseq);
- host->pwrseq = NULL;
}
static struct mmc_pwrseq_ops mmc_pwrseq_emmc_ops = {
return NOTIFY_DONE;
}
-int mmc_pwrseq_emmc_alloc(struct mmc_host *host, struct device *dev)
+struct mmc_pwrseq *mmc_pwrseq_emmc_alloc(struct mmc_host *host,
+ struct device *dev)
{
struct mmc_pwrseq_emmc *pwrseq;
int ret = 0;
pwrseq = kzalloc(sizeof(struct mmc_pwrseq_emmc), GFP_KERNEL);
if (!pwrseq)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
pwrseq->reset_gpio = gpiod_get_index(dev, "reset", 0, GPIOD_OUT_LOW);
if (IS_ERR(pwrseq->reset_gpio)) {
register_restart_handler(&pwrseq->reset_nb);
pwrseq->pwrseq.ops = &mmc_pwrseq_emmc_ops;
- host->pwrseq = &pwrseq->pwrseq;
- return 0;
+ return &pwrseq->pwrseq;
free:
kfree(pwrseq);
- return ret;
+ return ERR_PTR(ret);
}
clk_put(pwrseq->ext_clk);
kfree(pwrseq);
- host->pwrseq = NULL;
}
static struct mmc_pwrseq_ops mmc_pwrseq_simple_ops = {
.free = mmc_pwrseq_simple_free,
};
-int mmc_pwrseq_simple_alloc(struct mmc_host *host, struct device *dev)
+struct mmc_pwrseq *mmc_pwrseq_simple_alloc(struct mmc_host *host,
+ struct device *dev)
{
struct mmc_pwrseq_simple *pwrseq;
int i, nr_gpios, ret = 0;
pwrseq = kzalloc(sizeof(struct mmc_pwrseq_simple) + nr_gpios *
sizeof(struct gpio_desc *), GFP_KERNEL);
if (!pwrseq)
- return -ENOMEM;
+ return ERR_PTR(-ENOMEM);
pwrseq->ext_clk = clk_get(dev, "ext_clock");
if (IS_ERR(pwrseq->ext_clk) &&
PTR_ERR(pwrseq->reset_gpios[i]) != -ENOSYS) {
ret = PTR_ERR(pwrseq->reset_gpios[i]);
- while (--i)
+ while (i--)
gpiod_put(pwrseq->reset_gpios[i]);
goto clk_put;
pwrseq->nr_gpios = nr_gpios;
pwrseq->pwrseq.ops = &mmc_pwrseq_simple_ops;
- host->pwrseq = &pwrseq->pwrseq;
- return 0;
+ return &pwrseq->pwrseq;
clk_put:
if (!IS_ERR(pwrseq->ext_clk))
clk_put(pwrseq->ext_clk);
free:
kfree(pwrseq);
- return ret;
+ return ERR_PTR(ret);
}
int err;
if (card->type == MMC_TYPE_SDIO)
- return sdio_enable_wide(card);
-
- if ((card->host->caps & MMC_CAP_4_BIT_DATA) &&
- (card->scr.bus_widths & SD_SCR_BUS_WIDTH_4)) {
+ err = sdio_enable_wide(card);
+ else if ((card->host->caps & MMC_CAP_4_BIT_DATA) &&
+ (card->scr.bus_widths & SD_SCR_BUS_WIDTH_4)) {
err = mmc_app_set_bus_width(card, MMC_BUS_WIDTH_4);
if (err)
return err;
+ err = sdio_enable_wide(card);
+ if (err <= 0)
+ mmc_app_set_bus_width(card, MMC_BUS_WIDTH_1);
} else
return 0;
- err = sdio_enable_wide(card);
- if (err <= 0)
- mmc_app_set_bus_width(card, MMC_BUS_WIDTH_1);
+ if (err > 0) {
+ mmc_set_bus_width(card->host, MMC_BUS_WIDTH_4);
+ err = 0;
+ }
return err;
}
/*
* Switch to wider bus (if supported).
*/
- if (card->host->caps & MMC_CAP_4_BIT_DATA) {
+ if (card->host->caps & MMC_CAP_4_BIT_DATA)
err = sdio_enable_4bit_bus(card);
- if (err > 0) {
- mmc_set_bus_width(card->host, MMC_BUS_WIDTH_4);
- err = 0;
- }
- }
/* Set the driver strength for the card */
sdio_select_driver_type(card);
* Switch to wider bus (if supported).
*/
err = sdio_enable_4bit_bus(card);
- if (err > 0)
- mmc_set_bus_width(card->host, MMC_BUS_WIDTH_4);
- else if (err)
+ if (err)
goto remove;
}
finish:
} else if (mmc_card_keep_power(host) && mmc_card_wake_sdio_irq(host)) {
/* We may have switched to 1-bit mode during suspend */
err = sdio_enable_4bit_bus(host->card);
- if (err > 0) {
- mmc_set_bus_width(host, MMC_BUS_WIDTH_4);
- err = 0;
- }
}
if (!err && host->sdio_irqs) {
config MMC_SDHCI_OF_ESDHC
tristate "SDHCI OF support for the Freescale eSDHC controller"
depends on MMC_SDHCI_PLTFM
- depends on PPC_OF
+ depends on PPC
select MMC_SDHCI_BIG_ENDIAN_32BIT_BYTE_SWAPPER
help
This selects the Freescale eSDHC controller support.
config MMC_SDHCI_OF_HLWD
tristate "SDHCI OF support for the Nintendo Wii SDHCI controllers"
depends on MMC_SDHCI_PLTFM
- depends on PPC_OF
+ depends on PPC
select MMC_SDHCI_BIG_ENDIAN_32BIT_BYTE_SWAPPER
help
This selects the Secure Digital Host Controller Interface (SDHCI)
tristate "Marvell MMP2 SD Host Controller support (PXAV3)"
depends on CLKDEV_LOOKUP
depends on MMC_SDHCI_PLTFM
- depends on ARCH_MMP || COMPILE_TEST
+ depends on ARCH_BERLIN || ARCH_MMP || ARCH_MVEBU || COMPILE_TEST
default CPU_MMP2
help
This selects the Marvell(R) PXAV3 SD Host Controller.
config MMC_SDHCI_SPEAR
tristate "SDHCI support on ST SPEAr platform"
depends on MMC_SDHCI && PLAT_SPEAR
+ depends on OF
help
This selects the Secure Digital Host Controller Interface (SDHCI)
often referrered to as the HSMMC block in some of the ST SPEAR range
If unsure, say N.
+config MMC_SDHCI_IPROC
+ tristate "SDHCI platform support for the iProc SD/MMC Controller"
+ depends on ARCH_BCM_IPROC || COMPILE_TEST
+ depends on MMC_SDHCI_PLTFM
+ default ARCH_BCM_IPROC
+ select MMC_SDHCI_IO_ACCESSORS
+ help
+ This selects the iProc SD/MMC controller.
+
+ If you have an IPROC platform with SD or MMC devices,
+ say Y or M here.
+
+ If unsure, say N.
+
config MMC_MOXART
tristate "MOXART SD/MMC Host Controller support"
depends on ARCH_MOXART && MMC
obj-$(CONFIG_MMC_SDHCI_OF_HLWD) += sdhci-of-hlwd.o
obj-$(CONFIG_MMC_SDHCI_BCM_KONA) += sdhci-bcm-kona.o
obj-$(CONFIG_MMC_SDHCI_BCM2835) += sdhci-bcm2835.o
+obj-$(CONFIG_MMC_SDHCI_IPROC) += sdhci-iproc.o
obj-$(CONFIG_MMC_SDHCI_MSM) += sdhci-msm.o
obj-$(CONFIG_MMC_SDHCI_ST) += sdhci-st.o
#define ATMCI_REGS_SIZE 0x100
/* Register access macros */
-#define atmci_readl(port,reg) \
+#ifdef CONFIG_AVR32
+#define atmci_readl(port, reg) \
__raw_readl((port)->regs + reg)
-#define atmci_writel(port,reg,value) \
+#define atmci_writel(port, reg, value) \
__raw_writel((value), (port)->regs + reg)
+#else
+#define atmci_readl(port, reg) \
+ readl_relaxed((port)->regs + reg)
+#define atmci_writel(port, reg, value) \
+ writel_relaxed((value), (port)->regs + reg)
+#endif
/* On AVR chips the Peripheral DMA Controller is not connected to MCI. */
#ifdef CONFIG_AVR32
u8 ciu_div;
u32 sdr_timing;
u32 ddr_timing;
+ u32 hs400_timing;
+ u32 tuned_sample;
u32 cur_speed;
+ u32 dqs_delay;
+ u32 saved_dqs_en;
+ u32 saved_strobe_ctrl;
};
static struct dw_mci_exynos_compatible {
},
};
+static inline u8 dw_mci_exynos_get_ciu_div(struct dw_mci *host)
+{
+ struct dw_mci_exynos_priv_data *priv = host->priv;
+
+ if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS4412)
+ return EXYNOS4412_FIXED_CIU_CLK_DIV;
+ else if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS4210)
+ return EXYNOS4210_FIXED_CIU_CLK_DIV;
+ else if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
+ priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
+ return SDMMC_CLKSEL_GET_DIV(mci_readl(host, CLKSEL64)) + 1;
+ else
+ return SDMMC_CLKSEL_GET_DIV(mci_readl(host, CLKSEL)) + 1;
+}
+
static int dw_mci_exynos_priv_init(struct dw_mci *host)
{
struct dw_mci_exynos_priv_data *priv = host->priv;
SDMMC_MPSCTRL_NON_SECURE_WRITE_BIT);
}
+ if (priv->ctrl_type >= DW_MCI_TYPE_EXYNOS5420) {
+ priv->saved_strobe_ctrl = mci_readl(host, HS400_DLINE_CTRL);
+ priv->saved_dqs_en = mci_readl(host, HS400_DQS_EN);
+ priv->saved_dqs_en |= AXI_NON_BLOCKING_WR;
+ mci_writel(host, HS400_DQS_EN, priv->saved_dqs_en);
+ if (!priv->dqs_delay)
+ priv->dqs_delay =
+ DQS_CTRL_GET_RD_DELAY(priv->saved_strobe_ctrl);
+ }
+
return 0;
}
return 0;
}
+static void dw_mci_exynos_set_clksel_timing(struct dw_mci *host, u32 timing)
+{
+ struct dw_mci_exynos_priv_data *priv = host->priv;
+ u32 clksel;
+
+ if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
+ priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
+ clksel = mci_readl(host, CLKSEL64);
+ else
+ clksel = mci_readl(host, CLKSEL);
+
+ clksel = (clksel & ~SDMMC_CLKSEL_TIMING_MASK) | timing;
+
+ if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
+ priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
+ mci_writel(host, CLKSEL64, clksel);
+ else
+ mci_writel(host, CLKSEL, clksel);
+}
+
#ifdef CONFIG_PM_SLEEP
static int dw_mci_exynos_suspend(struct device *dev)
{
}
}
-static void dw_mci_exynos_set_ios(struct dw_mci *host, struct mmc_ios *ios)
+static void dw_mci_exynos_config_hs400(struct dw_mci *host, u32 timing)
{
struct dw_mci_exynos_priv_data *priv = host->priv;
- unsigned int wanted = ios->clock;
- unsigned long actual;
- u8 div = priv->ciu_div + 1;
+ u32 dqs, strobe;
- if (ios->timing == MMC_TIMING_MMC_DDR52) {
- if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
- priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
- mci_writel(host, CLKSEL64, priv->ddr_timing);
- else
- mci_writel(host, CLKSEL, priv->ddr_timing);
- /* Should be double rate for DDR mode */
- if (ios->bus_width == MMC_BUS_WIDTH_8)
- wanted <<= 1;
+ /*
+ * Not supported to configure register
+ * related to HS400
+ */
+ if (priv->ctrl_type < DW_MCI_TYPE_EXYNOS5420)
+ return;
+
+ dqs = priv->saved_dqs_en;
+ strobe = priv->saved_strobe_ctrl;
+
+ if (timing == MMC_TIMING_MMC_HS400) {
+ dqs |= DATA_STROBE_EN;
+ strobe = DQS_CTRL_RD_DELAY(strobe, priv->dqs_delay);
} else {
- if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
- priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
- mci_writel(host, CLKSEL64, priv->sdr_timing);
- else
- mci_writel(host, CLKSEL, priv->sdr_timing);
+ dqs &= ~DATA_STROBE_EN;
}
+ mci_writel(host, HS400_DQS_EN, dqs);
+ mci_writel(host, HS400_DLINE_CTRL, strobe);
+}
+
+static void dw_mci_exynos_adjust_clock(struct dw_mci *host, unsigned int wanted)
+{
+ struct dw_mci_exynos_priv_data *priv = host->priv;
+ unsigned long actual;
+ u8 div;
+ int ret;
/*
* Don't care if wanted clock is zero or
* ciu clock is unavailable
if (wanted < EXYNOS_CCLKIN_MIN)
wanted = EXYNOS_CCLKIN_MIN;
- if (wanted != priv->cur_speed) {
- int ret = clk_set_rate(host->ciu_clk, wanted * div);
- if (ret)
- dev_warn(host->dev,
- "failed to set clk-rate %u error: %d\n",
- wanted * div, ret);
- actual = clk_get_rate(host->ciu_clk);
- host->bus_hz = actual / div;
- priv->cur_speed = wanted;
- host->current_speed = 0;
+ if (wanted == priv->cur_speed)
+ return;
+
+ div = dw_mci_exynos_get_ciu_div(host);
+ ret = clk_set_rate(host->ciu_clk, wanted * div);
+ if (ret)
+ dev_warn(host->dev,
+ "failed to set clk-rate %u error: %d\n",
+ wanted * div, ret);
+ actual = clk_get_rate(host->ciu_clk);
+ host->bus_hz = actual / div;
+ priv->cur_speed = wanted;
+ host->current_speed = 0;
+}
+
+static void dw_mci_exynos_set_ios(struct dw_mci *host, struct mmc_ios *ios)
+{
+ struct dw_mci_exynos_priv_data *priv = host->priv;
+ unsigned int wanted = ios->clock;
+ u32 timing = ios->timing, clksel;
+
+ switch (timing) {
+ case MMC_TIMING_MMC_HS400:
+ /* Update tuned sample timing */
+ clksel = SDMMC_CLKSEL_UP_SAMPLE(
+ priv->hs400_timing, priv->tuned_sample);
+ wanted <<= 1;
+ break;
+ case MMC_TIMING_MMC_DDR52:
+ clksel = priv->ddr_timing;
+ /* Should be double rate for DDR mode */
+ if (ios->bus_width == MMC_BUS_WIDTH_8)
+ wanted <<= 1;
+ break;
+ default:
+ clksel = priv->sdr_timing;
}
+
+ /* Set clock timing for the requested speed mode*/
+ dw_mci_exynos_set_clksel_timing(host, clksel);
+
+ /* Configure setting for HS400 */
+ dw_mci_exynos_config_hs400(host, timing);
+
+ /* Configure clock rate */
+ dw_mci_exynos_adjust_clock(host, wanted);
}
static int dw_mci_exynos_parse_dt(struct dw_mci *host)
return ret;
priv->ddr_timing = SDMMC_CLKSEL_TIMING(timing[0], timing[1], div);
+
+ ret = of_property_read_u32_array(np,
+ "samsung,dw-mshc-hs400-timing", timing, 2);
+ if (!ret && of_property_read_u32(np,
+ "samsung,read-strobe-delay", &priv->dqs_delay))
+ dev_dbg(host->dev,
+ "read-strobe-delay is not found, assuming usage of default value\n");
+
+ priv->hs400_timing = SDMMC_CLKSEL_TIMING(timing[0], timing[1],
+ HS400_FIXED_CIU_CLK_DIV);
host->priv = priv;
return 0;
}
clksel = mci_readl(host, CLKSEL64);
else
clksel = mci_readl(host, CLKSEL);
- clksel = (clksel & ~0x7) | SDMMC_CLKSEL_CCLK_SAMPLE(sample);
+ clksel = SDMMC_CLKSEL_UP_SAMPLE(clksel, sample);
if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
mci_writel(host, CLKSEL64, clksel);
clksel = mci_readl(host, CLKSEL64);
else
clksel = mci_readl(host, CLKSEL);
+
sample = (clksel + 1) & 0x7;
- clksel = (clksel & ~0x7) | sample;
+ clksel = SDMMC_CLKSEL_UP_SAMPLE(clksel, sample);
+
if (priv->ctrl_type == DW_MCI_TYPE_EXYNOS7 ||
priv->ctrl_type == DW_MCI_TYPE_EXYNOS7_SMU)
mci_writel(host, CLKSEL64, clksel);
else
mci_writel(host, CLKSEL, clksel);
+
return sample;
}
static int dw_mci_exynos_execute_tuning(struct dw_mci_slot *slot)
{
struct dw_mci *host = slot->host;
+ struct dw_mci_exynos_priv_data *priv = host->priv;
struct mmc_host *mmc = slot->mmc;
u8 start_smpl, smpl, candiates = 0;
s8 found = -1;
} while (start_smpl != smpl);
found = dw_mci_exynos_get_best_clksmpl(candiates);
- if (found >= 0)
+ if (found >= 0) {
dw_mci_exynos_set_clksmpl(host, found);
- else
+ priv->tuned_sample = found;
+ } else {
ret = -EIO;
+ }
return ret;
}
+static int dw_mci_exynos_prepare_hs400_tuning(struct dw_mci *host,
+ struct mmc_ios *ios)
+{
+ struct dw_mci_exynos_priv_data *priv = host->priv;
+
+ dw_mci_exynos_set_clksel_timing(host, priv->hs400_timing);
+ dw_mci_exynos_adjust_clock(host, (ios->clock) << 1);
+
+ return 0;
+}
+
/* Common capabilities of Exynos4/Exynos5 SoC */
static unsigned long exynos_dwmmc_caps[4] = {
MMC_CAP_1_8V_DDR | MMC_CAP_8_BIT_DATA | MMC_CAP_CMD23,
.set_ios = dw_mci_exynos_set_ios,
.parse_dt = dw_mci_exynos_parse_dt,
.execute_tuning = dw_mci_exynos_execute_tuning,
+ .prepare_hs400_tuning = dw_mci_exynos_prepare_hs400_tuning,
};
static const struct of_device_id dw_mci_exynos_match[] = {
#ifndef _DW_MMC_EXYNOS_H_
#define _DW_MMC_EXYNOS_H_
-/* Extended Register's Offset */
#define SDMMC_CLKSEL 0x09C
#define SDMMC_CLKSEL64 0x0A8
+/* Extended Register's Offset */
+#define SDMMC_HS400_DQS_EN 0x180
+#define SDMMC_HS400_ASYNC_FIFO_CTRL 0x184
+#define SDMMC_HS400_DLINE_CTRL 0x188
+
/* CLKSEL register defines */
#define SDMMC_CLKSEL_CCLK_SAMPLE(x) (((x) & 7) << 0)
#define SDMMC_CLKSEL_CCLK_DRIVE(x) (((x) & 7) << 16)
#define SDMMC_CLKSEL_CCLK_DIVIDER(x) (((x) & 7) << 24)
#define SDMMC_CLKSEL_GET_DRV_WD3(x) (((x) >> 16) & 0x7)
+#define SDMMC_CLKSEL_GET_DIV(x) (((x) >> 24) & 0x7)
+#define SDMMC_CLKSEL_UP_SAMPLE(x, y) (((x) & ~SDMMC_CLKSEL_CCLK_SAMPLE(7)) |\
+ SDMMC_CLKSEL_CCLK_SAMPLE(y))
#define SDMMC_CLKSEL_TIMING(x, y, z) (SDMMC_CLKSEL_CCLK_SAMPLE(x) | \
SDMMC_CLKSEL_CCLK_DRIVE(y) | \
SDMMC_CLKSEL_CCLK_DIVIDER(z))
+#define SDMMC_CLKSEL_TIMING_MASK SDMMC_CLKSEL_TIMING(0x7, 0x7, 0x7)
#define SDMMC_CLKSEL_WAKEUP_INT BIT(11)
+/* RCLK_EN register defines */
+#define DATA_STROBE_EN BIT(0)
+#define AXI_NON_BLOCKING_WR BIT(7)
+
+/* DLINE_CTRL register defines */
+#define DQS_CTRL_RD_DELAY(x, y) (((x) & ~0x3FF) | ((y) & 0x3FF))
+#define DQS_CTRL_GET_RD_DELAY(x) ((x) & 0x3FF)
+
/* Protector Register */
#define SDMMC_EMMCP_BASE 0x1000
#define SDMMC_MPSECURITY (SDMMC_EMMCP_BASE + 0x0010)
/* Fixed clock divider */
#define EXYNOS4210_FIXED_CIU_CLK_DIV 2
#define EXYNOS4412_FIXED_CIU_CLK_DIV 4
+#define HS400_FIXED_CIU_CLK_DIV 1
/* Minimal required clock frequency for cclkin, unit: HZ */
#define EXYNOS_CCLKIN_MIN 50000000
return 0;
}
+/* Common capabilities of RK3288 SoC */
+static unsigned long dw_mci_rk3288_dwmmc_caps[4] = {
+ MMC_CAP_RUNTIME_RESUME, /* emmc */
+ MMC_CAP_RUNTIME_RESUME, /* sdmmc */
+ MMC_CAP_RUNTIME_RESUME, /* sdio0 */
+ MMC_CAP_RUNTIME_RESUME, /* sdio1 */
+};
static const struct dw_mci_drv_data rk2928_drv_data = {
.prepare_command = dw_mci_rockchip_prepare_command,
.init = dw_mci_rockchip_init,
};
static const struct dw_mci_drv_data rk3288_drv_data = {
+ .caps = dw_mci_rk3288_dwmmc_caps,
.prepare_command = dw_mci_rockchip_prepare_command,
.set_ios = dw_mci_rk3288_set_ios,
.setup_clock = dw_mci_rk3288_setup_clock,
u32 des2; /*Buffer sizes */
#define IDMAC_64ADDR_SET_BUFFER1_SIZE(d, s) \
- ((d)->des2 = ((d)->des2 & 0x03ffe000) | ((s) & 0x1fff))
+ ((d)->des2 = ((d)->des2 & cpu_to_le32(0x03ffe000)) | \
+ ((cpu_to_le32(s)) & cpu_to_le32(0x1fff)))
u32 des3; /* Reserved */
};
struct idmac_desc {
- u32 des0; /* Control Descriptor */
+ __le32 des0; /* Control Descriptor */
#define IDMAC_DES0_DIC BIT(1)
#define IDMAC_DES0_LD BIT(2)
#define IDMAC_DES0_FD BIT(3)
#define IDMAC_DES0_CES BIT(30)
#define IDMAC_DES0_OWN BIT(31)
- u32 des1; /* Buffer sizes */
+ __le32 des1; /* Buffer sizes */
#define IDMAC_SET_BUFFER1_SIZE(d, s) \
((d)->des1 = ((d)->des1 & 0x03ffe000) | ((s) & 0x1fff))
- u32 des2; /* buffer 1 physical address */
+ __le32 des2; /* buffer 1 physical address */
- u32 des3; /* buffer 2 physical address */
+ __le32 des3; /* buffer 2 physical address */
};
#endif /* CONFIG_MMC_DW_IDMAC */
static bool dw_mci_reset(struct dw_mci *host);
static bool dw_mci_ctrl_reset(struct dw_mci *host, u32 reset);
+static int dw_mci_card_busy(struct mmc_host *mmc);
#if defined(CONFIG_DEBUG_FS)
static int dw_mci_req_show(struct seq_file *s, void *v)
return cmdr;
}
+static void dw_mci_wait_while_busy(struct dw_mci *host, u32 cmd_flags)
+{
+ unsigned long timeout = jiffies + msecs_to_jiffies(500);
+
+ /*
+ * Databook says that before issuing a new data transfer command
+ * we need to check to see if the card is busy. Data transfer commands
+ * all have SDMMC_CMD_PRV_DAT_WAIT set, so we'll key off that.
+ *
+ * ...also allow sending for SDMMC_CMD_VOLT_SWITCH where busy is
+ * expected.
+ */
+ if ((cmd_flags & SDMMC_CMD_PRV_DAT_WAIT) &&
+ !(cmd_flags & SDMMC_CMD_VOLT_SWITCH)) {
+ while (mci_readl(host, STATUS) & SDMMC_STATUS_BUSY) {
+ if (time_after(jiffies, timeout)) {
+ /* Command will fail; we'll pass error then */
+ dev_err(host->dev, "Busy; trying anyway\n");
+ break;
+ }
+ udelay(10);
+ }
+ }
+}
+
static void dw_mci_start_command(struct dw_mci *host,
struct mmc_command *cmd, u32 cmd_flags)
{
mci_writel(host, CMDARG, cmd->arg);
wmb();
+ dw_mci_wait_while_busy(host, cmd_flags);
mci_writel(host, CMD, cmd_flags | SDMMC_CMD_START);
}
* Set the OWN bit and disable interrupts for this
* descriptor
*/
- desc->des0 = IDMAC_DES0_OWN | IDMAC_DES0_DIC |
- IDMAC_DES0_CH;
+ desc->des0 = cpu_to_le32(IDMAC_DES0_OWN |
+ IDMAC_DES0_DIC | IDMAC_DES0_CH);
/* Buffer length */
IDMAC_SET_BUFFER1_SIZE(desc, length);
/* Physical address to DMA to/from */
- desc->des2 = mem_addr;
+ desc->des2 = cpu_to_le32(mem_addr);
}
/* Set first descriptor */
desc = host->sg_cpu;
- desc->des0 |= IDMAC_DES0_FD;
+ desc->des0 |= cpu_to_le32(IDMAC_DES0_FD);
/* Set last descriptor */
desc = host->sg_cpu + (i - 1) * sizeof(struct idmac_desc);
- desc->des0 &= ~(IDMAC_DES0_CH | IDMAC_DES0_DIC);
- desc->des0 |= IDMAC_DES0_LD;
+ desc->des0 &= cpu_to_le32(~(IDMAC_DES0_CH | IDMAC_DES0_DIC));
+ desc->des0 |= cpu_to_le32(IDMAC_DES0_LD);
}
wmb();
/* Forward link the descriptor list */
for (i = 0, p = host->sg_cpu; i < host->ring_size - 1; i++, p++)
- p->des3 = host->sg_dma + (sizeof(struct idmac_desc) *
- (i + 1));
+ p->des3 = cpu_to_le32(host->sg_dma +
+ (sizeof(struct idmac_desc) * (i + 1)));
/* Set the last descriptor as the end-of-ring descriptor */
- p->des3 = host->sg_dma;
- p->des0 = IDMAC_DES0_ER;
+ p->des3 = cpu_to_le32(host->sg_dma);
+ p->des0 = cpu_to_le32(IDMAC_DES0_ER);
}
dw_mci_idmac_reset(host);
return;
if (host->timing != MMC_TIMING_MMC_HS200 &&
+ host->timing != MMC_TIMING_MMC_HS400 &&
host->timing != MMC_TIMING_UHS_SDR104)
goto disable;
mci_writel(host, CMDARG, arg);
wmb();
+ dw_mci_wait_while_busy(host, cmd);
mci_writel(host, CMD, SDMMC_CMD_START | cmd);
while (time_before(jiffies, timeout)) {
dw_mci_start_command(host, cmd, cmdflags);
+ if (cmd->opcode == SD_SWITCH_VOLTAGE) {
+ unsigned long irqflags;
+
+ /*
+ * Databook says to fail after 2ms w/ no response, but evidence
+ * shows that sometimes the cmd11 interrupt takes over 130ms.
+ * We'll set to 500ms, plus an extra jiffy just in case jiffies
+ * is just about to roll over.
+ *
+ * We do this whole thing under spinlock and only if the
+ * command hasn't already completed (indicating the the irq
+ * already ran so we don't want the timeout).
+ */
+ spin_lock_irqsave(&host->irq_lock, irqflags);
+ if (!test_bit(EVENT_CMD_COMPLETE, &host->pending_events))
+ mod_timer(&host->cmd11_timer,
+ jiffies + msecs_to_jiffies(500) + 1);
+ spin_unlock_irqrestore(&host->irq_lock, irqflags);
+ }
+
if (mrq->stop)
host->stop_cmdr = dw_mci_prepare_command(slot->mmc, mrq->stop);
else
regs = mci_readl(slot->host, UHS_REG);
/* DDR mode set */
- if (ios->timing == MMC_TIMING_MMC_DDR52)
+ if (ios->timing == MMC_TIMING_MMC_DDR52 ||
+ ios->timing == MMC_TIMING_MMC_HS400)
regs |= ((0x1 << slot->id) << 16);
else
regs &= ~((0x1 << slot->id) << 16);
if (drv_data && drv_data->set_ios)
drv_data->set_ios(slot->host, ios);
- /* Slot specific timing and width adjustment */
- dw_mci_setup_bus(slot, false);
-
- if (slot->host->state == STATE_WAITING_CMD11_DONE && ios->clock != 0)
- slot->host->state = STATE_IDLE;
-
switch (ios->power_mode) {
case MMC_POWER_UP:
if (!IS_ERR(mmc->supply.vmmc)) {
mci_writel(slot->host, PWREN, regs);
break;
case MMC_POWER_ON:
- if (!IS_ERR(mmc->supply.vqmmc) && !slot->host->vqmmc_enabled) {
- ret = regulator_enable(mmc->supply.vqmmc);
- if (ret < 0)
- dev_err(slot->host->dev,
- "failed to enable vqmmc regulator\n");
- else
+ if (!slot->host->vqmmc_enabled) {
+ if (!IS_ERR(mmc->supply.vqmmc)) {
+ ret = regulator_enable(mmc->supply.vqmmc);
+ if (ret < 0)
+ dev_err(slot->host->dev,
+ "failed to enable vqmmc\n");
+ else
+ slot->host->vqmmc_enabled = true;
+
+ } else {
+ /* Keep track so we don't reset again */
slot->host->vqmmc_enabled = true;
+ }
+
+ /* Reset our state machine after powering on */
+ dw_mci_ctrl_reset(slot->host,
+ SDMMC_CTRL_ALL_RESET_FLAGS);
}
+
+ /* Adjust clock / bus width after power is up */
+ dw_mci_setup_bus(slot, false);
+
break;
case MMC_POWER_OFF:
+ /* Turn clock off before power goes down */
+ dw_mci_setup_bus(slot, false);
+
if (!IS_ERR(mmc->supply.vmmc))
mmc_regulator_set_ocr(mmc, mmc->supply.vmmc, 0);
- if (!IS_ERR(mmc->supply.vqmmc) && slot->host->vqmmc_enabled) {
+ if (!IS_ERR(mmc->supply.vqmmc) && slot->host->vqmmc_enabled)
regulator_disable(mmc->supply.vqmmc);
- slot->host->vqmmc_enabled = false;
- }
+ slot->host->vqmmc_enabled = false;
regs = mci_readl(slot->host, PWREN);
regs &= ~(1 << slot->id);
default:
break;
}
+
+ if (slot->host->state == STATE_WAITING_CMD11_DONE && ios->clock != 0)
+ slot->host->state = STATE_IDLE;
}
static int dw_mci_card_busy(struct mmc_host *mmc)
return err;
}
+static int dw_mci_prepare_hs400_tuning(struct mmc_host *mmc, struct mmc_ios *ios)
+{
+ struct dw_mci_slot *slot = mmc_priv(mmc);
+ struct dw_mci *host = slot->host;
+ const struct dw_mci_drv_data *drv_data = host->drv_data;
+
+ if (drv_data && drv_data->prepare_hs400_tuning)
+ return drv_data->prepare_hs400_tuning(host, ios);
+
+ return 0;
+}
+
static const struct mmc_host_ops dw_mci_ops = {
.request = dw_mci_request,
.pre_req = dw_mci_pre_req,
.card_busy = dw_mci_card_busy,
.start_signal_voltage_switch = dw_mci_switch_voltage,
.init_card = dw_mci_init_card,
+ .prepare_hs400_tuning = dw_mci_prepare_hs400_tuning,
};
static void dw_mci_request_end(struct dw_mci *host, struct mmc_request *mrq)
if (test_and_clear_bit(EVENT_DATA_ERROR,
&host->pending_events)) {
dw_mci_stop_dma(host);
- send_stop_abort(host, data);
+ if (data->stop ||
+ !(host->data_status & (SDMMC_INT_DRTO |
+ SDMMC_INT_EBE)))
+ send_stop_abort(host, data);
state = STATE_DATA_ERROR;
break;
}
if (test_and_clear_bit(EVENT_DATA_ERROR,
&host->pending_events)) {
dw_mci_stop_dma(host);
- send_stop_abort(host, data);
+ if (data->stop ||
+ !(host->data_status & (SDMMC_INT_DRTO |
+ SDMMC_INT_EBE)))
+ send_stop_abort(host, data);
state = STATE_DATA_ERROR;
break;
}
buf += len;
cnt -= len;
if (host->part_buf_count == 2) {
- mci_writew(host, DATA(host->data_offset),
- host->part_buf16);
+ mci_fifo_writew(host->fifo_reg, host->part_buf16);
host->part_buf_count = 0;
}
}
cnt -= len;
/* push data from aligned buffer into fifo */
for (i = 0; i < items; ++i)
- mci_writew(host, DATA(host->data_offset),
- aligned_buf[i]);
+ mci_fifo_writew(host->fifo_reg, aligned_buf[i]);
}
} else
#endif
{
u16 *pdata = buf;
for (; cnt >= 2; cnt -= 2)
- mci_writew(host, DATA(host->data_offset), *pdata++);
+ mci_fifo_writew(host->fifo_reg, *pdata++);
buf = pdata;
}
/* put anything remaining in the part_buf */
/* Push data if we have reached the expected data length */
if ((data->bytes_xfered + init_cnt) ==
(data->blksz * data->blocks))
- mci_writew(host, DATA(host->data_offset),
- host->part_buf16);
+ mci_fifo_writew(host->fifo_reg, host->part_buf16);
}
}
int items = len >> 1;
int i;
for (i = 0; i < items; ++i)
- aligned_buf[i] = mci_readw(host,
- DATA(host->data_offset));
+ aligned_buf[i] = mci_fifo_readw(host->fifo_reg);
/* memcpy from aligned buffer into output buffer */
memcpy(buf, aligned_buf, len);
buf += len;
{
u16 *pdata = buf;
for (; cnt >= 2; cnt -= 2)
- *pdata++ = mci_readw(host, DATA(host->data_offset));
+ *pdata++ = mci_fifo_readw(host->fifo_reg);
buf = pdata;
}
if (cnt) {
- host->part_buf16 = mci_readw(host, DATA(host->data_offset));
+ host->part_buf16 = mci_fifo_readw(host->fifo_reg);
dw_mci_pull_final_bytes(host, buf, cnt);
}
}
buf += len;
cnt -= len;
if (host->part_buf_count == 4) {
- mci_writel(host, DATA(host->data_offset),
- host->part_buf32);
+ mci_fifo_writel(host->fifo_reg, host->part_buf32);
host->part_buf_count = 0;
}
}
cnt -= len;
/* push data from aligned buffer into fifo */
for (i = 0; i < items; ++i)
- mci_writel(host, DATA(host->data_offset),
- aligned_buf[i]);
+ mci_fifo_writel(host->fifo_reg, aligned_buf[i]);
}
} else
#endif
{
u32 *pdata = buf;
for (; cnt >= 4; cnt -= 4)
- mci_writel(host, DATA(host->data_offset), *pdata++);
+ mci_fifo_writel(host->fifo_reg, *pdata++);
buf = pdata;
}
/* put anything remaining in the part_buf */
/* Push data if we have reached the expected data length */
if ((data->bytes_xfered + init_cnt) ==
(data->blksz * data->blocks))
- mci_writel(host, DATA(host->data_offset),
- host->part_buf32);
+ mci_fifo_writel(host->fifo_reg, host->part_buf32);
}
}
int items = len >> 2;
int i;
for (i = 0; i < items; ++i)
- aligned_buf[i] = mci_readl(host,
- DATA(host->data_offset));
+ aligned_buf[i] = mci_fifo_readl(host->fifo_reg);
/* memcpy from aligned buffer into output buffer */
memcpy(buf, aligned_buf, len);
buf += len;
{
u32 *pdata = buf;
for (; cnt >= 4; cnt -= 4)
- *pdata++ = mci_readl(host, DATA(host->data_offset));
+ *pdata++ = mci_fifo_readl(host->fifo_reg);
buf = pdata;
}
if (cnt) {
- host->part_buf32 = mci_readl(host, DATA(host->data_offset));
+ host->part_buf32 = mci_fifo_readl(host->fifo_reg);
dw_mci_pull_final_bytes(host, buf, cnt);
}
}
cnt -= len;
if (host->part_buf_count == 8) {
- mci_writeq(host, DATA(host->data_offset),
- host->part_buf);
+ mci_fifo_writeq(host->fifo_reg, host->part_buf);
host->part_buf_count = 0;
}
}
cnt -= len;
/* push data from aligned buffer into fifo */
for (i = 0; i < items; ++i)
- mci_writeq(host, DATA(host->data_offset),
- aligned_buf[i]);
+ mci_fifo_writeq(host->fifo_reg, aligned_buf[i]);
}
} else
#endif
{
u64 *pdata = buf;
for (; cnt >= 8; cnt -= 8)
- mci_writeq(host, DATA(host->data_offset), *pdata++);
+ mci_fifo_writeq(host->fifo_reg, *pdata++);
buf = pdata;
}
/* put anything remaining in the part_buf */
/* Push data if we have reached the expected data length */
if ((data->bytes_xfered + init_cnt) ==
(data->blksz * data->blocks))
- mci_writeq(host, DATA(host->data_offset),
- host->part_buf);
+ mci_fifo_writeq(host->fifo_reg, host->part_buf);
}
}
int items = len >> 3;
int i;
for (i = 0; i < items; ++i)
- aligned_buf[i] = mci_readq(host,
- DATA(host->data_offset));
+ aligned_buf[i] = mci_fifo_readq(host->fifo_reg);
+
/* memcpy from aligned buffer into output buffer */
memcpy(buf, aligned_buf, len);
buf += len;
{
u64 *pdata = buf;
for (; cnt >= 8; cnt -= 8)
- *pdata++ = mci_readq(host, DATA(host->data_offset));
+ *pdata++ = mci_fifo_readq(host->fifo_reg);
buf = pdata;
}
if (cnt) {
- host->part_buf = mci_readq(host, DATA(host->data_offset));
+ host->part_buf = mci_fifo_readq(host->fifo_reg);
dw_mci_pull_final_bytes(host, buf, cnt);
}
}
/* Check volt switch first, since it can look like an error */
if ((host->state == STATE_SENDING_CMD11) &&
(pending & SDMMC_INT_VOLT_SWITCH)) {
+ unsigned long irqflags;
+
mci_writel(host, RINTSTS, SDMMC_INT_VOLT_SWITCH);
pending &= ~SDMMC_INT_VOLT_SWITCH;
+
+ /*
+ * Hold the lock; we know cmd11_timer can't be kicked
+ * off after the lock is released, so safe to delete.
+ */
+ spin_lock_irqsave(&host->irq_lock, irqflags);
dw_mci_cmd_interrupt(host, pending);
+ spin_unlock_irqrestore(&host->irq_lock, irqflags);
+
+ del_timer(&host->cmd11_timer);
}
if (pending & DW_MCI_CMD_ERROR_FLAGS) {
/* Handle SDIO Interrupts */
for (i = 0; i < host->num_slots; i++) {
struct dw_mci_slot *slot = host->slot[i];
+
+ if (!slot)
+ continue;
+
if (pending & SDMMC_INT_SDIO(slot->sdio_id)) {
mci_writel(host, RINTSTS,
SDMMC_INT_SDIO(slot->sdio_id));
return ret;
}
+static void dw_mci_cmd11_timer(unsigned long arg)
+{
+ struct dw_mci *host = (struct dw_mci *)arg;
+
+ if (host->state != STATE_SENDING_CMD11) {
+ dev_warn(host->dev, "Unexpected CMD11 timeout\n");
+ return;
+ }
+
+ host->cmd_status = SDMMC_INT_RTO;
+ set_bit(EVENT_CMD_COMPLETE, &host->pending_events);
+ tasklet_schedule(&host->tasklet);
+}
+
#ifdef CONFIG_OF
static struct dw_mci_of_quirks {
char *quirk;
}
#endif /* CONFIG_OF */
+static void dw_mci_enable_cd(struct dw_mci *host)
+{
+ struct dw_mci_board *brd = host->pdata;
+ unsigned long irqflags;
+ u32 temp;
+ int i;
+
+ /* No need for CD if broken card detection */
+ if (brd->quirks & DW_MCI_QUIRK_BROKEN_CARD_DETECTION)
+ return;
+
+ /* No need for CD if all slots have a non-error GPIO */
+ for (i = 0; i < host->num_slots; i++) {
+ struct dw_mci_slot *slot = host->slot[i];
+
+ if (IS_ERR_VALUE(mmc_gpio_get_cd(slot->mmc)))
+ break;
+ }
+ if (i == host->num_slots)
+ return;
+
+ spin_lock_irqsave(&host->irq_lock, irqflags);
+ temp = mci_readl(host, INTMASK);
+ temp |= SDMMC_INT_CD;
+ mci_writel(host, INTMASK, temp);
+ spin_unlock_irqrestore(&host->irq_lock, irqflags);
+}
+
int dw_mci_probe(struct dw_mci *host)
{
const struct dw_mci_drv_data *drv_data = host->drv_data;
}
}
+ setup_timer(&host->cmd11_timer,
+ dw_mci_cmd11_timer, (unsigned long)host);
+
host->quirks = host->pdata->quirks;
spin_lock_init(&host->lock);
dev_info(host->dev, "Version ID is %04x\n", host->verid);
if (host->verid < DW_MMC_240A)
- host->data_offset = DATA_OFFSET;
+ host->fifo_reg = host->regs + DATA_OFFSET;
else
- host->data_offset = DATA_240A_OFFSET;
+ host->fifo_reg = host->regs + DATA_240A_OFFSET;
tasklet_init(&host->tasklet, dw_mci_tasklet_func, (unsigned long)host);
ret = devm_request_irq(host->dev, host->irq, dw_mci_interrupt,
host->num_slots = ((mci_readl(host, HCON) >> 1) & 0x1F) + 1;
/*
- * Enable interrupts for command done, data over, data empty, card det,
+ * Enable interrupts for command done, data over, data empty,
* receive ready and error such as transmit, receive timeout, crc error
*/
mci_writel(host, RINTSTS, 0xFFFFFFFF);
mci_writel(host, INTMASK, SDMMC_INT_CMD_DONE | SDMMC_INT_DATA_OVER |
SDMMC_INT_TXDR | SDMMC_INT_RXDR |
- DW_MCI_ERROR_FLAGS | SDMMC_INT_CD);
+ DW_MCI_ERROR_FLAGS);
mci_writel(host, CTRL, SDMMC_CTRL_INT_ENABLE); /* Enable mci interrupt */
dev_info(host->dev, "DW MMC controller at irq %d, "
goto err_dmaunmap;
}
+ /* Now that slots are all setup, we can enable card detect */
+ dw_mci_enable_cd(host);
+
if (host->quirks & DW_MCI_QUIRK_IDMAC_DTO)
dev_info(host->dev, "Internal DMAC interrupt fix enabled.\n");
mci_writel(host, RINTSTS, 0xFFFFFFFF);
mci_writel(host, INTMASK, SDMMC_INT_CMD_DONE | SDMMC_INT_DATA_OVER |
SDMMC_INT_TXDR | SDMMC_INT_RXDR |
- DW_MCI_ERROR_FLAGS | SDMMC_INT_CD);
+ DW_MCI_ERROR_FLAGS);
mci_writel(host, CTRL, SDMMC_CTRL_INT_ENABLE);
for (i = 0; i < host->num_slots; i++) {
dw_mci_setup_bus(slot, true);
}
}
+
+ /* Now that slots are all setup, we can enable card detect */
+ dw_mci_enable_cd(host);
+
return 0;
}
EXPORT_SYMBOL(dw_mci_resume);
#define SDMMC_CTRL_ALL_RESET_FLAGS \
(SDMMC_CTRL_RESET | SDMMC_CTRL_FIFO_RESET | SDMMC_CTRL_DMA_RESET)
+/* FIFO register access macros. These should not change the data endian-ness
+ * as they are written to memory to be dealt with by the upper layers */
+#define mci_fifo_readw(__reg) __raw_readw(__reg)
+#define mci_fifo_readl(__reg) __raw_readl(__reg)
+#define mci_fifo_readq(__reg) __raw_readq(__reg)
+
+#define mci_fifo_writew(__value, __reg) __raw_writew(__reg, __value)
+#define mci_fifo_writel(__value, __reg) __raw_writel(__reg, __value)
+#define mci_fifo_writeq(__value, __reg) __raw_writeq(__reg, __value)
+
/* Register access macros */
#define mci_readl(dev, reg) \
- __raw_readl((dev)->regs + SDMMC_##reg)
+ readl_relaxed((dev)->regs + SDMMC_##reg)
#define mci_writel(dev, reg, value) \
- __raw_writel((value), (dev)->regs + SDMMC_##reg)
+ writel_relaxed((value), (dev)->regs + SDMMC_##reg)
/* 16-bit FIFO access macros */
#define mci_readw(dev, reg) \
- __raw_readw((dev)->regs + SDMMC_##reg)
+ readw_relaxed((dev)->regs + SDMMC_##reg)
#define mci_writew(dev, reg, value) \
- __raw_writew((value), (dev)->regs + SDMMC_##reg)
+ writew_relaxed((value), (dev)->regs + SDMMC_##reg)
/* 64-bit FIFO access macros */
#ifdef readq
#define mci_readq(dev, reg) \
- __raw_readq((dev)->regs + SDMMC_##reg)
+ readq_relaxed((dev)->regs + SDMMC_##reg)
#define mci_writeq(dev, reg, value) \
- __raw_writeq((value), (dev)->regs + SDMMC_##reg)
+ writeq_relaxed((value), (dev)->regs + SDMMC_##reg)
#else
/*
* Dummy readq implementation for architectures that don't define it.
(*(volatile u64 __force *)((dev)->regs + SDMMC_##reg))
#define mci_writeq(dev, reg, value) \
(*(volatile u64 __force *)((dev)->regs + SDMMC_##reg) = (value))
+
+#define __raw_writeq(__value, __reg) \
+ (*(volatile u64 __force *)(__reg) = (__value))
+#define __raw_readq(__reg) (*(volatile u64 __force *)(__reg))
#endif
extern int dw_mci_probe(struct dw_mci *host);
void (*set_ios)(struct dw_mci *host, struct mmc_ios *ios);
int (*parse_dt)(struct dw_mci *host);
int (*execute_tuning)(struct dw_mci_slot *slot);
+ int (*prepare_hs400_tuning)(struct dw_mci *host,
+ struct mmc_ios *ios);
};
#endif /* _DW_MMC_H_ */
return 0;
}
-static struct of_device_id mmc_spi_of_match_table[] = {
+static const struct of_device_id mmc_spi_of_match_table[] = {
{ .compatible = "mmc-spi-slot", },
{},
};
dev_dbg(mmc_dev(mmc), "clocking block at %u Hz\n", mmc->f_max);
/* Get regulators and the supported OCR mask */
- mmc_regulator_get_supply(mmc);
+ ret = mmc_regulator_get_supply(mmc);
+ if (ret == -EPROBE_DEFER)
+ goto clk_disable;
+
if (!mmc->ocr_avail)
mmc->ocr_avail = plat->ocr_mask;
else if (plat->ocr_mask)
struct omap_hsmmc_next next_data;
struct omap_hsmmc_platform_data *pdata;
- /* To handle board related suspend/resume functionality for MMC */
- int (*suspend)(struct device *dev);
- int (*resume)(struct device *dev);
-
/* return MMC cover switch state, can be NULL if not supported.
*
* possible return values:
*/
int (*get_cover_state)(struct device *dev);
- /* Card detection IRQs */
- int card_detect_irq;
-
int (*card_detect)(struct device *dev);
- int (*get_ro)(struct device *dev);
-
};
struct omap_mmc_of_data {
return mmc_gpio_get_cd(host->mmc);
}
-static int omap_hsmmc_get_wp(struct device *dev)
-{
- struct omap_hsmmc_host *host = dev_get_drvdata(dev);
-
- return mmc_gpio_get_ro(host->mmc);
-}
-
static int omap_hsmmc_get_cover_state(struct device *dev)
{
struct omap_hsmmc_host *host = dev_get_drvdata(dev);
#endif
-static irqreturn_t omap_hsmmc_detect(int irq, void *dev_id);
+static irqreturn_t omap_hsmmc_cover_irq(int irq, void *dev_id);
static int omap_hsmmc_gpio_init(struct mmc_host *mmc,
struct omap_hsmmc_host *host,
{
int ret;
- if (gpio_is_valid(pdata->switch_pin)) {
- if (pdata->cover)
- host->get_cover_state =
- omap_hsmmc_get_cover_state;
- else
- host->card_detect = omap_hsmmc_card_detect;
- host->card_detect_irq =
- gpio_to_irq(pdata->switch_pin);
- mmc_gpio_set_cd_isr(mmc, omap_hsmmc_detect);
- ret = mmc_gpio_request_cd(mmc, pdata->switch_pin, 0);
+ if (gpio_is_valid(pdata->gpio_cod)) {
+ ret = mmc_gpio_request_cd(mmc, pdata->gpio_cod, 0);
if (ret)
return ret;
- } else {
- pdata->switch_pin = -EINVAL;
+
+ host->get_cover_state = omap_hsmmc_get_cover_state;
+ mmc_gpio_set_cd_isr(mmc, omap_hsmmc_cover_irq);
+ } else if (gpio_is_valid(pdata->gpio_cd)) {
+ ret = mmc_gpio_request_cd(mmc, pdata->gpio_cd, 0);
+ if (ret)
+ return ret;
+
+ host->card_detect = omap_hsmmc_card_detect;
}
if (gpio_is_valid(pdata->gpio_wp)) {
- host->get_ro = omap_hsmmc_get_wp;
ret = mmc_gpio_request_ro(mmc, pdata->gpio_wp);
if (ret)
return ret;
- } else {
- pdata->gpio_wp = -EINVAL;
}
return 0;
return;
host->mrq = NULL;
mmc_request_done(host->mmc, mrq);
+ pm_runtime_mark_last_busy(host->dev);
+ pm_runtime_put_autosuspend(host->dev);
}
/*
}
/*
- * irq handler to notify the core about card insertion/removal
+ * irq handler when (cell-phone) cover is mounted/removed
*/
-static irqreturn_t omap_hsmmc_detect(int irq, void *dev_id)
+static irqreturn_t omap_hsmmc_cover_irq(int irq, void *dev_id)
{
struct omap_hsmmc_host *host = dev_id;
- int carddetect;
sysfs_notify(&host->mmc->class_dev.kobj, NULL, "cover_switch");
- if (host->card_detect)
- carddetect = host->card_detect(host->dev);
- else {
- omap_hsmmc_protect_card(host);
- carddetect = -ENOSYS;
- }
-
- if (carddetect)
- mmc_detect_change(host->mmc, (HZ * 200) / 1000);
- else
- mmc_detect_change(host->mmc, (HZ * 50) / 1000);
+ omap_hsmmc_protect_card(host);
+ mmc_detect_change(host->mmc, (HZ * 200) / 1000);
return IRQ_HANDLED;
}
host->mrq = NULL;
mmc_request_done(host->mmc, mrq);
+ pm_runtime_mark_last_busy(host->dev);
+ pm_runtime_put_autosuspend(host->dev);
}
}
BUG_ON(host->req_in_progress);
BUG_ON(host->dma_ch != -1);
+ pm_runtime_get_sync(host->dev);
if (host->protect_card) {
if (host->reqs_blocked < 3) {
/*
req->data->error = -EBADF;
req->cmd->retries = 0;
mmc_request_done(mmc, req);
+ pm_runtime_mark_last_busy(host->dev);
+ pm_runtime_put_autosuspend(host->dev);
return;
} else if (host->reqs_blocked)
host->reqs_blocked = 0;
req->data->error = err;
host->mrq = NULL;
mmc_request_done(mmc, req);
+ pm_runtime_mark_last_busy(host->dev);
+ pm_runtime_put_autosuspend(host->dev);
return;
}
if (req->sbc && !(host->flags & AUTO_CMD23)) {
return host->card_detect(host->dev);
}
-static int omap_hsmmc_get_ro(struct mmc_host *mmc)
-{
- struct omap_hsmmc_host *host = mmc_priv(mmc);
-
- if (!host->get_ro)
- return -ENOSYS;
- return host->get_ro(host->dev);
-}
-
static void omap_hsmmc_init_card(struct mmc_host *mmc, struct mmc_card *card)
{
struct omap_hsmmc_host *host = mmc_priv(mmc);
set_sd_bus_power(host);
}
-static int omap_hsmmc_enable_fclk(struct mmc_host *mmc)
-{
- struct omap_hsmmc_host *host = mmc_priv(mmc);
-
- pm_runtime_get_sync(host->dev);
-
- return 0;
-}
-
-static int omap_hsmmc_disable_fclk(struct mmc_host *mmc)
-{
- struct omap_hsmmc_host *host = mmc_priv(mmc);
-
- pm_runtime_mark_last_busy(host->dev);
- pm_runtime_put_autosuspend(host->dev);
-
- return 0;
-}
-
static int omap_hsmmc_multi_io_quirk(struct mmc_card *card,
unsigned int direction, int blk_size)
{
}
static struct mmc_host_ops omap_hsmmc_ops = {
- .enable = omap_hsmmc_enable_fclk,
- .disable = omap_hsmmc_disable_fclk,
.post_req = omap_hsmmc_post_req,
.pre_req = omap_hsmmc_pre_req,
.request = omap_hsmmc_request,
.set_ios = omap_hsmmc_set_ios,
.get_cd = omap_hsmmc_get_cd,
- .get_ro = omap_hsmmc_get_ro,
+ .get_ro = mmc_gpio_get_ro,
.init_card = omap_hsmmc_init_card,
.enable_sdio_irq = omap_hsmmc_enable_sdio_irq,
};
if (of_find_property(np, "ti,dual-volt", NULL))
pdata->controller_flags |= OMAP_HSMMC_SUPPORTS_DUAL_VOLT;
- pdata->switch_pin = -EINVAL;
+ pdata->gpio_cd = -EINVAL;
+ pdata->gpio_cod = -EINVAL;
pdata->gpio_wp = -EINVAL;
if (of_find_property(np, "ti,non-removable", NULL)) {
if (ret < 0)
goto err_slot_name;
}
- if (host->card_detect_irq && host->get_cover_state) {
+ if (host->get_cover_state) {
ret = device_create_file(&mmc->class_dev,
- &dev_attr_cover_switch);
+ &dev_attr_cover_switch);
if (ret < 0)
goto err_slot_name;
}
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int omap_hsmmc_suspend(struct device *dev)
{
struct omap_hsmmc_host *host = dev_get_drvdata(dev);
pm_runtime_put_autosuspend(host->dev);
return 0;
}
-
-#else
-#define omap_hsmmc_suspend NULL
-#define omap_hsmmc_resume NULL
#endif
static int omap_hsmmc_runtime_suspend(struct device *dev)
}
static struct dev_pm_ops omap_hsmmc_dev_pm_ops = {
- .suspend = omap_hsmmc_suspend,
- .resume = omap_hsmmc_resume,
+ SET_SYSTEM_SLEEP_PM_OPS(omap_hsmmc_suspend, omap_hsmmc_resume)
.runtime_suspend = omap_hsmmc_runtime_suspend,
.runtime_resume = omap_hsmmc_runtime_resume,
};
#include <linux/mmc/host.h>
#include <linux/mmc/pm.h>
#include <linux/mmc/slot-gpio.h>
-#include <linux/mmc/sdhci.h>
#include "sdhci.h"
struct sdhci_bcm_kona_dev {
struct mutex write_lock; /* protect back to back writes */
- struct clk *external_clk;
};
}
}
-/*
- * Get the base clock. Use central clock source for now. Not sure if different
- * clock speed to each dev is allowed
- */
-static unsigned int sdhci_bcm_kona_get_max_clk(struct sdhci_host *host)
-{
- struct sdhci_bcm_kona_dev *kona_dev;
- struct sdhci_pltfm_host *pltfm_priv = sdhci_priv(host);
- kona_dev = sdhci_pltfm_priv(pltfm_priv);
-
- return host->mmc->f_max;
-}
-
-static unsigned int sdhci_bcm_kona_get_timeout_clock(struct sdhci_host *host)
-{
- return sdhci_bcm_kona_get_max_clk(host);
-}
-
static void sdhci_bcm_kona_init_74_clocks(struct sdhci_host *host,
u8 power_mode)
{
static struct sdhci_ops sdhci_bcm_kona_ops = {
.set_clock = sdhci_set_clock,
- .get_max_clock = sdhci_bcm_kona_get_max_clk,
- .get_timeout_clock = sdhci_bcm_kona_get_timeout_clock,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .get_timeout_clock = sdhci_pltfm_clk_get_max_clock,
.platform_send_init_74_clocks = sdhci_bcm_kona_init_74_clocks,
.set_bus_width = sdhci_set_bus_width,
.reset = sdhci_reset,
goto err_pltfm_free;
}
- /* Get and enable the external clock */
- kona_dev->external_clk = devm_clk_get(dev, NULL);
- if (IS_ERR(kona_dev->external_clk)) {
- dev_err(dev, "Failed to get external clock\n");
- ret = PTR_ERR(kona_dev->external_clk);
+ /* Get and enable the core clock */
+ pltfm_priv->clk = devm_clk_get(dev, NULL);
+ if (IS_ERR(pltfm_priv->clk)) {
+ dev_err(dev, "Failed to get core clock\n");
+ ret = PTR_ERR(pltfm_priv->clk);
goto err_pltfm_free;
}
- if (clk_set_rate(kona_dev->external_clk, host->mmc->f_max) != 0) {
- dev_err(dev, "Failed to set rate external clock\n");
+ if (clk_set_rate(pltfm_priv->clk, host->mmc->f_max) != 0) {
+ dev_err(dev, "Failed to set rate core clock\n");
goto err_pltfm_free;
}
- if (clk_prepare_enable(kona_dev->external_clk) != 0) {
- dev_err(dev, "Failed to enable external clock\n");
+ if (clk_prepare_enable(pltfm_priv->clk) != 0) {
+ dev_err(dev, "Failed to enable core clock\n");
goto err_pltfm_free;
}
sdhci_bcm_kona_sd_reset(host);
err_clk_disable:
- clk_disable_unprepare(kona_dev->external_clk);
+ clk_disable_unprepare(pltfm_priv->clk);
err_pltfm_free:
sdhci_pltfm_free(pdev);
return ret;
}
-static int sdhci_bcm_kona_remove(struct platform_device *pdev)
-{
- struct sdhci_host *host = platform_get_drvdata(pdev);
- struct sdhci_pltfm_host *pltfm_priv = sdhci_priv(host);
- struct sdhci_bcm_kona_dev *kona_dev = sdhci_pltfm_priv(pltfm_priv);
- int dead = (readl(host->ioaddr + SDHCI_INT_STATUS) == 0xffffffff);
-
- sdhci_remove_host(host, dead);
-
- clk_disable_unprepare(kona_dev->external_clk);
-
- sdhci_pltfm_free(pdev);
-
- return 0;
-}
-
static struct platform_driver sdhci_bcm_kona_driver = {
.driver = {
.name = "sdhci-kona",
.of_match_table = sdhci_bcm_kona_of_match,
},
.probe = sdhci_bcm_kona_probe,
- .remove = sdhci_bcm_kona_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_bcm_kona_driver);
return ret;
}
-static int bcm2835_sdhci_remove(struct platform_device *pdev)
-{
- return sdhci_pltfm_unregister(pdev);
-}
-
static const struct of_device_id bcm2835_sdhci_of_match[] = {
{ .compatible = "brcm,bcm2835-sdhci" },
{ }
.pm = SDHCI_PLTFM_PMOPS,
},
.probe = bcm2835_sdhci_probe,
- .remove = bcm2835_sdhci_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(bcm2835_sdhci_driver);
return sdhci_pltfm_register(pdev, &sdhci_cns3xxx_pdata, 0);
}
-static int sdhci_cns3xxx_remove(struct platform_device *pdev)
-{
- return sdhci_pltfm_unregister(pdev);
-}
-
static struct platform_driver sdhci_cns3xxx_driver = {
.driver = {
.name = "sdhci-cns3xxx",
.pm = SDHCI_PLTFM_PMOPS,
},
.probe = sdhci_cns3xxx_probe,
- .remove = sdhci_cns3xxx_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_cns3xxx_driver);
#include "sdhci-pltfm.h"
-struct sdhci_dove_priv {
- struct clk *clk;
-};
-
static u16 sdhci_dove_readw(struct sdhci_host *host, int reg)
{
u16 ret;
{
struct sdhci_host *host;
struct sdhci_pltfm_host *pltfm_host;
- struct sdhci_dove_priv *priv;
int ret;
- priv = devm_kzalloc(&pdev->dev, sizeof(struct sdhci_dove_priv),
- GFP_KERNEL);
- if (!priv) {
- dev_err(&pdev->dev, "unable to allocate private data");
- return -ENOMEM;
- }
-
- priv->clk = devm_clk_get(&pdev->dev, NULL);
-
host = sdhci_pltfm_init(pdev, &sdhci_dove_pdata, 0);
if (IS_ERR(host))
return PTR_ERR(host);
pltfm_host = sdhci_priv(host);
- pltfm_host->priv = priv;
+ pltfm_host->clk = devm_clk_get(&pdev->dev, NULL);
- if (!IS_ERR(priv->clk))
- clk_prepare_enable(priv->clk);
+ if (!IS_ERR(pltfm_host->clk))
+ clk_prepare_enable(pltfm_host->clk);
ret = mmc_of_parse(host->mmc);
if (ret)
return 0;
err_sdhci_add:
- if (!IS_ERR(priv->clk))
- clk_disable_unprepare(priv->clk);
+ clk_disable_unprepare(pltfm_host->clk);
sdhci_pltfm_free(pdev);
return ret;
}
-static int sdhci_dove_remove(struct platform_device *pdev)
-{
- struct sdhci_host *host = platform_get_drvdata(pdev);
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_dove_priv *priv = pltfm_host->priv;
-
- sdhci_pltfm_unregister(pdev);
-
- if (!IS_ERR(priv->clk))
- clk_disable_unprepare(priv->clk);
-
- return 0;
-}
-
static const struct of_device_id sdhci_dove_of_match_table[] = {
{ .compatible = "marvell,dove-sdhci", },
{}
.of_match_table = sdhci_dove_of_match_table,
},
.probe = sdhci_dove_probe,
- .remove = sdhci_dove_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_dove_driver);
new_val |= ESDHC_VENDOR_SPEC_FRC_SDCLK_ON;
else
new_val &= ~ESDHC_VENDOR_SPEC_FRC_SDCLK_ON;
- writel(new_val, host->ioaddr + ESDHC_VENDOR_SPEC);
+ writel(new_val, host->ioaddr + ESDHC_VENDOR_SPEC);
return;
case SDHCI_HOST_CONTROL2:
new_val = readl(host->ioaddr + ESDHC_VENDOR_SPEC);
#ifdef CONFIG_OF
static int
sdhci_esdhc_imx_probe_dt(struct platform_device *pdev,
+ struct sdhci_host *host,
struct esdhc_platform_data *boarddata)
{
struct device_node *np = pdev->dev.of_node;
if (of_property_read_u32(np, "fsl,delay-line", &boarddata->delay_line))
boarddata->delay_line = 0;
+ mmc_of_parse_voltage(np, &host->ocr_mask);
+
return 0;
}
#else
static inline int
sdhci_esdhc_imx_probe_dt(struct platform_device *pdev,
+ struct sdhci_host *host,
struct esdhc_platform_data *boarddata)
{
return -ENODEV;
host->ioaddr + ESDHC_TUNING_CTRL);
boarddata = &imx_data->boarddata;
- if (sdhci_esdhc_imx_probe_dt(pdev, boarddata) < 0) {
+ if (sdhci_esdhc_imx_probe_dt(pdev, host, boarddata) < 0) {
if (!host->mmc->parent->platform_data) {
dev_err(mmc_dev(host->mmc), "no board data!\n");
err = -EINVAL;
host->mmc->parent->platform_data);
}
- /* write_protect */
- if (boarddata->wp_type == ESDHC_WP_GPIO) {
- err = mmc_gpio_request_ro(host->mmc, boarddata->wp_gpio);
- if (err) {
- dev_err(mmc_dev(host->mmc),
- "failed to request write-protect gpio!\n");
- goto disable_clk;
- }
- host->mmc->caps2 |= MMC_CAP2_RO_ACTIVE_HIGH;
- }
-
/* card_detect */
- switch (boarddata->cd_type) {
- case ESDHC_CD_GPIO:
- err = mmc_gpio_request_cd(host->mmc, boarddata->cd_gpio, 0);
- if (err) {
- dev_err(mmc_dev(host->mmc),
- "failed to request card-detect gpio!\n");
- goto disable_clk;
- }
- /* fall through */
-
- case ESDHC_CD_CONTROLLER:
- /* we have a working card_detect back */
+ if (boarddata->cd_type == ESDHC_CD_CONTROLLER)
host->quirks &= ~SDHCI_QUIRK_BROKEN_CARD_DETECTION;
- break;
-
- case ESDHC_CD_PERMANENT:
- host->mmc->caps |= MMC_CAP_NONREMOVABLE;
- break;
-
- case ESDHC_CD_NONE:
- break;
- }
switch (boarddata->max_bus_width) {
case 8:
host->quirks2 |= SDHCI_QUIRK2_NO_1_8_V;
}
+ /* call to generic mmc_of_parse to support additional capabilities */
+ err = mmc_of_parse(host->mmc);
+ if (err)
+ goto disable_clk;
+
err = sdhci_add_host(host);
if (err)
goto disable_clk;
--- /dev/null
+/*
+ * Copyright (C) 2014 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+/*
+ * iProc SDHCI platform driver
+ */
+
+#include <linux/delay.h>
+#include <linux/module.h>
+#include <linux/mmc/host.h>
+#include <linux/of.h>
+#include <linux/of_device.h>
+#include "sdhci-pltfm.h"
+
+struct sdhci_iproc_data {
+ const struct sdhci_pltfm_data *pdata;
+ u32 caps;
+ u32 caps1;
+};
+
+struct sdhci_iproc_host {
+ const struct sdhci_iproc_data *data;
+ u32 shadow_cmd;
+ u32 shadow_blk;
+};
+
+#define REG_OFFSET_IN_BITS(reg) ((reg) << 3 & 0x18)
+
+static inline u32 sdhci_iproc_readl(struct sdhci_host *host, int reg)
+{
+ u32 val = readl(host->ioaddr + reg);
+
+ pr_debug("%s: readl [0x%02x] 0x%08x\n",
+ mmc_hostname(host->mmc), reg, val);
+ return val;
+}
+
+static u16 sdhci_iproc_readw(struct sdhci_host *host, int reg)
+{
+ u32 val = sdhci_iproc_readl(host, (reg & ~3));
+ u16 word = val >> REG_OFFSET_IN_BITS(reg) & 0xffff;
+ return word;
+}
+
+static u8 sdhci_iproc_readb(struct sdhci_host *host, int reg)
+{
+ u32 val = sdhci_iproc_readl(host, (reg & ~3));
+ u8 byte = val >> REG_OFFSET_IN_BITS(reg) & 0xff;
+ return byte;
+}
+
+static inline void sdhci_iproc_writel(struct sdhci_host *host, u32 val, int reg)
+{
+ pr_debug("%s: writel [0x%02x] 0x%08x\n",
+ mmc_hostname(host->mmc), reg, val);
+
+ writel(val, host->ioaddr + reg);
+
+ if (host->clock <= 400000) {
+ /* Round up to micro-second four SD clock delay */
+ if (host->clock)
+ udelay((4 * 1000000 + host->clock - 1) / host->clock);
+ else
+ udelay(10);
+ }
+}
+
+/*
+ * The Arasan has a bugette whereby it may lose the content of successive
+ * writes to the same register that are within two SD-card clock cycles of
+ * each other (a clock domain crossing problem). The data
+ * register does not have this problem, which is just as well - otherwise we'd
+ * have to nobble the DMA engine too.
+ *
+ * This wouldn't be a problem with the code except that we can only write the
+ * controller with 32-bit writes. So two different 16-bit registers are
+ * written back to back creates the problem.
+ *
+ * In reality, this only happens when SDHCI_BLOCK_SIZE and SDHCI_BLOCK_COUNT
+ * are written followed by SDHCI_TRANSFER_MODE and SDHCI_COMMAND.
+ * The BLOCK_SIZE and BLOCK_COUNT are meaningless until a command issued so
+ * the work around can be further optimized. We can keep shadow values of
+ * BLOCK_SIZE, BLOCK_COUNT, and TRANSFER_MODE until a COMMAND is issued.
+ * Then, write the BLOCK_SIZE+BLOCK_COUNT in a single 32-bit write followed
+ * by the TRANSFER+COMMAND in another 32-bit write.
+ */
+static void sdhci_iproc_writew(struct sdhci_host *host, u16 val, int reg)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct sdhci_iproc_host *iproc_host = sdhci_pltfm_priv(pltfm_host);
+ u32 word_shift = REG_OFFSET_IN_BITS(reg);
+ u32 mask = 0xffff << word_shift;
+ u32 oldval, newval;
+
+ if (reg == SDHCI_COMMAND) {
+ /* Write the block now as we are issuing a command */
+ if (iproc_host->shadow_blk != 0) {
+ sdhci_iproc_writel(host, iproc_host->shadow_blk,
+ SDHCI_BLOCK_SIZE);
+ iproc_host->shadow_blk = 0;
+ }
+ oldval = iproc_host->shadow_cmd;
+ } else if (reg == SDHCI_BLOCK_SIZE || reg == SDHCI_BLOCK_COUNT) {
+ /* Block size and count are stored in shadow reg */
+ oldval = iproc_host->shadow_blk;
+ } else {
+ /* Read reg, all other registers are not shadowed */
+ oldval = sdhci_iproc_readl(host, (reg & ~3));
+ }
+ newval = (oldval & ~mask) | (val << word_shift);
+
+ if (reg == SDHCI_TRANSFER_MODE) {
+ /* Save the transfer mode until the command is issued */
+ iproc_host->shadow_cmd = newval;
+ } else if (reg == SDHCI_BLOCK_SIZE || reg == SDHCI_BLOCK_COUNT) {
+ /* Save the block info until the command is issued */
+ iproc_host->shadow_blk = newval;
+ } else {
+ /* Command or other regular 32-bit write */
+ sdhci_iproc_writel(host, newval, reg & ~3);
+ }
+}
+
+static void sdhci_iproc_writeb(struct sdhci_host *host, u8 val, int reg)
+{
+ u32 oldval = sdhci_iproc_readl(host, (reg & ~3));
+ u32 byte_shift = REG_OFFSET_IN_BITS(reg);
+ u32 mask = 0xff << byte_shift;
+ u32 newval = (oldval & ~mask) | (val << byte_shift);
+
+ sdhci_iproc_writel(host, newval, reg & ~3);
+}
+
+static const struct sdhci_ops sdhci_iproc_ops = {
+ .read_l = sdhci_iproc_readl,
+ .read_w = sdhci_iproc_readw,
+ .read_b = sdhci_iproc_readb,
+ .write_l = sdhci_iproc_writel,
+ .write_w = sdhci_iproc_writew,
+ .write_b = sdhci_iproc_writeb,
+ .set_clock = sdhci_set_clock,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+ .set_bus_width = sdhci_set_bus_width,
+ .reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
+};
+
+static const struct sdhci_pltfm_data sdhci_iproc_pltfm_data = {
+ .quirks = SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK,
+ .quirks2 = SDHCI_QUIRK2_ACMD23_BROKEN,
+ .ops = &sdhci_iproc_ops,
+};
+
+static const struct sdhci_iproc_data iproc_data = {
+ .pdata = &sdhci_iproc_pltfm_data,
+ .caps = 0x05E90000,
+ .caps1 = 0x00000064,
+};
+
+static const struct of_device_id sdhci_iproc_of_match[] = {
+ { .compatible = "brcm,sdhci-iproc-cygnus", .data = &iproc_data },
+ { }
+};
+MODULE_DEVICE_TABLE(of, sdhci_iproc_of_match);
+
+static int sdhci_iproc_probe(struct platform_device *pdev)
+{
+ const struct of_device_id *match;
+ const struct sdhci_iproc_data *iproc_data;
+ struct sdhci_host *host;
+ struct sdhci_iproc_host *iproc_host;
+ struct sdhci_pltfm_host *pltfm_host;
+ int ret;
+
+ match = of_match_device(sdhci_iproc_of_match, &pdev->dev);
+ if (!match)
+ return -EINVAL;
+ iproc_data = match->data;
+
+ host = sdhci_pltfm_init(pdev, iproc_data->pdata, sizeof(*iproc_host));
+ if (IS_ERR(host))
+ return PTR_ERR(host);
+
+ pltfm_host = sdhci_priv(host);
+ iproc_host = sdhci_pltfm_priv(pltfm_host);
+
+ iproc_host->data = iproc_data;
+
+ mmc_of_parse(host->mmc);
+ sdhci_get_of_property(pdev);
+
+ /* Enable EMMC 1/8V DDR capable */
+ host->mmc->caps |= MMC_CAP_1_8V_DDR;
+
+ pltfm_host->clk = devm_clk_get(&pdev->dev, NULL);
+ if (IS_ERR(pltfm_host->clk)) {
+ ret = PTR_ERR(pltfm_host->clk);
+ goto err;
+ }
+
+ if (iproc_host->data->pdata->quirks & SDHCI_QUIRK_MISSING_CAPS) {
+ host->caps = iproc_host->data->caps;
+ host->caps1 = iproc_host->data->caps1;
+ }
+
+ return sdhci_add_host(host);
+
+err:
+ sdhci_pltfm_free(pdev);
+ return ret;
+}
+
+static int sdhci_iproc_remove(struct platform_device *pdev)
+{
+ return sdhci_pltfm_unregister(pdev);
+}
+
+static struct platform_driver sdhci_iproc_driver = {
+ .driver = {
+ .name = "sdhci-iproc",
+ .of_match_table = sdhci_iproc_of_match,
+ .pm = SDHCI_PLTFM_PMOPS,
+ },
+ .probe = sdhci_iproc_probe,
+ .remove = sdhci_iproc_remove,
+};
+module_platform_driver(sdhci_iproc_driver);
+
+MODULE_AUTHOR("Broadcom");
+MODULE_DESCRIPTION("IPROC SDHCI driver");
+MODULE_LICENSE("GPL v2");
#include "sdhci-pltfm.h"
+#define CORE_MCI_VERSION 0x50
+#define CORE_VERSION_MAJOR_SHIFT 28
+#define CORE_VERSION_MAJOR_MASK (0xf << CORE_VERSION_MAJOR_SHIFT)
+#define CORE_VERSION_MINOR_MASK 0xff
+
#define CORE_HC_MODE 0x78
#define HC_MODE_EN 0x1
#define CORE_POWER 0x0
#define CORE_VENDOR_SPEC 0x10c
#define CORE_CLK_PWRSAVE BIT(1)
+#define CORE_VENDOR_SPEC_CAPABILITIES0 0x11c
+
#define CDR_SELEXT_SHIFT 20
#define CDR_SELEXT_MASK (0xf << CDR_SELEXT_SHIFT)
#define CMUX_SHIFT_PHASE_SHIFT 24
struct sdhci_msm_host *msm_host;
struct resource *core_memres;
int ret;
- u16 host_version;
+ u16 host_version, core_minor;
+ u32 core_version, caps;
+ u8 core_major;
msm_host = devm_kzalloc(&pdev->dev, sizeof(*msm_host), GFP_KERNEL);
if (!msm_host)
host_version, ((host_version & SDHCI_VENDOR_VER_MASK) >>
SDHCI_VENDOR_VER_SHIFT));
+ core_version = readl_relaxed(msm_host->core_mem + CORE_MCI_VERSION);
+ core_major = (core_version & CORE_VERSION_MAJOR_MASK) >>
+ CORE_VERSION_MAJOR_SHIFT;
+ core_minor = core_version & CORE_VERSION_MINOR_MASK;
+ dev_dbg(&pdev->dev, "MCI Version: 0x%08x, major: 0x%04x, minor: 0x%02x\n",
+ core_version, core_major, core_minor);
+
+ /*
+ * Support for some capabilities is not advertised by newer
+ * controller versions and must be explicitly enabled.
+ */
+ if (core_major >= 1 && core_minor != 0x11 && core_minor != 0x12) {
+ caps = readl_relaxed(host->ioaddr + SDHCI_CAPABILITIES);
+ caps |= SDHCI_CAN_VDD_300 | SDHCI_CAN_DO_8BIT;
+ writel_relaxed(caps, host->ioaddr +
+ CORE_VENDOR_SPEC_CAPABILITIES0);
+ }
+
ret = sdhci_add_host(host);
if (ret)
goto clk_disable;
pltfm_host->priv = sdhci_arasan;
pltfm_host->clk = clk_xin;
+ ret = mmc_of_parse(host->mmc);
+ if (ret) {
+ dev_err(&pdev->dev, "parsing dt failed (%u)\n", ret);
+ goto clk_disable_all;
+ }
+
ret = sdhci_add_host(host);
if (ret)
goto err_pltfm_free;
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
struct sdhci_arasan_data *sdhci_arasan = pltfm_host->priv;
- clk_disable_unprepare(pltfm_host->clk);
clk_disable_unprepare(sdhci_arasan->clk_ahb);
return sdhci_pltfm_unregister(pdev);
return ret;
}
-static int sdhci_esdhc_remove(struct platform_device *pdev)
-{
- return sdhci_pltfm_unregister(pdev);
-}
-
static const struct of_device_id sdhci_esdhc_of_match[] = {
{ .compatible = "fsl,mpc8379-esdhc" },
{ .compatible = "fsl,mpc8536-esdhc" },
.pm = ESDHC_PMOPS,
},
.probe = sdhci_esdhc_probe,
- .remove = sdhci_esdhc_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_esdhc_driver);
return sdhci_pltfm_register(pdev, &sdhci_hlwd_pdata, 0);
}
-static int sdhci_hlwd_remove(struct platform_device *pdev)
-{
- return sdhci_pltfm_unregister(pdev);
-}
-
static const struct of_device_id sdhci_hlwd_of_match[] = {
{ .compatible = "nintendo,hollywood-sdhci" },
{ }
.pm = SDHCI_PLTFM_PMOPS,
},
.probe = sdhci_hlwd_probe,
- .remove = sdhci_hlwd_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_hlwd_driver);
static const struct sdhci_pci_fixes sdhci_rtsx = {
.quirks2 = SDHCI_QUIRK2_PRESET_VALUE_BROKEN |
+ SDHCI_QUIRK2_BROKEN_64_BIT_DMA |
SDHCI_QUIRK2_BROKEN_DDR50,
.probe_slot = rtsx_probe_slot,
};
u32 bus_width;
int size;
- if (of_device_is_available(np)) {
- if (of_get_property(np, "sdhci,auto-cmd12", NULL))
- host->quirks |= SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12;
+ if (of_get_property(np, "sdhci,auto-cmd12", NULL))
+ host->quirks |= SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12;
- if (of_get_property(np, "sdhci,1-bit-only", NULL) ||
- (of_property_read_u32(np, "bus-width", &bus_width) == 0 &&
- bus_width == 1))
- host->quirks |= SDHCI_QUIRK_FORCE_1_BIT_DATA;
+ if (of_get_property(np, "sdhci,1-bit-only", NULL) ||
+ (of_property_read_u32(np, "bus-width", &bus_width) == 0 &&
+ bus_width == 1))
+ host->quirks |= SDHCI_QUIRK_FORCE_1_BIT_DATA;
- if (sdhci_of_wp_inverted(np))
- host->quirks |= SDHCI_QUIRK_INVERTED_WRITE_PROTECT;
+ if (sdhci_of_wp_inverted(np))
+ host->quirks |= SDHCI_QUIRK_INVERTED_WRITE_PROTECT;
- if (of_get_property(np, "broken-cd", NULL))
- host->quirks |= SDHCI_QUIRK_BROKEN_CARD_DETECTION;
+ if (of_get_property(np, "broken-cd", NULL))
+ host->quirks |= SDHCI_QUIRK_BROKEN_CARD_DETECTION;
- if (of_get_property(np, "no-1-8-v", NULL))
- host->quirks2 |= SDHCI_QUIRK2_NO_1_8_V;
+ if (of_get_property(np, "no-1-8-v", NULL))
+ host->quirks2 |= SDHCI_QUIRK2_NO_1_8_V;
- if (of_device_is_compatible(np, "fsl,p2020-rev1-esdhc"))
- host->quirks |= SDHCI_QUIRK_BROKEN_DMA;
+ if (of_device_is_compatible(np, "fsl,p2020-rev1-esdhc"))
+ host->quirks |= SDHCI_QUIRK_BROKEN_DMA;
- if (of_device_is_compatible(np, "fsl,p2020-esdhc") ||
- of_device_is_compatible(np, "fsl,p1010-esdhc") ||
- of_device_is_compatible(np, "fsl,t4240-esdhc") ||
- of_device_is_compatible(np, "fsl,mpc8536-esdhc"))
- host->quirks |= SDHCI_QUIRK_BROKEN_TIMEOUT_VAL;
+ if (of_device_is_compatible(np, "fsl,p2020-esdhc") ||
+ of_device_is_compatible(np, "fsl,p1010-esdhc") ||
+ of_device_is_compatible(np, "fsl,t4240-esdhc") ||
+ of_device_is_compatible(np, "fsl,mpc8536-esdhc"))
+ host->quirks |= SDHCI_QUIRK_BROKEN_TIMEOUT_VAL;
- clk = of_get_property(np, "clock-frequency", &size);
- if (clk && size == sizeof(*clk) && *clk)
- pltfm_host->clock = be32_to_cpup(clk);
+ clk = of_get_property(np, "clock-frequency", &size);
+ if (clk && size == sizeof(*clk) && *clk)
+ pltfm_host->clock = be32_to_cpup(clk);
- if (of_find_property(np, "keep-power-in-suspend", NULL))
- host->mmc->pm_caps |= MMC_PM_KEEP_POWER;
+ if (of_find_property(np, "keep-power-in-suspend", NULL))
+ host->mmc->pm_caps |= MMC_PM_KEEP_POWER;
- if (of_find_property(np, "enable-sdio-wakeup", NULL))
- host->mmc->pm_caps |= MMC_PM_WAKE_SDIO_IRQ;
- }
+ if (of_find_property(np, "enable-sdio-wakeup", NULL))
+ host->mmc->pm_caps |= MMC_PM_WAKE_SDIO_IRQ;
}
#else
void sdhci_get_of_property(struct platform_device *pdev) {}
int sdhci_pltfm_unregister(struct platform_device *pdev)
{
struct sdhci_host *host = platform_get_drvdata(pdev);
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
int dead = (readl(host->ioaddr + SDHCI_INT_STATUS) == 0xffffffff);
sdhci_remove_host(host, dead);
+ clk_disable_unprepare(pltfm_host->clk);
sdhci_pltfm_free(pdev);
return 0;
#define SIRF_TUNING_COUNT 128
struct sdhci_sirf_priv {
- struct clk *clk;
int gpio_cd;
};
-static unsigned int sdhci_sirf_get_max_clk(struct sdhci_host *host)
-{
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_sirf_priv *priv = sdhci_pltfm_priv(pltfm_host);
- return clk_get_rate(priv->clk);
-}
-
static void sdhci_sirf_set_bus_width(struct sdhci_host *host, int width)
{
u8 ctrl;
int tuning_seq_cnt = 3;
u8 phase, tuned_phases[SIRF_TUNING_COUNT];
u8 tuned_phase_cnt = 0;
- int rc, longest_range = 0;
+ int rc = 0, longest_range = 0;
int start = -1, end = 0, tuning_value = -1, range = 0;
u16 clock_setting;
struct mmc_host *mmc = host->mmc;
phase = 0;
do {
sdhci_writel(host,
- clock_setting | phase | (phase << 7) | (phase << 16),
+ clock_setting | phase,
SDHCI_CLK_DELAY_SETTING);
if (!mmc_send_tuning(mmc)) {
*/
phase = tuning_value;
sdhci_writel(host,
- clock_setting | phase | (phase << 7) | (phase << 16),
+ clock_setting | phase,
SDHCI_CLK_DELAY_SETTING);
dev_dbg(mmc_dev(mmc), "%s: Setting the tuning phase to %d\n",
static struct sdhci_ops sdhci_sirf_ops = {
.platform_execute_tuning = sdhci_sirf_execute_tuning,
.set_clock = sdhci_set_clock,
- .get_max_clock = sdhci_sirf_get_max_clk,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
.set_bus_width = sdhci_sirf_set_bus_width,
.reset = sdhci_reset,
.set_uhs_signaling = sdhci_set_uhs_signaling,
return PTR_ERR(host);
pltfm_host = sdhci_priv(host);
+ pltfm_host->clk = clk;
priv = sdhci_pltfm_priv(pltfm_host);
- priv->clk = clk;
priv->gpio_cd = gpio_cd;
sdhci_get_of_property(pdev);
- ret = clk_prepare_enable(priv->clk);
+ ret = clk_prepare_enable(pltfm_host->clk);
if (ret)
goto err_clk_prepare;
err_request_cd:
sdhci_remove_host(host, 0);
err_sdhci_add:
- clk_disable_unprepare(priv->clk);
+ clk_disable_unprepare(pltfm_host->clk);
err_clk_prepare:
sdhci_pltfm_free(pdev);
return ret;
}
-static int sdhci_sirf_remove(struct platform_device *pdev)
-{
- struct sdhci_host *host = platform_get_drvdata(pdev);
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_sirf_priv *priv = sdhci_pltfm_priv(pltfm_host);
-
- sdhci_pltfm_unregister(pdev);
-
- clk_disable_unprepare(priv->clk);
- return 0;
-}
-
#ifdef CONFIG_PM_SLEEP
static int sdhci_sirf_suspend(struct device *dev)
{
struct sdhci_host *host = dev_get_drvdata(dev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_sirf_priv *priv = sdhci_pltfm_priv(pltfm_host);
int ret;
ret = sdhci_suspend_host(host);
if (ret)
return ret;
- clk_disable(priv->clk);
+ clk_disable(pltfm_host->clk);
return 0;
}
{
struct sdhci_host *host = dev_get_drvdata(dev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_sirf_priv *priv = sdhci_pltfm_priv(pltfm_host);
int ret;
- ret = clk_enable(priv->clk);
+ ret = clk_enable(pltfm_host->clk);
if (ret) {
dev_dbg(dev, "Resume: Error enabling clock\n");
return ret;
#endif
},
.probe = sdhci_sirf_probe,
- .remove = sdhci_sirf_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_sirf_driver);
#include <linux/pm.h>
#include <linux/slab.h>
#include <linux/mmc/host.h>
-#include <linux/mmc/sdhci-spear.h>
#include <linux/mmc/slot-gpio.h>
#include <linux/io.h>
#include "sdhci.h"
struct spear_sdhci {
struct clk *clk;
- struct sdhci_plat_data *data;
+ int card_int_gpio;
};
/* sdhci ops */
.set_uhs_signaling = sdhci_set_uhs_signaling,
};
-#ifdef CONFIG_OF
-static struct sdhci_plat_data *sdhci_probe_config_dt(struct platform_device *pdev)
+static void sdhci_probe_config_dt(struct device_node *np,
+ struct spear_sdhci *host)
{
- struct device_node *np = pdev->dev.of_node;
- struct sdhci_plat_data *pdata = NULL;
int cd_gpio;
cd_gpio = of_get_named_gpio(np, "cd-gpios", 0);
if (!gpio_is_valid(cd_gpio))
cd_gpio = -1;
- /* If pdata is required */
- if (cd_gpio != -1) {
- pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
- if (!pdata)
- dev_err(&pdev->dev, "DT: kzalloc failed\n");
- else
- pdata->card_int_gpio = cd_gpio;
- }
-
- return pdata;
-}
-#else
-static struct sdhci_plat_data *sdhci_probe_config_dt(struct platform_device *pdev)
-{
- return ERR_PTR(-ENOSYS);
+ host->card_int_gpio = cd_gpio;
}
-#endif
static int sdhci_probe(struct platform_device *pdev)
{
- struct device_node *np = pdev->dev.of_node;
struct sdhci_host *host;
struct resource *iomem;
struct spear_sdhci *sdhci;
dev_dbg(&pdev->dev, "Error setting desired clk, clk=%lu\n",
clk_get_rate(sdhci->clk));
- if (np) {
- sdhci->data = sdhci_probe_config_dt(pdev);
- if (IS_ERR(sdhci->data)) {
- dev_err(&pdev->dev, "DT: Failed to get pdata\n");
- goto disable_clk;
- }
- } else {
- sdhci->data = dev_get_platdata(&pdev->dev);
- }
-
+ sdhci_probe_config_dt(pdev->dev.of_node, sdhci);
/*
* It is optional to use GPIOs for sdhci card detection. If
- * sdhci->data is NULL, then use original sdhci lines otherwise
+ * sdhci->card_int_gpio < 0, then use original sdhci lines otherwise
* GPIO lines. We use the built-in GPIO support for this.
*/
- if (sdhci->data && sdhci->data->card_int_gpio >= 0) {
- ret = mmc_gpio_request_cd(host->mmc,
- sdhci->data->card_int_gpio, 0);
+ if (sdhci->card_int_gpio >= 0) {
+ ret = mmc_gpio_request_cd(host->mmc, sdhci->card_int_gpio, 0);
if (ret < 0) {
dev_dbg(&pdev->dev,
"failed to request card-detect gpio%d\n",
- sdhci->data->card_int_gpio);
+ sdhci->card_int_gpio);
goto disable_clk;
}
}
#include <linux/module.h>
#include <linux/err.h>
#include <linux/mmc/host.h>
-
+#include <linux/reset.h>
#include "sdhci-pltfm.h"
+struct st_mmc_platform_data {
+ struct reset_control *rstc;
+ void __iomem *top_ioaddr;
+};
+
+/* MMCSS glue logic to setup the HC on some ST SoCs (e.g. STiH407 family) */
+
+#define ST_MMC_CCONFIG_REG_1 0x400
+#define ST_MMC_CCONFIG_TIMEOUT_CLK_UNIT BIT(24)
+#define ST_MMC_CCONFIG_TIMEOUT_CLK_FREQ BIT(12)
+#define ST_MMC_CCONFIG_TUNING_COUNT_DEFAULT BIT(8)
+#define ST_MMC_CCONFIG_ASYNC_WAKEUP BIT(0)
+#define ST_MMC_CCONFIG_1_DEFAULT \
+ ((ST_MMC_CCONFIG_TIMEOUT_CLK_UNIT) | \
+ (ST_MMC_CCONFIG_TIMEOUT_CLK_FREQ) | \
+ (ST_MMC_CCONFIG_TUNING_COUNT_DEFAULT))
+
+#define ST_MMC_CCONFIG_REG_2 0x404
+#define ST_MMC_CCONFIG_HIGH_SPEED BIT(28)
+#define ST_MMC_CCONFIG_ADMA2 BIT(24)
+#define ST_MMC_CCONFIG_8BIT BIT(20)
+#define ST_MMC_CCONFIG_MAX_BLK_LEN 16
+#define MAX_BLK_LEN_1024 1
+#define MAX_BLK_LEN_2048 2
+#define BASE_CLK_FREQ_200 0xc8
+#define BASE_CLK_FREQ_100 0x64
+#define BASE_CLK_FREQ_50 0x32
+#define ST_MMC_CCONFIG_2_DEFAULT \
+ (ST_MMC_CCONFIG_HIGH_SPEED | ST_MMC_CCONFIG_ADMA2 | \
+ ST_MMC_CCONFIG_8BIT | \
+ (MAX_BLK_LEN_1024 << ST_MMC_CCONFIG_MAX_BLK_LEN))
+
+#define ST_MMC_CCONFIG_REG_3 0x408
+#define ST_MMC_CCONFIG_EMMC_SLOT_TYPE BIT(28)
+#define ST_MMC_CCONFIG_64BIT BIT(24)
+#define ST_MMC_CCONFIG_ASYNCH_INTR_SUPPORT BIT(20)
+#define ST_MMC_CCONFIG_1P8_VOLT BIT(16)
+#define ST_MMC_CCONFIG_3P0_VOLT BIT(12)
+#define ST_MMC_CCONFIG_3P3_VOLT BIT(8)
+#define ST_MMC_CCONFIG_SUSP_RES_SUPPORT BIT(4)
+#define ST_MMC_CCONFIG_SDMA BIT(0)
+#define ST_MMC_CCONFIG_3_DEFAULT \
+ (ST_MMC_CCONFIG_ASYNCH_INTR_SUPPORT | \
+ ST_MMC_CCONFIG_3P3_VOLT | \
+ ST_MMC_CCONFIG_SUSP_RES_SUPPORT | \
+ ST_MMC_CCONFIG_SDMA)
+
+#define ST_MMC_CCONFIG_REG_4 0x40c
+#define ST_MMC_CCONFIG_D_DRIVER BIT(20)
+#define ST_MMC_CCONFIG_C_DRIVER BIT(16)
+#define ST_MMC_CCONFIG_A_DRIVER BIT(12)
+#define ST_MMC_CCONFIG_DDR50 BIT(8)
+#define ST_MMC_CCONFIG_SDR104 BIT(4)
+#define ST_MMC_CCONFIG_SDR50 BIT(0)
+#define ST_MMC_CCONFIG_4_DEFAULT 0
+
+#define ST_MMC_CCONFIG_REG_5 0x410
+#define ST_MMC_CCONFIG_TUNING_FOR_SDR50 BIT(8)
+#define RETUNING_TIMER_CNT_MAX 0xf
+#define ST_MMC_CCONFIG_5_DEFAULT 0
+
+/* I/O configuration for Arasan IP */
+#define ST_MMC_GP_OUTPUT 0x450
+#define ST_MMC_GP_OUTPUT_CD BIT(12)
+
+#define ST_MMC_STATUS_R 0x460
+
+#define ST_TOP_MMC_DLY_FIX_OFF(x) (x - 0x8)
+
+/* TOP config registers to manage static and dynamic delay */
+#define ST_TOP_MMC_TX_CLK_DLY ST_TOP_MMC_DLY_FIX_OFF(0x8)
+#define ST_TOP_MMC_RX_CLK_DLY ST_TOP_MMC_DLY_FIX_OFF(0xc)
+/* MMC delay control register */
+#define ST_TOP_MMC_DLY_CTRL ST_TOP_MMC_DLY_FIX_OFF(0x18)
+#define ST_TOP_MMC_DLY_CTRL_DLL_BYPASS_CMD BIT(0)
+#define ST_TOP_MMC_DLY_CTRL_DLL_BYPASS_PH_SEL BIT(1)
+#define ST_TOP_MMC_DLY_CTRL_TX_DLL_ENABLE BIT(8)
+#define ST_TOP_MMC_DLY_CTRL_RX_DLL_ENABLE BIT(9)
+#define ST_TOP_MMC_DLY_CTRL_ATUNE_NOT_CFG_DLY BIT(10)
+#define ST_TOP_MMC_START_DLL_LOCK BIT(11)
+
+/* register to provide the phase-shift value for DLL */
+#define ST_TOP_MMC_TX_DLL_STEP_DLY ST_TOP_MMC_DLY_FIX_OFF(0x1c)
+#define ST_TOP_MMC_RX_DLL_STEP_DLY ST_TOP_MMC_DLY_FIX_OFF(0x20)
+#define ST_TOP_MMC_RX_CMD_STEP_DLY ST_TOP_MMC_DLY_FIX_OFF(0x24)
+
+/* phase shift delay on the tx clk 2.188ns */
+#define ST_TOP_MMC_TX_DLL_STEP_DLY_VALID 0x6
+
+#define ST_TOP_MMC_DLY_MAX 0xf
+
+#define ST_TOP_MMC_DYN_DLY_CONF \
+ (ST_TOP_MMC_DLY_CTRL_TX_DLL_ENABLE | \
+ ST_TOP_MMC_DLY_CTRL_ATUNE_NOT_CFG_DLY | \
+ ST_TOP_MMC_START_DLL_LOCK)
+
+/*
+ * For clock speeds greater than 90MHz, we need to check that the
+ * DLL procedure has finished before switching to ultra-speed modes.
+ */
+#define CLK_TO_CHECK_DLL_LOCK 90000000
+
+static inline void st_mmcss_set_static_delay(void __iomem *ioaddr)
+{
+ if (!ioaddr)
+ return;
+
+ writel_relaxed(0x0, ioaddr + ST_TOP_MMC_DLY_CTRL);
+ writel_relaxed(ST_TOP_MMC_DLY_MAX,
+ ioaddr + ST_TOP_MMC_TX_CLK_DLY);
+}
+
+/**
+ * st_mmcss_cconfig: configure the Arasan HC inside the flashSS.
+ * @np: dt device node.
+ * @host: sdhci host
+ * Description: this function is to configure the Arasan host controller.
+ * On some ST SoCs, i.e. STiH407 family, the MMC devices inside a dedicated
+ * flashSS sub-system which needs to be configured to be compliant to eMMC 4.5
+ * or eMMC4.3. This has to be done before registering the sdhci host.
+ */
+static void st_mmcss_cconfig(struct device_node *np, struct sdhci_host *host)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct mmc_host *mhost = host->mmc;
+ u32 cconf2, cconf3, cconf4, cconf5;
+
+ if (!of_device_is_compatible(np, "st,sdhci-stih407"))
+ return;
+
+ cconf2 = ST_MMC_CCONFIG_2_DEFAULT;
+ cconf3 = ST_MMC_CCONFIG_3_DEFAULT;
+ cconf4 = ST_MMC_CCONFIG_4_DEFAULT;
+ cconf5 = ST_MMC_CCONFIG_5_DEFAULT;
+
+ writel_relaxed(ST_MMC_CCONFIG_1_DEFAULT,
+ host->ioaddr + ST_MMC_CCONFIG_REG_1);
+
+ /* Set clock frequency, default to 50MHz if max-frequency is not
+ * provided */
+
+ switch (mhost->f_max) {
+ case 200000000:
+ clk_set_rate(pltfm_host->clk, mhost->f_max);
+ cconf2 |= BASE_CLK_FREQ_200;
+ break;
+ case 100000000:
+ clk_set_rate(pltfm_host->clk, mhost->f_max);
+ cconf2 |= BASE_CLK_FREQ_100;
+ break;
+ default:
+ clk_set_rate(pltfm_host->clk, 50000000);
+ cconf2 |= BASE_CLK_FREQ_50;
+ }
+
+ writel_relaxed(cconf2, host->ioaddr + ST_MMC_CCONFIG_REG_2);
+
+ if (mhost->caps & MMC_CAP_NONREMOVABLE)
+ cconf3 |= ST_MMC_CCONFIG_EMMC_SLOT_TYPE;
+ else
+ /* CARD _D ET_CTRL */
+ writel_relaxed(ST_MMC_GP_OUTPUT_CD,
+ host->ioaddr + ST_MMC_GP_OUTPUT);
+
+ if (mhost->caps & MMC_CAP_UHS_SDR50) {
+ /* use 1.8V */
+ cconf3 |= ST_MMC_CCONFIG_1P8_VOLT;
+ cconf4 |= ST_MMC_CCONFIG_SDR50;
+ /* Use tuning */
+ cconf5 |= ST_MMC_CCONFIG_TUNING_FOR_SDR50;
+ /* Max timeout for retuning */
+ cconf5 |= RETUNING_TIMER_CNT_MAX;
+ }
+
+ if (mhost->caps & MMC_CAP_UHS_SDR104) {
+ /*
+ * SDR104 implies the HC can support HS200 mode, so
+ * it's mandatory to use 1.8V
+ */
+ cconf3 |= ST_MMC_CCONFIG_1P8_VOLT;
+ cconf4 |= ST_MMC_CCONFIG_SDR104;
+ /* Max timeout for retuning */
+ cconf5 |= RETUNING_TIMER_CNT_MAX;
+ }
+
+ if (mhost->caps & MMC_CAP_UHS_DDR50)
+ cconf4 |= ST_MMC_CCONFIG_DDR50;
+
+ writel_relaxed(cconf3, host->ioaddr + ST_MMC_CCONFIG_REG_3);
+ writel_relaxed(cconf4, host->ioaddr + ST_MMC_CCONFIG_REG_4);
+ writel_relaxed(cconf5, host->ioaddr + ST_MMC_CCONFIG_REG_5);
+}
+
+static inline void st_mmcss_set_dll(void __iomem *ioaddr)
+{
+ if (!ioaddr)
+ return;
+
+ writel_relaxed(ST_TOP_MMC_DYN_DLY_CONF, ioaddr + ST_TOP_MMC_DLY_CTRL);
+ writel_relaxed(ST_TOP_MMC_TX_DLL_STEP_DLY_VALID,
+ ioaddr + ST_TOP_MMC_TX_DLL_STEP_DLY);
+}
+
+static int st_mmcss_lock_dll(void __iomem *ioaddr)
+{
+ unsigned long curr, value;
+ unsigned long finish = jiffies + HZ;
+
+ /* Checks if the DLL procedure is finished */
+ do {
+ curr = jiffies;
+ value = readl(ioaddr + ST_MMC_STATUS_R);
+ if (value & 0x1)
+ return 0;
+
+ cpu_relax();
+ } while (!time_after_eq(curr, finish));
+
+ return -EBUSY;
+}
+
+static int sdhci_st_set_dll_for_clock(struct sdhci_host *host)
+{
+ int ret = 0;
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct st_mmc_platform_data *pdata = pltfm_host->priv;
+
+ if (host->clock > CLK_TO_CHECK_DLL_LOCK) {
+ st_mmcss_set_dll(pdata->top_ioaddr);
+ ret = st_mmcss_lock_dll(host->ioaddr);
+ }
+
+ return ret;
+}
+
+static void sdhci_st_set_uhs_signaling(struct sdhci_host *host,
+ unsigned int uhs)
+{
+ struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct st_mmc_platform_data *pdata = pltfm_host->priv;
+ u16 ctrl_2 = sdhci_readw(host, SDHCI_HOST_CONTROL2);
+ int ret = 0;
+
+ /* Select Bus Speed Mode for host */
+ ctrl_2 &= ~SDHCI_CTRL_UHS_MASK;
+ switch (uhs) {
+ /*
+ * Set V18_EN -- UHS modes do not work without this.
+ * does not change signaling voltage
+ */
+
+ case MMC_TIMING_UHS_SDR12:
+ st_mmcss_set_static_delay(pdata->top_ioaddr);
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR12 | SDHCI_CTRL_VDD_180;
+ break;
+ case MMC_TIMING_UHS_SDR25:
+ st_mmcss_set_static_delay(pdata->top_ioaddr);
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR25 | SDHCI_CTRL_VDD_180;
+ break;
+ case MMC_TIMING_UHS_SDR50:
+ st_mmcss_set_static_delay(pdata->top_ioaddr);
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR50 | SDHCI_CTRL_VDD_180;
+ ret = sdhci_st_set_dll_for_clock(host);
+ break;
+ case MMC_TIMING_UHS_SDR104:
+ case MMC_TIMING_MMC_HS200:
+ st_mmcss_set_static_delay(pdata->top_ioaddr);
+ ctrl_2 |= SDHCI_CTRL_UHS_SDR104 | SDHCI_CTRL_VDD_180;
+ ret = sdhci_st_set_dll_for_clock(host);
+ break;
+ case MMC_TIMING_UHS_DDR50:
+ case MMC_TIMING_MMC_DDR52:
+ st_mmcss_set_static_delay(pdata->top_ioaddr);
+ ctrl_2 |= SDHCI_CTRL_UHS_DDR50 | SDHCI_CTRL_VDD_180;
+ break;
+ }
+
+ if (ret)
+ dev_warn(mmc_dev(host->mmc), "Error setting dll for clock "
+ "(uhs %d)\n", uhs);
+
+ dev_dbg(mmc_dev(host->mmc), "uhs %d, ctrl_2 %04X\n", uhs, ctrl_2);
+
+ sdhci_writew(host, ctrl_2, SDHCI_HOST_CONTROL2);
+}
+
static u32 sdhci_st_readl(struct sdhci_host *host, int reg)
{
u32 ret;
.set_bus_width = sdhci_set_bus_width,
.read_l = sdhci_st_readl,
.reset = sdhci_reset,
+ .set_uhs_signaling = sdhci_st_set_uhs_signaling,
};
static const struct sdhci_pltfm_data sdhci_st_pdata = {
.ops = &sdhci_st_ops,
.quirks = SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC |
- SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
+ SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN |
+ SDHCI_QUIRK_NO_HISPD_BIT,
+ .quirks2 = SDHCI_QUIRK2_PRESET_VALUE_BROKEN |
+ SDHCI_QUIRK2_STOP_WITH_TC,
};
static int sdhci_st_probe(struct platform_device *pdev)
{
+ struct device_node *np = pdev->dev.of_node;
struct sdhci_host *host;
+ struct st_mmc_platform_data *pdata;
struct sdhci_pltfm_host *pltfm_host;
struct clk *clk;
int ret = 0;
u16 host_version;
+ struct resource *res;
+
+ pdata = devm_kzalloc(&pdev->dev, sizeof(*pdata), GFP_KERNEL);
+ if (!pdata)
+ return -ENOMEM;
clk = devm_clk_get(&pdev->dev, "mmc");
if (IS_ERR(clk)) {
return PTR_ERR(clk);
}
+ pdata->rstc = devm_reset_control_get(&pdev->dev, NULL);
+ if (IS_ERR(pdata->rstc))
+ pdata->rstc = NULL;
+ else
+ reset_control_deassert(pdata->rstc);
+
host = sdhci_pltfm_init(pdev, &sdhci_st_pdata, 0);
if (IS_ERR(host)) {
dev_err(&pdev->dev, "Failed sdhci_pltfm_init\n");
- return PTR_ERR(host);
+ ret = PTR_ERR(host);
+ goto err_pltfm_init;
}
ret = mmc_of_parse(host->mmc);
clk_prepare_enable(clk);
+ /* Configure the FlashSS Top registers for setting eMMC TX/RX delay */
+ res = platform_get_resource_byname(pdev, IORESOURCE_MEM,
+ "top-mmc-delay");
+ pdata->top_ioaddr = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(pdata->top_ioaddr)) {
+ dev_warn(&pdev->dev, "FlashSS Top Dly registers not available");
+ pdata->top_ioaddr = NULL;
+ }
+
pltfm_host = sdhci_priv(host);
+ pltfm_host->priv = pdata;
pltfm_host->clk = clk;
+ /* Configure the Arasan HC inside the flashSS */
+ st_mmcss_cconfig(np, host);
+
ret = sdhci_add_host(host);
if (ret) {
dev_err(&pdev->dev, "Failed sdhci_add_host\n");
clk_disable_unprepare(clk);
err_of:
sdhci_pltfm_free(pdev);
+err_pltfm_init:
+ if (pdata->rstc)
+ reset_control_assert(pdata->rstc);
return ret;
}
{
struct sdhci_host *host = platform_get_drvdata(pdev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct st_mmc_platform_data *pdata = pltfm_host->priv;
+ int ret;
- clk_disable_unprepare(pltfm_host->clk);
+ ret = sdhci_pltfm_unregister(pdev);
+
+ if (pdata->rstc)
+ reset_control_assert(pdata->rstc);
- return sdhci_pltfm_unregister(pdev);
+ return ret;
}
#ifdef CONFIG_PM_SLEEP
{
struct sdhci_host *host = dev_get_drvdata(dev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct st_mmc_platform_data *pdata = pltfm_host->priv;
int ret = sdhci_suspend_host(host);
if (ret)
goto out;
+ if (pdata->rstc)
+ reset_control_assert(pdata->rstc);
+
clk_disable_unprepare(pltfm_host->clk);
out:
return ret;
{
struct sdhci_host *host = dev_get_drvdata(dev);
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
+ struct st_mmc_platform_data *pdata = pltfm_host->priv;
+ struct device_node *np = dev->of_node;
clk_prepare_enable(pltfm_host->clk);
+ if (pdata->rstc)
+ reset_control_deassert(pdata->rstc);
+
+ st_mmcss_cconfig(np, host);
+
return sdhci_resume_host(host);
}
#endif
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_device.h>
-#include <linux/of_gpio.h>
-#include <linux/gpio.h>
#include <linux/mmc/card.h>
#include <linux/mmc/host.h>
#include <linux/mmc/slot-gpio.h>
+#include <linux/gpio/consumer.h>
#include "sdhci-pltfm.h"
#define NVQUIRK_DISABLE_SDR50 BIT(3)
#define NVQUIRK_DISABLE_SDR104 BIT(4)
#define NVQUIRK_DISABLE_DDR50 BIT(5)
-#define NVQUIRK_SHADOW_XFER_MODE_REG BIT(6)
struct sdhci_tegra_soc_data {
const struct sdhci_pltfm_data *pdata;
struct sdhci_tegra {
const struct sdhci_tegra_soc_data *soc_data;
- int power_gpio;
+ struct gpio_desc *power_gpio;
};
static u16 tegra_sdhci_readw(struct sdhci_host *host, int reg)
static void tegra_sdhci_writew(struct sdhci_host *host, u16 val, int reg)
{
struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_tegra *tegra_host = pltfm_host->priv;
- const struct sdhci_tegra_soc_data *soc_data = tegra_host->soc_data;
- if (soc_data->nvquirks & NVQUIRK_SHADOW_XFER_MODE_REG) {
- switch (reg) {
- case SDHCI_TRANSFER_MODE:
- /*
- * Postpone this write, we must do it together with a
- * command write that is down below.
- */
- pltfm_host->xfer_mode_shadow = val;
- return;
- case SDHCI_COMMAND:
- writel((val << 16) | pltfm_host->xfer_mode_shadow,
- host->ioaddr + SDHCI_TRANSFER_MODE);
- return;
- }
+ switch (reg) {
+ case SDHCI_TRANSFER_MODE:
+ /*
+ * Postpone this write, we must do it together with a
+ * command write that is down below.
+ */
+ pltfm_host->xfer_mode_shadow = val;
+ return;
+ case SDHCI_COMMAND:
+ writel((val << 16) | pltfm_host->xfer_mode_shadow,
+ host->ioaddr + SDHCI_TRANSFER_MODE);
+ return;
}
writew(val, host->ioaddr + reg);
static const struct sdhci_ops tegra_sdhci_ops = {
.get_ro = tegra_sdhci_get_ro,
.read_w = tegra_sdhci_readw,
- .write_w = tegra_sdhci_writew,
.write_l = tegra_sdhci_writel,
.set_clock = sdhci_set_clock,
.set_bus_width = tegra_sdhci_set_bus_width,
NVQUIRK_DISABLE_SDR104,
};
+static const struct sdhci_ops tegra114_sdhci_ops = {
+ .get_ro = tegra_sdhci_get_ro,
+ .read_w = tegra_sdhci_readw,
+ .write_w = tegra_sdhci_writew,
+ .write_l = tegra_sdhci_writel,
+ .set_clock = sdhci_set_clock,
+ .set_bus_width = tegra_sdhci_set_bus_width,
+ .reset = tegra_sdhci_reset,
+ .set_uhs_signaling = sdhci_set_uhs_signaling,
+ .get_max_clock = sdhci_pltfm_clk_get_max_clock,
+};
+
static const struct sdhci_pltfm_data sdhci_tegra114_pdata = {
.quirks = SDHCI_QUIRK_BROKEN_TIMEOUT_VAL |
SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK |
SDHCI_QUIRK_NO_HISPD_BIT |
SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC |
SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN,
- .ops = &tegra_sdhci_ops,
+ .ops = &tegra114_sdhci_ops,
};
static struct sdhci_tegra_soc_data soc_data_tegra114 = {
.pdata = &sdhci_tegra114_pdata,
.nvquirks = NVQUIRK_DISABLE_SDR50 |
NVQUIRK_DISABLE_DDR50 |
- NVQUIRK_DISABLE_SDR104 |
- NVQUIRK_SHADOW_XFER_MODE_REG,
+ NVQUIRK_DISABLE_SDR104,
};
static const struct of_device_id sdhci_tegra_dt_match[] = {
};
MODULE_DEVICE_TABLE(of, sdhci_tegra_dt_match);
-static int sdhci_tegra_parse_dt(struct device *dev)
-{
- struct device_node *np = dev->of_node;
- struct sdhci_host *host = dev_get_drvdata(dev);
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_tegra *tegra_host = pltfm_host->priv;
-
- tegra_host->power_gpio = of_get_named_gpio(np, "power-gpios", 0);
- return mmc_of_parse(host->mmc);
-}
-
static int sdhci_tegra_probe(struct platform_device *pdev)
{
const struct of_device_id *match;
tegra_host->soc_data = soc_data;
pltfm_host->priv = tegra_host;
- rc = sdhci_tegra_parse_dt(&pdev->dev);
+ rc = mmc_of_parse(host->mmc);
if (rc)
goto err_parse_dt;
- if (gpio_is_valid(tegra_host->power_gpio)) {
- rc = gpio_request(tegra_host->power_gpio, "sdhci_power");
- if (rc) {
- dev_err(mmc_dev(host->mmc),
- "failed to allocate power gpio\n");
- goto err_power_req;
- }
- gpio_direction_output(tegra_host->power_gpio, 1);
+ tegra_host->power_gpio = devm_gpiod_get_optional(&pdev->dev, "power",
+ GPIOD_OUT_HIGH);
+ if (IS_ERR(tegra_host->power_gpio)) {
+ rc = PTR_ERR(tegra_host->power_gpio);
+ goto err_power_req;
}
- clk = clk_get(mmc_dev(host->mmc), NULL);
+ clk = devm_clk_get(mmc_dev(host->mmc), NULL);
if (IS_ERR(clk)) {
dev_err(mmc_dev(host->mmc), "clk err\n");
rc = PTR_ERR(clk);
err_add_host:
clk_disable_unprepare(pltfm_host->clk);
- clk_put(pltfm_host->clk);
err_clk_get:
- if (gpio_is_valid(tegra_host->power_gpio))
- gpio_free(tegra_host->power_gpio);
err_power_req:
err_parse_dt:
err_alloc_tegra_host:
return rc;
}
-static int sdhci_tegra_remove(struct platform_device *pdev)
-{
- struct sdhci_host *host = platform_get_drvdata(pdev);
- struct sdhci_pltfm_host *pltfm_host = sdhci_priv(host);
- struct sdhci_tegra *tegra_host = pltfm_host->priv;
- int dead = (readl(host->ioaddr + SDHCI_INT_STATUS) == 0xffffffff);
-
- sdhci_remove_host(host, dead);
-
- if (gpio_is_valid(tegra_host->power_gpio))
- gpio_free(tegra_host->power_gpio);
-
- clk_disable_unprepare(pltfm_host->clk);
- clk_put(pltfm_host->clk);
-
- sdhci_pltfm_free(pdev);
-
- return 0;
-}
-
static struct platform_driver sdhci_tegra_driver = {
.driver = {
.name = "sdhci-tegra",
.pm = SDHCI_PLTFM_PMOPS,
},
.probe = sdhci_tegra_probe,
- .remove = sdhci_tegra_remove,
+ .remove = sdhci_pltfm_unregister,
};
module_platform_driver(sdhci_tegra_driver);
#include <linux/mmc/mmc.h>
#include <linux/mmc/host.h>
#include <linux/mmc/card.h>
+#include <linux/mmc/sdio.h>
#include <linux/mmc/slot-gpio.h>
#include "sdhci.h"
static int sdhci_pre_dma_transfer(struct sdhci_host *host,
struct mmc_data *data,
struct sdhci_host_next *next);
+static int sdhci_do_get_cd(struct sdhci_host *host);
#ifdef CONFIG_PM
static int sdhci_runtime_pm_get(struct sdhci_host *host);
* If we are sending CMD23, CMD12 never gets sent
* on successful completion (so no Auto-CMD12).
*/
- if (!host->mrq->sbc && (host->flags & SDHCI_AUTO_CMD12))
+ if (!host->mrq->sbc && (host->flags & SDHCI_AUTO_CMD12) &&
+ (cmd->opcode != SD_IO_RW_EXTENDED))
mode |= SDHCI_TRNS_AUTO_CMD12;
else if (host->mrq->sbc && (host->flags & SDHCI_AUTO_CMD23)) {
mode |= SDHCI_TRNS_AUTO_CMD23;
sdhci_runtime_pm_get(host);
- present = mmc_gpio_get_cd(host->mmc);
+ /* Firstly check card presence */
+ present = sdhci_do_get_cd(host);
spin_lock_irqsave(&host->lock, flags);
host->mrq = mrq;
- /*
- * Firstly check card presence from cd-gpio. The return could
- * be one of the following possibilities:
- * negative: cd-gpio is not available
- * zero: cd-gpio is used, and card is removed
- * one: cd-gpio is used, and card is present
- */
- if (present < 0) {
- /* If polling, assume that the card is always present. */
- if (host->quirks & SDHCI_QUIRK_BROKEN_CARD_DETECTION)
- present = 1;
- else
- present = sdhci_readl(host, SDHCI_PRESENT_STATE) &
- SDHCI_CARD_PRESENT;
- }
-
if (!present || host->flags & SDHCI_DEVICE_DEAD) {
host->mrq->cmd->error = -ENOMEDIUM;
tasklet_schedule(&host->finish_tasklet);
/* Auto-CMD23 stuff only works in ADMA or PIO. */
if ((host->version >= SDHCI_SPEC_300) &&
((host->flags & SDHCI_USE_ADMA) ||
- !(host->flags & SDHCI_USE_SDMA))) {
+ !(host->flags & SDHCI_USE_SDMA)) &&
+ !(host->quirks2 & SDHCI_QUIRK2_ACMD23_BROKEN)) {
host->flags |= SDHCI_AUTO_CMD23;
DBG("%s: Auto-CMD23 available\n", mmc_hostname(mmc));
} else {
#include <linux/types.h>
#include <linux/io.h>
-#include <linux/mmc/sdhci.h>
+#include <linux/mmc/host.h>
/*
* Controller registers
*/
#define SDHCI_MAX_SEGS 128
+struct sdhci_host_next {
+ unsigned int sg_count;
+ s32 cookie;
+};
+
+struct sdhci_host {
+ /* Data set by hardware interface driver */
+ const char *hw_name; /* Hardware bus name */
+
+ unsigned int quirks; /* Deviations from spec. */
+
+/* Controller doesn't honor resets unless we touch the clock register */
+#define SDHCI_QUIRK_CLOCK_BEFORE_RESET (1<<0)
+/* Controller has bad caps bits, but really supports DMA */
+#define SDHCI_QUIRK_FORCE_DMA (1<<1)
+/* Controller doesn't like to be reset when there is no card inserted. */
+#define SDHCI_QUIRK_NO_CARD_NO_RESET (1<<2)
+/* Controller doesn't like clearing the power reg before a change */
+#define SDHCI_QUIRK_SINGLE_POWER_WRITE (1<<3)
+/* Controller has flaky internal state so reset it on each ios change */
+#define SDHCI_QUIRK_RESET_CMD_DATA_ON_IOS (1<<4)
+/* Controller has an unusable DMA engine */
+#define SDHCI_QUIRK_BROKEN_DMA (1<<5)
+/* Controller has an unusable ADMA engine */
+#define SDHCI_QUIRK_BROKEN_ADMA (1<<6)
+/* Controller can only DMA from 32-bit aligned addresses */
+#define SDHCI_QUIRK_32BIT_DMA_ADDR (1<<7)
+/* Controller can only DMA chunk sizes that are a multiple of 32 bits */
+#define SDHCI_QUIRK_32BIT_DMA_SIZE (1<<8)
+/* Controller can only ADMA chunks that are a multiple of 32 bits */
+#define SDHCI_QUIRK_32BIT_ADMA_SIZE (1<<9)
+/* Controller needs to be reset after each request to stay stable */
+#define SDHCI_QUIRK_RESET_AFTER_REQUEST (1<<10)
+/* Controller needs voltage and power writes to happen separately */
+#define SDHCI_QUIRK_NO_SIMULT_VDD_AND_POWER (1<<11)
+/* Controller provides an incorrect timeout value for transfers */
+#define SDHCI_QUIRK_BROKEN_TIMEOUT_VAL (1<<12)
+/* Controller has an issue with buffer bits for small transfers */
+#define SDHCI_QUIRK_BROKEN_SMALL_PIO (1<<13)
+/* Controller does not provide transfer-complete interrupt when not busy */
+#define SDHCI_QUIRK_NO_BUSY_IRQ (1<<14)
+/* Controller has unreliable card detection */
+#define SDHCI_QUIRK_BROKEN_CARD_DETECTION (1<<15)
+/* Controller reports inverted write-protect state */
+#define SDHCI_QUIRK_INVERTED_WRITE_PROTECT (1<<16)
+/* Controller does not like fast PIO transfers */
+#define SDHCI_QUIRK_PIO_NEEDS_DELAY (1<<18)
+/* Controller has to be forced to use block size of 2048 bytes */
+#define SDHCI_QUIRK_FORCE_BLK_SZ_2048 (1<<20)
+/* Controller cannot do multi-block transfers */
+#define SDHCI_QUIRK_NO_MULTIBLOCK (1<<21)
+/* Controller can only handle 1-bit data transfers */
+#define SDHCI_QUIRK_FORCE_1_BIT_DATA (1<<22)
+/* Controller needs 10ms delay between applying power and clock */
+#define SDHCI_QUIRK_DELAY_AFTER_POWER (1<<23)
+/* Controller uses SDCLK instead of TMCLK for data timeouts */
+#define SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK (1<<24)
+/* Controller reports wrong base clock capability */
+#define SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN (1<<25)
+/* Controller cannot support End Attribute in NOP ADMA descriptor */
+#define SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC (1<<26)
+/* Controller is missing device caps. Use caps provided by host */
+#define SDHCI_QUIRK_MISSING_CAPS (1<<27)
+/* Controller uses Auto CMD12 command to stop the transfer */
+#define SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12 (1<<28)
+/* Controller doesn't have HISPD bit field in HI-SPEED SD card */
+#define SDHCI_QUIRK_NO_HISPD_BIT (1<<29)
+/* Controller treats ADMA descriptors with length 0000h incorrectly */
+#define SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC (1<<30)
+/* The read-only detection via SDHCI_PRESENT_STATE register is unstable */
+#define SDHCI_QUIRK_UNSTABLE_RO_DETECT (1<<31)
+
+ unsigned int quirks2; /* More deviations from spec. */
+
+#define SDHCI_QUIRK2_HOST_OFF_CARD_ON (1<<0)
+#define SDHCI_QUIRK2_HOST_NO_CMD23 (1<<1)
+/* The system physically doesn't support 1.8v, even if the host does */
+#define SDHCI_QUIRK2_NO_1_8_V (1<<2)
+#define SDHCI_QUIRK2_PRESET_VALUE_BROKEN (1<<3)
+#define SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON (1<<4)
+/* Controller has a non-standard host control register */
+#define SDHCI_QUIRK2_BROKEN_HOST_CONTROL (1<<5)
+/* Controller does not support HS200 */
+#define SDHCI_QUIRK2_BROKEN_HS200 (1<<6)
+/* Controller does not support DDR50 */
+#define SDHCI_QUIRK2_BROKEN_DDR50 (1<<7)
+/* Stop command (CMD12) can set Transfer Complete when not using MMC_RSP_BUSY */
+#define SDHCI_QUIRK2_STOP_WITH_TC (1<<8)
+/* Controller does not support 64-bit DMA */
+#define SDHCI_QUIRK2_BROKEN_64_BIT_DMA (1<<9)
+/* need clear transfer mode register before send cmd */
+#define SDHCI_QUIRK2_CLEAR_TRANSFERMODE_REG_BEFORE_CMD (1<<10)
+/* Capability register bit-63 indicates HS400 support */
+#define SDHCI_QUIRK2_CAPS_BIT63_FOR_HS400 (1<<11)
+/* forced tuned clock */
+#define SDHCI_QUIRK2_TUNING_WORK_AROUND (1<<12)
+/* disable the block count for single block transactions */
+#define SDHCI_QUIRK2_SUPPORT_SINGLE (1<<13)
+/* Controller broken with using ACMD23 */
+#define SDHCI_QUIRK2_ACMD23_BROKEN (1<<14)
+
+ int irq; /* Device IRQ */
+ void __iomem *ioaddr; /* Mapped address */
+
+ const struct sdhci_ops *ops; /* Low level hw interface */
+
+ /* Internal data */
+ struct mmc_host *mmc; /* MMC structure */
+ u64 dma_mask; /* custom DMA mask */
+
+#if defined(CONFIG_LEDS_CLASS) || defined(CONFIG_LEDS_CLASS_MODULE)
+ struct led_classdev led; /* LED control */
+ char led_name[32];
+#endif
+
+ spinlock_t lock; /* Mutex */
+
+ int flags; /* Host attributes */
+#define SDHCI_USE_SDMA (1<<0) /* Host is SDMA capable */
+#define SDHCI_USE_ADMA (1<<1) /* Host is ADMA capable */
+#define SDHCI_REQ_USE_DMA (1<<2) /* Use DMA for this req. */
+#define SDHCI_DEVICE_DEAD (1<<3) /* Device unresponsive */
+#define SDHCI_SDR50_NEEDS_TUNING (1<<4) /* SDR50 needs tuning */
+#define SDHCI_NEEDS_RETUNING (1<<5) /* Host needs retuning */
+#define SDHCI_AUTO_CMD12 (1<<6) /* Auto CMD12 support */
+#define SDHCI_AUTO_CMD23 (1<<7) /* Auto CMD23 support */
+#define SDHCI_PV_ENABLED (1<<8) /* Preset value enabled */
+#define SDHCI_SDIO_IRQ_ENABLED (1<<9) /* SDIO irq enabled */
+#define SDHCI_SDR104_NEEDS_TUNING (1<<10) /* SDR104/HS200 needs tuning */
+#define SDHCI_USING_RETUNING_TIMER (1<<11) /* Host is using a retuning timer for the card */
+#define SDHCI_USE_64_BIT_DMA (1<<12) /* Use 64-bit DMA */
+#define SDHCI_HS400_TUNING (1<<13) /* Tuning for HS400 */
+
+ unsigned int version; /* SDHCI spec. version */
+
+ unsigned int max_clk; /* Max possible freq (MHz) */
+ unsigned int timeout_clk; /* Timeout freq (KHz) */
+ unsigned int clk_mul; /* Clock Muliplier value */
+
+ unsigned int clock; /* Current clock (MHz) */
+ u8 pwr; /* Current voltage */
+
+ bool runtime_suspended; /* Host is runtime suspended */
+ bool bus_on; /* Bus power prevents runtime suspend */
+ bool preset_enabled; /* Preset is enabled */
+
+ struct mmc_request *mrq; /* Current request */
+ struct mmc_command *cmd; /* Current command */
+ struct mmc_data *data; /* Current data request */
+ unsigned int data_early:1; /* Data finished before cmd */
+ unsigned int busy_handle:1; /* Handling the order of Busy-end */
+
+ struct sg_mapping_iter sg_miter; /* SG state for PIO */
+ unsigned int blocks; /* remaining PIO blocks */
+
+ int sg_count; /* Mapped sg entries */
+
+ void *adma_table; /* ADMA descriptor table */
+ void *align_buffer; /* Bounce buffer */
+
+ size_t adma_table_sz; /* ADMA descriptor table size */
+ size_t align_buffer_sz; /* Bounce buffer size */
+
+ dma_addr_t adma_addr; /* Mapped ADMA descr. table */
+ dma_addr_t align_addr; /* Mapped bounce buffer */
+
+ unsigned int desc_sz; /* ADMA descriptor size */
+ unsigned int align_sz; /* ADMA alignment */
+ unsigned int align_mask; /* ADMA alignment mask */
+
+ struct tasklet_struct finish_tasklet; /* Tasklet structures */
+
+ struct timer_list timer; /* Timer for timeouts */
+
+ u32 caps; /* Alternative CAPABILITY_0 */
+ u32 caps1; /* Alternative CAPABILITY_1 */
+
+ unsigned int ocr_avail_sdio; /* OCR bit masks */
+ unsigned int ocr_avail_sd;
+ unsigned int ocr_avail_mmc;
+ u32 ocr_mask; /* available voltages */
+
+ unsigned timing; /* Current timing */
+
+ u32 thread_isr;
+
+ /* cached registers */
+ u32 ier;
+
+ wait_queue_head_t buf_ready_int; /* Waitqueue for Buffer Read Ready interrupt */
+ unsigned int tuning_done; /* Condition flag set when CMD19 succeeds */
+
+ unsigned int tuning_count; /* Timer count for re-tuning */
+ unsigned int tuning_mode; /* Re-tuning mode supported by host */
+#define SDHCI_TUNING_MODE_1 0
+ struct timer_list tuning_timer; /* Timer for tuning */
+
+ struct sdhci_host_next next_data;
+ unsigned long private[0] ____cacheline_aligned;
+};
+
struct sdhci_ops {
#ifdef CONFIG_MMC_SDHCI_IO_ACCESSORS
u32 (*read_l)(struct sdhci_host *host, int reg);
struct mmc_command *cmd = mrq->cmd;
u32 opc = cmd->opcode;
u32 mask;
+ unsigned long flags;
switch (opc) {
/* response busy check */
/* set arg */
sh_mmcif_writel(host->addr, MMCIF_CE_ARG, cmd->arg);
/* set cmd */
+ spin_lock_irqsave(&host->lock, flags);
sh_mmcif_writel(host->addr, MMCIF_CE_CMD_SET, opc);
host->wait_for = MMCIF_WAIT_FOR_CMD;
schedule_delayed_work(&host->timeout_work, host->timeout);
+ spin_unlock_irqrestore(&host->lock, flags);
}
static void sh_mmcif_stop_cmd(struct sh_mmcif_host *host,
struct sh_mmcif_host *host = dev_id;
struct mmc_request *mrq;
bool wait = false;
+ unsigned long flags;
+ int wait_work;
+
+ spin_lock_irqsave(&host->lock, flags);
+ wait_work = host->wait_for;
+ spin_unlock_irqrestore(&host->lock, flags);
cancel_delayed_work_sync(&host->timeout_work);
* All handlers return true, if processing continues, and false, if the
* request has to be completed - successfully or not
*/
- switch (host->wait_for) {
+ switch (wait_work) {
case MMCIF_WAIT_FOR_REQUEST:
/* We're too late, the timeout has already kicked in */
mutex_unlock(&host->thread_lock);
/* Don't run after mmc_remove_host() */
return;
- dev_err(&host->pd->dev, "Timeout waiting for %u on CMD%u\n",
- host->wait_for, mrq->cmd->opcode);
-
spin_lock_irqsave(&host->lock, flags);
if (host->state == STATE_IDLE) {
spin_unlock_irqrestore(&host->lock, flags);
return;
}
+ dev_err(&host->pd->dev, "Timeout waiting for %u on CMD%u\n",
+ host->wait_for, mrq->cmd->opcode);
+
host->state = STATE_TIMEOUT;
spin_unlock_irqrestore(&host->lock, flags);
struct mmc_data *data)
{
struct sunxi_idma_des *pdes = (struct sunxi_idma_des *)host->sg_cpu;
- struct sunxi_idma_des *pdes_pa = (struct sunxi_idma_des *)host->sg_dma;
+ dma_addr_t next_desc = host->sg_dma;
int i, max_len = (1 << host->idma_des_size_bits);
for (i = 0; i < data->sg_len; i++) {
else
pdes[i].buf_size = data->sg[i].length;
+ next_desc += sizeof(struct sunxi_idma_des);
pdes[i].buf_addr_ptr1 = sg_dma_address(&data->sg[i]);
- pdes[i].buf_addr_ptr2 = (u32)&pdes_pa[i + 1];
+ pdes[i].buf_addr_ptr2 = (u32)next_desc;
}
pdes[0].config |= SDXC_IDMAC_DES0_FD;
return PTR_ERR(host->clk_sample);
}
- host->reset = devm_reset_control_get(&pdev->dev, "ahb");
+ host->reset = devm_reset_control_get_optional(&pdev->dev, "ahb");
+ if (PTR_ERR(host->reset) == -EPROBE_DEFER)
+ return PTR_ERR(host->reset);
ret = clk_prepare_enable(host->clk_ahb);
if (ret) {
mmc->f_min = 400000;
mmc->f_max = 50000000;
mmc->caps |= MMC_CAP_MMC_HIGHSPEED | MMC_CAP_SD_HIGHSPEED |
- MMC_CAP_ERASE;
+ MMC_CAP_ERASE | MMC_CAP_SDIO_IRQ;
ret = mmc_of_parse(mmc);
if (ret)
void tmio_mmc_host_free(struct tmio_mmc_host *host)
{
mmc_free_host(host->mmc);
-
- host->mmc = NULL;
}
EXPORT_SYMBOL(tmio_mmc_host_free);
.max_blk_size = 2048,
};
-static struct of_device_id wmt_mci_dt_ids[] = {
+static const struct of_device_id wmt_mci_dt_ids[] = {
{ .compatible = "wm,wm8505-sdhc", .data = &wm8505_caps },
{ /* Sentinel */ },
};
config MTD_NAND_HISI504
tristate "Support for NAND controller on Hisilicon SoC Hip04"
+ depends on HAS_DMA
help
Enables support for NAND controller on Hisilicon SoC Hip04.
hisi_nfc_host_init(host);
- ret = devm_request_irq(dev, irq, hinfc_irq_handle, IRQF_DISABLED,
- "nandc", host);
+ ret = devm_request_irq(dev, irq, hinfc_irq_handle, 0x0, "nandc", host);
if (ret) {
dev_err(dev, "failed to request IRQ\n");
goto err_res;
nand_writel(info, NDCR, ndcr | int_mask);
}
+static void drain_fifo(struct pxa3xx_nand_info *info, void *data, int len)
+{
+ if (info->ecc_bch) {
+ int timeout;
+
+ /*
+ * According to the datasheet, when reading from NDDB
+ * with BCH enabled, after each 32 bytes reads, we
+ * have to make sure that the NDSR.RDDREQ bit is set.
+ *
+ * Drain the FIFO 8 32 bits reads at a time, and skip
+ * the polling on the last read.
+ */
+ while (len > 8) {
+ __raw_readsl(info->mmio_base + NDDB, data, 8);
+
+ for (timeout = 0;
+ !(nand_readl(info, NDSR) & NDSR_RDDREQ);
+ timeout++) {
+ if (timeout >= 5) {
+ dev_err(&info->pdev->dev,
+ "Timeout on RDDREQ while draining the FIFO\n");
+ return;
+ }
+
+ mdelay(1);
+ }
+
+ data += 32;
+ len -= 8;
+ }
+ }
+
+ __raw_readsl(info->mmio_base + NDDB, data, len);
+}
+
static void handle_data_pio(struct pxa3xx_nand_info *info)
{
unsigned int do_bytes = min(info->data_size, info->chunk_size);
DIV_ROUND_UP(info->oob_size, 4));
break;
case STATE_PIO_READING:
- __raw_readsl(info->mmio_base + NDDB,
- info->data_buff + info->data_buff_pos,
- DIV_ROUND_UP(do_bytes, 4));
+ drain_fifo(info,
+ info->data_buff + info->data_buff_pos,
+ DIV_ROUND_UP(do_bytes, 4));
if (info->oob_size > 0)
- __raw_readsl(info->mmio_base + NDDB,
- info->oob_buff + info->oob_buff_pos,
- DIV_ROUND_UP(info->oob_size, 4));
+ drain_fifo(info,
+ info->oob_buff + info->oob_buff_pos,
+ DIV_ROUND_UP(info->oob_size, 4));
break;
default:
dev_err(&info->pdev->dev, "%s: invalid state %d\n", __func__,
int ret, irq, cs;
pdata = dev_get_platdata(&pdev->dev);
+ if (pdata->num_cs <= 0)
+ return -ENODEV;
info = devm_kzalloc(&pdev->dev, sizeof(*info) + (sizeof(*mtd) +
sizeof(*host)) * pdata->num_cs, GFP_KERNEL);
if (!info)
ubi_warn(ubi, "corrupted VID header at PEB %d, LEB %d:%d",
pnum, vol_id, lnum);
err = -EBADMSG;
- } else
+ } else {
err = -EINVAL;
ubi_ro_mode(ubi);
+ }
}
goto out_free;
} else if (err == UBI_IO_BITFLIPS)
making it transparent to the connected L2 switch.
Ipvlan devices can be added using the "ip" command from the
- iproute2 package starting with the iproute2-X.Y.ZZ release:
+ iproute2 package starting with the iproute2-3.19 release:
"ip link add link <main-dev> [ NAME ] type ipvlan"
config LTPC
tristate "Apple/Farallon LocalTalk PC support"
- depends on DEV_APPLETALK && (ISA || EISA) && ISA_DMA_API
+ depends on DEV_APPLETALK && (ISA || EISA) && ISA_DMA_API && VIRT_TO_BUS
help
This allows you to use the AppleTalk PC card to connect to LocalTalk
networks. The card is also known as the Farallon PhoneNet PC card.
/* Find out if any slaves have the same mapping as this skb. */
bond_for_each_slave_rcu(bond, slave, iter) {
if (slave->queue_id == skb->queue_mapping) {
- if (bond_slave_can_tx(slave)) {
+ if (bond_slave_is_up(slave) &&
+ slave->link == BOND_LINK_UP) {
bond_dev_queue_xmit(bond, skb, slave->dev);
return 0;
}
config CAN_XILINXCAN
tristate "Xilinx CAN"
- depends on ARCH_ZYNQ || MICROBLAZE || COMPILE_TEST
+ depends on ARCH_ZYNQ || ARM64 || MICROBLAZE || COMPILE_TEST
depends on COMMON_CLK && HAS_IOMEM
---help---
Xilinx CAN driver. This driver supports both soft AXI CAN IP and
skb->pkt_type = PACKET_BROADCAST;
skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb_reset_mac_header(skb);
+ skb_reset_network_header(skb);
+ skb_reset_transport_header(skb);
+
can_skb_reserve(skb);
can_skb_prv(skb)->ifindex = dev->ifindex;
skb->pkt_type = PACKET_BROADCAST;
skb->ip_summed = CHECKSUM_UNNECESSARY;
+ skb_reset_mac_header(skb);
+ skb_reset_network_header(skb);
+ skb_reset_transport_header(skb);
+
can_skb_reserve(skb);
can_skb_prv(skb)->ifindex = dev->ifindex;
rx_state = unlikely(reg_esr & FLEXCAN_ESR_RX_WRN) ?
CAN_STATE_ERROR_WARNING : CAN_STATE_ERROR_ACTIVE;
new_state = max(tx_state, rx_state);
- } else if (unlikely(flt == FLEXCAN_ESR_FLT_CONF_PASSIVE)) {
+ } else {
__flexcan_get_berr_counter(dev, &bec);
- new_state = CAN_STATE_ERROR_PASSIVE;
+ new_state = flt == FLEXCAN_ESR_FLT_CONF_PASSIVE ?
+ CAN_STATE_ERROR_PASSIVE : CAN_STATE_BUS_OFF;
rx_state = bec.rxerr >= bec.txerr ? new_state : 0;
tx_state = bec.rxerr <= bec.txerr ? new_state : 0;
- } else {
- new_state = CAN_STATE_BUS_OFF;
}
/* state hasn't changed */
const struct flexcan_devtype_data *devtype_data;
struct net_device *dev;
struct flexcan_priv *priv;
+ struct regulator *reg_xceiver;
struct resource *mem;
struct clk *clk_ipg = NULL, *clk_per = NULL;
void __iomem *base;
int err, irq;
u32 clock_freq = 0;
+ reg_xceiver = devm_regulator_get(&pdev->dev, "xceiver");
+ if (PTR_ERR(reg_xceiver) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ else if (IS_ERR(reg_xceiver))
+ reg_xceiver = NULL;
+
if (pdev->dev.of_node)
of_property_read_u32(pdev->dev.of_node,
"clock-frequency", &clock_freq);
priv->pdata = dev_get_platdata(&pdev->dev);
priv->devtype_data = devtype_data;
- priv->reg_xceiver = devm_regulator_get(&pdev->dev, "xceiver");
- if (IS_ERR(priv->reg_xceiver))
- priv->reg_xceiver = NULL;
+ priv->reg_xceiver = reg_xceiver;
netif_napi_add(dev, &priv->napi, flexcan_poll, FLEXCAN_NAPI_WEIGHT);
}
dev = kzalloc(sizeof(*dev), GFP_KERNEL);
+ if (!dev)
+ return -ENOMEM;
init_usb_anchor(&dev->rx_submitted);
atomic_set(&dev->active_channels, 0);
* Copyright (C) 2015 Valeo S.A.
*/
+#include <linux/spinlock.h>
+#include <linux/kernel.h>
#include <linux/completion.h>
#include <linux/module.h>
#include <linux/netdevice.h>
#include <linux/can/dev.h>
#include <linux/can/error.h>
-#define MAX_TX_URBS 16
#define MAX_RX_URBS 4
#define START_TIMEOUT 1000 /* msecs */
#define STOP_TIMEOUT 1000 /* msecs */
};
};
+/* Context for an outstanding, not yet ACKed, transmission */
struct kvaser_usb_tx_urb_context {
struct kvaser_usb_net_priv *priv;
u32 echo_index;
struct usb_endpoint_descriptor *bulk_in, *bulk_out;
struct usb_anchor rx_submitted;
+ /* @max_tx_urbs: Firmware-reported maximum number of oustanding,
+ * not yet ACKed, transmissions on this device. This value is
+ * also used as a sentinel for marking free tx contexts.
+ */
u32 fw_version;
unsigned int nchannels;
+ unsigned int max_tx_urbs;
enum kvaser_usb_family family;
bool rxinitdone;
struct kvaser_usb_net_priv {
struct can_priv can;
-
- atomic_t active_tx_urbs;
- struct usb_anchor tx_submitted;
- struct kvaser_usb_tx_urb_context tx_contexts[MAX_TX_URBS];
-
- struct completion start_comp, stop_comp;
+ struct can_berr_counter bec;
struct kvaser_usb *dev;
struct net_device *netdev;
int channel;
- struct can_berr_counter bec;
+ struct completion start_comp, stop_comp;
+ struct usb_anchor tx_submitted;
+
+ spinlock_t tx_contexts_lock;
+ int active_tx_contexts;
+ struct kvaser_usb_tx_urb_context tx_contexts[];
};
static const struct usb_device_id kvaser_usb_table[] = {
while (pos <= actual_len - MSG_HEADER_LEN) {
tmp = buf + pos;
- if (!tmp->len)
- break;
+ /* Handle messages crossing the USB endpoint max packet
+ * size boundary. Check kvaser_usb_read_bulk_callback()
+ * for further details.
+ */
+ if (tmp->len == 0) {
+ pos = round_up(pos, le16_to_cpu(dev->bulk_in->
+ wMaxPacketSize));
+ continue;
+ }
if (pos + tmp->len > actual_len) {
dev_err(dev->udev->dev.parent,
switch (dev->family) {
case KVASER_LEAF:
dev->fw_version = le32_to_cpu(msg.u.leaf.softinfo.fw_version);
+ dev->max_tx_urbs =
+ le16_to_cpu(msg.u.leaf.softinfo.max_outstanding_tx);
break;
case KVASER_USBCAN:
dev->fw_version = le32_to_cpu(msg.u.usbcan.softinfo.fw_version);
+ dev->max_tx_urbs =
+ le16_to_cpu(msg.u.usbcan.softinfo.max_outstanding_tx);
break;
}
struct kvaser_usb_net_priv *priv;
struct sk_buff *skb;
struct can_frame *cf;
+ unsigned long flags;
u8 channel, tid;
channel = msg->u.tx_acknowledge_header.channel;
stats = &priv->netdev->stats;
- context = &priv->tx_contexts[tid % MAX_TX_URBS];
+ context = &priv->tx_contexts[tid % dev->max_tx_urbs];
/* Sometimes the state change doesn't come after a bus-off event */
if (priv->can.restart_ms &&
stats->tx_packets++;
stats->tx_bytes += context->dlc;
- can_get_echo_skb(priv->netdev, context->echo_index);
- context->echo_index = MAX_TX_URBS;
- atomic_dec(&priv->active_tx_urbs);
+ spin_lock_irqsave(&priv->tx_contexts_lock, flags);
+ can_get_echo_skb(priv->netdev, context->echo_index);
+ context->echo_index = dev->max_tx_urbs;
+ --priv->active_tx_contexts;
netif_wake_queue(priv->netdev);
+
+ spin_unlock_irqrestore(&priv->tx_contexts_lock, flags);
}
static void kvaser_usb_simple_msg_callback(struct urb *urb)
netdev_err(netdev, "Error transmitting URB\n");
usb_unanchor_urb(urb);
usb_free_urb(urb);
- kfree(buf);
return err;
}
return 0;
}
-static void kvaser_usb_unlink_tx_urbs(struct kvaser_usb_net_priv *priv)
-{
- int i;
-
- usb_kill_anchored_urbs(&priv->tx_submitted);
- atomic_set(&priv->active_tx_urbs, 0);
-
- for (i = 0; i < MAX_TX_URBS; i++)
- priv->tx_contexts[i].echo_index = MAX_TX_URBS;
-}
-
static void kvaser_usb_rx_error_update_can_state(struct kvaser_usb_net_priv *priv,
const struct kvaser_usb_error_summary *es,
struct can_frame *cf)
while (pos <= urb->actual_length - MSG_HEADER_LEN) {
msg = urb->transfer_buffer + pos;
- if (!msg->len)
- break;
+ /* The Kvaser firmware can only read and write messages that
+ * does not cross the USB's endpoint wMaxPacketSize boundary.
+ * If a follow-up command crosses such boundary, firmware puts
+ * a placeholder zero-length command in its place then aligns
+ * the real command to the next max packet size.
+ *
+ * Handle such cases or we're going to miss a significant
+ * number of events in case of a heavy rx load on the bus.
+ */
+ if (msg->len == 0) {
+ pos = round_up(pos, le16_to_cpu(dev->bulk_in->
+ wMaxPacketSize));
+ continue;
+ }
if (pos + msg->len > urb->actual_length) {
dev_err(dev->udev->dev.parent, "Format error\n");
}
kvaser_usb_handle_message(dev, msg);
-
pos += msg->len;
}
return err;
}
+static void kvaser_usb_reset_tx_urb_contexts(struct kvaser_usb_net_priv *priv)
+{
+ int i, max_tx_urbs;
+
+ max_tx_urbs = priv->dev->max_tx_urbs;
+
+ priv->active_tx_contexts = 0;
+ for (i = 0; i < max_tx_urbs; i++)
+ priv->tx_contexts[i].echo_index = max_tx_urbs;
+}
+
+/* This method might sleep. Do not call it in the atomic context
+ * of URB completions.
+ */
+static void kvaser_usb_unlink_tx_urbs(struct kvaser_usb_net_priv *priv)
+{
+ usb_kill_anchored_urbs(&priv->tx_submitted);
+ kvaser_usb_reset_tx_urb_contexts(priv);
+}
+
static void kvaser_usb_unlink_all_urbs(struct kvaser_usb *dev)
{
int i;
struct urb *urb;
void *buf;
struct kvaser_msg *msg;
- int i, err;
- int ret = NETDEV_TX_OK;
+ int i, err, ret = NETDEV_TX_OK;
u8 *msg_tx_can_flags = NULL; /* GCC */
+ unsigned long flags;
if (can_dropped_invalid_skb(netdev, skb))
return NETDEV_TX_OK;
if (!buf) {
stats->tx_dropped++;
dev_kfree_skb(skb);
- goto nobufmem;
+ goto freeurb;
}
msg = buf;
if (cf->can_id & CAN_RTR_FLAG)
*msg_tx_can_flags |= MSG_FLAG_REMOTE_FRAME;
- for (i = 0; i < ARRAY_SIZE(priv->tx_contexts); i++) {
- if (priv->tx_contexts[i].echo_index == MAX_TX_URBS) {
+ spin_lock_irqsave(&priv->tx_contexts_lock, flags);
+ for (i = 0; i < dev->max_tx_urbs; i++) {
+ if (priv->tx_contexts[i].echo_index == dev->max_tx_urbs) {
context = &priv->tx_contexts[i];
+
+ context->echo_index = i;
+ can_put_echo_skb(skb, netdev, context->echo_index);
+ ++priv->active_tx_contexts;
+ if (priv->active_tx_contexts >= dev->max_tx_urbs)
+ netif_stop_queue(netdev);
+
break;
}
}
+ spin_unlock_irqrestore(&priv->tx_contexts_lock, flags);
/* This should never happen; it implies a flow control bug */
if (!context) {
netdev_warn(netdev, "cannot find free context\n");
+
+ kfree(buf);
ret = NETDEV_TX_BUSY;
- goto releasebuf;
+ goto freeurb;
}
context->priv = priv;
- context->echo_index = i;
context->dlc = cf->can_dlc;
msg->u.tx_can.tid = context->echo_index;
kvaser_usb_write_bulk_callback, context);
usb_anchor_urb(urb, &priv->tx_submitted);
- can_put_echo_skb(skb, netdev, context->echo_index);
-
- atomic_inc(&priv->active_tx_urbs);
-
- if (atomic_read(&priv->active_tx_urbs) >= MAX_TX_URBS)
- netif_stop_queue(netdev);
-
err = usb_submit_urb(urb, GFP_ATOMIC);
if (unlikely(err)) {
+ spin_lock_irqsave(&priv->tx_contexts_lock, flags);
+
can_free_echo_skb(netdev, context->echo_index);
+ context->echo_index = dev->max_tx_urbs;
+ --priv->active_tx_contexts;
+ netif_wake_queue(netdev);
+
+ spin_unlock_irqrestore(&priv->tx_contexts_lock, flags);
- atomic_dec(&priv->active_tx_urbs);
usb_unanchor_urb(urb);
stats->tx_dropped++;
else
netdev_warn(netdev, "Failed tx_urb %d\n", err);
- goto releasebuf;
+ goto freeurb;
}
- usb_free_urb(urb);
-
- return NETDEV_TX_OK;
+ ret = NETDEV_TX_OK;
-releasebuf:
- kfree(buf);
-nobufmem:
+freeurb:
usb_free_urb(urb);
return ret;
}
struct kvaser_usb *dev = usb_get_intfdata(intf);
struct net_device *netdev;
struct kvaser_usb_net_priv *priv;
- int i, err;
+ int err;
err = kvaser_usb_send_simple_msg(dev, CMD_RESET_CHIP, channel);
if (err)
return err;
- netdev = alloc_candev(sizeof(*priv), MAX_TX_URBS);
+ netdev = alloc_candev(sizeof(*priv) +
+ dev->max_tx_urbs * sizeof(*priv->tx_contexts),
+ dev->max_tx_urbs);
if (!netdev) {
dev_err(&intf->dev, "Cannot alloc candev\n");
return -ENOMEM;
priv = netdev_priv(netdev);
+ init_usb_anchor(&priv->tx_submitted);
init_completion(&priv->start_comp);
init_completion(&priv->stop_comp);
- init_usb_anchor(&priv->tx_submitted);
- atomic_set(&priv->active_tx_urbs, 0);
-
- for (i = 0; i < ARRAY_SIZE(priv->tx_contexts); i++)
- priv->tx_contexts[i].echo_index = MAX_TX_URBS;
-
priv->dev = dev;
priv->netdev = netdev;
priv->channel = channel;
+ spin_lock_init(&priv->tx_contexts_lock);
+ kvaser_usb_reset_tx_urb_contexts(priv);
+
priv->can.state = CAN_STATE_STOPPED;
priv->can.clock.freq = CAN_USB_CLOCK;
priv->can.bittiming_const = &kvaser_usb_bittiming_const;
return err;
}
+ dev_dbg(&intf->dev, "Firmware version: %d.%d.%d\n",
+ ((dev->fw_version >> 24) & 0xff),
+ ((dev->fw_version >> 16) & 0xff),
+ (dev->fw_version & 0xffff));
+
+ dev_dbg(&intf->dev, "Max oustanding tx = %d URBs\n", dev->max_tx_urbs);
+
err = kvaser_usb_get_card_info(dev);
if (err) {
dev_err(&intf->dev,
return err;
}
- dev_dbg(&intf->dev, "Firmware version: %d.%d.%d\n",
- ((dev->fw_version >> 24) & 0xff),
- ((dev->fw_version >> 16) & 0xff),
- (dev->fw_version & 0xffff));
-
for (i = 0; i < dev->nchannels; i++) {
err = kvaser_usb_init_one(intf, id, i);
if (err) {
#define PUCAN_CMD_FILTER_STD 0x008
#define PUCAN_CMD_TX_ABORT 0x009
#define PUCAN_CMD_WR_ERR_CNT 0x00a
-#define PUCAN_CMD_RX_FRAME_ENABLE 0x00b
-#define PUCAN_CMD_RX_FRAME_DISABLE 0x00c
+#define PUCAN_CMD_SET_EN_OPTION 0x00b
+#define PUCAN_CMD_CLR_DIS_OPTION 0x00c
#define PUCAN_CMD_END_OF_COLLECTION 0x3ff
/* uCAN received messages list */
u16 unused;
};
-/* uCAN RX_FRAME_ENABLE command fields */
-#define PUCAN_FLTEXT_ERROR 0x0001
-#define PUCAN_FLTEXT_BUSLOAD 0x0002
+/* uCAN SET_EN/CLR_DIS _OPTION command fields */
+#define PUCAN_OPTION_ERROR 0x0001
+#define PUCAN_OPTION_BUSLOAD 0x0002
+#define PUCAN_OPTION_CANDFDISO 0x0004
-struct __packed pucan_filter_ext {
+struct __packed pucan_options {
__le16 opcode_channel;
- __le16 ext_mask;
+ __le16 options;
u32 unused;
};
u8 unused[5];
};
-/* Extended usage of uCAN commands CMD_RX_FRAME_xxxABLE for PCAN-USB Pro FD */
+/* Extended usage of uCAN commands CMD_xxx_xx_OPTION for PCAN-USB Pro FD */
#define PCAN_UFD_FLTEXT_CALIBRATION 0x8000
-struct __packed pcan_ufd_filter_ext {
+struct __packed pcan_ufd_options {
__le16 opcode_channel;
- __le16 ext_mask;
+ __le16 ucan_mask;
u16 unused;
__le16 usb_mask;
};
/* moves the pointer forward */
pc += sizeof(struct pucan_wr_err_cnt);
+ /* add command to switch from ISO to non-ISO mode, if fw allows it */
+ if (dev->can.ctrlmode_supported & CAN_CTRLMODE_FD_NON_ISO) {
+ struct pucan_options *puo = (struct pucan_options *)pc;
+
+ puo->opcode_channel =
+ (dev->can.ctrlmode & CAN_CTRLMODE_FD_NON_ISO) ?
+ pucan_cmd_opcode_channel(dev,
+ PUCAN_CMD_CLR_DIS_OPTION) :
+ pucan_cmd_opcode_channel(dev, PUCAN_CMD_SET_EN_OPTION);
+
+ puo->options = cpu_to_le16(PUCAN_OPTION_CANDFDISO);
+
+ /* to be sure that no other extended bits will be taken into
+ * account
+ */
+ puo->unused = 0;
+
+ /* moves the pointer forward */
+ pc += sizeof(struct pucan_options);
+ }
+
/* next, go back to operational mode */
cmd = (struct pucan_command *)pc;
cmd->opcode_channel = pucan_cmd_opcode_channel(dev,
return pcan_usb_fd_send_cmd(dev, cmd);
}
-/* set/unset notifications filter:
+/* set/unset options
*
- * onoff sets(1)/unset(0) notifications
- * mask each bit defines a kind of notification to set/unset
+ * onoff set(1)/unset(0) options
+ * mask each bit defines a kind of options to set/unset
*/
-static int pcan_usb_fd_set_filter_ext(struct peak_usb_device *dev,
- bool onoff, u16 ext_mask, u16 usb_mask)
+static int pcan_usb_fd_set_options(struct peak_usb_device *dev,
+ bool onoff, u16 ucan_mask, u16 usb_mask)
{
- struct pcan_ufd_filter_ext *cmd = pcan_usb_fd_cmd_buffer(dev);
+ struct pcan_ufd_options *cmd = pcan_usb_fd_cmd_buffer(dev);
cmd->opcode_channel = pucan_cmd_opcode_channel(dev,
- (onoff) ? PUCAN_CMD_RX_FRAME_ENABLE :
- PUCAN_CMD_RX_FRAME_DISABLE);
+ (onoff) ? PUCAN_CMD_SET_EN_OPTION :
+ PUCAN_CMD_CLR_DIS_OPTION);
- cmd->ext_mask = cpu_to_le16(ext_mask);
+ cmd->ucan_mask = cpu_to_le16(ucan_mask);
cmd->usb_mask = cpu_to_le16(usb_mask);
/* send the command */
&pcan_usb_pro_fd);
/* enable USB calibration messages */
- err = pcan_usb_fd_set_filter_ext(dev, 1,
- PUCAN_FLTEXT_ERROR,
- PCAN_UFD_FLTEXT_CALIBRATION);
+ err = pcan_usb_fd_set_options(dev, 1,
+ PUCAN_OPTION_ERROR,
+ PCAN_UFD_FLTEXT_CALIBRATION);
}
pdev->usb_if->dev_opened_count++;
/* turn off special msgs for that interface if no other dev opened */
if (pdev->usb_if->dev_opened_count == 1)
- pcan_usb_fd_set_filter_ext(dev, 0,
- PUCAN_FLTEXT_ERROR,
- PCAN_UFD_FLTEXT_CALIBRATION);
+ pcan_usb_fd_set_options(dev, 0,
+ PUCAN_OPTION_ERROR,
+ PCAN_UFD_FLTEXT_CALIBRATION);
pdev->usb_if->dev_opened_count--;
return 0;
pdev->usb_if->fw_info.fw_version[2],
dev->adapter->ctrl_count);
- /* the currently supported hw is non-ISO */
- dev->can.ctrlmode = CAN_CTRLMODE_FD_NON_ISO;
+ /* check for ability to switch between ISO/non-ISO modes */
+ if (pdev->usb_if->fw_info.fw_version[0] >= 2) {
+ /* firmware >= 2.x supports ISO/non-ISO switching */
+ dev->can.ctrlmode_supported |= CAN_CTRLMODE_FD_NON_ISO;
+ } else {
+ /* firmware < 2.x only supports fixed(!) non-ISO */
+ dev->can.ctrlmode |= CAN_CTRLMODE_FD_NON_ISO;
+ }
/* tell the hardware the can driver is running */
err = pcan_usb_fd_drv_loaded(dev, 1);
pdev->usb_if = ppdev->usb_if;
pdev->cmd_buffer_addr = ppdev->cmd_buffer_addr;
+
+ /* do a copy of the ctrlmode[_supported] too */
+ dev->can.ctrlmode = ppdev->dev.can.ctrlmode;
+ dev->can.ctrlmode_supported = ppdev->dev.can.ctrlmode_supported;
}
pdev->usb_if->dev[dev->ctrl_idx] = dev;
if (dev->ctrl_idx == 0) {
/* turn off calibration message if any device were opened */
if (pdev->usb_if->dev_opened_count > 0)
- pcan_usb_fd_set_filter_ext(dev, 0,
- PUCAN_FLTEXT_ERROR,
- PCAN_UFD_FLTEXT_CALIBRATION);
+ pcan_usb_fd_set_options(dev, 0,
+ PUCAN_OPTION_ERROR,
+ PCAN_UFD_FLTEXT_CALIBRATION);
/* tell USB adapter that the driver is being unloaded */
pcan_usb_fd_drv_loaded(dev, 0);
{ \
u32 indir, dir; \
spin_lock(&priv->indir_lock); \
- indir = reg_readl(priv, REG_DIR_DATA_READ); \
dir = __raw_readl(priv->name + off); \
+ indir = reg_readl(priv, REG_DIR_DATA_READ); \
spin_unlock(&priv->indir_lock); \
return (u64)indir << 32 | dir; \
} \
link->open++;
info->link_status = 0x00;
- init_timer(&info->watchdog);
- info->watchdog.function = ei_watchdog;
- info->watchdog.data = (u_long)dev;
- info->watchdog.expires = jiffies + HZ;
- add_timer(&info->watchdog);
+ setup_timer(&info->watchdog, ei_watchdog, (u_long)dev);
+ mod_timer(&info->watchdog, jiffies + HZ);
return ax_open(dev);
} /* axnet_open */
info->phy_id = info->eth_phy;
info->link_status = 0x00;
- init_timer(&info->watchdog);
- info->watchdog.function = ei_watchdog;
- info->watchdog.data = (u_long)dev;
- info->watchdog.expires = jiffies + HZ;
- add_timer(&info->watchdog);
+ setup_timer(&info->watchdog, ei_watchdog, (u_long)dev);
+ mod_timer(&info->watchdog, jiffies + HZ);
return ei_open(dev);
} /* pcnet_open */
u16 pktlength;
u16 pktstatus;
- while ((rxstatus = priv->dmaops->get_rx_status(priv)) != 0) {
+ while (((rxstatus = priv->dmaops->get_rx_status(priv)) != 0) &&
+ (count < limit)) {
pktstatus = rxstatus >> 16;
pktlength = rxstatus & 0xffff;
struct altera_tse_private *priv =
container_of(napi, struct altera_tse_private, napi);
int rxcomplete = 0;
- int txcomplete = 0;
unsigned long int flags;
- txcomplete = tse_tx_complete(priv);
+ tse_tx_complete(priv);
rxcomplete = tse_rx(priv, budget);
- if (rxcomplete >= budget || txcomplete > 0)
- return rxcomplete;
+ if (rxcomplete < budget) {
- napi_gro_flush(napi, false);
- __napi_complete(napi);
+ napi_gro_flush(napi, false);
+ __napi_complete(napi);
- netdev_dbg(priv->dev,
- "NAPI Complete, did %d packets with budget %d\n",
- txcomplete+rxcomplete, budget);
+ netdev_dbg(priv->dev,
+ "NAPI Complete, did %d packets with budget %d\n",
+ rxcomplete, budget);
- spin_lock_irqsave(&priv->rxdma_irq_lock, flags);
- priv->dmaops->enable_rxirq(priv);
- priv->dmaops->enable_txirq(priv);
- spin_unlock_irqrestore(&priv->rxdma_irq_lock, flags);
- return rxcomplete + txcomplete;
+ spin_lock_irqsave(&priv->rxdma_irq_lock, flags);
+ priv->dmaops->enable_rxirq(priv);
+ priv->dmaops->enable_txirq(priv);
+ spin_unlock_irqrestore(&priv->rxdma_irq_lock, flags);
+ }
+ return rxcomplete;
}
/* DMA TX & RX FIFO interrupt routing
{
struct net_device *dev = dev_id;
struct altera_tse_private *priv;
- unsigned long int flags;
if (unlikely(!dev)) {
pr_err("%s: invalid dev pointer\n", __func__);
}
priv = netdev_priv(dev);
- /* turn off desc irqs and enable napi rx */
- spin_lock_irqsave(&priv->rxdma_irq_lock, flags);
+ spin_lock(&priv->rxdma_irq_lock);
+ /* reset IRQs */
+ priv->dmaops->clear_rxirq(priv);
+ priv->dmaops->clear_txirq(priv);
+ spin_unlock(&priv->rxdma_irq_lock);
if (likely(napi_schedule_prep(&priv->napi))) {
+ spin_lock(&priv->rxdma_irq_lock);
priv->dmaops->disable_rxirq(priv);
priv->dmaops->disable_txirq(priv);
+ spin_unlock(&priv->rxdma_irq_lock);
__napi_schedule(&priv->napi);
}
- /* reset IRQs */
- priv->dmaops->clear_rxirq(priv);
- priv->dmaops->clear_txirq(priv);
-
- spin_unlock_irqrestore(&priv->rxdma_irq_lock, flags);
return IRQ_HANDLED;
}
}
if (of_property_read_u32(pdev->dev.of_node, "tx-fifo-depth",
- &priv->rx_fifo_depth)) {
+ &priv->tx_fifo_depth)) {
dev_err(&pdev->dev, "cannot obtain tx-fifo-depth\n");
ret = -ENXIO;
goto err_free_netdev;
{
struct pcnet32_private *lp;
int i, media;
- int fdx, mii, fset, dxsuflo;
+ int fdx, mii, fset, dxsuflo, sram;
int chip_version;
char *chipname;
struct net_device *dev;
}
/* initialize variables */
- fdx = mii = fset = dxsuflo = 0;
+ fdx = mii = fset = dxsuflo = sram = 0;
chip_version = (chip_version >> 12) & 0xffff;
switch (chip_version) {
chipname = "PCnet/FAST III 79C973"; /* PCI */
fdx = 1;
mii = 1;
+ sram = 1;
break;
case 0x2626:
chipname = "PCnet/Home 79C978"; /* PCI */
chipname = "PCnet/FAST III 79C975"; /* PCI */
fdx = 1;
mii = 1;
+ sram = 1;
break;
case 0x2628:
chipname = "PCnet/PRO 79C976";
dxsuflo = 1;
}
+ /*
+ * The Am79C973/Am79C975 controllers come with 12K of SRAM
+ * which we can use for the Tx/Rx buffers but most importantly,
+ * the use of SRAM allow us to use the BCR18:NOUFLO bit to avoid
+ * Tx fifo underflows.
+ */
+ if (sram) {
+ /*
+ * The SRAM is being configured in two steps. First we
+ * set the SRAM size in the BCR25:SRAM_SIZE bits. According
+ * to the datasheet, each bit corresponds to a 512-byte
+ * page so we can have at most 24 pages. The SRAM_SIZE
+ * holds the value of the upper 8 bits of the 16-bit SRAM size.
+ * The low 8-bits start at 0x00 and end at 0xff. So the
+ * address range is from 0x0000 up to 0x17ff. Therefore,
+ * the SRAM_SIZE is set to 0x17. The next step is to set
+ * the BCR26:SRAM_BND midway through so the Tx and Rx
+ * buffers can share the SRAM equally.
+ */
+ a->write_bcr(ioaddr, 25, 0x17);
+ a->write_bcr(ioaddr, 26, 0xc);
+ /* And finally enable the NOUFLO bit */
+ a->write_bcr(ioaddr, 18, a->read_bcr(ioaddr, 18) | (1 << 11));
+ }
+
dev = alloc_etherdev(sizeof(*lp));
if (!dev) {
ret = -ENOMEM;
}
}
+static int xgbe_request_irqs(struct xgbe_prv_data *pdata)
+{
+ struct xgbe_channel *channel;
+ struct net_device *netdev = pdata->netdev;
+ unsigned int i;
+ int ret;
+
+ ret = devm_request_irq(pdata->dev, pdata->dev_irq, xgbe_isr, 0,
+ netdev->name, pdata);
+ if (ret) {
+ netdev_alert(netdev, "error requesting irq %d\n",
+ pdata->dev_irq);
+ return ret;
+ }
+
+ if (!pdata->per_channel_irq)
+ return 0;
+
+ channel = pdata->channel;
+ for (i = 0; i < pdata->channel_count; i++, channel++) {
+ snprintf(channel->dma_irq_name,
+ sizeof(channel->dma_irq_name) - 1,
+ "%s-TxRx-%u", netdev_name(netdev),
+ channel->queue_index);
+
+ ret = devm_request_irq(pdata->dev, channel->dma_irq,
+ xgbe_dma_isr, 0,
+ channel->dma_irq_name, channel);
+ if (ret) {
+ netdev_alert(netdev, "error requesting irq %d\n",
+ channel->dma_irq);
+ goto err_irq;
+ }
+ }
+
+ return 0;
+
+err_irq:
+ /* Using an unsigned int, 'i' will go to UINT_MAX and exit */
+ for (i--, channel--; i < pdata->channel_count; i--, channel--)
+ devm_free_irq(pdata->dev, channel->dma_irq, channel);
+
+ devm_free_irq(pdata->dev, pdata->dev_irq, pdata);
+
+ return ret;
+}
+
+static void xgbe_free_irqs(struct xgbe_prv_data *pdata)
+{
+ struct xgbe_channel *channel;
+ unsigned int i;
+
+ devm_free_irq(pdata->dev, pdata->dev_irq, pdata);
+
+ if (!pdata->per_channel_irq)
+ return;
+
+ channel = pdata->channel;
+ for (i = 0; i < pdata->channel_count; i++, channel++)
+ devm_free_irq(pdata->dev, channel->dma_irq, channel);
+}
+
void xgbe_init_tx_coalesce(struct xgbe_prv_data *pdata)
{
struct xgbe_hw_if *hw_if = &pdata->hw_if;
return -EINVAL;
}
- phy_stop(pdata->phydev);
-
spin_lock_irqsave(&pdata->lock, flags);
if (caller == XGMAC_DRIVER_CONTEXT)
netif_device_detach(netdev);
netif_tx_stop_all_queues(netdev);
- xgbe_napi_disable(pdata, 0);
- /* Powerdown Tx/Rx */
hw_if->powerdown_tx(pdata);
hw_if->powerdown_rx(pdata);
+ xgbe_napi_disable(pdata, 0);
+
+ phy_stop(pdata->phydev);
+
pdata->power_down = 1;
spin_unlock_irqrestore(&pdata->lock, flags);
phy_start(pdata->phydev);
- /* Enable Tx/Rx */
+ xgbe_napi_enable(pdata, 0);
+
hw_if->powerup_tx(pdata);
hw_if->powerup_rx(pdata);
if (caller == XGMAC_DRIVER_CONTEXT)
netif_device_attach(netdev);
- xgbe_napi_enable(pdata, 0);
netif_tx_start_all_queues(netdev);
spin_unlock_irqrestore(&pdata->lock, flags);
{
struct xgbe_hw_if *hw_if = &pdata->hw_if;
struct net_device *netdev = pdata->netdev;
+ int ret;
DBGPR("-->xgbe_start\n");
phy_start(pdata->phydev);
+ xgbe_napi_enable(pdata, 1);
+
+ ret = xgbe_request_irqs(pdata);
+ if (ret)
+ goto err_napi;
+
hw_if->enable_tx(pdata);
hw_if->enable_rx(pdata);
xgbe_init_tx_timers(pdata);
- xgbe_napi_enable(pdata, 1);
netif_tx_start_all_queues(netdev);
DBGPR("<--xgbe_start\n");
return 0;
+
+err_napi:
+ xgbe_napi_disable(pdata, 1);
+
+ phy_stop(pdata->phydev);
+
+ hw_if->exit(pdata);
+
+ return ret;
}
static void xgbe_stop(struct xgbe_prv_data *pdata)
DBGPR("-->xgbe_stop\n");
- phy_stop(pdata->phydev);
-
netif_tx_stop_all_queues(netdev);
- xgbe_napi_disable(pdata, 1);
xgbe_stop_tx_timers(pdata);
hw_if->disable_tx(pdata);
hw_if->disable_rx(pdata);
+ xgbe_free_irqs(pdata);
+
+ xgbe_napi_disable(pdata, 1);
+
+ phy_stop(pdata->phydev);
+
+ hw_if->exit(pdata);
+
channel = pdata->channel;
for (i = 0; i < pdata->channel_count; i++, channel++) {
if (!channel->tx_ring)
static void xgbe_restart_dev(struct xgbe_prv_data *pdata)
{
- struct xgbe_channel *channel;
- struct xgbe_hw_if *hw_if = &pdata->hw_if;
- unsigned int i;
-
DBGPR("-->xgbe_restart_dev\n");
/* If not running, "restart" will happen on open */
return;
xgbe_stop(pdata);
- synchronize_irq(pdata->dev_irq);
- if (pdata->per_channel_irq) {
- channel = pdata->channel;
- for (i = 0; i < pdata->channel_count; i++, channel++)
- synchronize_irq(channel->dma_irq);
- }
xgbe_free_tx_data(pdata);
xgbe_free_rx_data(pdata);
- /* Issue software reset to device */
- hw_if->exit(pdata);
-
xgbe_start(pdata);
DBGPR("<--xgbe_restart_dev\n");
static int xgbe_open(struct net_device *netdev)
{
struct xgbe_prv_data *pdata = netdev_priv(netdev);
- struct xgbe_hw_if *hw_if = &pdata->hw_if;
struct xgbe_desc_if *desc_if = &pdata->desc_if;
- struct xgbe_channel *channel = NULL;
- unsigned int i = 0;
int ret;
DBGPR("-->xgbe_open\n");
INIT_WORK(&pdata->restart_work, xgbe_restart);
INIT_WORK(&pdata->tx_tstamp_work, xgbe_tx_tstamp);
- /* Request interrupts */
- ret = devm_request_irq(pdata->dev, pdata->dev_irq, xgbe_isr, 0,
- netdev->name, pdata);
- if (ret) {
- netdev_alert(netdev, "error requesting irq %d\n",
- pdata->dev_irq);
- goto err_rings;
- }
-
- if (pdata->per_channel_irq) {
- channel = pdata->channel;
- for (i = 0; i < pdata->channel_count; i++, channel++) {
- snprintf(channel->dma_irq_name,
- sizeof(channel->dma_irq_name) - 1,
- "%s-TxRx-%u", netdev_name(netdev),
- channel->queue_index);
-
- ret = devm_request_irq(pdata->dev, channel->dma_irq,
- xgbe_dma_isr, 0,
- channel->dma_irq_name, channel);
- if (ret) {
- netdev_alert(netdev,
- "error requesting irq %d\n",
- channel->dma_irq);
- goto err_irq;
- }
- }
- }
-
ret = xgbe_start(pdata);
if (ret)
- goto err_start;
+ goto err_rings;
DBGPR("<--xgbe_open\n");
return 0;
-err_start:
- hw_if->exit(pdata);
-
-err_irq:
- if (pdata->per_channel_irq) {
- /* Using an unsigned int, 'i' will go to UINT_MAX and exit */
- for (i--, channel--; i < pdata->channel_count; i--, channel--)
- devm_free_irq(pdata->dev, channel->dma_irq, channel);
- }
-
- devm_free_irq(pdata->dev, pdata->dev_irq, pdata);
-
err_rings:
desc_if->free_ring_resources(pdata);
static int xgbe_close(struct net_device *netdev)
{
struct xgbe_prv_data *pdata = netdev_priv(netdev);
- struct xgbe_hw_if *hw_if = &pdata->hw_if;
struct xgbe_desc_if *desc_if = &pdata->desc_if;
- struct xgbe_channel *channel;
- unsigned int i;
DBGPR("-->xgbe_close\n");
/* Stop the device */
xgbe_stop(pdata);
- /* Issue software reset to device */
- hw_if->exit(pdata);
-
/* Free the ring descriptors and buffers */
desc_if->free_ring_resources(pdata);
- /* Release the interrupts */
- devm_free_irq(pdata->dev, pdata->dev_irq, pdata);
- if (pdata->per_channel_irq) {
- channel = pdata->channel;
- for (i = 0; i < pdata->channel_count; i++, channel++)
- devm_free_irq(pdata->dev, channel->dma_irq, channel);
- }
-
/* Free the channel and ring structures */
xgbe_free_channels(pdata);
if (!xgene_ring_mgr_init(pdata))
return -ENODEV;
- if (!efi_enabled(EFI_BOOT)) {
+ if (pdata->clk) {
clk_prepare_enable(pdata->clk);
clk_disable_unprepare(pdata->clk);
clk_prepare_enable(pdata->clk);
#ifdef CONFIG_ACPI
static const struct acpi_device_id xgene_enet_acpi_match[] = {
{ "APMC0D05", },
+ { "APMC0D30", },
+ { "APMC0D31", },
{ }
};
MODULE_DEVICE_TABLE(acpi, xgene_enet_acpi_match);
#ifdef CONFIG_OF
static struct of_device_id xgene_enet_of_match[] = {
{.compatible = "apm,xgene-enet",},
+ {.compatible = "apm,xgene1-sgenet",},
+ {.compatible = "apm,xgene1-xgenet",},
{},
};
{
struct bcm_enet_priv *priv;
struct net_device *dev;
- int tx_work_done, rx_work_done;
+ int rx_work_done;
priv = container_of(napi, struct bcm_enet_priv, napi);
dev = priv->net_dev;
ENETDMAC_IR, priv->tx_chan);
/* reclaim sent skb */
- tx_work_done = bcm_enet_tx_reclaim(dev, 0);
+ bcm_enet_tx_reclaim(dev, 0);
spin_lock(&priv->rx_lock);
rx_work_done = bcm_enet_receive_queue(dev, budget);
spin_unlock(&priv->rx_lock);
- if (rx_work_done >= budget || tx_work_done > 0) {
- /* rx/tx queue is not yet empty/clean */
+ if (rx_work_done >= budget) {
+ /* rx queue is not yet empty/clean */
return rx_work_done;
}
/* RBUF misc statistics */
STAT_RBUF("rbuf_ovflow_cnt", mib.rbuf_ovflow_cnt, RBUF_OVFL_DISC_CNTR),
STAT_RBUF("rbuf_err_cnt", mib.rbuf_err_cnt, RBUF_ERR_PKT_CNTR),
- STAT_MIB_RX("alloc_rx_buff_failed", mib.alloc_rx_buff_failed),
- STAT_MIB_RX("rx_dma_failed", mib.rx_dma_failed),
- STAT_MIB_TX("tx_dma_failed", mib.tx_dma_failed),
+ STAT_MIB_SOFT("alloc_rx_buff_failed", mib.alloc_rx_buff_failed),
+ STAT_MIB_SOFT("rx_dma_failed", mib.rx_dma_failed),
+ STAT_MIB_SOFT("tx_dma_failed", mib.tx_dma_failed),
};
#define BCM_SYSPORT_STATS_LEN ARRAY_SIZE(bcm_sysport_gstrings_stats)
s = &bcm_sysport_gstrings_stats[i];
switch (s->type) {
case BCM_SYSPORT_STAT_NETDEV:
+ case BCM_SYSPORT_STAT_SOFT:
continue;
case BCM_SYSPORT_STAT_MIB_RX:
case BCM_SYSPORT_STAT_MIB_TX:
BCM_SYSPORT_STAT_RUNT,
BCM_SYSPORT_STAT_RXCHK,
BCM_SYSPORT_STAT_RBUF,
+ BCM_SYSPORT_STAT_SOFT,
};
/* Macros to help define ethtool statistics */
#define STAT_MIB_RX(str, m) STAT_MIB(str, m, BCM_SYSPORT_STAT_MIB_RX)
#define STAT_MIB_TX(str, m) STAT_MIB(str, m, BCM_SYSPORT_STAT_MIB_TX)
#define STAT_RUNT(str, m) STAT_MIB(str, m, BCM_SYSPORT_STAT_RUNT)
+#define STAT_MIB_SOFT(str, m) STAT_MIB(str, m, BCM_SYSPORT_STAT_SOFT)
#define STAT_RXCHK(str, m, ofs) { \
.stat_string = str, \
slot->skb = skb;
slot->dma_addr = dma_addr;
- if (slot->dma_addr & 0xC0000000)
- bgmac_warn(bgmac, "DMA address using 0xC0000000 bit(s), it may need translation trick\n");
-
return 0;
}
ring->mmio_base);
goto err_dma_free;
}
- if (ring->dma_base & 0xC0000000)
- bgmac_warn(bgmac, "DMA address using 0xC0000000 bit(s), it may need translation trick\n");
ring->unaligned = bgmac_dma_unaligned(bgmac, ring,
BGMAC_DMA_RING_TX);
err = -ENOMEM;
goto err_dma_free;
}
- if (ring->dma_base & 0xC0000000)
- bgmac_warn(bgmac, "DMA address using 0xC0000000 bit(s), it may need translation trick\n");
ring->unaligned = bgmac_dma_unaligned(bgmac, ring,
BGMAC_DMA_RING_RX);
int stats_state;
/* used for synchronization of concurrent threads statistics handling */
- spinlock_t stats_lock;
+ struct mutex stats_lock;
/* used by dmae command loader */
struct dmae_command stats_dmae;
int fp_array_size;
u32 dump_preset_idx;
- bool stats_started;
- struct semaphore stats_sema;
u8 phys_port_id[ETH_ALEN];
u32 xmac_val;
u32 emac_addr;
u32 emac_val;
- u32 umac_addr;
- u32 umac_val;
+ u32 umac_addr[2];
+ u32 umac_val[2];
u32 bmac_addr;
u32 bmac_val[2];
};
return 0;
}
+/* previous driver DMAE transaction may have occurred when pre-boot stage ended
+ * and boot began, or when kdump kernel was loaded. Either case would invalidate
+ * the addresses of the transaction, resulting in was-error bit set in the pci
+ * causing all hw-to-host pcie transactions to timeout. If this happened we want
+ * to clear the interrupt which detected this from the pglueb and the was done
+ * bit
+ */
+static void bnx2x_clean_pglue_errors(struct bnx2x *bp)
+{
+ if (!CHIP_IS_E1x(bp))
+ REG_WR(bp, PGLUE_B_REG_WAS_ERROR_PF_7_0_CLR,
+ 1 << BP_ABS_FUNC(bp));
+}
+
static int bnx2x_init_hw_func(struct bnx2x *bp)
{
int port = BP_PORT(bp);
bnx2x_init_block(bp, BLOCK_PGLUE_B, init_phase);
- if (!CHIP_IS_E1x(bp))
- REG_WR(bp, PGLUE_B_REG_WAS_ERROR_PF_7_0_CLR, func);
+ bnx2x_clean_pglue_errors(bp);
bnx2x_init_block(bp, BLOCK_ATC, init_phase);
bnx2x_init_block(bp, BLOCK_DMAE, init_phase);
return base + (BP_ABS_FUNC(bp)) * stride;
}
+static bool bnx2x_prev_unload_close_umac(struct bnx2x *bp,
+ u8 port, u32 reset_reg,
+ struct bnx2x_mac_vals *vals)
+{
+ u32 mask = MISC_REGISTERS_RESET_REG_2_UMAC0 << port;
+ u32 base_addr;
+
+ if (!(mask & reset_reg))
+ return false;
+
+ BNX2X_DEV_INFO("Disable umac Rx %02x\n", port);
+ base_addr = port ? GRCBASE_UMAC1 : GRCBASE_UMAC0;
+ vals->umac_addr[port] = base_addr + UMAC_REG_COMMAND_CONFIG;
+ vals->umac_val[port] = REG_RD(bp, vals->umac_addr[port]);
+ REG_WR(bp, vals->umac_addr[port], 0);
+
+ return true;
+}
+
static void bnx2x_prev_unload_close_mac(struct bnx2x *bp,
struct bnx2x_mac_vals *vals)
{
u8 port = BP_PORT(bp);
/* reset addresses as they also mark which values were changed */
- vals->bmac_addr = 0;
- vals->umac_addr = 0;
- vals->xmac_addr = 0;
- vals->emac_addr = 0;
+ memset(vals, 0, sizeof(*vals));
reset_reg = REG_RD(bp, MISC_REG_RESET_REG_2);
REG_WR(bp, vals->xmac_addr, 0);
mac_stopped = true;
}
- mask = MISC_REGISTERS_RESET_REG_2_UMAC0 << port;
- if (mask & reset_reg) {
- BNX2X_DEV_INFO("Disable umac Rx\n");
- base_addr = BP_PORT(bp) ? GRCBASE_UMAC1 : GRCBASE_UMAC0;
- vals->umac_addr = base_addr + UMAC_REG_COMMAND_CONFIG;
- vals->umac_val = REG_RD(bp, vals->umac_addr);
- REG_WR(bp, vals->umac_addr, 0);
- mac_stopped = true;
- }
+
+ mac_stopped |= bnx2x_prev_unload_close_umac(bp, 0,
+ reset_reg, vals);
+ mac_stopped |= bnx2x_prev_unload_close_umac(bp, 1,
+ reset_reg, vals);
}
if (mac_stopped)
/* Close the MAC Rx to prevent BRB from filling up */
bnx2x_prev_unload_close_mac(bp, &mac_vals);
- /* close LLH filters towards the BRB */
+ /* close LLH filters for both ports towards the BRB */
+ bnx2x_set_rx_filter(&bp->link_params, 0);
+ bp->link_params.port ^= 1;
bnx2x_set_rx_filter(&bp->link_params, 0);
+ bp->link_params.port ^= 1;
/* Check if the UNDI driver was previously loaded */
if (bnx2x_prev_is_after_undi(bp)) {
if (mac_vals.xmac_addr)
REG_WR(bp, mac_vals.xmac_addr, mac_vals.xmac_val);
- if (mac_vals.umac_addr)
- REG_WR(bp, mac_vals.umac_addr, mac_vals.umac_val);
+ if (mac_vals.umac_addr[0])
+ REG_WR(bp, mac_vals.umac_addr[0], mac_vals.umac_val[0]);
+ if (mac_vals.umac_addr[1])
+ REG_WR(bp, mac_vals.umac_addr[1], mac_vals.umac_val[1]);
if (mac_vals.emac_addr)
REG_WR(bp, mac_vals.emac_addr, mac_vals.emac_val);
if (mac_vals.bmac_addr) {
return bnx2x_prev_mcp_done(bp);
}
-/* previous driver DMAE transaction may have occurred when pre-boot stage ended
- * and boot began, or when kdump kernel was loaded. Either case would invalidate
- * the addresses of the transaction, resulting in was-error bit set in the pci
- * causing all hw-to-host pcie transactions to timeout. If this happened we want
- * to clear the interrupt which detected this from the pglueb and the was done
- * bit
- */
-static void bnx2x_prev_interrupted_dmae(struct bnx2x *bp)
-{
- if (!CHIP_IS_E1x(bp)) {
- u32 val = REG_RD(bp, PGLUE_B_REG_PGLUE_B_INT_STS);
- if (val & PGLUE_B_PGLUE_B_INT_STS_REG_WAS_ERROR_ATTN) {
- DP(BNX2X_MSG_SP,
- "'was error' bit was found to be set in pglueb upon startup. Clearing\n");
- REG_WR(bp, PGLUE_B_REG_WAS_ERROR_PF_7_0_CLR,
- 1 << BP_FUNC(bp));
- }
- }
-}
-
static int bnx2x_prev_unload(struct bnx2x *bp)
{
int time_counter = 10;
/* clear hw from errors which may have resulted from an interrupted
* dmae transaction.
*/
- bnx2x_prev_interrupted_dmae(bp);
+ bnx2x_clean_pglue_errors(bp);
/* Release previously held locks */
hw_lock_reg = (BP_FUNC(bp) <= 5) ?
mutex_init(&bp->port.phy_mutex);
mutex_init(&bp->fw_mb_mutex);
mutex_init(&bp->drv_info_mutex);
+ mutex_init(&bp->stats_lock);
bp->drv_info_mng_owner = false;
- spin_lock_init(&bp->stats_lock);
- sema_init(&bp->stats_sema, 1);
INIT_DELAYED_WORK(&bp->sp_task, bnx2x_sp_task);
INIT_DELAYED_WORK(&bp->sp_rtnl_task, bnx2x_sp_rtnl_task);
pci_write_config_dword(bp->pdev, PCICFG_GRC_ADDRESS,
PCICFG_VENDOR_ID_OFFSET);
+ /* Set PCIe reset type to fundamental for EEH recovery */
+ pdev->needs_freset = 1;
+
/* AER (Advanced Error reporting) configuration */
rc = pci_enable_pcie_error_reporting(pdev);
if (!rc)
NETIF_F_TSO | NETIF_F_TSO_ECN | NETIF_F_TSO6 |
NETIF_F_RXCSUM | NETIF_F_LRO | NETIF_F_GRO |
NETIF_F_RXHASH | NETIF_F_HW_VLAN_CTAG_TX;
- if (!CHIP_IS_E1x(bp)) {
+ if (!chip_is_e1x) {
dev->hw_features |= NETIF_F_GSO_GRE | NETIF_F_GSO_UDP_TUNNEL |
NETIF_F_GSO_IPIP | NETIF_F_GSO_SIT;
dev->hw_enc_features =
cancel_delayed_work_sync(&bp->sp_task);
cancel_delayed_work_sync(&bp->period_task);
- spin_lock_bh(&bp->stats_lock);
+ mutex_lock(&bp->stats_lock);
bp->stats_state = STATS_STATE_DISABLED;
- spin_unlock_bh(&bp->stats_lock);
+ mutex_unlock(&bp->stats_lock);
bnx2x_save_statistics(bp);
cookie.vf = vf;
cookie.state = VF_ACQUIRED;
- bnx2x_stats_safe_exec(bp, bnx2x_set_vf_state, &cookie);
+ rc = bnx2x_stats_safe_exec(bp, bnx2x_set_vf_state, &cookie);
+ if (rc)
+ goto op_err;
}
DP(BNX2X_MSG_IOV, "set state to acquired\n");
*/
static void bnx2x_storm_stats_post(struct bnx2x *bp)
{
- if (!bp->stats_pending) {
- int rc;
+ int rc;
- spin_lock_bh(&bp->stats_lock);
-
- if (bp->stats_pending) {
- spin_unlock_bh(&bp->stats_lock);
- return;
- }
-
- bp->fw_stats_req->hdr.drv_stats_counter =
- cpu_to_le16(bp->stats_counter++);
+ if (bp->stats_pending)
+ return;
- DP(BNX2X_MSG_STATS, "Sending statistics ramrod %d\n",
- le16_to_cpu(bp->fw_stats_req->hdr.drv_stats_counter));
+ bp->fw_stats_req->hdr.drv_stats_counter =
+ cpu_to_le16(bp->stats_counter++);
- /* adjust the ramrod to include VF queues statistics */
- bnx2x_iov_adjust_stats_req(bp);
- bnx2x_dp_stats(bp);
+ DP(BNX2X_MSG_STATS, "Sending statistics ramrod %d\n",
+ le16_to_cpu(bp->fw_stats_req->hdr.drv_stats_counter));
- /* send FW stats ramrod */
- rc = bnx2x_sp_post(bp, RAMROD_CMD_ID_COMMON_STAT_QUERY, 0,
- U64_HI(bp->fw_stats_req_mapping),
- U64_LO(bp->fw_stats_req_mapping),
- NONE_CONNECTION_TYPE);
- if (rc == 0)
- bp->stats_pending = 1;
+ /* adjust the ramrod to include VF queues statistics */
+ bnx2x_iov_adjust_stats_req(bp);
+ bnx2x_dp_stats(bp);
- spin_unlock_bh(&bp->stats_lock);
- }
+ /* send FW stats ramrod */
+ rc = bnx2x_sp_post(bp, RAMROD_CMD_ID_COMMON_STAT_QUERY, 0,
+ U64_HI(bp->fw_stats_req_mapping),
+ U64_LO(bp->fw_stats_req_mapping),
+ NONE_CONNECTION_TYPE);
+ if (rc == 0)
+ bp->stats_pending = 1;
}
static void bnx2x_hw_stats_post(struct bnx2x *bp)
*/
/* should be called under stats_sema */
-static void __bnx2x_stats_pmf_update(struct bnx2x *bp)
+static void bnx2x_stats_pmf_update(struct bnx2x *bp)
{
struct dmae_command *dmae;
u32 opcode;
}
/* should be called under stats_sema */
-static void __bnx2x_stats_start(struct bnx2x *bp)
+static void bnx2x_stats_start(struct bnx2x *bp)
{
if (IS_PF(bp)) {
if (bp->port.pmf)
bnx2x_hw_stats_post(bp);
bnx2x_storm_stats_post(bp);
}
-
- bp->stats_started = true;
-}
-
-static void bnx2x_stats_start(struct bnx2x *bp)
-{
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
- __bnx2x_stats_start(bp);
- up(&bp->stats_sema);
}
static void bnx2x_stats_pmf_start(struct bnx2x *bp)
{
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
bnx2x_stats_comp(bp);
- __bnx2x_stats_pmf_update(bp);
- __bnx2x_stats_start(bp);
- up(&bp->stats_sema);
-}
-
-static void bnx2x_stats_pmf_update(struct bnx2x *bp)
-{
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
- __bnx2x_stats_pmf_update(bp);
- up(&bp->stats_sema);
+ bnx2x_stats_pmf_update(bp);
+ bnx2x_stats_start(bp);
}
static void bnx2x_stats_restart(struct bnx2x *bp)
*/
if (IS_VF(bp))
return;
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
+
bnx2x_stats_comp(bp);
- __bnx2x_stats_start(bp);
- up(&bp->stats_sema);
+ bnx2x_stats_start(bp);
}
static void bnx2x_bmac_stats_update(struct bnx2x *bp)
{
u32 *stats_comp = bnx2x_sp(bp, stats_comp);
- /* we run update from timer context, so give up
- * if somebody is in the middle of transition
- */
- if (down_trylock(&bp->stats_sema))
+ if (bnx2x_edebug_stats_stopped(bp))
return;
- if (bnx2x_edebug_stats_stopped(bp) || !bp->stats_started)
- goto out;
-
if (IS_PF(bp)) {
if (*stats_comp != DMAE_COMP_VAL)
- goto out;
+ return;
if (bp->port.pmf)
bnx2x_hw_stats_update(bp);
BNX2X_ERR("storm stats were not updated for 3 times\n");
bnx2x_panic();
}
- goto out;
+ return;
}
} else {
/* vf doesn't collect HW statistics, and doesn't get completions
/* vf is done */
if (IS_VF(bp))
- goto out;
+ return;
if (netif_msg_timer(bp)) {
struct bnx2x_eth_stats *estats = &bp->eth_stats;
bnx2x_hw_stats_post(bp);
bnx2x_storm_stats_post(bp);
-
-out:
- up(&bp->stats_sema);
}
static void bnx2x_port_stats_stop(struct bnx2x *bp)
static void bnx2x_stats_stop(struct bnx2x *bp)
{
- int update = 0;
-
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
-
- bp->stats_started = false;
+ bool update = false;
bnx2x_stats_comp(bp);
bnx2x_hw_stats_post(bp);
bnx2x_stats_comp(bp);
}
-
- up(&bp->stats_sema);
}
static void bnx2x_stats_do_nothing(struct bnx2x *bp)
void bnx2x_stats_handle(struct bnx2x *bp, enum bnx2x_stats_event event)
{
- enum bnx2x_stats_state state;
- void (*action)(struct bnx2x *bp);
+ enum bnx2x_stats_state state = bp->stats_state;
+
if (unlikely(bp->panic))
return;
- spin_lock_bh(&bp->stats_lock);
- state = bp->stats_state;
+ /* Statistics update run from timer context, and we don't want to stop
+ * that context in case someone is in the middle of a transition.
+ * For other events, wait a bit until lock is taken.
+ */
+ if (!mutex_trylock(&bp->stats_lock)) {
+ if (event == STATS_EVENT_UPDATE)
+ return;
+
+ DP(BNX2X_MSG_STATS,
+ "Unlikely stats' lock contention [event %d]\n", event);
+ mutex_lock(&bp->stats_lock);
+ }
+
+ bnx2x_stats_stm[state][event].action(bp);
bp->stats_state = bnx2x_stats_stm[state][event].next_state;
- action = bnx2x_stats_stm[state][event].action;
- spin_unlock_bh(&bp->stats_lock);
- action(bp);
+ mutex_unlock(&bp->stats_lock);
if ((event != STATS_EVENT_UPDATE) || netif_msg_timer(bp))
DP(BNX2X_MSG_STATS, "state %d -> event %d -> state %d\n",
}
}
-void bnx2x_stats_safe_exec(struct bnx2x *bp,
- void (func_to_exec)(void *cookie),
- void *cookie){
- if (down_timeout(&bp->stats_sema, HZ/10))
- BNX2X_ERR("Unable to acquire stats lock\n");
+int bnx2x_stats_safe_exec(struct bnx2x *bp,
+ void (func_to_exec)(void *cookie),
+ void *cookie)
+{
+ int cnt = 10, rc = 0;
+
+ /* Wait for statistics to end [while blocking further requests],
+ * then run supplied function 'safely'.
+ */
+ mutex_lock(&bp->stats_lock);
+
bnx2x_stats_comp(bp);
+ while (bp->stats_pending && cnt--)
+ if (bnx2x_storm_stats_update(bp))
+ usleep_range(1000, 2000);
+ if (bp->stats_pending) {
+ BNX2X_ERR("Failed to wait for stats pending to clear [possibly FW is stuck]\n");
+ rc = -EBUSY;
+ goto out;
+ }
+
func_to_exec(cookie);
- __bnx2x_stats_start(bp);
- up(&bp->stats_sema);
+
+out:
+ /* No need to restart statistics - if they're enabled, the timer
+ * will restart the statistics.
+ */
+ mutex_unlock(&bp->stats_lock);
+
+ return rc;
}
void bnx2x_memset_stats(struct bnx2x *bp);
void bnx2x_stats_init(struct bnx2x *bp);
void bnx2x_stats_handle(struct bnx2x *bp, enum bnx2x_stats_event event);
-void bnx2x_stats_safe_exec(struct bnx2x *bp,
- void (func_to_exec)(void *cookie),
- void *cookie);
+int bnx2x_stats_safe_exec(struct bnx2x *bp,
+ void (func_to_exec)(void *cookie),
+ void *cookie);
/**
* bnx2x_save_statistics - save statistics when unloading.
BCMGENET_STAT_MIB_TX,
BCMGENET_STAT_RUNT,
BCMGENET_STAT_MISC,
+ BCMGENET_STAT_SOFT,
};
struct bcmgenet_stats {
#define STAT_GENET_MIB_RX(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_MIB_RX)
#define STAT_GENET_MIB_TX(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_MIB_TX)
#define STAT_GENET_RUNT(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_RUNT)
+#define STAT_GENET_SOFT_MIB(str, m) STAT_GENET_MIB(str, m, BCMGENET_STAT_SOFT)
#define STAT_GENET_MISC(str, m, offset) { \
.stat_string = str, \
UMAC_RBUF_OVFL_CNT),
STAT_GENET_MISC("rbuf_err_cnt", mib.rbuf_err_cnt, UMAC_RBUF_ERR_CNT),
STAT_GENET_MISC("mdf_err_cnt", mib.mdf_err_cnt, UMAC_MDF_ERR_CNT),
- STAT_GENET_MIB_RX("alloc_rx_buff_failed", mib.alloc_rx_buff_failed),
- STAT_GENET_MIB_RX("rx_dma_failed", mib.rx_dma_failed),
- STAT_GENET_MIB_TX("tx_dma_failed", mib.tx_dma_failed),
+ STAT_GENET_SOFT_MIB("alloc_rx_buff_failed", mib.alloc_rx_buff_failed),
+ STAT_GENET_SOFT_MIB("rx_dma_failed", mib.rx_dma_failed),
+ STAT_GENET_SOFT_MIB("tx_dma_failed", mib.tx_dma_failed),
};
#define BCMGENET_STATS_LEN ARRAY_SIZE(bcmgenet_gstrings_stats)
s = &bcmgenet_gstrings_stats[i];
switch (s->type) {
case BCMGENET_STAT_NETDEV:
+ case BCMGENET_STAT_SOFT:
continue;
case BCMGENET_STAT_MIB_RX:
case BCMGENET_STAT_MIB_TX:
}
/* Unlocked version of the reclaim routine */
-static void __bcmgenet_tx_reclaim(struct net_device *dev,
- struct bcmgenet_tx_ring *ring)
+static unsigned int __bcmgenet_tx_reclaim(struct net_device *dev,
+ struct bcmgenet_tx_ring *ring)
{
struct bcmgenet_priv *priv = netdev_priv(dev);
int last_tx_cn, last_c_index, num_tx_bds;
struct enet_cb *tx_cb_ptr;
struct netdev_queue *txq;
+ unsigned int pkts_compl = 0;
unsigned int bds_compl;
unsigned int c_index;
tx_cb_ptr = ring->cbs + last_c_index;
bds_compl = 0;
if (tx_cb_ptr->skb) {
+ pkts_compl++;
bds_compl = skb_shinfo(tx_cb_ptr->skb)->nr_frags + 1;
dev->stats.tx_bytes += tx_cb_ptr->skb->len;
dma_unmap_single(&dev->dev,
last_c_index &= (num_tx_bds - 1);
}
- if (ring->free_bds > (MAX_SKB_FRAGS + 1))
- ring->int_disable(priv, ring);
-
- if (netif_tx_queue_stopped(txq))
- netif_tx_wake_queue(txq);
+ if (ring->free_bds > (MAX_SKB_FRAGS + 1)) {
+ if (netif_tx_queue_stopped(txq))
+ netif_tx_wake_queue(txq);
+ }
ring->c_index = c_index;
+
+ return pkts_compl;
}
-static void bcmgenet_tx_reclaim(struct net_device *dev,
+static unsigned int bcmgenet_tx_reclaim(struct net_device *dev,
struct bcmgenet_tx_ring *ring)
{
+ unsigned int released;
unsigned long flags;
spin_lock_irqsave(&ring->lock, flags);
- __bcmgenet_tx_reclaim(dev, ring);
+ released = __bcmgenet_tx_reclaim(dev, ring);
spin_unlock_irqrestore(&ring->lock, flags);
+
+ return released;
+}
+
+static int bcmgenet_tx_poll(struct napi_struct *napi, int budget)
+{
+ struct bcmgenet_tx_ring *ring =
+ container_of(napi, struct bcmgenet_tx_ring, napi);
+ unsigned int work_done = 0;
+
+ work_done = bcmgenet_tx_reclaim(ring->priv->dev, ring);
+
+ if (work_done == 0) {
+ napi_complete(napi);
+ ring->int_enable(ring->priv, ring);
+
+ return 0;
+ }
+
+ return budget;
}
static void bcmgenet_tx_reclaim_all(struct net_device *dev)
bcmgenet_tdma_ring_writel(priv, ring->index,
ring->prod_index, TDMA_PROD_INDEX);
- if (ring->free_bds <= (MAX_SKB_FRAGS + 1)) {
+ if (ring->free_bds <= (MAX_SKB_FRAGS + 1))
netif_tx_stop_queue(txq);
- ring->int_enable(priv, ring);
- }
out:
spin_unlock_irqrestore(&ring->lock, flags);
struct device *kdev = &priv->pdev->dev;
int ret;
u32 reg, cpu_mask_clear;
+ int index;
dev_dbg(&priv->pdev->dev, "bcmgenet: init_umac\n");
bcmgenet_intr_disable(priv);
- cpu_mask_clear = UMAC_IRQ_RXDMA_BDONE;
+ cpu_mask_clear = UMAC_IRQ_RXDMA_BDONE | UMAC_IRQ_TXDMA_BDONE;
dev_dbg(kdev, "%s:Enabling RXDMA_BDONE interrupt\n", __func__);
bcmgenet_intrl2_0_writel(priv, cpu_mask_clear, INTRL2_CPU_MASK_CLEAR);
+ for (index = 0; index < priv->hw_params->tx_queues; index++)
+ bcmgenet_intrl2_1_writel(priv, (1 << index),
+ INTRL2_CPU_MASK_CLEAR);
+
/* Enable rx/tx engine.*/
dev_dbg(kdev, "done init umac\n");
unsigned int first_bd;
spin_lock_init(&ring->lock);
+ ring->priv = priv;
+ netif_napi_add(priv->dev, &ring->napi, bcmgenet_tx_poll, 64);
ring->index = index;
if (index == DESC_INDEX) {
ring->queue = 0;
TDMA_WRITE_PTR);
bcmgenet_tdma_ring_writel(priv, index, end_ptr * words_per_bd - 1,
DMA_END_ADDR);
+
+ napi_enable(&ring->napi);
+}
+
+static void bcmgenet_fini_tx_ring(struct bcmgenet_priv *priv,
+ unsigned int index)
+{
+ struct bcmgenet_tx_ring *ring = &priv->tx_rings[index];
+
+ napi_disable(&ring->napi);
+ netif_napi_del(&ring->napi);
}
/* Initialize a RDMA ring */
return ret;
}
-static void bcmgenet_fini_dma(struct bcmgenet_priv *priv)
+static void __bcmgenet_fini_dma(struct bcmgenet_priv *priv)
{
int i;
kfree(priv->tx_cbs);
}
+static void bcmgenet_fini_dma(struct bcmgenet_priv *priv)
+{
+ int i;
+
+ bcmgenet_fini_tx_ring(priv, DESC_INDEX);
+
+ for (i = 0; i < priv->hw_params->tx_queues; i++)
+ bcmgenet_fini_tx_ring(priv, i);
+
+ __bcmgenet_fini_dma(priv);
+}
+
/* init_edma: Initialize DMA control register */
static int bcmgenet_init_dma(struct bcmgenet_priv *priv)
{
priv->tx_cbs = kcalloc(priv->num_tx_bds, sizeof(struct enet_cb),
GFP_KERNEL);
if (!priv->tx_cbs) {
- bcmgenet_fini_dma(priv);
+ __bcmgenet_fini_dma(priv);
return -ENOMEM;
}
struct bcmgenet_priv, napi);
unsigned int work_done;
- /* tx reclaim */
- bcmgenet_tx_reclaim(priv->dev, &priv->tx_rings[DESC_INDEX]);
-
work_done = bcmgenet_desc_rx(priv, budget);
/* Advancing our consumer index*/
static irqreturn_t bcmgenet_isr1(int irq, void *dev_id)
{
struct bcmgenet_priv *priv = dev_id;
+ struct bcmgenet_tx_ring *ring;
unsigned int index;
/* Save irq status for bottom-half processing. */
priv->irq1_stat =
bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_STAT) &
- ~priv->int1_mask;
+ ~bcmgenet_intrl2_1_readl(priv, INTRL2_CPU_MASK_STATUS);
/* clear interrupts */
bcmgenet_intrl2_1_writel(priv, priv->irq1_stat, INTRL2_CPU_CLEAR);
netif_dbg(priv, intr, priv->dev,
"%s: IRQ=0x%x\n", __func__, priv->irq1_stat);
+
/* Check the MBDONE interrupts.
* packet is done, reclaim descriptors
*/
- if (priv->irq1_stat & 0x0000ffff) {
- index = 0;
- for (index = 0; index < 16; index++) {
- if (priv->irq1_stat & (1 << index))
- bcmgenet_tx_reclaim(priv->dev,
- &priv->tx_rings[index]);
+ for (index = 0; index < priv->hw_params->tx_queues; index++) {
+ if (!(priv->irq1_stat & BIT(index)))
+ continue;
+
+ ring = &priv->tx_rings[index];
+
+ if (likely(napi_schedule_prep(&ring->napi))) {
+ ring->int_disable(priv, ring);
+ __napi_schedule(&ring->napi);
}
}
+
return IRQ_HANDLED;
}
}
if (priv->irq0_stat &
(UMAC_IRQ_TXDMA_BDONE | UMAC_IRQ_TXDMA_PDONE)) {
- /* Tx reclaim */
- bcmgenet_tx_reclaim(priv->dev, &priv->tx_rings[DESC_INDEX]);
+ struct bcmgenet_tx_ring *ring = &priv->tx_rings[DESC_INDEX];
+
+ if (likely(napi_schedule_prep(&ring->napi))) {
+ ring->int_disable(priv, ring);
+ __napi_schedule(&ring->napi);
+ }
}
if (priv->irq0_stat & (UMAC_IRQ_PHY_DET_R |
UMAC_IRQ_PHY_DET_F |
struct bcmgenet_tx_ring {
spinlock_t lock; /* ring lock */
+ struct napi_struct napi; /* NAPI per tx queue */
unsigned int index; /* ring index */
unsigned int queue; /* queue index */
struct enet_cb *cbs; /* tx ring buffer control block*/
struct bcmgenet_tx_ring *);
void (*int_disable)(struct bcmgenet_priv *priv,
struct bcmgenet_tx_ring *);
+ struct bcmgenet_priv *priv;
};
/* device context */
if (wol->wolopts & ~(WAKE_MAGIC | WAKE_MAGICSECURE))
return -EINVAL;
+ reg = bcmgenet_umac_readl(priv, UMAC_MPD_CTRL);
if (wol->wolopts & WAKE_MAGICSECURE) {
bcmgenet_umac_writel(priv, get_unaligned_be16(&wol->sopass[0]),
UMAC_MPD_PW_MS);
bcmgenet_umac_writel(priv, get_unaligned_be32(&wol->sopass[2]),
UMAC_MPD_PW_LS);
- reg = bcmgenet_umac_readl(priv, UMAC_MPD_CTRL);
reg |= MPD_PW_EN;
- bcmgenet_umac_writel(priv, reg, UMAC_MPD_CTRL);
+ } else {
+ reg &= ~MPD_PW_EN;
}
+ bcmgenet_umac_writel(priv, reg, UMAC_MPD_CTRL);
/* Flag the device and relevant IRQ as wakeup capable */
if (wol->wolopts) {
};
#if defined(CONFIG_OF)
-static struct macb_config pc302gem_config = {
+static const struct macb_config pc302gem_config = {
.caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE,
.dma_burst_length = 16,
};
-static struct macb_config sama5d3_config = {
+static const struct macb_config sama5d3_config = {
.caps = MACB_CAPS_SG_DISABLED | MACB_CAPS_GIGABIT_MODE_AVAILABLE,
.dma_burst_length = 16,
};
-static struct macb_config sama5d4_config = {
+static const struct macb_config sama5d4_config = {
.caps = 0,
.dma_burst_length = 4,
};
if (bp->pdev->dev.of_node) {
match = of_match_node(macb_dt_ids, bp->pdev->dev.of_node);
if (match && match->data) {
- config = (const struct macb_config *)match->data;
+ config = match->data;
bp->caps = config->caps;
/*
/* Bitfields in MID */
#define MACB_IDNUM_OFFSET 16
-#define MACB_IDNUM_SIZE 16
+#define MACB_IDNUM_SIZE 12
#define MACB_REV_OFFSET 0
#define MACB_REV_SIZE 16
}
static unsigned int clip_addr_hash(struct clip_tbl *ctbl, const u32 *addr,
- int addr_len)
+ u8 v6)
{
- return addr_len == 4 ? ipv4_clip_hash(ctbl, addr) :
- ipv6_clip_hash(ctbl, addr);
+ return v6 ? ipv6_clip_hash(ctbl, addr) :
+ ipv4_clip_hash(ctbl, addr);
}
static int clip6_get_mbox(const struct net_device *dev,
struct clip_entry *ce, *cte;
u32 *addr = (u32 *)lip;
int hash;
- int addr_len;
- int ret = 0;
+ int ret = -1;
if (!ctbl)
return 0;
- if (v6)
- addr_len = 16;
- else
- addr_len = 4;
-
- hash = clip_addr_hash(ctbl, addr, addr_len);
+ hash = clip_addr_hash(ctbl, addr, v6);
read_lock_bh(&ctbl->lock);
list_for_each_entry(cte, &ctbl->hash_list[hash], list) {
- if (addr_len == cte->addr_len &&
- memcmp(lip, cte->addr, cte->addr_len) == 0) {
+ if (cte->addr6.sin6_family == AF_INET6 && v6)
+ ret = memcmp(lip, cte->addr6.sin6_addr.s6_addr,
+ sizeof(struct in6_addr));
+ else if (cte->addr.sin_family == AF_INET && !v6)
+ ret = memcmp(lip, (char *)(&cte->addr.sin_addr),
+ sizeof(struct in_addr));
+ if (!ret) {
ce = cte;
read_unlock_bh(&ctbl->lock);
goto found;
spin_lock_init(&ce->lock);
atomic_set(&ce->refcnt, 0);
atomic_dec(&ctbl->nfree);
- ce->addr_len = addr_len;
- memcpy(ce->addr, lip, addr_len);
list_add_tail(&ce->list, &ctbl->hash_list[hash]);
if (v6) {
+ ce->addr6.sin6_family = AF_INET6;
+ memcpy(ce->addr6.sin6_addr.s6_addr,
+ lip, sizeof(struct in6_addr));
ret = clip6_get_mbox(dev, (const struct in6_addr *)lip);
if (ret) {
write_unlock_bh(&ctbl->lock);
return ret;
}
+ } else {
+ ce->addr.sin_family = AF_INET;
+ memcpy((char *)(&ce->addr.sin_addr), lip,
+ sizeof(struct in_addr));
}
} else {
write_unlock_bh(&ctbl->lock);
struct clip_entry *ce, *cte;
u32 *addr = (u32 *)lip;
int hash;
- int addr_len;
-
- if (v6)
- addr_len = 16;
- else
- addr_len = 4;
+ int ret = -1;
- hash = clip_addr_hash(ctbl, addr, addr_len);
+ hash = clip_addr_hash(ctbl, addr, v6);
read_lock_bh(&ctbl->lock);
list_for_each_entry(cte, &ctbl->hash_list[hash], list) {
- if (addr_len == cte->addr_len &&
- memcmp(lip, cte->addr, cte->addr_len) == 0) {
+ if (cte->addr6.sin6_family == AF_INET6 && v6)
+ ret = memcmp(lip, cte->addr6.sin6_addr.s6_addr,
+ sizeof(struct in6_addr));
+ else if (cte->addr.sin_family == AF_INET && !v6)
+ ret = memcmp(lip, (char *)(&cte->addr.sin_addr),
+ sizeof(struct in_addr));
+ if (!ret) {
ce = cte;
read_unlock_bh(&ctbl->lock);
goto found;
for (i = 0 ; i < ctbl->clipt_size; ++i) {
list_for_each_entry(ce, &ctbl->hash_list[i], list) {
ip[0] = '\0';
- if (ce->addr_len == 16)
- sprintf(ip, "%pI6c", ce->addr);
- else
- sprintf(ip, "%pI4c", ce->addr);
+ sprintf(ip, "%pISc", &ce->addr);
seq_printf(seq, "%-25s %u\n", ip,
atomic_read(&ce->refcnt));
}
spinlock_t lock; /* Hold while modifying clip reference */
atomic_t refcnt;
struct list_head list;
- u32 addr[4];
- int addr_len;
+ union {
+ struct sockaddr_in addr;
+ struct sockaddr_in6 addr6;
+ };
};
struct clip_tbl {
enum {
INGQ_EXTRAS = 2, /* firmware event queue and */
/* forwarded interrupts */
- MAX_EGRQ = MAX_ETH_QSETS*2 + MAX_OFLD_QSETS*2
- + MAX_CTRL_QUEUES + MAX_RDMA_QUEUES + MAX_ISCSI_QUEUES,
MAX_INGQ = MAX_ETH_QSETS + MAX_OFLD_QSETS + MAX_RDMA_QUEUES
+ MAX_RDMA_CIQS + MAX_ISCSI_QUEUES + INGQ_EXTRAS,
};
unsigned int idma_qid[2]; /* SGE IDMA Hung Ingress Queue ID */
unsigned int egr_start;
+ unsigned int egr_sz;
unsigned int ingr_start;
- void *egr_map[MAX_EGRQ]; /* qid->queue egress queue map */
- struct sge_rspq *ingr_map[MAX_INGQ]; /* qid->queue ingress queue map */
- DECLARE_BITMAP(starving_fl, MAX_EGRQ);
- DECLARE_BITMAP(txq_maperr, MAX_EGRQ);
+ unsigned int ingr_sz;
+ void **egr_map; /* qid->queue egress queue map */
+ struct sge_rspq **ingr_map; /* qid->queue ingress queue map */
+ unsigned long *starving_fl;
+ unsigned long *txq_maperr;
struct timer_list rx_timer; /* refills starving FLs */
struct timer_list tx_timer; /* checks Tx queues */
};
#define T4_MEMORY_WRITE 0
#define T4_MEMORY_READ 1
int t4_memory_rw(struct adapter *adap, int win, int mtype, u32 addr, u32 len,
- __be32 *buf, int dir);
+ void *buf, int dir);
static inline int t4_memory_write(struct adapter *adap, int mtype, u32 addr,
u32 len, __be32 *buf)
{
unsigned int qtimer_val(const struct adapter *adap,
const struct sge_rspq *q);
+
+int t4_init_devlog_params(struct adapter *adapter);
int t4_init_sge_params(struct adapter *adapter);
int t4_init_tp_params(struct adapter *adap);
int t4_filter_field_shift(const struct adapter *adap, int filter_sel);
"0.9375" };
int i;
- u16 incr[NMTUS][NCCTRL_WIN];
+ u16 (*incr)[NCCTRL_WIN];
struct adapter *adap = seq->private;
+ incr = kmalloc(sizeof(*incr) * NMTUS, GFP_KERNEL);
+ if (!incr)
+ return -ENOMEM;
+
t4_read_cong_tbl(adap, incr);
for (i = 0; i < NCCTRL_WIN; ++i) {
adap->params.a_wnd[i],
dec_fac[adap->params.b_wnd[i]]);
}
+
+ kfree(incr);
return 0;
}
{
int i;
- for (i = 0; i < ARRAY_SIZE(adap->sge.ingr_map); i++) {
+ for (i = 0; i < adap->sge.ingr_sz; i++) {
struct sge_rspq *q = adap->sge.ingr_map[i];
if (q && q->handler) {
}
}
+/* Disable interrupt and napi handler */
+static void disable_interrupts(struct adapter *adap)
+{
+ if (adap->flags & FULL_INIT_DONE) {
+ t4_intr_disable(adap);
+ if (adap->flags & USING_MSIX) {
+ free_msix_queue_irqs(adap);
+ free_irq(adap->msix_info[0].vec, adap);
+ } else {
+ free_irq(adap->pdev->irq, adap);
+ }
+ quiesce_rx(adap);
+ }
+}
+
/*
* Enable NAPI scheduling and interrupt generation for all Rx queues.
*/
{
int i;
- for (i = 0; i < ARRAY_SIZE(adap->sge.ingr_map); i++) {
+ for (i = 0; i < adap->sge.ingr_sz; i++) {
struct sge_rspq *q = adap->sge.ingr_map[i];
if (!q)
int err, msi_idx, i, j;
struct sge *s = &adap->sge;
- bitmap_zero(s->starving_fl, MAX_EGRQ);
- bitmap_zero(s->txq_maperr, MAX_EGRQ);
+ bitmap_zero(s->starving_fl, s->egr_sz);
+ bitmap_zero(s->txq_maperr, s->egr_sz);
if (adap->flags & USING_MSIX)
msi_idx = 1; /* vector 0 is for non-queue interrupts */
msi_idx = -((int)s->intrq.abs_id + 1);
}
+ /* NOTE: If you add/delete any Ingress/Egress Queue allocations in here,
+ * don't forget to update the following which need to be
+ * synchronized to and changes here.
+ *
+ * 1. The calculations of MAX_INGQ in cxgb4.h.
+ *
+ * 2. Update enable_msix/name_msix_vecs/request_msix_queue_irqs
+ * to accommodate any new/deleted Ingress Queues
+ * which need MSI-X Vectors.
+ *
+ * 3. Update sge_qinfo_show() to include information on the
+ * new/deleted queues.
+ */
err = t4_sge_alloc_rxq(adap, &s->fw_evtq, true, adap->port[0],
msi_idx, NULL, fwevtq_handler);
if (err) {
static void cxgb_down(struct adapter *adapter)
{
- t4_intr_disable(adapter);
cancel_work_sync(&adapter->tid_release_task);
cancel_work_sync(&adapter->db_full_task);
cancel_work_sync(&adapter->db_drop_task);
adapter->tid_release_task_busy = false;
adapter->tid_release_head = NULL;
- if (adapter->flags & USING_MSIX) {
- free_msix_queue_irqs(adapter);
- free_irq(adapter->msix_info[0].vec, adapter);
- } else
- free_irq(adapter->pdev->irq, adapter);
- quiesce_rx(adapter);
t4_sge_stop(adapter);
t4_free_sge_resources(adapter);
adapter->flags &= ~FULL_INIT_DONE;
if (ret < 0)
return ret;
- ret = t4_cfg_pfvf(adap, adap->fn, adap->fn, 0, MAX_EGRQ, 64, MAX_INGQ,
- 0, 0, 4, 0xf, 0xf, 16, FW_CMD_CAP_PF, FW_CMD_CAP_PF);
+ ret = t4_cfg_pfvf(adap, adap->fn, adap->fn, 0, adap->sge.egr_sz, 64,
+ MAX_INGQ, 0, 0, 4, 0xf, 0xf, 16, FW_CMD_CAP_PF,
+ FW_CMD_CAP_PF);
if (ret < 0)
return ret;
enum dev_state state;
u32 params[7], val[7];
struct fw_caps_config_cmd caps_cmd;
- struct fw_devlog_cmd devlog_cmd;
- u32 devlog_meminfo;
int reset = 1;
+ /* Grab Firmware Device Log parameters as early as possible so we have
+ * access to it for debugging, etc.
+ */
+ ret = t4_init_devlog_params(adap);
+ if (ret < 0)
+ return ret;
+
/* Contact FW, advertising Master capability */
ret = t4_fw_hello(adap, adap->mbox, adap->mbox, MASTER_MAY, &state);
if (ret < 0) {
if (ret < 0)
goto bye;
- /* Read firmware device log parameters. We really need to find a way
- * to get these parameters initialized with some default values (which
- * are likely to be correct) for the case where we either don't
- * attache to the firmware or it's crashed when we probe the adapter.
- * That way we'll still be able to perform early firmware startup
- * debugging ... If the request to get the Firmware's Device Log
- * parameters fails, we'll live so we don't make that a fatal error.
- */
- memset(&devlog_cmd, 0, sizeof(devlog_cmd));
- devlog_cmd.op_to_write = htonl(FW_CMD_OP_V(FW_DEVLOG_CMD) |
- FW_CMD_REQUEST_F | FW_CMD_READ_F);
- devlog_cmd.retval_len16 = htonl(FW_LEN16(devlog_cmd));
- ret = t4_wr_mbox(adap, adap->mbox, &devlog_cmd, sizeof(devlog_cmd),
- &devlog_cmd);
- if (ret == 0) {
- devlog_meminfo =
- ntohl(devlog_cmd.memtype_devlog_memaddr16_devlog);
- adap->params.devlog.memtype =
- FW_DEVLOG_CMD_MEMTYPE_DEVLOG_G(devlog_meminfo);
- adap->params.devlog.start =
- FW_DEVLOG_CMD_MEMADDR16_DEVLOG_G(devlog_meminfo) << 4;
- adap->params.devlog.size = ntohl(devlog_cmd.memsize_devlog);
- }
-
/*
* Find out what ports are available to us. Note that we need to do
* this before calling adap_init0_no_config() since it needs nports
adap->tids.nftids = val[4] - val[3] + 1;
adap->sge.ingr_start = val[5];
+ /* qids (ingress/egress) returned from firmware can be anywhere
+ * in the range from EQ(IQFLINT)_START to EQ(IQFLINT)_END.
+ * Hence driver needs to allocate memory for this range to
+ * store the queue info. Get the highest IQFLINT/EQ index returned
+ * in FW_EQ_*_CMD.alloc command.
+ */
+ params[0] = FW_PARAM_PFVF(EQ_END);
+ params[1] = FW_PARAM_PFVF(IQFLINT_END);
+ ret = t4_query_params(adap, adap->mbox, adap->fn, 0, 2, params, val);
+ if (ret < 0)
+ goto bye;
+ adap->sge.egr_sz = val[0] - adap->sge.egr_start + 1;
+ adap->sge.ingr_sz = val[1] - adap->sge.ingr_start + 1;
+
+ adap->sge.egr_map = kcalloc(adap->sge.egr_sz,
+ sizeof(*adap->sge.egr_map), GFP_KERNEL);
+ if (!adap->sge.egr_map) {
+ ret = -ENOMEM;
+ goto bye;
+ }
+
+ adap->sge.ingr_map = kcalloc(adap->sge.ingr_sz,
+ sizeof(*adap->sge.ingr_map), GFP_KERNEL);
+ if (!adap->sge.ingr_map) {
+ ret = -ENOMEM;
+ goto bye;
+ }
+
+ /* Allocate the memory for the vaious egress queue bitmaps
+ * ie starving_fl and txq_maperr.
+ */
+ adap->sge.starving_fl = kcalloc(BITS_TO_LONGS(adap->sge.egr_sz),
+ sizeof(long), GFP_KERNEL);
+ if (!adap->sge.starving_fl) {
+ ret = -ENOMEM;
+ goto bye;
+ }
+
+ adap->sge.txq_maperr = kcalloc(BITS_TO_LONGS(adap->sge.egr_sz),
+ sizeof(long), GFP_KERNEL);
+ if (!adap->sge.txq_maperr) {
+ ret = -ENOMEM;
+ goto bye;
+ }
+
params[0] = FW_PARAM_PFVF(CLIP_START);
params[1] = FW_PARAM_PFVF(CLIP_END);
ret = t4_query_params(adap, adap->mbox, adap->fn, 0, 2, params, val);
* happened to HW/FW, stop issuing commands.
*/
bye:
+ kfree(adap->sge.egr_map);
+ kfree(adap->sge.ingr_map);
+ kfree(adap->sge.starving_fl);
+ kfree(adap->sge.txq_maperr);
if (ret != -ETIMEDOUT && ret != -EIO)
t4_fw_bye(adap, adap->mbox);
return ret;
netif_carrier_off(dev);
}
spin_unlock(&adap->stats_lock);
+ disable_interrupts(adap);
if (adap->flags & FULL_INIT_DONE)
cxgb_down(adap);
rtnl_unlock();
t4_free_mem(adapter->l2t);
t4_free_mem(adapter->tids.tid_tab);
+ kfree(adapter->sge.egr_map);
+ kfree(adapter->sge.ingr_map);
+ kfree(adapter->sge.starving_fl);
+ kfree(adapter->sge.txq_maperr);
disable_msi(adapter);
for_each_port(adapter, i)
if (is_offload(adapter))
detach_ulds(adapter);
+ disable_interrupts(adapter);
+
for_each_port(adapter, i)
if (adapter->port[i]->reg_state == NETREG_REGISTERED)
unregister_netdev(adapter->port[i]);
struct adapter *adap = (struct adapter *)data;
struct sge *s = &adap->sge;
- for (i = 0; i < ARRAY_SIZE(s->starving_fl); i++)
+ for (i = 0; i < BITS_TO_LONGS(s->egr_sz); i++)
for (m = s->starving_fl[i]; m; m &= m - 1) {
struct sge_eth_rxq *rxq;
unsigned int id = __ffs(m) + i * BITS_PER_LONG;
struct adapter *adap = (struct adapter *)data;
struct sge *s = &adap->sge;
- for (i = 0; i < ARRAY_SIZE(s->txq_maperr); i++)
+ for (i = 0; i < BITS_TO_LONGS(s->egr_sz); i++)
for (m = s->txq_maperr[i]; m; m &= m - 1) {
unsigned long id = __ffs(m) + i * BITS_PER_LONG;
struct sge_ofld_txq *txq = s->egr_map[id];
free_rspq_fl(adap, &adap->sge.intrq, NULL);
/* clear the reverse egress queue map */
- memset(adap->sge.egr_map, 0, sizeof(adap->sge.egr_map));
+ memset(adap->sge.egr_map, 0,
+ adap->sge.egr_sz * sizeof(*adap->sge.egr_map));
}
void t4_sge_start(struct adapter *adap)
* @mtype: memory type: MEM_EDC0, MEM_EDC1 or MEM_MC
* @addr: address within indicated memory type
* @len: amount of memory to transfer
- * @buf: host memory buffer
+ * @hbuf: host memory buffer
* @dir: direction of transfer T4_MEMORY_READ (1) or T4_MEMORY_WRITE (0)
*
* Reads/writes an [almost] arbitrary memory region in the firmware: the
* caller's responsibility to perform appropriate byte order conversions.
*/
int t4_memory_rw(struct adapter *adap, int win, int mtype, u32 addr,
- u32 len, __be32 *buf, int dir)
+ u32 len, void *hbuf, int dir)
{
u32 pos, offset, resid, memoffset;
u32 edc_size, mc_size, win_pf, mem_reg, mem_aperture, mem_base;
+ u32 *buf;
/* Argument sanity checks ...
*/
- if (addr & 0x3)
+ if (addr & 0x3 || (uintptr_t)hbuf & 0x3)
return -EINVAL;
+ buf = (u32 *)hbuf;
/* It's convenient to be able to handle lengths which aren't a
* multiple of 32-bits because we often end up transferring files to
/* Transfer data to/from the adapter as long as there's an integral
* number of 32-bit transfers to complete.
+ *
+ * A note on Endianness issues:
+ *
+ * The "register" reads and writes below from/to the PCI-E Memory
+ * Window invoke the standard adapter Big-Endian to PCI-E Link
+ * Little-Endian "swizzel." As a result, if we have the following
+ * data in adapter memory:
+ *
+ * Memory: ... | b0 | b1 | b2 | b3 | ...
+ * Address: i+0 i+1 i+2 i+3
+ *
+ * Then a read of the adapter memory via the PCI-E Memory Window
+ * will yield:
+ *
+ * x = readl(i)
+ * 31 0
+ * [ b3 | b2 | b1 | b0 ]
+ *
+ * If this value is stored into local memory on a Little-Endian system
+ * it will show up correctly in local memory as:
+ *
+ * ( ..., b0, b1, b2, b3, ... )
+ *
+ * But on a Big-Endian system, the store will show up in memory
+ * incorrectly swizzled as:
+ *
+ * ( ..., b3, b2, b1, b0, ... )
+ *
+ * So we need to account for this in the reads and writes to the
+ * PCI-E Memory Window below by undoing the register read/write
+ * swizzels.
*/
while (len > 0) {
if (dir == T4_MEMORY_READ)
- *buf++ = (__force __be32) t4_read_reg(adap,
- mem_base + offset);
+ *buf++ = le32_to_cpu((__force __le32)t4_read_reg(adap,
+ mem_base + offset));
else
t4_write_reg(adap, mem_base + offset,
- (__force u32) *buf++);
+ (__force u32)cpu_to_le32(*buf++));
offset += sizeof(__be32);
len -= sizeof(__be32);
*/
if (resid) {
union {
- __be32 word;
+ u32 word;
char byte[4];
} last;
unsigned char *bp;
int i;
if (dir == T4_MEMORY_READ) {
- last.word = (__force __be32) t4_read_reg(adap,
- mem_base + offset);
+ last.word = le32_to_cpu(
+ (__force __le32)t4_read_reg(adap,
+ mem_base + offset));
for (bp = (unsigned char *)buf, i = resid; i < 4; i++)
bp[i] = last.byte[i];
} else {
for (i = resid; i < 4; i++)
last.byte[i] = 0;
t4_write_reg(adap, mem_base + offset,
- (__force u32) last.word);
+ (__force u32)cpu_to_le32(last.word));
}
}
}
/* Installed successfully, update the cached header too. */
- memcpy(card_fw, fs_fw, sizeof(*card_fw));
+ *card_fw = *fs_fw;
card_fw_usable = 1;
*reset = 0; /* already reset as part of load_fw */
}
return 0;
}
+/**
+ * t4_init_devlog_params - initialize adapter->params.devlog
+ * @adap: the adapter
+ *
+ * Initialize various fields of the adapter's Firmware Device Log
+ * Parameters structure.
+ */
+int t4_init_devlog_params(struct adapter *adap)
+{
+ struct devlog_params *dparams = &adap->params.devlog;
+ u32 pf_dparams;
+ unsigned int devlog_meminfo;
+ struct fw_devlog_cmd devlog_cmd;
+ int ret;
+
+ /* If we're dealing with newer firmware, the Device Log Paramerters
+ * are stored in a designated register which allows us to access the
+ * Device Log even if we can't talk to the firmware.
+ */
+ pf_dparams =
+ t4_read_reg(adap, PCIE_FW_REG(PCIE_FW_PF_A, PCIE_FW_PF_DEVLOG));
+ if (pf_dparams) {
+ unsigned int nentries, nentries128;
+
+ dparams->memtype = PCIE_FW_PF_DEVLOG_MEMTYPE_G(pf_dparams);
+ dparams->start = PCIE_FW_PF_DEVLOG_ADDR16_G(pf_dparams) << 4;
+
+ nentries128 = PCIE_FW_PF_DEVLOG_NENTRIES128_G(pf_dparams);
+ nentries = (nentries128 + 1) * 128;
+ dparams->size = nentries * sizeof(struct fw_devlog_e);
+
+ return 0;
+ }
+
+ /* Otherwise, ask the firmware for it's Device Log Parameters.
+ */
+ memset(&devlog_cmd, 0, sizeof(devlog_cmd));
+ devlog_cmd.op_to_write = htonl(FW_CMD_OP_V(FW_DEVLOG_CMD) |
+ FW_CMD_REQUEST_F | FW_CMD_READ_F);
+ devlog_cmd.retval_len16 = htonl(FW_LEN16(devlog_cmd));
+ ret = t4_wr_mbox(adap, adap->mbox, &devlog_cmd, sizeof(devlog_cmd),
+ &devlog_cmd);
+ if (ret)
+ return ret;
+
+ devlog_meminfo = ntohl(devlog_cmd.memtype_devlog_memaddr16_devlog);
+ dparams->memtype = FW_DEVLOG_CMD_MEMTYPE_DEVLOG_G(devlog_meminfo);
+ dparams->start = FW_DEVLOG_CMD_MEMADDR16_DEVLOG_G(devlog_meminfo) << 4;
+ dparams->size = ntohl(devlog_cmd.memsize_devlog);
+
+ return 0;
+}
+
/**
* t4_init_sge_params - initialize adap->params.sge
* @adapter: the adapter
#define MC_BIST_STATUS_REG(reg_addr, idx) ((reg_addr) + (idx) * 4)
#define EDC_BIST_STATUS_REG(reg_addr, idx) ((reg_addr) + (idx) * 4)
+#define PCIE_FW_REG(reg_addr, idx) ((reg_addr) + (idx) * 4)
+
#define SGE_PF_KDOORBELL_A 0x0
#define QID_S 15
#define PFNUM_V(x) ((x) << PFNUM_S)
#define PCIE_FW_A 0x30b8
+#define PCIE_FW_PF_A 0x30bc
#define PCIE_CORE_UTL_SYSTEM_BUS_AGENT_STATUS_A 0x5908
FW_RI_BIND_MW_WR = 0x18,
FW_RI_FR_NSMR_WR = 0x19,
FW_RI_INV_LSTAG_WR = 0x1a,
- FW_LASTC2E_WR = 0x40
+ FW_LASTC2E_WR = 0x70
};
struct fw_wr_hdr {
FW_MEMTYPE_CF_EXTMEM = 0x2,
FW_MEMTYPE_CF_FLASH = 0x4,
FW_MEMTYPE_CF_INTERNAL = 0x5,
+ FW_MEMTYPE_CF_EXTMEM1 = 0x6,
};
struct fw_caps_config_cmd {
FW_PARAMS_MNEM_PFVF = 2, /* function params */
FW_PARAMS_MNEM_REG = 3, /* limited register access */
FW_PARAMS_MNEM_DMAQ = 4, /* dma queue params */
+ FW_PARAMS_MNEM_CHNET = 5, /* chnet params */
FW_PARAMS_MNEM_LAST
};
FW_DEVLOG_FACILITY_FCOE = 0x2E,
FW_DEVLOG_FACILITY_FOISCSI = 0x30,
FW_DEVLOG_FACILITY_FOFCOE = 0x32,
- FW_DEVLOG_FACILITY_MAX = 0x32,
+ FW_DEVLOG_FACILITY_CHNET = 0x34,
+ FW_DEVLOG_FACILITY_MAX = 0x34,
};
/* log message format */
(((x) >> FW_DEVLOG_CMD_MEMADDR16_DEVLOG_S) & \
FW_DEVLOG_CMD_MEMADDR16_DEVLOG_M)
+/* P C I E F W P F 7 R E G I S T E R */
+
+/* PF7 stores the Firmware Device Log parameters which allows Host Drivers to
+ * access the "devlog" which needing to contact firmware. The encoding is
+ * mostly the same as that returned by the DEVLOG command except for the size
+ * which is encoded as the number of entries in multiples-1 of 128 here rather
+ * than the memory size as is done in the DEVLOG command. Thus, 0 means 128
+ * and 15 means 2048. This of course in turn constrains the allowed values
+ * for the devlog size ...
+ */
+#define PCIE_FW_PF_DEVLOG 7
+
+#define PCIE_FW_PF_DEVLOG_NENTRIES128_S 28
+#define PCIE_FW_PF_DEVLOG_NENTRIES128_M 0xf
+#define PCIE_FW_PF_DEVLOG_NENTRIES128_V(x) \
+ ((x) << PCIE_FW_PF_DEVLOG_NENTRIES128_S)
+#define PCIE_FW_PF_DEVLOG_NENTRIES128_G(x) \
+ (((x) >> PCIE_FW_PF_DEVLOG_NENTRIES128_S) & \
+ PCIE_FW_PF_DEVLOG_NENTRIES128_M)
+
+#define PCIE_FW_PF_DEVLOG_ADDR16_S 4
+#define PCIE_FW_PF_DEVLOG_ADDR16_M 0xffffff
+#define PCIE_FW_PF_DEVLOG_ADDR16_V(x) ((x) << PCIE_FW_PF_DEVLOG_ADDR16_S)
+#define PCIE_FW_PF_DEVLOG_ADDR16_G(x) \
+ (((x) >> PCIE_FW_PF_DEVLOG_ADDR16_S) & PCIE_FW_PF_DEVLOG_ADDR16_M)
+
+#define PCIE_FW_PF_DEVLOG_MEMTYPE_S 0
+#define PCIE_FW_PF_DEVLOG_MEMTYPE_M 0xf
+#define PCIE_FW_PF_DEVLOG_MEMTYPE_V(x) ((x) << PCIE_FW_PF_DEVLOG_MEMTYPE_S)
+#define PCIE_FW_PF_DEVLOG_MEMTYPE_G(x) \
+ (((x) >> PCIE_FW_PF_DEVLOG_MEMTYPE_S) & PCIE_FW_PF_DEVLOG_MEMTYPE_M)
+
#endif /* _T4FW_INTERFACE_H_ */
#define __T4FW_VERSION_H__
#define T4FW_VERSION_MAJOR 0x01
-#define T4FW_VERSION_MINOR 0x0C
-#define T4FW_VERSION_MICRO 0x19
+#define T4FW_VERSION_MINOR 0x0D
+#define T4FW_VERSION_MICRO 0x20
#define T4FW_VERSION_BUILD 0x00
#define T5FW_VERSION_MAJOR 0x01
-#define T5FW_VERSION_MINOR 0x0C
-#define T5FW_VERSION_MICRO 0x19
+#define T5FW_VERSION_MINOR 0x0D
+#define T5FW_VERSION_MICRO 0x20
#define T5FW_VERSION_BUILD 0x00
#endif
? (tq->pidx - 1)
: (tq->size - 1));
__be64 *src = (__be64 *)&tq->desc[index];
- __be64 __iomem *dst = (__be64 *)(tq->bar2_addr +
+ __be64 __iomem *dst = (__be64 __iomem *)(tq->bar2_addr +
SGE_UDB_WCDOORBELL);
unsigned int count = EQ_UNIT / sizeof(__be64);
* DMA.
*/
while (count) {
- writeq(*src, dst);
+ /* the (__force u64) is because the compiler
+ * doesn't understand the endian swizzling
+ * going on
+ */
+ writeq((__force u64)*src, dst);
src++;
dst++;
count--;
BUG_ON(DIV_ROUND_UP(ETHTXQ_MAX_HDR, TXD_PER_EQ_UNIT) > 1);
wr = (void *)&txq->q.desc[txq->q.pidx];
wr->equiq_to_len16 = cpu_to_be32(wr_mid);
- wr->r3[0] = cpu_to_be64(0);
- wr->r3[1] = cpu_to_be64(0);
+ wr->r3[0] = cpu_to_be32(0);
+ wr->r3[1] = cpu_to_be32(0);
skb_copy_from_linear_data(skb, (void *)wr->ethmacdst, fw_hdr_copy_len);
end = (u64 *)wr + flits;
if (rpl) {
/* request bit in high-order BE word */
- WARN_ON((be32_to_cpu(*(const u32 *)cmd)
+ WARN_ON((be32_to_cpu(*(const __be32 *)cmd)
& FW_CMD_REQUEST_F) == 0);
get_mbox_rpl(adapter, rpl, size, mbox_data);
- WARN_ON((be32_to_cpu(*(u32 *)rpl)
+ WARN_ON((be32_to_cpu(*(__be32 *)rpl)
& FW_CMD_REQUEST_F) != 0);
}
t4_write_reg(adapter, mbox_ctl,
* o The BAR2 Queue ID.
* o The BAR2 Queue ID Offset into the BAR2 page.
*/
- bar2_page_offset = ((qid >> qpp_shift) << page_shift);
+ bar2_page_offset = ((u64)(qid >> qpp_shift) << page_shift);
bar2_qid = qid & qpp_mask;
bar2_qid_offset = bar2_qid * SGE_UDB_SIZE;
}
if (ENIC_TEST_INTR(pba, notify_intr)) {
- vnic_intr_return_all_credits(&enic->intr[notify_intr]);
enic_notify_check(enic);
+ vnic_intr_return_all_credits(&enic->intr[notify_intr]);
}
if (ENIC_TEST_INTR(pba, err_intr)) {
struct enic *enic = data;
unsigned int intr = enic_msix_notify_intr(enic);
- vnic_intr_return_all_credits(&enic->intr[intr]);
enic_notify_check(enic);
+ vnic_intr_return_all_credits(&enic->intr[intr]);
return IRQ_HANDLED;
}
(unsigned int)tp->rx_ring[i].buffer1,
(unsigned int)tp->rx_ring[i].buffer2,
buf[0], buf[1], buf[2]);
- for (j = 0; buf[j] != 0xee && j < 1600; j++)
+ for (j = 0; ((j < 1600) && buf[j] != 0xee); j++)
if (j < 100)
pr_cont(" %02x", buf[j]);
pr_cont(" j=%d\n", j);
u16 vlan_tag;
u32 tx_rate;
u32 plink_tracking;
+ u32 privileges;
};
enum vf_state {
u8 __iomem *csr; /* CSR BAR used only for BE2/3 */
u8 __iomem *db; /* Door Bell */
+ u8 __iomem *pcicfg; /* On SH,BEx only. Shadow of PCI config space */
struct mutex mbox_lock; /* For serializing mbox cmds to BE card */
struct be_dma_mem mbox_mem;
{
int num_eqs, i = 0;
- if (lancer_chip(adapter) && num > 8) {
- while (num) {
- num_eqs = min(num, 8);
- __be_cmd_modify_eqd(adapter, &set_eqd[i], num_eqs);
- i += num_eqs;
- num -= num_eqs;
- }
- } else {
- __be_cmd_modify_eqd(adapter, set_eqd, num);
+ while (num) {
+ num_eqs = min(num, 8);
+ __be_cmd_modify_eqd(adapter, &set_eqd[i], num_eqs);
+ i += num_eqs;
+ num -= num_eqs;
}
return 0;
/* Uses sycnhronous mcc */
int be_cmd_vlan_config(struct be_adapter *adapter, u32 if_id, u16 *vtag_array,
- u32 num)
+ u32 num, u32 domain)
{
struct be_mcc_wrb *wrb;
struct be_cmd_req_vlan_config *req;
be_wrb_cmd_hdr_prepare(&req->hdr, CMD_SUBSYSTEM_COMMON,
OPCODE_COMMON_NTWK_VLAN_CONFIG, sizeof(*req),
wrb, NULL);
+ req->hdr.domain = domain;
req->interface_id = if_id;
req->untagged = BE_IF_FLAGS_UNTAGGED & be_if_cap_flags(adapter) ? 1 : 0;
int be_cmd_get_fw_ver(struct be_adapter *adapter);
int be_cmd_modify_eqd(struct be_adapter *adapter, struct be_set_eqd *, int num);
int be_cmd_vlan_config(struct be_adapter *adapter, u32 if_id, u16 *vtag_array,
- u32 num);
+ u32 num, u32 domain);
int be_cmd_rx_filter(struct be_adapter *adapter, u32 flags, u32 status);
int be_cmd_set_flow_control(struct be_adapter *adapter, u32 tx_fc, u32 rx_fc);
int be_cmd_get_flow_control(struct be_adapter *adapter, u32 *tx_fc, u32 *rx_fc);
for_each_set_bit(i, adapter->vids, VLAN_N_VID)
vids[num++] = cpu_to_le16(i);
- status = be_cmd_vlan_config(adapter, adapter->if_handle, vids, num);
+ status = be_cmd_vlan_config(adapter, adapter->if_handle, vids, num, 0);
if (status) {
dev_err(dev, "Setting HW VLAN filtering failed\n");
/* Set to VLAN promisc mode as setting VLAN filter failed */
return 0;
}
+static int be_set_vf_tvt(struct be_adapter *adapter, int vf, u16 vlan)
+{
+ struct be_vf_cfg *vf_cfg = &adapter->vf_cfg[vf];
+ u16 vids[BE_NUM_VLANS_SUPPORTED];
+ int vf_if_id = vf_cfg->if_handle;
+ int status;
+
+ /* Enable Transparent VLAN Tagging */
+ status = be_cmd_set_hsw_config(adapter, vlan, vf + 1, vf_if_id, 0);
+ if (status)
+ return status;
+
+ /* Clear pre-programmed VLAN filters on VF if any, if TVT is enabled */
+ vids[0] = 0;
+ status = be_cmd_vlan_config(adapter, vf_if_id, vids, 1, vf + 1);
+ if (!status)
+ dev_info(&adapter->pdev->dev,
+ "Cleared guest VLANs on VF%d", vf);
+
+ /* After TVT is enabled, disallow VFs to program VLAN filters */
+ if (vf_cfg->privileges & BE_PRIV_FILTMGMT) {
+ status = be_cmd_set_fn_privileges(adapter, vf_cfg->privileges &
+ ~BE_PRIV_FILTMGMT, vf + 1);
+ if (!status)
+ vf_cfg->privileges &= ~BE_PRIV_FILTMGMT;
+ }
+ return 0;
+}
+
+static int be_clear_vf_tvt(struct be_adapter *adapter, int vf)
+{
+ struct be_vf_cfg *vf_cfg = &adapter->vf_cfg[vf];
+ struct device *dev = &adapter->pdev->dev;
+ int status;
+
+ /* Reset Transparent VLAN Tagging. */
+ status = be_cmd_set_hsw_config(adapter, BE_RESET_VLAN_TAG_ID, vf + 1,
+ vf_cfg->if_handle, 0);
+ if (status)
+ return status;
+
+ /* Allow VFs to program VLAN filtering */
+ if (!(vf_cfg->privileges & BE_PRIV_FILTMGMT)) {
+ status = be_cmd_set_fn_privileges(adapter, vf_cfg->privileges |
+ BE_PRIV_FILTMGMT, vf + 1);
+ if (!status) {
+ vf_cfg->privileges |= BE_PRIV_FILTMGMT;
+ dev_info(dev, "VF%d: FILTMGMT priv enabled", vf);
+ }
+ }
+
+ dev_info(dev,
+ "Disable/re-enable i/f in VM to clear Transparent VLAN tag");
+ return 0;
+}
+
static int be_set_vf_vlan(struct net_device *netdev, int vf, u16 vlan, u8 qos)
{
struct be_adapter *adapter = netdev_priv(netdev);
struct be_vf_cfg *vf_cfg = &adapter->vf_cfg[vf];
- int status = 0;
+ int status;
if (!sriov_enabled(adapter))
return -EPERM;
if (vlan || qos) {
vlan |= qos << VLAN_PRIO_SHIFT;
- if (vf_cfg->vlan_tag != vlan)
- status = be_cmd_set_hsw_config(adapter, vlan, vf + 1,
- vf_cfg->if_handle, 0);
+ status = be_set_vf_tvt(adapter, vf, vlan);
} else {
- /* Reset Transparent Vlan Tagging. */
- status = be_cmd_set_hsw_config(adapter, BE_RESET_VLAN_TAG_ID,
- vf + 1, vf_cfg->if_handle, 0);
+ status = be_clear_vf_tvt(adapter, vf);
}
if (status) {
dev_err(&adapter->pdev->dev,
- "VLAN %d config on VF %d failed : %#x\n", vlan,
- vf, status);
+ "VLAN %d config on VF %d failed : %#x\n", vlan, vf,
+ status);
return be_cmd_status(status);
}
vf_cfg->vlan_tag = vlan;
-
return 0;
}
}
}
} else {
- pci_read_config_dword(adapter->pdev,
- PCICFG_UE_STATUS_LOW, &ue_lo);
- pci_read_config_dword(adapter->pdev,
- PCICFG_UE_STATUS_HIGH, &ue_hi);
- pci_read_config_dword(adapter->pdev,
- PCICFG_UE_STATUS_LOW_MASK, &ue_lo_mask);
- pci_read_config_dword(adapter->pdev,
- PCICFG_UE_STATUS_HI_MASK, &ue_hi_mask);
+ ue_lo = ioread32(adapter->pcicfg + PCICFG_UE_STATUS_LOW);
+ ue_hi = ioread32(adapter->pcicfg + PCICFG_UE_STATUS_HIGH);
+ ue_lo_mask = ioread32(adapter->pcicfg +
+ PCICFG_UE_STATUS_LOW_MASK);
+ ue_hi_mask = ioread32(adapter->pcicfg +
+ PCICFG_UE_STATUS_HI_MASK);
ue_lo = (ue_lo & ~ue_lo_mask);
ue_hi = (ue_hi & ~ue_hi_mask);
u32 cap_flags, u32 vf)
{
u32 en_flags;
- int status;
en_flags = BE_IF_FLAGS_UNTAGGED | BE_IF_FLAGS_BROADCAST |
BE_IF_FLAGS_MULTICAST | BE_IF_FLAGS_PASS_L3L4_ERRORS |
en_flags &= cap_flags;
- status = be_cmd_if_create(adapter, cap_flags, en_flags,
- if_handle, vf);
-
- return status;
+ return be_cmd_if_create(adapter, cap_flags, en_flags, if_handle, vf);
}
static int be_vfs_if_create(struct be_adapter *adapter)
if (!BE3_chip(adapter)) {
status = be_cmd_get_profile_config(adapter, &res,
vf + 1);
- if (!status)
+ if (!status) {
cap_flags = res.if_cap_flags;
+ /* Prevent VFs from enabling VLAN promiscuous
+ * mode
+ */
+ cap_flags &= ~BE_IF_FLAGS_VLAN_PROMISCUOUS;
+ }
}
status = be_if_create(adapter, &vf_cfg->if_handle,
struct device *dev = &adapter->pdev->dev;
struct be_vf_cfg *vf_cfg;
int status, old_vfs, vf;
- u32 privileges;
old_vfs = pci_num_vf(adapter->pdev);
for_all_vfs(adapter, vf_cfg, vf) {
/* Allow VFs to programs MAC/VLAN filters */
- status = be_cmd_get_fn_privileges(adapter, &privileges, vf + 1);
- if (!status && !(privileges & BE_PRIV_FILTMGMT)) {
+ status = be_cmd_get_fn_privileges(adapter, &vf_cfg->privileges,
+ vf + 1);
+ if (!status && !(vf_cfg->privileges & BE_PRIV_FILTMGMT)) {
status = be_cmd_set_fn_privileges(adapter,
- privileges |
+ vf_cfg->privileges |
BE_PRIV_FILTMGMT,
vf + 1);
- if (!status)
+ if (!status) {
+ vf_cfg->privileges |= BE_PRIV_FILTMGMT;
dev_info(dev, "VF%d has FILTMGMT privilege\n",
vf);
+ }
}
/* Allow full available bandwidth */
static int be_map_pci_bars(struct be_adapter *adapter)
{
+ struct pci_dev *pdev = adapter->pdev;
u8 __iomem *addr;
if (BEx_chip(adapter) && be_physfn(adapter)) {
- adapter->csr = pci_iomap(adapter->pdev, 2, 0);
+ adapter->csr = pci_iomap(pdev, 2, 0);
if (!adapter->csr)
return -ENOMEM;
}
- addr = pci_iomap(adapter->pdev, db_bar(adapter), 0);
+ addr = pci_iomap(pdev, db_bar(adapter), 0);
if (!addr)
goto pci_map_err;
adapter->db = addr;
+ if (skyhawk_chip(adapter) || BEx_chip(adapter)) {
+ if (be_physfn(adapter)) {
+ /* PCICFG is the 2nd BAR in BE2 */
+ addr = pci_iomap(pdev, BE2_chip(adapter) ? 1 : 0, 0);
+ if (!addr)
+ goto pci_map_err;
+ adapter->pcicfg = addr;
+ } else {
+ adapter->pcicfg = adapter->db + SRIOV_VF_PCICFG_OFFSET;
+ }
+ }
+
be_roce_map_pci_bars(adapter);
return 0;
pci_map_err:
- dev_err(&adapter->pdev->dev, "Error in mapping PCI BARs\n");
+ dev_err(&pdev->dev, "Error in mapping PCI BARs\n");
be_unmap_pci_bars(adapter);
return -ENOMEM;
}
fec_enet_tx_queue(struct net_device *ndev, u16 queue_id)
{
struct fec_enet_private *fep;
- struct bufdesc *bdp, *bdp_t;
+ struct bufdesc *bdp;
unsigned short status;
struct sk_buff *skb;
struct fec_enet_priv_tx_q *txq;
struct netdev_queue *nq;
int index = 0;
- int i, bdnum;
int entries_free;
fep = netdev_priv(ndev);
if (bdp == txq->cur_tx)
break;
- bdp_t = bdp;
- bdnum = 1;
- index = fec_enet_get_bd_index(txq->tx_bd_base, bdp_t, fep);
- skb = txq->tx_skbuff[index];
- while (!skb) {
- bdp_t = fec_enet_get_nextdesc(bdp_t, fep, queue_id);
- index = fec_enet_get_bd_index(txq->tx_bd_base, bdp_t, fep);
- skb = txq->tx_skbuff[index];
- bdnum++;
- }
- if (skb_shinfo(skb)->nr_frags &&
- (status = bdp_t->cbd_sc) & BD_ENET_TX_READY)
- break;
+ index = fec_enet_get_bd_index(txq->tx_bd_base, bdp, fep);
- for (i = 0; i < bdnum; i++) {
- if (!IS_TSO_HEADER(txq, bdp->cbd_bufaddr))
- dma_unmap_single(&fep->pdev->dev, bdp->cbd_bufaddr,
- bdp->cbd_datlen, DMA_TO_DEVICE);
- bdp->cbd_bufaddr = 0;
- if (i < bdnum - 1)
- bdp = fec_enet_get_nextdesc(bdp, fep, queue_id);
- }
+ skb = txq->tx_skbuff[index];
txq->tx_skbuff[index] = NULL;
+ if (!IS_TSO_HEADER(txq, bdp->cbd_bufaddr))
+ dma_unmap_single(&fep->pdev->dev, bdp->cbd_bufaddr,
+ bdp->cbd_datlen, DMA_TO_DEVICE);
+ bdp->cbd_bufaddr = 0;
+ if (!skb) {
+ bdp = fec_enet_get_nextdesc(bdp, fep, queue_id);
+ continue;
+ }
/* Check for errors. */
if (status & (BD_ENET_TX_HB | BD_ENET_TX_LC |
vlan_packet_rcvd = true;
- skb_copy_to_linear_data_offset(skb, VLAN_HLEN,
- data, (2 * ETH_ALEN));
+ memmove(skb->data + VLAN_HLEN, data, ETH_ALEN * 2);
skb_pull(skb, VLAN_HLEN);
}
writel(int_events, fep->hwp + FEC_IEVENT);
fec_enet_collect_events(fep, int_events);
- if (fep->work_tx || fep->work_rx) {
+ if ((fep->work_tx || fep->work_rx) && fep->link) {
ret = IRQ_HANDLED;
if (napi_schedule_prep(&fep->napi)) {
struct fec_enet_private *fep = netdev_priv(ndev);
struct device_node *node;
int err = -ENXIO, i;
+ u32 mii_speed, holdtime;
/*
* The i.MX28 dual fec interfaces are not equal.
* Reference Manual has an error on this, and gets fixed on i.MX6Q
* document.
*/
- fep->phy_speed = DIV_ROUND_UP(clk_get_rate(fep->clk_ipg), 5000000);
+ mii_speed = DIV_ROUND_UP(clk_get_rate(fep->clk_ipg), 5000000);
if (fep->quirks & FEC_QUIRK_ENET_MAC)
- fep->phy_speed--;
- fep->phy_speed <<= 1;
+ mii_speed--;
+ if (mii_speed > 63) {
+ dev_err(&pdev->dev,
+ "fec clock (%lu) to fast to get right mii speed\n",
+ clk_get_rate(fep->clk_ipg));
+ err = -EINVAL;
+ goto err_out;
+ }
+
+ /*
+ * The i.MX28 and i.MX6 types have another filed in the MSCR (aka
+ * MII_SPEED) register that defines the MDIO output hold time. Earlier
+ * versions are RAZ there, so just ignore the difference and write the
+ * register always.
+ * The minimal hold time according to IEE802.3 (clause 22) is 10 ns.
+ * HOLDTIME + 1 is the number of clk cycles the fec is holding the
+ * output.
+ * The HOLDTIME bitfield takes values between 0 and 7 (inclusive).
+ * Given that ceil(clkrate / 5000000) <= 64, the calculation for
+ * holdtime cannot result in a value greater than 3.
+ */
+ holdtime = DIV_ROUND_UP(clk_get_rate(fep->clk_ipg), 100000000) - 1;
+
+ fep->phy_speed = mii_speed << 1 | holdtime << 8;
+
writel(fep->phy_speed, fep->hwp + FEC_MII_SPEED);
fep->mii_bus = mdiobus_alloc();
regulator_disable(fep->reg_phy);
if (fep->ptp_clock)
ptp_clock_unregister(fep->ptp_clock);
- fec_enet_clk_enable(ndev, false);
of_node_put(fep->phy_node);
free_netdev(ndev);
return 0;
}
+static int gfar_of_group_count(struct device_node *np)
+{
+ struct device_node *child;
+ int num = 0;
+
+ for_each_available_child_of_node(np, child)
+ if (!of_node_cmp(child->name, "queue-group"))
+ num++;
+
+ return num;
+}
+
static int gfar_of_init(struct platform_device *ofdev, struct net_device **pdev)
{
const char *model;
num_rx_qs = 1;
} else { /* MQ_MG_MODE */
/* get the actual number of supported groups */
- unsigned int num_grps = of_get_available_child_count(np);
+ unsigned int num_grps = gfar_of_group_count(np);
if (num_grps == 0 || num_grps > MAXGROUPS) {
dev_err(&ofdev->dev, "Invalid # of int groups(%d)\n",
/* Parse and initialize group specific information */
if (priv->mode == MQ_MG_MODE) {
- for_each_child_of_node(np, child) {
+ for_each_available_child_of_node(np, child) {
+ if (of_node_cmp(child->name, "queue-group"))
+ continue;
+
err = gfar_parse_group(child, priv, model);
if (err)
goto err_grp_init;
struct phy_device *phydev = priv->phydev;
if (unlikely(phydev->link != priv->oldlink ||
- phydev->duplex != priv->oldduplex ||
- phydev->speed != priv->oldspeed))
+ (phydev->link && (phydev->duplex != priv->oldduplex ||
+ phydev->speed != priv->oldspeed))))
gfar_update_link_state(priv);
}
ugeth->phy_interface = phy_interface;
ugeth->max_speed = max_speed;
+ /* Carrier starts down, phylib will bring it up */
+ netif_carrier_off(dev);
+
err = register_netdev(dev);
if (err) {
if (netif_msg_probe(ugeth))
device_remove_file(&dev->dev, &dev_attr_remove_port);
}
+static int ehea_reboot_notifier(struct notifier_block *nb,
+ unsigned long action, void *unused)
+{
+ if (action == SYS_RESTART) {
+ pr_info("Reboot: freeing all eHEA resources\n");
+ ibmebus_unregister_driver(&ehea_driver);
+ }
+ return NOTIFY_DONE;
+}
+
+static struct notifier_block ehea_reboot_nb = {
+ .notifier_call = ehea_reboot_notifier,
+};
+
+static int ehea_mem_notifier(struct notifier_block *nb,
+ unsigned long action, void *data)
+{
+ int ret = NOTIFY_BAD;
+ struct memory_notify *arg = data;
+
+ mutex_lock(&dlpar_mem_lock);
+
+ switch (action) {
+ case MEM_CANCEL_OFFLINE:
+ pr_info("memory offlining canceled");
+ /* Fall through: re-add canceled memory block */
+
+ case MEM_ONLINE:
+ pr_info("memory is going online");
+ set_bit(__EHEA_STOP_XFER, &ehea_driver_flags);
+ if (ehea_add_sect_bmap(arg->start_pfn, arg->nr_pages))
+ goto out_unlock;
+ ehea_rereg_mrs();
+ break;
+
+ case MEM_GOING_OFFLINE:
+ pr_info("memory is going offline");
+ set_bit(__EHEA_STOP_XFER, &ehea_driver_flags);
+ if (ehea_rem_sect_bmap(arg->start_pfn, arg->nr_pages))
+ goto out_unlock;
+ ehea_rereg_mrs();
+ break;
+
+ default:
+ break;
+ }
+
+ ehea_update_firmware_handles();
+ ret = NOTIFY_OK;
+
+out_unlock:
+ mutex_unlock(&dlpar_mem_lock);
+ return ret;
+}
+
+static struct notifier_block ehea_mem_nb = {
+ .notifier_call = ehea_mem_notifier,
+};
+
+static void ehea_crash_handler(void)
+{
+ int i;
+
+ if (ehea_fw_handles.arr)
+ for (i = 0; i < ehea_fw_handles.num_entries; i++)
+ ehea_h_free_resource(ehea_fw_handles.arr[i].adh,
+ ehea_fw_handles.arr[i].fwh,
+ FORCE_FREE);
+
+ if (ehea_bcmc_regs.arr)
+ for (i = 0; i < ehea_bcmc_regs.num_entries; i++)
+ ehea_h_reg_dereg_bcmc(ehea_bcmc_regs.arr[i].adh,
+ ehea_bcmc_regs.arr[i].port_id,
+ ehea_bcmc_regs.arr[i].reg_type,
+ ehea_bcmc_regs.arr[i].macaddr,
+ 0, H_DEREG_BCMC);
+}
+
+static atomic_t ehea_memory_hooks_registered;
+
+/* Register memory hooks on probe of first adapter */
+static int ehea_register_memory_hooks(void)
+{
+ int ret = 0;
+
+ if (atomic_inc_and_test(&ehea_memory_hooks_registered))
+ return 0;
+
+ ret = ehea_create_busmap();
+ if (ret) {
+ pr_info("ehea_create_busmap failed\n");
+ goto out;
+ }
+
+ ret = register_reboot_notifier(&ehea_reboot_nb);
+ if (ret) {
+ pr_info("register_reboot_notifier failed\n");
+ goto out;
+ }
+
+ ret = register_memory_notifier(&ehea_mem_nb);
+ if (ret) {
+ pr_info("register_memory_notifier failed\n");
+ goto out2;
+ }
+
+ ret = crash_shutdown_register(ehea_crash_handler);
+ if (ret) {
+ pr_info("crash_shutdown_register failed\n");
+ goto out3;
+ }
+
+ return 0;
+
+out3:
+ unregister_memory_notifier(&ehea_mem_nb);
+out2:
+ unregister_reboot_notifier(&ehea_reboot_nb);
+out:
+ return ret;
+}
+
+static void ehea_unregister_memory_hooks(void)
+{
+ if (atomic_read(&ehea_memory_hooks_registered))
+ return;
+
+ unregister_reboot_notifier(&ehea_reboot_nb);
+ if (crash_shutdown_unregister(ehea_crash_handler))
+ pr_info("failed unregistering crash handler\n");
+ unregister_memory_notifier(&ehea_mem_nb);
+}
+
static int ehea_probe_adapter(struct platform_device *dev)
{
struct ehea_adapter *adapter;
int ret;
int i;
+ ret = ehea_register_memory_hooks();
+ if (ret)
+ return ret;
+
if (!dev || !dev->dev.of_node) {
pr_err("Invalid ibmebus device probed\n");
return -EINVAL;
return 0;
}
-static void ehea_crash_handler(void)
-{
- int i;
-
- if (ehea_fw_handles.arr)
- for (i = 0; i < ehea_fw_handles.num_entries; i++)
- ehea_h_free_resource(ehea_fw_handles.arr[i].adh,
- ehea_fw_handles.arr[i].fwh,
- FORCE_FREE);
-
- if (ehea_bcmc_regs.arr)
- for (i = 0; i < ehea_bcmc_regs.num_entries; i++)
- ehea_h_reg_dereg_bcmc(ehea_bcmc_regs.arr[i].adh,
- ehea_bcmc_regs.arr[i].port_id,
- ehea_bcmc_regs.arr[i].reg_type,
- ehea_bcmc_regs.arr[i].macaddr,
- 0, H_DEREG_BCMC);
-}
-
-static int ehea_mem_notifier(struct notifier_block *nb,
- unsigned long action, void *data)
-{
- int ret = NOTIFY_BAD;
- struct memory_notify *arg = data;
-
- mutex_lock(&dlpar_mem_lock);
-
- switch (action) {
- case MEM_CANCEL_OFFLINE:
- pr_info("memory offlining canceled");
- /* Readd canceled memory block */
- case MEM_ONLINE:
- pr_info("memory is going online");
- set_bit(__EHEA_STOP_XFER, &ehea_driver_flags);
- if (ehea_add_sect_bmap(arg->start_pfn, arg->nr_pages))
- goto out_unlock;
- ehea_rereg_mrs();
- break;
- case MEM_GOING_OFFLINE:
- pr_info("memory is going offline");
- set_bit(__EHEA_STOP_XFER, &ehea_driver_flags);
- if (ehea_rem_sect_bmap(arg->start_pfn, arg->nr_pages))
- goto out_unlock;
- ehea_rereg_mrs();
- break;
- default:
- break;
- }
-
- ehea_update_firmware_handles();
- ret = NOTIFY_OK;
-
-out_unlock:
- mutex_unlock(&dlpar_mem_lock);
- return ret;
-}
-
-static struct notifier_block ehea_mem_nb = {
- .notifier_call = ehea_mem_notifier,
-};
-
-static int ehea_reboot_notifier(struct notifier_block *nb,
- unsigned long action, void *unused)
-{
- if (action == SYS_RESTART) {
- pr_info("Reboot: freeing all eHEA resources\n");
- ibmebus_unregister_driver(&ehea_driver);
- }
- return NOTIFY_DONE;
-}
-
-static struct notifier_block ehea_reboot_nb = {
- .notifier_call = ehea_reboot_notifier,
-};
-
static int check_module_parm(void)
{
int ret = 0;
if (ret)
goto out;
- ret = ehea_create_busmap();
- if (ret)
- goto out;
-
- ret = register_reboot_notifier(&ehea_reboot_nb);
- if (ret)
- pr_info("failed registering reboot notifier\n");
-
- ret = register_memory_notifier(&ehea_mem_nb);
- if (ret)
- pr_info("failed registering memory remove notifier\n");
-
- ret = crash_shutdown_register(ehea_crash_handler);
- if (ret)
- pr_info("failed registering crash handler\n");
-
ret = ibmebus_register_driver(&ehea_driver);
if (ret) {
pr_err("failed registering eHEA device driver on ebus\n");
- goto out2;
+ goto out;
}
ret = driver_create_file(&ehea_driver.driver,
if (ret) {
pr_err("failed to register capabilities attribute, ret=%d\n",
ret);
- goto out3;
+ goto out2;
}
return ret;
-out3:
- ibmebus_unregister_driver(&ehea_driver);
out2:
- unregister_memory_notifier(&ehea_mem_nb);
- unregister_reboot_notifier(&ehea_reboot_nb);
- crash_shutdown_unregister(ehea_crash_handler);
+ ibmebus_unregister_driver(&ehea_driver);
out:
return ret;
}
static void __exit ehea_module_exit(void)
{
- int ret;
-
driver_remove_file(&ehea_driver.driver, &driver_attr_capabilities);
ibmebus_unregister_driver(&ehea_driver);
- unregister_reboot_notifier(&ehea_reboot_nb);
- ret = crash_shutdown_unregister(ehea_crash_handler);
- if (ret)
- pr_info("failed unregistering crash handler\n");
- unregister_memory_notifier(&ehea_mem_nb);
+ ehea_unregister_memory_hooks();
kfree(ehea_fw_handles.arr);
kfree(ehea_bcmc_regs.arr);
ehea_destroy_busmap();
ibmveth_replenish_task(adapter);
if (frames_processed < budget) {
+ napi_complete(napi);
+
/* We think we are done - reenable interrupts,
* then check once more to make sure we are done.
*/
BUG_ON(lpar_rc != H_SUCCESS);
- napi_complete(napi);
-
if (ibmveth_rxq_pending_buffer(adapter) &&
napi_reschedule(napi)) {
lpar_rc = h_vio_signal(adapter->vdev->unit_address,
return ret;
}
+static int ibmveth_set_mac_addr(struct net_device *dev, void *p)
+{
+ struct ibmveth_adapter *adapter = netdev_priv(dev);
+ struct sockaddr *addr = p;
+ u64 mac_address;
+ int rc;
+
+ if (!is_valid_ether_addr(addr->sa_data))
+ return -EADDRNOTAVAIL;
+
+ mac_address = ibmveth_encode_mac_addr(addr->sa_data);
+ rc = h_change_logical_lan_mac(adapter->vdev->unit_address, mac_address);
+ if (rc) {
+ netdev_err(adapter->netdev, "h_change_logical_lan_mac failed with rc=%d\n", rc);
+ return rc;
+ }
+
+ ether_addr_copy(dev->dev_addr, addr->sa_data);
+
+ return 0;
+}
+
static const struct net_device_ops ibmveth_netdev_ops = {
.ndo_open = ibmveth_open,
.ndo_stop = ibmveth_close,
.ndo_fix_features = ibmveth_fix_features,
.ndo_set_features = ibmveth_set_features,
.ndo_validate_addr = eth_validate_addr,
- .ndo_set_mac_address = eth_mac_addr,
+ .ndo_set_mac_address = ibmveth_set_mac_addr,
#ifdef CONFIG_NET_POLL_CONTROLLER
.ndo_poll_controller = ibmveth_poll_controller,
#endif
* The grst delay value is in 100ms units, and we'll wait a
* couple counts longer to be sure we don't just miss the end.
*/
- grst_del = rd32(hw, I40E_GLGEN_RSTCTL) & I40E_GLGEN_RSTCTL_GRSTDEL_MASK
- >> I40E_GLGEN_RSTCTL_GRSTDEL_SHIFT;
+ grst_del = (rd32(hw, I40E_GLGEN_RSTCTL) &
+ I40E_GLGEN_RSTCTL_GRSTDEL_MASK) >>
+ I40E_GLGEN_RSTCTL_GRSTDEL_SHIFT;
for (cnt = 0; cnt < grst_del + 2; cnt++) {
reg = rd32(hw, I40E_GLGEN_RSTAT);
if (!(reg & I40E_GLGEN_RSTAT_DEVSTATE_MASK))
status = i40e_asq_send_command(hw, &desc, NULL, 0, cmd_details);
- if (!status)
+ if (!status && filter_index)
*filter_index = resp->index;
return status;
u32 val;
val = rd32(hw, I40E_PRTDCB_GENC);
- *delay = (u16)(val & I40E_PRTDCB_GENC_PFCLDA_MASK >>
+ *delay = (u16)((val & I40E_PRTDCB_GENC_PFCLDA_MASK) >>
I40E_PRTDCB_GENC_PFCLDA_SHIFT);
}
if (!cmd_buf)
return count;
bytes_not_copied = copy_from_user(cmd_buf, buffer, count);
- if (bytes_not_copied < 0)
+ if (bytes_not_copied < 0) {
+ kfree(cmd_buf);
return bytes_not_copied;
+ }
if (bytes_not_copied > 0)
count -= bytes_not_copied;
cmd_buf[count] = '\0';
vsi->tc_config.numtc = numtc;
vsi->tc_config.enabled_tc = enabled_tc ? enabled_tc : 1;
/* Number of queues per enabled TC */
- num_tc_qps = vsi->alloc_queue_pairs/numtc;
+ /* In MFP case we can have a much lower count of MSIx
+ * vectors available and so we need to lower the used
+ * q count.
+ */
+ qcount = min_t(int, vsi->alloc_queue_pairs, pf->num_lan_msix);
+ num_tc_qps = qcount / numtc;
num_tc_qps = min_t(int, num_tc_qps, I40E_MAX_QUEUES_PER_TC);
/* Setup queue offset/count for all TCs for given VSI */
u16 qoffset, qcount;
int i, n;
- if (!(vsi->back->flags & I40E_FLAG_DCB_ENABLED))
- return;
+ if (!(vsi->back->flags & I40E_FLAG_DCB_ENABLED)) {
+ /* Reset the TC information */
+ for (i = 0; i < vsi->num_queue_pairs; i++) {
+ rx_ring = vsi->rx_rings[i];
+ tx_ring = vsi->tx_rings[i];
+ rx_ring->dcb_tc = 0;
+ tx_ring->dcb_tc = 0;
+ }
+ }
for (n = 0; n < I40E_MAX_TRAFFIC_CLASS; n++) {
if (!(vsi->tc_config.enabled_tc & (1 << n)))
{
int i;
+ i40e_stop_misc_vector(pf);
+ if (pf->flags & I40E_FLAG_MSIX_ENABLED) {
+ synchronize_irq(pf->msix_entries[0].vector);
+ free_irq(pf->msix_entries[0].vector, pf);
+ }
+
i40e_put_lump(pf->irq_pile, 0, I40E_PILE_VALID_BIT-1);
for (i = 0; i < pf->num_alloc_vsi; i++)
if (pf->vsi[i])
/* Wait for the PF's Tx queues to be disabled */
ret = i40e_pf_wait_txq_disabled(pf);
- if (!ret)
+ if (ret) {
+ /* Schedule PF reset to recover */
+ set_bit(__I40E_PF_RESET_REQUESTED, &pf->state);
+ i40e_service_event_schedule(pf);
+ } else {
i40e_pf_unquiesce_all_vsi(pf);
+ }
+
exit:
return ret;
}
int i, v;
/* If we're down or resetting, just bail */
- if (test_bit(__I40E_CONFIG_BUSY, &pf->state))
+ if (test_bit(__I40E_DOWN, &pf->state) ||
+ test_bit(__I40E_CONFIG_BUSY, &pf->state))
return;
/* for each VSI/netdev
set_bit(__I40E_DOWN, &pf->state);
del_timer_sync(&pf->service_timer);
cancel_work_sync(&pf->service_task);
+ i40e_fdir_teardown(pf);
if (pf->flags & I40E_FLAG_SRIOV_ENABLED) {
i40e_free_vfs(pf);
if (pf->vsi[pf->lan_vsi])
i40e_vsi_release(pf->vsi[pf->lan_vsi]);
- i40e_stop_misc_vector(pf);
- if (pf->flags & I40E_FLAG_MSIX_ENABLED) {
- synchronize_irq(pf->msix_entries[0].vector);
- free_irq(pf->msix_entries[0].vector, pf);
- }
-
/* shutdown and destroy the HMC */
if (pf->hw.hmc.hmc_obj) {
ret_code = i40e_shutdown_lan_hmc(&pf->hw);
wr32(hw, I40E_PFPM_APM, (pf->wol_en ? I40E_PFPM_APM_APME_MASK : 0));
wr32(hw, I40E_PFPM_WUFC, (pf->wol_en ? I40E_PFPM_WUFC_MAG_MASK : 0));
+ i40e_clear_interrupt_scheme(pf);
+
if (system_state == SYSTEM_POWER_OFF) {
pci_wake_from_d3(pdev, pf->wol_en);
pci_set_power_state(pdev, PCI_D3hot);
{
i40e_status status;
enum i40e_nvmupd_cmd upd_cmd;
+ bool retry_attempt = false;
upd_cmd = i40e_nvmupd_validate_command(hw, cmd, errno);
+retry:
switch (upd_cmd) {
case I40E_NVMUPD_WRITE_CON:
status = i40e_nvmupd_nvm_write(hw, cmd, bytes, errno);
*errno = -ESRCH;
break;
}
+
+ /* In some circumstances, a multi-write transaction takes longer
+ * than the default 3 minute timeout on the write semaphore. If
+ * the write failed with an EBUSY status, this is likely the problem,
+ * so here we try to reacquire the semaphore then retry the write.
+ * We only do one retry, then give up.
+ */
+ if (status && (hw->aq.asq_last_status == I40E_AQ_RC_EBUSY) &&
+ !retry_attempt) {
+ i40e_status old_status = status;
+ u32 old_asq_status = hw->aq.asq_last_status;
+ u32 gtime;
+
+ gtime = rd32(hw, I40E_GLVFGEN_TIMER);
+ if (gtime >= hw->nvm.hw_semaphore_timeout) {
+ i40e_debug(hw, I40E_DEBUG_ALL,
+ "NVMUPD: write semaphore expired (%d >= %lld), retrying\n",
+ gtime, hw->nvm.hw_semaphore_timeout);
+ i40e_release_nvm(hw);
+ status = i40e_acquire_nvm(hw, I40E_RESOURCE_WRITE);
+ if (status) {
+ i40e_debug(hw, I40E_DEBUG_ALL,
+ "NVMUPD: write semaphore reacquire failed aq_err = %d\n",
+ hw->aq.asq_last_status);
+ status = old_status;
+ hw->aq.asq_last_status = old_asq_status;
+ } else {
+ retry_attempt = true;
+ goto retry;
+ }
+ }
+ }
+
return status;
}
}
}
+/**
+ * i40e_get_head - Retrieve head from head writeback
+ * @tx_ring: tx ring to fetch head of
+ *
+ * Returns value of Tx ring head based on value stored
+ * in head write-back location
+ **/
+static inline u32 i40e_get_head(struct i40e_ring *tx_ring)
+{
+ void *head = (struct i40e_tx_desc *)tx_ring->desc + tx_ring->count;
+
+ return le32_to_cpu(*(volatile __le32 *)head);
+}
+
/**
* i40e_get_tx_pending - how many tx descriptors not processed
* @tx_ring: the ring of descriptors
**/
static u32 i40e_get_tx_pending(struct i40e_ring *ring)
{
- u32 ntu = ((ring->next_to_clean <= ring->next_to_use)
- ? ring->next_to_use
- : ring->next_to_use + ring->count);
- return ntu - ring->next_to_clean;
+ u32 head, tail;
+
+ head = i40e_get_head(ring);
+ tail = readl(ring->tail);
+
+ if (head != tail)
+ return (head < tail) ?
+ tail - head : (tail + ring->count - head);
+
+ return 0;
}
/**
**/
static bool i40e_check_tx_hang(struct i40e_ring *tx_ring)
{
+ u32 tx_done = tx_ring->stats.packets;
+ u32 tx_done_old = tx_ring->tx_stats.tx_done_old;
u32 tx_pending = i40e_get_tx_pending(tx_ring);
struct i40e_pf *pf = tx_ring->vsi->back;
bool ret = false;
* run the check_tx_hang logic with a transmit completion
* pending but without time to complete it yet.
*/
- if ((tx_ring->tx_stats.tx_done_old == tx_ring->stats.packets) &&
- (tx_pending >= I40E_MIN_DESC_PENDING)) {
+ if ((tx_done_old == tx_done) && tx_pending) {
/* make sure it is true for two checks in a row */
ret = test_and_set_bit(__I40E_HANG_CHECK_ARMED,
&tx_ring->state);
- } else if ((tx_ring->tx_stats.tx_done_old == tx_ring->stats.packets) &&
- (tx_pending < I40E_MIN_DESC_PENDING) &&
- (tx_pending > 0)) {
+ } else if (tx_done_old == tx_done &&
+ (tx_pending < I40E_MIN_DESC_PENDING) && (tx_pending > 0)) {
if (I40E_DEBUG_FLOW & pf->hw.debug_mask)
dev_info(tx_ring->dev, "HW needs some more descs to do a cacheline flush. tx_pending %d, queue %d",
tx_pending, tx_ring->queue_index);
pf->tx_sluggish_count++;
} else {
/* update completed stats and disarm the hang check */
- tx_ring->tx_stats.tx_done_old = tx_ring->stats.packets;
+ tx_ring->tx_stats.tx_done_old = tx_done;
clear_bit(__I40E_HANG_CHECK_ARMED, &tx_ring->state);
}
return ret;
}
-/**
- * i40e_get_head - Retrieve head from head writeback
- * @tx_ring: tx ring to fetch head of
- *
- * Returns value of Tx ring head based on value stored
- * in head write-back location
- **/
-static inline u32 i40e_get_head(struct i40e_ring *tx_ring)
-{
- void *head = (struct i40e_tx_desc *)tx_ring->desc + tx_ring->count;
-
- return le32_to_cpu(*(volatile __le32 *)head);
-}
-
#define WB_STRIDE 0x3
/**
return __i40e_maybe_stop_tx(tx_ring, size);
}
+/**
+ * i40e_chk_linearize - Check if there are more than 8 fragments per packet
+ * @skb: send buffer
+ * @tx_flags: collected send information
+ * @hdr_len: size of the packet header
+ *
+ * Note: Our HW can't scatter-gather more than 8 fragments to build
+ * a packet on the wire and so we need to figure out the cases where we
+ * need to linearize the skb.
+ **/
+static bool i40e_chk_linearize(struct sk_buff *skb, u32 tx_flags,
+ const u8 hdr_len)
+{
+ struct skb_frag_struct *frag;
+ bool linearize = false;
+ unsigned int size = 0;
+ u16 num_frags;
+ u16 gso_segs;
+
+ num_frags = skb_shinfo(skb)->nr_frags;
+ gso_segs = skb_shinfo(skb)->gso_segs;
+
+ if (tx_flags & (I40E_TX_FLAGS_TSO | I40E_TX_FLAGS_FSO)) {
+ u16 j = 1;
+
+ if (num_frags < (I40E_MAX_BUFFER_TXD))
+ goto linearize_chk_done;
+ /* try the simple math, if we have too many frags per segment */
+ if (DIV_ROUND_UP((num_frags + gso_segs), gso_segs) >
+ I40E_MAX_BUFFER_TXD) {
+ linearize = true;
+ goto linearize_chk_done;
+ }
+ frag = &skb_shinfo(skb)->frags[0];
+ size = hdr_len;
+ /* we might still have more fragments per segment */
+ do {
+ size += skb_frag_size(frag);
+ frag++; j++;
+ if (j == I40E_MAX_BUFFER_TXD) {
+ if (size < skb_shinfo(skb)->gso_size) {
+ linearize = true;
+ break;
+ }
+ j = 1;
+ size -= skb_shinfo(skb)->gso_size;
+ if (size)
+ j++;
+ size += hdr_len;
+ }
+ num_frags--;
+ } while (num_frags);
+ } else {
+ if (num_frags >= I40E_MAX_BUFFER_TXD)
+ linearize = true;
+ }
+
+linearize_chk_done:
+ return linearize;
+}
+
/**
* i40e_tx_map - Build the Tx descriptor
* @tx_ring: ring to send buffer on
if (tsyn)
tx_flags |= I40E_TX_FLAGS_TSYN;
+ if (i40e_chk_linearize(skb, tx_flags, hdr_len))
+ if (skb_linearize(skb))
+ goto out_drop;
+
skb_tx_timestamp(skb);
/* always enable CRC insertion offload */
#define i40e_rx_desc i40e_32byte_rx_desc
+#define I40E_MAX_BUFFER_TXD 8
#define I40E_MIN_TX_LEN 17
#define I40E_MAX_DATA_PER_TXD 8192
}
}
+/**
+ * i40e_get_head - Retrieve head from head writeback
+ * @tx_ring: tx ring to fetch head of
+ *
+ * Returns value of Tx ring head based on value stored
+ * in head write-back location
+ **/
+static inline u32 i40e_get_head(struct i40e_ring *tx_ring)
+{
+ void *head = (struct i40e_tx_desc *)tx_ring->desc + tx_ring->count;
+
+ return le32_to_cpu(*(volatile __le32 *)head);
+}
+
/**
* i40e_get_tx_pending - how many tx descriptors not processed
* @tx_ring: the ring of descriptors
**/
static u32 i40e_get_tx_pending(struct i40e_ring *ring)
{
- u32 ntu = ((ring->next_to_clean <= ring->next_to_use)
- ? ring->next_to_use
- : ring->next_to_use + ring->count);
- return ntu - ring->next_to_clean;
+ u32 head, tail;
+
+ head = i40e_get_head(ring);
+ tail = readl(ring->tail);
+
+ if (head != tail)
+ return (head < tail) ?
+ tail - head : (tail + ring->count - head);
+
+ return 0;
}
/**
**/
static bool i40e_check_tx_hang(struct i40e_ring *tx_ring)
{
+ u32 tx_done = tx_ring->stats.packets;
+ u32 tx_done_old = tx_ring->tx_stats.tx_done_old;
u32 tx_pending = i40e_get_tx_pending(tx_ring);
bool ret = false;
* run the check_tx_hang logic with a transmit completion
* pending but without time to complete it yet.
*/
- if ((tx_ring->tx_stats.tx_done_old == tx_ring->stats.packets) &&
- (tx_pending >= I40E_MIN_DESC_PENDING)) {
+ if ((tx_done_old == tx_done) && tx_pending) {
/* make sure it is true for two checks in a row */
ret = test_and_set_bit(__I40E_HANG_CHECK_ARMED,
&tx_ring->state);
- } else if (!(tx_ring->tx_stats.tx_done_old == tx_ring->stats.packets) ||
- !(tx_pending < I40E_MIN_DESC_PENDING) ||
- !(tx_pending > 0)) {
+ } else if (tx_done_old == tx_done &&
+ (tx_pending < I40E_MIN_DESC_PENDING) && (tx_pending > 0)) {
/* update completed stats and disarm the hang check */
- tx_ring->tx_stats.tx_done_old = tx_ring->stats.packets;
+ tx_ring->tx_stats.tx_done_old = tx_done;
clear_bit(__I40E_HANG_CHECK_ARMED, &tx_ring->state);
}
return ret;
}
-/**
- * i40e_get_head - Retrieve head from head writeback
- * @tx_ring: tx ring to fetch head of
- *
- * Returns value of Tx ring head based on value stored
- * in head write-back location
- **/
-static inline u32 i40e_get_head(struct i40e_ring *tx_ring)
-{
- void *head = (struct i40e_tx_desc *)tx_ring->desc + tx_ring->count;
-
- return le32_to_cpu(*(volatile __le32 *)head);
-}
-
#define WB_STRIDE 0x3
/**
if (err < 0)
return err;
- if (protocol == htons(ETH_P_IP)) {
- iph = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
+ iph = skb->encapsulation ? inner_ip_hdr(skb) : ip_hdr(skb);
+ ipv6h = skb->encapsulation ? inner_ipv6_hdr(skb) : ipv6_hdr(skb);
+
+ if (iph->version == 4) {
tcph = skb->encapsulation ? inner_tcp_hdr(skb) : tcp_hdr(skb);
iph->tot_len = 0;
iph->check = 0;
tcph->check = ~csum_tcpudp_magic(iph->saddr, iph->daddr,
0, IPPROTO_TCP, 0);
- } else if (skb_is_gso_v6(skb)) {
-
- ipv6h = skb->encapsulation ? inner_ipv6_hdr(skb)
- : ipv6_hdr(skb);
+ } else if (ipv6h->version == 6) {
tcph = skb->encapsulation ? inner_tcp_hdr(skb) : tcp_hdr(skb);
ipv6h->payload_len = 0;
tcph->check = ~csum_ipv6_magic(&ipv6h->saddr, &ipv6h->daddr,
I40E_TX_CTX_EXT_IP_IPV4_NO_CSUM;
}
} else if (tx_flags & I40E_TX_FLAGS_IPV6) {
- if (tx_flags & I40E_TX_FLAGS_TSO) {
- *cd_tunneling |= I40E_TX_CTX_EXT_IP_IPV6;
+ *cd_tunneling |= I40E_TX_CTX_EXT_IP_IPV6;
+ if (tx_flags & I40E_TX_FLAGS_TSO)
ip_hdr(skb)->check = 0;
- } else {
- *cd_tunneling |=
- I40E_TX_CTX_EXT_IP_IPV4_NO_CSUM;
- }
}
/* Now set the ctx descriptor fields */
((skb_inner_network_offset(skb) -
skb_transport_offset(skb)) >> 1) <<
I40E_TXD_CTX_QW0_NATLEN_SHIFT;
+ if (this_ip_hdr->version == 6) {
+ tx_flags &= ~I40E_TX_FLAGS_IPV4;
+ tx_flags |= I40E_TX_FLAGS_IPV6;
+ }
+
} else {
network_hdr_len = skb_network_header_len(skb);
context_desc->type_cmd_tso_mss = cpu_to_le64(cd_type_cmd_tso_mss);
}
+ /**
+ * i40e_chk_linearize - Check if there are more than 8 fragments per packet
+ * @skb: send buffer
+ * @tx_flags: collected send information
+ * @hdr_len: size of the packet header
+ *
+ * Note: Our HW can't scatter-gather more than 8 fragments to build
+ * a packet on the wire and so we need to figure out the cases where we
+ * need to linearize the skb.
+ **/
+static bool i40e_chk_linearize(struct sk_buff *skb, u32 tx_flags,
+ const u8 hdr_len)
+{
+ struct skb_frag_struct *frag;
+ bool linearize = false;
+ unsigned int size = 0;
+ u16 num_frags;
+ u16 gso_segs;
+
+ num_frags = skb_shinfo(skb)->nr_frags;
+ gso_segs = skb_shinfo(skb)->gso_segs;
+
+ if (tx_flags & (I40E_TX_FLAGS_TSO | I40E_TX_FLAGS_FSO)) {
+ u16 j = 1;
+
+ if (num_frags < (I40E_MAX_BUFFER_TXD))
+ goto linearize_chk_done;
+ /* try the simple math, if we have too many frags per segment */
+ if (DIV_ROUND_UP((num_frags + gso_segs), gso_segs) >
+ I40E_MAX_BUFFER_TXD) {
+ linearize = true;
+ goto linearize_chk_done;
+ }
+ frag = &skb_shinfo(skb)->frags[0];
+ size = hdr_len;
+ /* we might still have more fragments per segment */
+ do {
+ size += skb_frag_size(frag);
+ frag++; j++;
+ if (j == I40E_MAX_BUFFER_TXD) {
+ if (size < skb_shinfo(skb)->gso_size) {
+ linearize = true;
+ break;
+ }
+ j = 1;
+ size -= skb_shinfo(skb)->gso_size;
+ if (size)
+ j++;
+ size += hdr_len;
+ }
+ num_frags--;
+ } while (num_frags);
+ } else {
+ if (num_frags >= I40E_MAX_BUFFER_TXD)
+ linearize = true;
+ }
+
+linearize_chk_done:
+ return linearize;
+}
+
/**
* i40e_tx_map - Build the Tx descriptor
* @tx_ring: ring to send buffer on
else if (tso)
tx_flags |= I40E_TX_FLAGS_TSO;
+ if (i40e_chk_linearize(skb, tx_flags, hdr_len))
+ if (skb_linearize(skb))
+ goto out_drop;
+
skb_tx_timestamp(skb);
/* always enable CRC insertion offload */
#define i40e_rx_desc i40e_32byte_rx_desc
+#define I40E_MAX_BUFFER_TXD 8
#define I40E_MIN_TX_LEN 17
#define I40E_MAX_DATA_PER_TXD 8192
static int mvneta_ioctl(struct net_device *dev, struct ifreq *ifr, int cmd)
{
struct mvneta_port *pp = netdev_priv(dev);
- int ret;
if (!pp->phy_dev)
return -ENOTSUPP;
- ret = phy_mii_ioctl(pp->phy_dev, ifr, cmd);
- if (!ret)
- mvneta_adjust_link(dev);
-
- return ret;
+ return phy_mii_ioctl(pp->phy_dev, ifr, cmd);
}
/* Ethtool methods */
* on the host, we deprecate the error message for this
* specific command/input_mod/opcode_mod/fw-status to be debug.
*/
- if (op == MLX4_CMD_SET_PORT && in_modifier == 1 &&
+ if (op == MLX4_CMD_SET_PORT &&
+ (in_modifier == 1 || in_modifier == 2) &&
op_modifier == 0 && context->fw_status == CMD_STAT_BAD_SIZE)
mlx4_dbg(dev, "command 0x%x failed: fw status = 0x%x\n",
op, context->fw_status);
goto reset_slave;
slave_state[slave].vhcr_dma = ((u64) param) << 48;
priv->mfunc.master.slave_state[slave].cookie = 0;
- mutex_init(&priv->mfunc.master.gen_eqe_mutex[slave]);
break;
case MLX4_COMM_CMD_VHCR1:
if (slave_state[slave].last_cmd != MLX4_COMM_CMD_VHCR0)
for (i = 0; i < dev->num_slaves; ++i) {
s_state = &priv->mfunc.master.slave_state[i];
s_state->last_cmd = MLX4_COMM_CMD_RESET;
+ mutex_init(&priv->mfunc.master.gen_eqe_mutex[i]);
for (j = 0; j < MLX4_EVENT_TYPES_NUM; ++j)
s_state->event_eq[j].eqn = -1;
__raw_writel((__force u32) 0,
/* Schedule multicast task to populate multicast list */
queue_work(mdev->workqueue, &priv->rx_mode_task);
- mlx4_set_stats_bitmap(mdev->dev, &priv->stats_bitmap);
-
#ifdef CONFIG_MLX4_EN_VXLAN
if (priv->mdev->dev->caps.tunnel_offload_mode == MLX4_TUNNEL_OFFLOAD_MODE_VXLAN)
vxlan_get_rx_port(dev);
netif_carrier_off(dev);
mlx4_en_set_default_moderation(priv);
- err = register_netdev(dev);
- if (err) {
- en_err(priv, "Netdev registration failed for port %d\n", port);
- goto out;
- }
- priv->registered = 1;
-
en_warn(priv, "Using %d TX rings\n", prof->tx_ring_num);
en_warn(priv, "Using %d RX rings\n", prof->rx_ring_num);
queue_delayed_work(mdev->workqueue, &priv->service_task,
SERVICE_TASK_DELAY);
+ mlx4_set_stats_bitmap(mdev->dev, &priv->stats_bitmap);
+
+ err = register_netdev(dev);
+ if (err) {
+ en_err(priv, "Netdev registration failed for port %d\n", port);
+ goto out;
+ }
+
+ priv->registered = 1;
+
return 0;
out:
{
u32 loopback_ok = 0;
int i;
-
+ bool gro_enabled;
priv->loopback_ok = 0;
priv->validate_loopback = 1;
+ gro_enabled = priv->dev->features & NETIF_F_GRO;
mlx4_en_update_loopback_state(priv->dev, priv->dev->features);
+ priv->dev->features &= ~NETIF_F_GRO;
/* xmit */
if (mlx4_en_test_loopback_xmit(priv)) {
mlx4_en_test_loopback_exit:
priv->validate_loopback = 0;
+
+ if (gro_enabled)
+ priv->dev->features |= NETIF_F_GRO;
+
mlx4_en_update_loopback_state(priv->dev, priv->dev->features);
return !loopback_ok;
}
/* All active slaves need to receive the event */
if (slave == ALL_SLAVES) {
- for (i = 0; i < dev->num_slaves; i++) {
- if (i != dev->caps.function &&
- master->slave_state[i].active)
- if (mlx4_GEN_EQE(dev, i, eqe))
- mlx4_warn(dev, "Failed to generate event for slave %d\n",
- i);
+ for (i = 0; i <= dev->persist->num_vfs; i++) {
+ if (mlx4_GEN_EQE(dev, i, eqe))
+ mlx4_warn(dev, "Failed to generate event for slave %d\n",
+ i);
}
} else {
if (mlx4_GEN_EQE(dev, slave, eqe))
struct mlx4_eqe *eqe)
{
struct mlx4_priv *priv = mlx4_priv(dev);
- struct mlx4_slave_state *s_slave =
- &priv->mfunc.master.slave_state[slave];
- if (!s_slave->active) {
- /*mlx4_warn(dev, "Trying to pass event to inactive slave\n");*/
+ if (slave < 0 || slave > dev->persist->num_vfs ||
+ slave == dev->caps.function ||
+ !priv->mfunc.master.slave_state[slave].active)
return;
- }
slave_event(dev, slave, eqe);
}
unsigned long rx_chksum_none;
unsigned long rx_chksum_complete;
unsigned long tx_chksum_offload;
-#define NUM_PORT_STATS 9
+#define NUM_PORT_STATS 10
};
struct mlx4_en_perf_stats {
EXPORT_SYMBOL_GPL(mlx4_qp_alloc);
-#define MLX4_UPDATE_QP_SUPPORTED_ATTRS MLX4_UPDATE_QP_SMAC
int mlx4_update_qp(struct mlx4_dev *dev, u32 qpn,
enum mlx4_update_qp_attr attr,
struct mlx4_update_qp_params *params)
struct mlx4_vport_oper_state *vp_oper;
struct mlx4_priv *priv;
u32 qp_type;
- int port;
+ int port, err = 0;
port = (qpc->pri_path.sched_queue & 0x40) ? 2 : 1;
priv = mlx4_priv(dev);
} else {
struct mlx4_update_qp_params params = {.flags = 0};
- mlx4_update_qp(dev, qpn, MLX4_UPDATE_QP_VSD, ¶ms);
+ err = mlx4_update_qp(dev, qpn, MLX4_UPDATE_QP_VSD, ¶ms);
+ if (err)
+ goto out;
}
}
qpc->pri_path.feup |= MLX4_FSM_FORCE_ETH_SRC_MAC;
qpc->pri_path.grh_mylmc = (0x80 & qpc->pri_path.grh_mylmc) + vp_oper->mac_idx;
}
- return 0;
+out:
+ return err;
}
static int mpt_mask(struct mlx4_dev *dev)
if (!priv->mfunc.master.slave_state)
return -EINVAL;
+ /* check for slave valid, slave not PF, and slave active */
+ if (slave < 0 || slave > dev->persist->num_vfs ||
+ slave == dev->caps.function ||
+ !priv->mfunc.master.slave_state[slave].active)
+ return 0;
+
event_eq = &priv->mfunc.master.slave_state[slave].event_eq[eqe->type];
/* Create the event only if the slave is registered */
if (mac->phydev)
phy_start(mac->phydev);
- init_timer(&mac->tx->clean_timer);
- mac->tx->clean_timer.function = pasemi_mac_tx_timer;
- mac->tx->clean_timer.data = (unsigned long)mac->tx;
- mac->tx->clean_timer.expires = jiffies+HZ;
- add_timer(&mac->tx->clean_timer);
+ setup_timer(&mac->tx->clean_timer, pasemi_mac_tx_timer,
+ (unsigned long)mac->tx);
+ mod_timer(&mac->tx->clean_timer, jiffies + HZ);
return 0;
} __attribute__ ((aligned(64)));
-/* Note: sizeof(rcv_desc) should always be a mutliple of 2 */
+/* Note: sizeof(rcv_desc) should always be a multiple of 2 */
struct rcv_desc {
__le16 reference_handle;
__le16 reserved;
#define NETXEN_IMAGE_START 0x43000 /* compressed image */
#define NETXEN_SECONDARY_START 0x200000 /* backup images */
#define NETXEN_PXE_START 0x3E0000 /* PXE boot rom */
-#define NETXEN_USER_START 0x3E8000 /* Firmare info */
+#define NETXEN_USER_START 0x3E8000 /* Firmware info */
#define NETXEN_FIXED_START 0x3F0000 /* backup of crbinit */
#define NETXEN_USER_START_OLD NETXEN_PXE_START /* very old flash */
#define QLCNIC_BRDCFG_START 0x4000 /* board config */
#define QLCNIC_BOOTLD_START 0x10000 /* bootld */
#define QLCNIC_IMAGE_START 0x43000 /* compressed image */
-#define QLCNIC_USER_START 0x3E8000 /* Firmare info */
+#define QLCNIC_USER_START 0x3E8000 /* Firmware info */
#define QLCNIC_FW_VERSION_OFFSET (QLCNIC_USER_START+0x408)
#define QLCNIC_FW_SIZE_OFFSET (QLCNIC_USER_START+0x40c)
int rc = -EINVAL;
if (!rtl_fw_format_ok(tp, rtl_fw)) {
- netif_err(tp, ifup, dev, "invalid firwmare\n");
+ netif_err(tp, ifup, dev, "invalid firmware\n");
goto out;
}
RTL_W8(ChipCmd, CmdReset);
rtl_udelay_loop_wait_low(tp, &rtl_chipcmd_cond, 100, 100);
-
- netdev_reset_queue(tp->dev);
}
static void rtl_request_uncached_firmware(struct rtl8169_private *tp)
u32 status, len;
u32 opts[2];
int frags;
- bool stop_queue;
if (unlikely(!TX_FRAGS_READY_FOR(tp, skb_shinfo(skb)->nr_frags))) {
netif_err(tp, drv, dev, "BUG! Tx Ring full when queue awake!\n");
txd->opts2 = cpu_to_le32(opts[1]);
- netdev_sent_queue(dev, skb->len);
-
skb_tx_timestamp(skb);
/* Force memory writes to complete before releasing descriptor */
tp->cur_tx += frags + 1;
- stop_queue = !TX_FRAGS_READY_FOR(tp, MAX_SKB_FRAGS);
+ RTL_W8(TxPoll, NPQ);
- if (!skb->xmit_more || stop_queue ||
- netif_xmit_stopped(netdev_get_tx_queue(dev, 0))) {
- RTL_W8(TxPoll, NPQ);
-
- mmiowb();
- }
+ mmiowb();
- if (stop_queue) {
+ if (!TX_FRAGS_READY_FOR(tp, MAX_SKB_FRAGS)) {
/* Avoid wrongly optimistic queue wake-up: rtl_tx thread must
* not miss a ring update when it notices a stopped queue.
*/
static void rtl_tx(struct net_device *dev, struct rtl8169_private *tp)
{
unsigned int dirty_tx, tx_left;
- unsigned int bytes_compl = 0, pkts_compl = 0;
dirty_tx = tp->dirty_tx;
smp_rmb();
rtl8169_unmap_tx_skb(&tp->pci_dev->dev, tx_skb,
tp->TxDescArray + entry);
if (status & LastFrag) {
- pkts_compl++;
- bytes_compl += tx_skb->skb->len;
+ u64_stats_update_begin(&tp->tx_stats.syncp);
+ tp->tx_stats.packets++;
+ tp->tx_stats.bytes += tx_skb->skb->len;
+ u64_stats_update_end(&tp->tx_stats.syncp);
dev_kfree_skb_any(tx_skb->skb);
tx_skb->skb = NULL;
}
}
if (tp->dirty_tx != dirty_tx) {
- netdev_completed_queue(tp->dev, pkts_compl, bytes_compl);
-
- u64_stats_update_begin(&tp->tx_stats.syncp);
- tp->tx_stats.packets += pkts_compl;
- tp->tx_stats.bytes += bytes_compl;
- u64_stats_update_end(&tp->tx_stats.syncp);
-
tp->dirty_tx = dirty_tx;
/* Sync with rtl8169_start_xmit:
* - publish dirty_tx ring index (write barrier)
.tpauser = 1,
.hw_swap = 1,
.rmiimode = 1,
- .shift_rd0 = 1,
};
static void sh_eth_set_rate_sh7724(struct net_device *ndev)
msleep(2); /* max frame time at 10 Mbps < 1250 us */
sh_eth_get_stats(ndev);
sh_eth_reset(ndev);
+
+ /* Set MAC address again */
+ update_mac_address(ndev);
}
/* free Tx skb function */
txdesc = &mdp->tx_ring[entry];
if (txdesc->status & cpu_to_edmac(mdp, TD_TACT))
break;
+ /* TACT bit must be checked before all the following reads */
+ rmb();
/* Free the original skb. */
if (mdp->tx_skbuff[entry]) {
dma_unmap_single(&ndev->dev, txdesc->addr,
limit = boguscnt;
rxdesc = &mdp->rx_ring[entry];
while (!(rxdesc->status & cpu_to_edmac(mdp, RD_RACT))) {
+ /* RACT bit must be checked before all the following reads */
+ rmb();
desc_status = edmac_to_cpu(mdp, rxdesc->status);
pkt_len = rxdesc->frame_length;
/* In case of almost all GETHER/ETHERs, the Receive Frame State
* (RFS) bits in the Receive Descriptor 0 are from bit 9 to
- * bit 0. However, in case of the R8A7740, R8A779x, and
- * R7S72100 the RFS bits are from bit 25 to bit 16. So, the
+ * bit 0. However, in case of the R8A7740 and R7S72100
+ * the RFS bits are from bit 25 to bit 16. So, the
* driver needs right shifting by 16.
*/
if (mdp->cd->shift_rd0)
skb_checksum_none_assert(skb);
rxdesc->addr = dma_addr;
}
+ wmb(); /* RACT bit must be set after all the above writes */
if (entry >= mdp->num_rx_ring - 1)
rxdesc->status |=
cpu_to_edmac(mdp, RD_RACT | RD_RFP | RD_RDEL);
/* If we don't need to check status, don't. -KDU */
if (!(sh_eth_read(ndev, EDRRR) & EDRRR_R)) {
/* fix the values for the next receiving if RDE is set */
- if (intr_status & EESR_RDE) {
+ if (intr_status & EESR_RDE && mdp->reg_offset[RDFAR] != 0) {
u32 count = (sh_eth_read(ndev, RDFAR) -
sh_eth_read(ndev, RDLAR)) >> 4;
}
spin_unlock_irqrestore(&mdp->lock, flags);
- if (skb_padto(skb, ETH_ZLEN))
+ if (skb_put_padto(skb, ETH_ZLEN))
return NETDEV_TX_OK;
entry = mdp->cur_tx % mdp->num_tx_ring;
}
txdesc->buffer_length = skb->len;
+ wmb(); /* TACT bit must be set after all the above writes */
if (entry >= mdp->num_tx_ring - 1)
txdesc->status |= cpu_to_edmac(mdp, TD_TACT | TD_TDLE);
else
u64 val = rocker_read64(rocker_port->rocker, PORT_PHYS_ENABLE);
if (enable)
- val |= 1 << rocker_port->lport;
+ val |= 1ULL << rocker_port->lport;
else
- val &= ~(1 << rocker_port->lport);
+ val &= ~(1ULL << rocker_port->lport);
rocker_write64(rocker_port->rocker, PORT_PHYS_ENABLE, val);
}
alloc_size = sizeof(struct rocker_port *) * rocker->port_count;
rocker->ports = kmalloc(alloc_size, GFP_KERNEL);
+ if (!rocker->ports)
+ return -ENOMEM;
for (i = 0; i < rocker->port_count; i++) {
err = rocker_probe_port(rocker, i);
if (err)
struct net_device *master = netdev_master_upper_dev_get(dev);
int err = 0;
+ /* There are currently three cases handled here:
+ * 1. Joining a bridge
+ * 2. Leaving a previously joined bridge
+ * 3. Other, e.g. being added to or removed from a bond or openvswitch,
+ * in which case nothing is done
+ */
if (master && master->rtnl_link_ops &&
!strcmp(master->rtnl_link_ops->kind, "bridge"))
err = rocker_port_bridge_join(rocker_port, master);
- else
+ else if (rocker_port_is_bridged(rocker_port))
err = rocker_port_bridge_leave(rocker_port);
return err;
smc->packets_waiting = 0;
smc_reset(dev);
- init_timer(&smc->media);
- smc->media.function = media_check;
- smc->media.data = (u_long) dev;
- smc->media.expires = jiffies + HZ;
- add_timer(&smc->media);
+ setup_timer(&smc->media, media_check, (u_long)dev);
+ mod_timer(&smc->media, jiffies + HZ);
return 0;
} /* smc_open */
#include "smc91x.h"
+#if defined(CONFIG_ASSABET_NEPONSET)
+#include <mach/assabet.h>
+#include <mach/neponset.h>
+#endif
+
#ifndef SMC_NOWAIT
# define SMC_NOWAIT 0
#endif
const struct of_device_id *match = NULL;
struct smc_local *lp;
struct net_device *ndev;
- struct resource *res;
+ struct resource *res, *ires;
unsigned int __iomem *addr;
unsigned long irq_flags = SMC_IRQ_FLAGS;
- unsigned long irq_resflags;
int ret;
ndev = alloc_etherdev(sizeof(struct smc_local));
goto out_free_netdev;
}
- ndev->irq = platform_get_irq(pdev, 0);
- if (ndev->irq <= 0) {
+ ires = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
+ if (!ires) {
ret = -ENODEV;
goto out_release_io;
}
- /*
- * If this platform does not specify any special irqflags, or if
- * the resource supplies a trigger, override the irqflags with
- * the trigger flags from the resource.
- */
- irq_resflags = irqd_get_trigger_type(irq_get_irq_data(ndev->irq));
- if (irq_flags == -1 || irq_resflags & IRQF_TRIGGER_MASK)
- irq_flags = irq_resflags & IRQF_TRIGGER_MASK;
+
+ ndev->irq = ires->start;
+
+ if (irq_flags == -1 || ires->flags & IRQF_TRIGGER_MASK)
+ irq_flags = ires->flags & IRQF_TRIGGER_MASK;
ret = smc_request_attrib(pdev, ndev);
if (ret)
goto out_release_io;
-#if defined(CONFIG_SA1100_ASSABET)
- neponset_ncr_set(NCR_ENET_OSC_EN);
+#if defined(CONFIG_ASSABET_NEPONSET)
+ if (machine_is_assabet() && machine_has_neponset())
+ neponset_ncr_set(NCR_ENET_OSC_EN);
#endif
platform_set_drvdata(pdev, ndev);
ret = smc_enable_device(pdev);
* Define your architecture specific bus configuration parameters here.
*/
-#if defined(CONFIG_ARCH_LUBBOCK) ||\
- defined(CONFIG_MACH_MAINSTONE) ||\
- defined(CONFIG_MACH_ZYLONITE) ||\
- defined(CONFIG_MACH_LITTLETON) ||\
- defined(CONFIG_MACH_ZYLONITE2) ||\
- defined(CONFIG_ARCH_VIPER) ||\
- defined(CONFIG_MACH_STARGATE2) ||\
- defined(CONFIG_ARCH_VERSATILE)
+#if defined(CONFIG_ARM)
#include <asm/mach-types.h>
/* We actually can't write halfwords properly if not word aligned */
static inline void SMC_outw(u16 val, void __iomem *ioaddr, int reg)
{
- if ((machine_is_mainstone() || machine_is_stargate2()) && reg & 2) {
- unsigned int v = val << 16;
- v |= readl(ioaddr + (reg & ~2)) & 0xffff;
- writel(v, ioaddr + (reg & ~2));
- } else {
- writew(val, ioaddr + reg);
- }
-}
-
-#elif defined(CONFIG_SA1100_PLEB)
-/* We can only do 16-bit reads and writes in the static memory space. */
-#define SMC_CAN_USE_8BIT 1
-#define SMC_CAN_USE_16BIT 1
-#define SMC_CAN_USE_32BIT 0
-#define SMC_IO_SHIFT 0
-#define SMC_NOWAIT 1
-
-#define SMC_inb(a, r) readb((a) + (r))
-#define SMC_insb(a, r, p, l) readsb((a) + (r), p, (l))
-#define SMC_inw(a, r) readw((a) + (r))
-#define SMC_insw(a, r, p, l) readsw((a) + (r), p, l)
-#define SMC_outb(v, a, r) writeb(v, (a) + (r))
-#define SMC_outsb(a, r, p, l) writesb((a) + (r), p, (l))
-#define SMC_outw(v, a, r) writew(v, (a) + (r))
-#define SMC_outsw(a, r, p, l) writesw((a) + (r), p, l)
-
-#define SMC_IRQ_FLAGS (-1)
-
-#elif defined(CONFIG_SA1100_ASSABET)
-
-#include <mach/neponset.h>
-
-/* We can only do 8-bit reads and writes in the static memory space. */
-#define SMC_CAN_USE_8BIT 1
-#define SMC_CAN_USE_16BIT 0
-#define SMC_CAN_USE_32BIT 0
-#define SMC_NOWAIT 1
-
-/* The first two address lines aren't connected... */
-#define SMC_IO_SHIFT 2
-
-#define SMC_inb(a, r) readb((a) + (r))
-#define SMC_outb(v, a, r) writeb(v, (a) + (r))
-#define SMC_insb(a, r, p, l) readsb((a) + (r), p, (l))
-#define SMC_outsb(a, r, p, l) writesb((a) + (r), p, (l))
-#define SMC_IRQ_FLAGS (-1) /* from resource */
-
-#elif defined(CONFIG_MACH_LOGICPD_PXA270) || \
- defined(CONFIG_MACH_NOMADIK_8815NHK)
-
-#define SMC_CAN_USE_8BIT 0
-#define SMC_CAN_USE_16BIT 1
-#define SMC_CAN_USE_32BIT 0
-#define SMC_IO_SHIFT 0
-#define SMC_NOWAIT 1
-
-#define SMC_inw(a, r) readw((a) + (r))
-#define SMC_outw(v, a, r) writew(v, (a) + (r))
-#define SMC_insw(a, r, p, l) readsw((a) + (r), p, l)
-#define SMC_outsw(a, r, p, l) writesw((a) + (r), p, l)
-
-#elif defined(CONFIG_ARCH_INNOKOM) || \
- defined(CONFIG_ARCH_PXA_IDP) || \
- defined(CONFIG_ARCH_RAMSES) || \
- defined(CONFIG_ARCH_PCM027)
-
-#define SMC_CAN_USE_8BIT 1
-#define SMC_CAN_USE_16BIT 1
-#define SMC_CAN_USE_32BIT 1
-#define SMC_IO_SHIFT 0
-#define SMC_NOWAIT 1
-#define SMC_USE_PXA_DMA 1
-
-#define SMC_inb(a, r) readb((a) + (r))
-#define SMC_inw(a, r) readw((a) + (r))
-#define SMC_inl(a, r) readl((a) + (r))
-#define SMC_outb(v, a, r) writeb(v, (a) + (r))
-#define SMC_outl(v, a, r) writel(v, (a) + (r))
-#define SMC_insl(a, r, p, l) readsl((a) + (r), p, l)
-#define SMC_outsl(a, r, p, l) writesl((a) + (r), p, l)
-#define SMC_insw(a, r, p, l) readsw((a) + (r), p, l)
-#define SMC_outsw(a, r, p, l) writesw((a) + (r), p, l)
-#define SMC_IRQ_FLAGS (-1) /* from resource */
-
-/* We actually can't write halfwords properly if not word aligned */
-static inline void
-SMC_outw(u16 val, void __iomem *ioaddr, int reg)
-{
- if (reg & 2) {
+ if ((machine_is_mainstone() || machine_is_stargate2() ||
+ machine_is_pxa_idp()) && reg & 2) {
unsigned int v = val << 16;
v |= readl(ioaddr + (reg & ~2)) & 0xffff;
writel(v, ioaddr + (reg & ~2));
#define RPC_LSA_DEFAULT RPC_LED_100_10
#define RPC_LSB_DEFAULT RPC_LED_TX_RX
-#elif defined(CONFIG_ARCH_MSM)
-
-#define SMC_CAN_USE_8BIT 0
-#define SMC_CAN_USE_16BIT 1
-#define SMC_CAN_USE_32BIT 0
-#define SMC_NOWAIT 1
-
-#define SMC_inw(a, r) readw((a) + (r))
-#define SMC_outw(v, a, r) writew(v, (a) + (r))
-#define SMC_insw(a, r, p, l) readsw((a) + (r), p, l)
-#define SMC_outsw(a, r, p, l) writesw((a) + (r), p, l)
-
-#define SMC_IRQ_FLAGS IRQF_TRIGGER_HIGH
-
#elif defined(CONFIG_COLDFIRE)
#define SMC_CAN_USE_8BIT 0
spin_lock_irqsave(&priv->lock, flags);
if (!priv->eee_active) {
priv->eee_active = 1;
- init_timer(&priv->eee_ctrl_timer);
- priv->eee_ctrl_timer.function = stmmac_eee_ctrl_timer;
- priv->eee_ctrl_timer.data = (unsigned long)priv;
- priv->eee_ctrl_timer.expires = STMMAC_LPI_T(eee_timer);
- add_timer(&priv->eee_ctrl_timer);
+ setup_timer(&priv->eee_ctrl_timer,
+ stmmac_eee_ctrl_timer,
+ (unsigned long)priv);
+ mod_timer(&priv->eee_ctrl_timer,
+ STMMAC_LPI_T(eee_timer));
priv->hw->mac->set_eee_timer(priv->hw,
STMMAC_DEFAULT_LIT_LS,
struct stmmac_priv *priv = NULL;
struct plat_stmmacenet_data *plat_dat = NULL;
const char *mac = NULL;
+ int irq, wol_irq, lpi_irq;
+
+ /* Get IRQ information early to have an ability to ask for deferred
+ * probe if needed before we went too far with resource allocation.
+ */
+ irq = platform_get_irq_byname(pdev, "macirq");
+ if (irq < 0) {
+ if (irq != -EPROBE_DEFER) {
+ dev_err(dev,
+ "MAC IRQ configuration information not found\n");
+ }
+ return irq;
+ }
+
+ /* On some platforms e.g. SPEAr the wake up irq differs from the mac irq
+ * The external wake up irq can be passed through the platform code
+ * named as "eth_wake_irq"
+ *
+ * In case the wake up interrupt is not passed from the platform
+ * so the driver will continue to use the mac irq (ndev->irq)
+ */
+ wol_irq = platform_get_irq_byname(pdev, "eth_wake_irq");
+ if (wol_irq < 0) {
+ if (wol_irq == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ wol_irq = irq;
+ }
+
+ lpi_irq = platform_get_irq_byname(pdev, "eth_lpi");
+ if (lpi_irq == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
addr = devm_ioremap_resource(dev, res);
return PTR_ERR(priv);
}
+ /* Copy IRQ values to priv structure which is now avaialble */
+ priv->dev->irq = irq;
+ priv->wol_irq = wol_irq;
+ priv->lpi_irq = lpi_irq;
+
/* Get MAC address if available (DT) */
if (mac)
memcpy(priv->dev->dev_addr, mac, ETH_ALEN);
- /* Get the MAC information */
- priv->dev->irq = platform_get_irq_byname(pdev, "macirq");
- if (priv->dev->irq < 0) {
- if (priv->dev->irq != -EPROBE_DEFER) {
- netdev_err(priv->dev,
- "MAC IRQ configuration information not found\n");
- }
- return priv->dev->irq;
- }
-
- /*
- * On some platforms e.g. SPEAr the wake up irq differs from the mac irq
- * The external wake up irq can be passed through the platform code
- * named as "eth_wake_irq"
- *
- * In case the wake up interrupt is not passed from the platform
- * so the driver will continue to use the mac irq (ndev->irq)
- */
- priv->wol_irq = platform_get_irq_byname(pdev, "eth_wake_irq");
- if (priv->wol_irq < 0) {
- if (priv->wol_irq == -EPROBE_DEFER)
- return -EPROBE_DEFER;
- priv->wol_irq = priv->dev->irq;
- }
-
- priv->lpi_irq = platform_get_irq_byname(pdev, "eth_lpi");
- if (priv->lpi_irq == -EPROBE_DEFER)
- return -EPROBE_DEFER;
-
platform_set_drvdata(pdev, priv->dev);
pr_debug("STMMAC platform driver registration completed");
*flow_type = IP_USER_FLOW;
break;
default:
- return 0;
+ return -EINVAL;
}
- return 1;
+ return 0;
}
static int niu_ethflow_to_class(int flow_type, u64 *class)
class = (tp->key[0] & TCAM_V4KEY0_CLASS_CODE) >>
TCAM_V4KEY0_CLASS_CODE_SHIFT;
ret = niu_class_to_ethflow(class, &fsp->flow_type);
-
if (ret < 0) {
netdev_info(np->dev, "niu%d: niu_class_to_ethflow failed\n",
parent->index);
- ret = -EINVAL;
goto out;
}
cpsw_ale_add_mcast(priv->ale, priv->ndev->broadcast,
port_mask, ALE_VLAN, slave->port_vlan, 0);
cpsw_ale_add_ucast(priv->ale, priv->mac_addr,
- priv->host_port, ALE_VLAN, slave->port_vlan);
+ priv->host_port, ALE_VLAN | ALE_SECURE, slave->port_vlan);
}
static void soft_reset_slave(struct cpsw_slave *slave)
return 0;
}
+#ifdef CONFIG_PM_SLEEP
static int cpsw_suspend(struct device *dev)
{
struct platform_device *pdev = to_platform_device(dev);
}
return 0;
}
+#endif
-static const struct dev_pm_ops cpsw_pm_ops = {
- .suspend = cpsw_suspend,
- .resume = cpsw_resume,
-};
+static SIMPLE_DEV_PM_OPS(cpsw_pm_ops, cpsw_suspend, cpsw_resume);
static const struct of_device_id cpsw_of_mtable[] = {
{ .compatible = "ti,cpsw", },
return 0;
}
+#ifdef CONFIG_PM_SLEEP
static int davinci_mdio_suspend(struct device *dev)
{
struct davinci_mdio_data *data = dev_get_drvdata(dev);
return 0;
}
+#endif
static const struct dev_pm_ops davinci_mdio_pm_ops = {
- .suspend_late = davinci_mdio_suspend,
- .resume_early = davinci_mdio_resume,
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(davinci_mdio_suspend, davinci_mdio_resume)
};
#if IS_ENABLED(CONFIG_OF)
}
if (rx_count < budget) {
+ napi_complete(napi);
w5100_write(priv, W5100_IMR, IR_S0);
mmiowb();
- napi_complete(napi);
}
return rx_count;
}
if (rx_count < budget) {
+ napi_complete(napi);
w5300_write(priv, W5300_IMR, IR_S0);
mmiowb();
- napi_complete(napi);
}
return rx_count;
int i;
static const u8 allmulti[] = { 0x01, 0x00, 0x00, 0x00, 0x00, 0x00 };
- if (dev->flags & IFF_ALLMULTI) {
+ if ((dev->flags & IFF_ALLMULTI) && !(dev->flags & IFF_PROMISC)) {
for (i = 0; i < ETH_ALEN; i++) {
__raw_writel(allmulti[i], &port->regs->mcast_addr[i]);
__raw_writel(allmulti[i], &port->regs->mcast_mask[i]);
rx_handler_result_t ipvlan_handle_frame(struct sk_buff **pskb);
int ipvlan_queue_xmit(struct sk_buff *skb, struct net_device *dev);
void ipvlan_ht_addr_add(struct ipvl_dev *ipvlan, struct ipvl_addr *addr);
-bool ipvlan_addr_busy(struct ipvl_dev *ipvlan, void *iaddr, bool is_v6);
+struct ipvl_addr *ipvlan_find_addr(const struct ipvl_dev *ipvlan,
+ const void *iaddr, bool is_v6);
+bool ipvlan_addr_busy(struct ipvl_port *port, void *iaddr, bool is_v6);
struct ipvl_addr *ipvlan_ht_addr_lookup(const struct ipvl_port *port,
const void *iaddr, bool is_v6);
void ipvlan_ht_addr_del(struct ipvl_addr *addr, bool sync);
hash = (addr->atype == IPVL_IPV6) ?
ipvlan_get_v6_hash(&addr->ip6addr) :
ipvlan_get_v4_hash(&addr->ip4addr);
- hlist_add_head_rcu(&addr->hlnode, &port->hlhead[hash]);
+ if (hlist_unhashed(&addr->hlnode))
+ hlist_add_head_rcu(&addr->hlnode, &port->hlhead[hash]);
}
void ipvlan_ht_addr_del(struct ipvl_addr *addr, bool sync)
{
- hlist_del_rcu(&addr->hlnode);
+ hlist_del_init_rcu(&addr->hlnode);
if (sync)
synchronize_rcu();
}
-bool ipvlan_addr_busy(struct ipvl_dev *ipvlan, void *iaddr, bool is_v6)
+struct ipvl_addr *ipvlan_find_addr(const struct ipvl_dev *ipvlan,
+ const void *iaddr, bool is_v6)
{
- struct ipvl_port *port = ipvlan->port;
struct ipvl_addr *addr;
list_for_each_entry(addr, &ipvlan->addrs, anode) {
ipv6_addr_equal(&addr->ip6addr, iaddr)) ||
(!is_v6 && addr->atype == IPVL_IPV4 &&
addr->ip4addr.s_addr == ((struct in_addr *)iaddr)->s_addr))
- return true;
+ return addr;
}
+ return NULL;
+}
+
+bool ipvlan_addr_busy(struct ipvl_port *port, void *iaddr, bool is_v6)
+{
+ struct ipvl_dev *ipvlan;
- if (ipvlan_ht_addr_lookup(port, iaddr, is_v6))
- return true;
+ ASSERT_RTNL();
+ list_for_each_entry(ipvlan, &port->ipvlans, pnode) {
+ if (ipvlan_find_addr(ipvlan, iaddr, is_v6))
+ return true;
+ }
return false;
}
if (skb->protocol == htons(ETH_P_PAUSE))
return;
- list_for_each_entry(ipvlan, &port->ipvlans, pnode) {
+ rcu_read_lock();
+ list_for_each_entry_rcu(ipvlan, &port->ipvlans, pnode) {
if (local && (ipvlan == in_dev))
continue;
mcast_acct:
ipvlan_count_rx(ipvlan, len, ret == NET_RX_SUCCESS, true);
}
+ rcu_read_unlock();
/* Locally generated? ...Forward a copy to the main-device as
* well. On the RX side we'll ignore it (wont give it to any
if (ipvlan->ipv6cnt > 0 || ipvlan->ipv4cnt > 0) {
list_for_each_entry_safe(addr, next, &ipvlan->addrs, anode) {
ipvlan_ht_addr_del(addr, !dev->dismantle);
- list_del_rcu(&addr->anode);
+ list_del(&addr->anode);
}
}
list_del_rcu(&ipvlan->pnode);
{
struct ipvl_addr *addr;
- if (ipvlan_addr_busy(ipvlan, ip6_addr, true)) {
+ if (ipvlan_addr_busy(ipvlan->port, ip6_addr, true)) {
netif_err(ipvlan, ifup, ipvlan->dev,
"Failed to add IPv6=%pI6c addr for %s intf\n",
ip6_addr, ipvlan->dev->name);
addr->master = ipvlan;
memcpy(&addr->ip6addr, ip6_addr, sizeof(struct in6_addr));
addr->atype = IPVL_IPV6;
- list_add_tail_rcu(&addr->anode, &ipvlan->addrs);
+ list_add_tail(&addr->anode, &ipvlan->addrs);
ipvlan->ipv6cnt++;
- ipvlan_ht_addr_add(ipvlan, addr);
+ /* If the interface is not up, the address will be added to the hash
+ * list by ipvlan_open.
+ */
+ if (netif_running(ipvlan->dev))
+ ipvlan_ht_addr_add(ipvlan, addr);
return 0;
}
{
struct ipvl_addr *addr;
- addr = ipvlan_ht_addr_lookup(ipvlan->port, ip6_addr, true);
+ addr = ipvlan_find_addr(ipvlan, ip6_addr, true);
if (!addr)
return;
ipvlan_ht_addr_del(addr, true);
- list_del_rcu(&addr->anode);
+ list_del(&addr->anode);
ipvlan->ipv6cnt--;
WARN_ON(ipvlan->ipv6cnt < 0);
kfree_rcu(addr, rcu);
{
struct ipvl_addr *addr;
- if (ipvlan_addr_busy(ipvlan, ip4_addr, false)) {
+ if (ipvlan_addr_busy(ipvlan->port, ip4_addr, false)) {
netif_err(ipvlan, ifup, ipvlan->dev,
"Failed to add IPv4=%pI4 on %s intf.\n",
ip4_addr, ipvlan->dev->name);
addr->master = ipvlan;
memcpy(&addr->ip4addr, ip4_addr, sizeof(struct in_addr));
addr->atype = IPVL_IPV4;
- list_add_tail_rcu(&addr->anode, &ipvlan->addrs);
+ list_add_tail(&addr->anode, &ipvlan->addrs);
ipvlan->ipv4cnt++;
- ipvlan_ht_addr_add(ipvlan, addr);
+ /* If the interface is not up, the address will be added to the hash
+ * list by ipvlan_open.
+ */
+ if (netif_running(ipvlan->dev))
+ ipvlan_ht_addr_add(ipvlan, addr);
ipvlan_set_broadcast_mac_filter(ipvlan, true);
return 0;
{
struct ipvl_addr *addr;
- addr = ipvlan_ht_addr_lookup(ipvlan->port, ip4_addr, false);
+ addr = ipvlan_find_addr(ipvlan, ip4_addr, false);
if (!addr)
return;
ipvlan_ht_addr_del(addr, true);
- list_del_rcu(&addr->anode);
+ list_del(&addr->anode);
ipvlan->ipv4cnt--;
WARN_ON(ipvlan->ipv4cnt < 0);
if (!ipvlan->ipv4cnt)
} /* else everything is zero */
}
+/* Neighbour code has some assumptions on HH_DATA_MOD alignment */
+#define MACVTAP_RESERVE HH_DATA_OFF(ETH_HLEN)
+
/* Get packet from user space buffer */
static ssize_t macvtap_get_user(struct macvtap_queue *q, struct msghdr *m,
struct iov_iter *from, int noblock)
{
- int good_linear = SKB_MAX_HEAD(NET_IP_ALIGN);
+ int good_linear = SKB_MAX_HEAD(MACVTAP_RESERVE);
struct sk_buff *skb;
struct macvlan_dev *vlan;
unsigned long total_len = iov_iter_count(from);
linear = macvtap16_to_cpu(q, vnet_hdr.hdr_len);
}
- skb = macvtap_alloc_skb(&q->sk, NET_IP_ALIGN, copylen,
+ skb = macvtap_alloc_skb(&q->sk, MACVTAP_RESERVE, copylen,
linear, noblock, &err);
if (!skb)
goto err;
#define XGBE_PHY_CDR_RATE_PROPERTY "amd,serdes-cdr-rate"
#define XGBE_PHY_PQ_SKEW_PROPERTY "amd,serdes-pq-skew"
#define XGBE_PHY_TX_AMP_PROPERTY "amd,serdes-tx-amp"
+#define XGBE_PHY_DFE_CFG_PROPERTY "amd,serdes-dfe-tap-config"
+#define XGBE_PHY_DFE_ENA_PROPERTY "amd,serdes-dfe-tap-enable"
#define XGBE_PHY_SPEEDS 3
#define XGBE_PHY_SPEED_1000 0
#define SPEED_10000_BLWC 0
#define SPEED_10000_CDR 0x7
#define SPEED_10000_PLL 0x1
-#define SPEED_10000_PQ 0x1e
+#define SPEED_10000_PQ 0x12
#define SPEED_10000_RATE 0x0
#define SPEED_10000_TXAMP 0xa
#define SPEED_10000_WORD 0x7
+#define SPEED_10000_DFE_TAP_CONFIG 0x1
+#define SPEED_10000_DFE_TAP_ENABLE 0x7f
#define SPEED_2500_BLWC 1
#define SPEED_2500_CDR 0x2
#define SPEED_2500_RATE 0x1
#define SPEED_2500_TXAMP 0xf
#define SPEED_2500_WORD 0x1
+#define SPEED_2500_DFE_TAP_CONFIG 0x3
+#define SPEED_2500_DFE_TAP_ENABLE 0x0
#define SPEED_1000_BLWC 1
#define SPEED_1000_CDR 0x2
#define SPEED_1000_RATE 0x3
#define SPEED_1000_TXAMP 0xf
#define SPEED_1000_WORD 0x1
+#define SPEED_1000_DFE_TAP_CONFIG 0x3
+#define SPEED_1000_DFE_TAP_ENABLE 0x0
/* SerDes RxTx register offsets */
+#define RXTX_REG6 0x0018
#define RXTX_REG20 0x0050
+#define RXTX_REG22 0x0058
#define RXTX_REG114 0x01c8
+#define RXTX_REG129 0x0204
/* SerDes RxTx register entry bit positions and sizes */
+#define RXTX_REG6_RESETB_RXD_INDEX 8
+#define RXTX_REG6_RESETB_RXD_WIDTH 1
#define RXTX_REG20_BLWC_ENA_INDEX 2
#define RXTX_REG20_BLWC_ENA_WIDTH 1
#define RXTX_REG114_PQ_REG_INDEX 9
#define RXTX_REG114_PQ_REG_WIDTH 7
+#define RXTX_REG129_RXDFE_CONFIG_INDEX 14
+#define RXTX_REG129_RXDFE_CONFIG_WIDTH 2
/* Bit setting and getting macros
* The get macro will extract the current bit field value from within
SPEED_10000_TXAMP,
};
+static const u32 amd_xgbe_phy_serdes_dfe_tap_cfg[] = {
+ SPEED_1000_DFE_TAP_CONFIG,
+ SPEED_2500_DFE_TAP_CONFIG,
+ SPEED_10000_DFE_TAP_CONFIG,
+};
+
+static const u32 amd_xgbe_phy_serdes_dfe_tap_ena[] = {
+ SPEED_1000_DFE_TAP_ENABLE,
+ SPEED_2500_DFE_TAP_ENABLE,
+ SPEED_10000_DFE_TAP_ENABLE,
+};
+
enum amd_xgbe_phy_an {
AMD_XGBE_AN_READY = 0,
AMD_XGBE_AN_PAGE_RECEIVED,
u32 serdes_cdr_rate[XGBE_PHY_SPEEDS];
u32 serdes_pq_skew[XGBE_PHY_SPEEDS];
u32 serdes_tx_amp[XGBE_PHY_SPEEDS];
+ u32 serdes_dfe_tap_cfg[XGBE_PHY_SPEEDS];
+ u32 serdes_dfe_tap_ena[XGBE_PHY_SPEEDS];
/* Auto-negotiation state machine support */
struct mutex an_mutex;
status = XSIR0_IOREAD(priv, SIR0_STATUS);
if (XSIR_GET_BITS(status, SIR0_STATUS, RX_READY) &&
XSIR_GET_BITS(status, SIR0_STATUS, TX_READY))
- return;
+ goto rx_reset;
}
netdev_dbg(phydev->attached_dev, "SerDes rx/tx not ready (%#hx)\n",
status);
+
+rx_reset:
+ /* Perform Rx reset for the DFE changes */
+ XRXTX_IOWRITE_BITS(priv, RXTX_REG6, RESETB_RXD, 0);
+ XRXTX_IOWRITE_BITS(priv, RXTX_REG6, RESETB_RXD, 1);
}
static int amd_xgbe_phy_xgmii_mode(struct phy_device *phydev)
priv->serdes_blwc[XGBE_PHY_SPEED_10000]);
XRXTX_IOWRITE_BITS(priv, RXTX_REG114, PQ_REG,
priv->serdes_pq_skew[XGBE_PHY_SPEED_10000]);
+ XRXTX_IOWRITE_BITS(priv, RXTX_REG129, RXDFE_CONFIG,
+ priv->serdes_dfe_tap_cfg[XGBE_PHY_SPEED_10000]);
+ XRXTX_IOWRITE(priv, RXTX_REG22,
+ priv->serdes_dfe_tap_ena[XGBE_PHY_SPEED_10000]);
amd_xgbe_phy_serdes_complete_ratechange(phydev);
priv->serdes_blwc[XGBE_PHY_SPEED_2500]);
XRXTX_IOWRITE_BITS(priv, RXTX_REG114, PQ_REG,
priv->serdes_pq_skew[XGBE_PHY_SPEED_2500]);
+ XRXTX_IOWRITE_BITS(priv, RXTX_REG129, RXDFE_CONFIG,
+ priv->serdes_dfe_tap_cfg[XGBE_PHY_SPEED_2500]);
+ XRXTX_IOWRITE(priv, RXTX_REG22,
+ priv->serdes_dfe_tap_ena[XGBE_PHY_SPEED_2500]);
amd_xgbe_phy_serdes_complete_ratechange(phydev);
priv->serdes_blwc[XGBE_PHY_SPEED_1000]);
XRXTX_IOWRITE_BITS(priv, RXTX_REG114, PQ_REG,
priv->serdes_pq_skew[XGBE_PHY_SPEED_1000]);
+ XRXTX_IOWRITE_BITS(priv, RXTX_REG129, RXDFE_CONFIG,
+ priv->serdes_dfe_tap_cfg[XGBE_PHY_SPEED_1000]);
+ XRXTX_IOWRITE(priv, RXTX_REG22,
+ priv->serdes_dfe_tap_ena[XGBE_PHY_SPEED_1000]);
amd_xgbe_phy_serdes_complete_ratechange(phydev);
sizeof(priv->serdes_tx_amp));
}
+ if (device_property_present(phy_dev, XGBE_PHY_DFE_CFG_PROPERTY)) {
+ ret = device_property_read_u32_array(phy_dev,
+ XGBE_PHY_DFE_CFG_PROPERTY,
+ priv->serdes_dfe_tap_cfg,
+ XGBE_PHY_SPEEDS);
+ if (ret) {
+ dev_err(dev, "invalid %s property\n",
+ XGBE_PHY_DFE_CFG_PROPERTY);
+ goto err_sir1;
+ }
+ } else {
+ memcpy(priv->serdes_dfe_tap_cfg,
+ amd_xgbe_phy_serdes_dfe_tap_cfg,
+ sizeof(priv->serdes_dfe_tap_cfg));
+ }
+
+ if (device_property_present(phy_dev, XGBE_PHY_DFE_ENA_PROPERTY)) {
+ ret = device_property_read_u32_array(phy_dev,
+ XGBE_PHY_DFE_ENA_PROPERTY,
+ priv->serdes_dfe_tap_ena,
+ XGBE_PHY_SPEEDS);
+ if (ret) {
+ dev_err(dev, "invalid %s property\n",
+ XGBE_PHY_DFE_ENA_PROPERTY);
+ goto err_sir1;
+ }
+ } else {
+ memcpy(priv->serdes_dfe_tap_ena,
+ amd_xgbe_phy_serdes_dfe_tap_ena,
+ sizeof(priv->serdes_dfe_tap_ena));
+ }
+
phydev->priv = priv;
if (!priv->adev || acpi_disabled)
return idx < MAX_NUM_SETTINGS ? idx : MAX_NUM_SETTINGS - 1;
}
+/**
+ * phy_check_valid - check if there is a valid PHY setting which matches
+ * speed, duplex, and feature mask
+ * @speed: speed to match
+ * @duplex: duplex to match
+ * @features: A mask of the valid settings
+ *
+ * Description: Returns true if there is a valid setting, false otherwise.
+ */
+static inline bool phy_check_valid(int speed, int duplex, u32 features)
+{
+ unsigned int idx;
+
+ idx = phy_find_valid(phy_find_setting(speed, duplex), features);
+
+ return settings[idx].speed == speed && settings[idx].duplex == duplex &&
+ (settings[idx].setting & features);
+}
+
/**
* phy_sanitize_settings - make sure the PHY is set to supported speed and duplex
* @phydev: the target phy_device struct
int eee_lp, eee_cap, eee_adv;
u32 lp, cap, adv;
int status;
- unsigned int idx;
/* Read phy status to properly get the right settings */
status = phy_read_status(phydev);
adv = mmd_eee_adv_to_ethtool_adv_t(eee_adv);
lp = mmd_eee_adv_to_ethtool_adv_t(eee_lp);
- idx = phy_find_setting(phydev->speed, phydev->duplex);
- if (!(lp & adv & settings[idx].setting))
+ if (!phy_check_valid(phydev->speed, phydev->duplex, lp & adv))
goto eee_exit_err;
if (clk_stop_enable) {
static struct team_port *team_port_get_rcu(const struct net_device *dev)
{
- struct team_port *port = rcu_dereference(dev->rx_handler_data);
-
- return team_port_exists(dev) ? port : NULL;
+ return rcu_dereference(dev->rx_handler_data);
}
static struct team_port *team_port_get_rtnl(const struct net_device *dev)
if (dev->type == ARPHRD_ETHER && !is_valid_ether_addr(addr->sa_data))
return -EADDRNOTAVAIL;
memcpy(dev->dev_addr, addr->sa_data, dev->addr_len);
- rcu_read_lock();
- list_for_each_entry_rcu(port, &team->port_list, list)
+ mutex_lock(&team->lock);
+ list_for_each_entry(port, &team->port_list, list)
if (team->ops.port_change_dev_addr)
team->ops.port_change_dev_addr(team, port);
- rcu_read_unlock();
+ mutex_unlock(&team->lock);
return 0;
}
* Linksys USB200M
* Netgear FA120
* Sitecom LN-029
+ * Sitecom LN-028
* Intellinet USB 2.0 Ethernet
* ST Lab USB 2.0 Ethernet
* TrendNet TU2-ET100
memcpy(skb_tail_pointer(skb), &padbytes, sizeof(padbytes));
skb_put(skb, sizeof(padbytes));
}
+
+ usbnet_set_skb_tx_stats(skb, 1, 0);
return skb;
}
// Sitecom LN-031 "USB 2.0 10/100/1000 Ethernet adapter"
USB_DEVICE (0x0df6, 0x0056),
.driver_info = (unsigned long) &ax88178_info,
+}, {
+ // Sitecom LN-028 "USB 2.0 10/100/1000 Ethernet adapter"
+ USB_DEVICE (0x0df6, 0x061c),
+ .driver_info = (unsigned long) &ax88178_info,
}, {
// corega FEther USB2-TX
USB_DEVICE (0x07aa, 0x0017),
#define DELL_VENDOR_ID 0x413C
#define REALTEK_VENDOR_ID 0x0bda
#define SAMSUNG_VENDOR_ID 0x04e8
+#define LENOVO_VENDOR_ID 0x17ef
static const struct usb_device_id products[] = {
/* BLACKLIST !!
.driver_info = 0,
},
+/* Lenovo Thinkpad USB 3.0 Ethernet Adapters (based on Realtek RTL8153) */
+{
+ USB_DEVICE_AND_INTERFACE_INFO(LENOVO_VENDOR_ID, 0x7205, USB_CLASS_COMM,
+ USB_CDC_SUBCLASS_ETHERNET, USB_CDC_PROTO_NONE),
+ .driver_info = 0,
+},
+
/* WHITELIST!!!
*
* CDC Ether uses two interfaces, not necessarily consecutive.
/* return skb */
ctx->tx_curr_skb = NULL;
- dev->net->stats.tx_packets += ctx->tx_curr_frame_num;
/* keep private stats: framing overhead and number of NTBs */
ctx->tx_overhead += skb_out->len - ctx->tx_curr_frame_payload;
ctx->tx_ntbs++;
- /* usbnet has already counted all the framing overhead.
+ /* usbnet will count all the framing overhead by default.
* Adjust the stats so that the tx_bytes counter show real
* payload data instead.
*/
- dev->net->stats.tx_bytes -= skb_out->len - ctx->tx_curr_frame_payload;
+ usbnet_set_skb_tx_stats(skb_out, n,
+ ctx->tx_curr_frame_payload - skb_out->len);
return skb_out;
};
#define CMD_PACKET_SIZE 64
-/* first command after power on can take around 8 seconds */
-#define CMD_TIMEOUT 15000
+#define CMD_TIMEOUT 100
#define CMD_REPLY_RETRY 5
#define CX82310_MTU 1514
ret = usb_bulk_msg(udev, usb_sndbulkpipe(udev, CMD_EP), buf,
CMD_PACKET_SIZE, &actual_len, CMD_TIMEOUT);
if (ret < 0) {
- dev_err(&dev->udev->dev, "send command %#x: error %d\n",
- cmd, ret);
+ if (cmd != CMD_GET_LINK_STATUS)
+ dev_err(&dev->udev->dev, "send command %#x: error %d\n",
+ cmd, ret);
goto end;
}
buf, CMD_PACKET_SIZE, &actual_len,
CMD_TIMEOUT);
if (ret < 0) {
- dev_err(&dev->udev->dev,
- "reply receive error %d\n", ret);
+ if (cmd != CMD_GET_LINK_STATUS)
+ dev_err(&dev->udev->dev,
+ "reply receive error %d\n",
+ ret);
goto end;
}
if (actual_len > 0)
int ret;
char buf[15];
struct usb_device *udev = dev->udev;
+ u8 link[3];
+ int timeout = 50;
/* avoid ADSL modems - continue only if iProduct is "USB NET CARD" */
if (usb_string(udev, udev->descriptor.iProduct, buf, sizeof(buf)) > 0
if (!dev->partial_data)
return -ENOMEM;
+ /* wait for firmware to become ready (indicated by the link being up) */
+ while (--timeout) {
+ ret = cx82310_cmd(dev, CMD_GET_LINK_STATUS, true, NULL, 0,
+ link, sizeof(link));
+ /* the command can time out during boot - it's not an error */
+ if (!ret && link[0] == 1 && link[2] == 1)
+ break;
+ msleep(500);
+ };
+ if (!timeout) {
+ dev_err(&udev->dev, "firmware not ready in time\n");
+ return -ETIMEDOUT;
+ }
+
/* enable ethernet mode (?) */
ret = cx82310_cmd(dev, CMD_ETHERNET_MODE, true, "\x01", 1, NULL, 0);
if (ret) {
.tx_fixup = cx82310_tx_fixup,
};
+#define USB_DEVICE_CLASS(vend, prod, cl, sc, pr) \
+ .match_flags = USB_DEVICE_ID_MATCH_DEVICE | \
+ USB_DEVICE_ID_MATCH_DEV_INFO, \
+ .idVendor = (vend), \
+ .idProduct = (prod), \
+ .bDeviceClass = (cl), \
+ .bDeviceSubClass = (sc), \
+ .bDeviceProtocol = (pr)
+
static const struct usb_device_id products[] = {
{
- USB_DEVICE_AND_INTERFACE_INFO(0x0572, 0xcb01, 0xff, 0, 0),
+ USB_DEVICE_CLASS(0x0572, 0xcb01, 0xff, 0, 0),
.driver_info = (unsigned long) &cx82310_info
},
{ },
}
cprev = cnow;
}
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
remove_wait_queue(&tiocmget->waitq, &wait);
return ret;
}, {
USB_DEVICE(0x050d, 0x258a), /* Belkin F5U258/F5U279 (PL-25A1) */
.driver_info = (unsigned long) &prolific_info,
+}, {
+ USB_DEVICE(0x3923, 0x7825), /* National Instruments USB
+ * Host-to-Host Cable
+ */
+ .driver_info = (unsigned long) &prolific_info,
},
{ }, // END
/* Define these values to match your device */
#define VENDOR_ID_REALTEK 0x0bda
#define VENDOR_ID_SAMSUNG 0x04e8
+#define VENDOR_ID_LENOVO 0x17ef
#define MCU_TYPE_PLA 0x0100
#define MCU_TYPE_USB 0x0000
{REALTEK_USB_DEVICE(VENDOR_ID_REALTEK, 0x8152)},
{REALTEK_USB_DEVICE(VENDOR_ID_REALTEK, 0x8153)},
{REALTEK_USB_DEVICE(VENDOR_ID_SAMSUNG, 0xa101)},
+ {REALTEK_USB_DEVICE(VENDOR_ID_LENOVO, 0x7205)},
{}
};
skb_put(skb, sizeof(padbytes));
}
+ usbnet_set_skb_tx_stats(skb, 1, 0);
return skb;
}
struct usbnet *dev = entry->dev;
if (urb->status == 0) {
- if (!(dev->driver_info->flags & FLAG_MULTI_PACKET))
- dev->net->stats.tx_packets++;
+ dev->net->stats.tx_packets += entry->packets;
dev->net->stats.tx_bytes += entry->length;
} else {
dev->net->stats.tx_errors++;
} else
urb->transfer_flags |= URB_ZERO_PACKET;
}
- entry->length = urb->transfer_buffer_length = length;
+ urb->transfer_buffer_length = length;
+
+ if (info->flags & FLAG_MULTI_PACKET) {
+ /* Driver has set number of packets and a length delta.
+ * Calculate the complete length and ensure that it's
+ * positive.
+ */
+ entry->length += length;
+ if (WARN_ON_ONCE(entry->length <= 0))
+ entry->length = length;
+ } else {
+ usbnet_set_skb_tx_stats(skb, 1, length);
+ }
spin_lock_irqsave(&dev->txq.lock, flags);
retval = usb_autopm_get_interface_async(dev->intf);
{
int i;
- for (i = 0; i < vi->max_queue_pairs; i++)
+ for (i = 0; i < vi->max_queue_pairs; i++) {
+ napi_hash_del(&vi->rq[i].napi);
netif_napi_del(&vi->rq[i].napi);
+ }
kfree(vi->rq);
kfree(vi->sq);
cancel_delayed_work_sync(&vi->refill);
if (netif_running(vi->dev)) {
- for (i = 0; i < vi->max_queue_pairs; i++) {
+ for (i = 0; i < vi->max_queue_pairs; i++)
napi_disable(&vi->rq[i].napi);
- napi_hash_del(&vi->rq[i].napi);
- netif_napi_del(&vi->rq[i].napi);
- }
}
remove_vq_common(vi);
goto drop;
flags &= ~VXLAN_HF_RCO;
- vni &= VXLAN_VID_MASK;
+ vni &= VXLAN_VNI_MASK;
}
/* For backwards compatibility, only allow reserved fields to be
flags &= ~VXLAN_GBP_USED_BITS;
}
- if (flags || (vni & ~VXLAN_VID_MASK)) {
+ if (flags || vni & ~VXLAN_VNI_MASK) {
/* If there are any unprocessed flags remaining treat
* this as a malformed packet. This behavior diverges from
* VXLAN RFC (RFC7348) which stipulates that bits in reserved
spin_lock_irqsave(&cosa->lock, flags);
add_wait_queue(&chan->rxwaitq, &wait);
while (!chan->rx_status) {
- current->state = TASK_INTERRUPTIBLE;
+ set_current_state(TASK_INTERRUPTIBLE);
spin_unlock_irqrestore(&cosa->lock, flags);
schedule();
spin_lock_irqsave(&cosa->lock, flags);
if (signal_pending(current) && chan->rx_status == 0) {
chan->rx_status = 1;
remove_wait_queue(&chan->rxwaitq, &wait);
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
spin_unlock_irqrestore(&cosa->lock, flags);
mutex_unlock(&chan->rlock);
return -ERESTARTSYS;
}
}
remove_wait_queue(&chan->rxwaitq, &wait);
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
kbuf = chan->rxdata;
count = chan->rxsize;
spin_unlock_irqrestore(&cosa->lock, flags);
spin_lock_irqsave(&cosa->lock, flags);
add_wait_queue(&chan->txwaitq, &wait);
while (!chan->tx_status) {
- current->state = TASK_INTERRUPTIBLE;
+ set_current_state(TASK_INTERRUPTIBLE);
spin_unlock_irqrestore(&cosa->lock, flags);
schedule();
spin_lock_irqsave(&cosa->lock, flags);
if (signal_pending(current) && chan->tx_status == 0) {
chan->tx_status = 1;
remove_wait_queue(&chan->txwaitq, &wait);
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
chan->tx_status = 1;
spin_unlock_irqrestore(&cosa->lock, flags);
up(&chan->wsem);
}
}
remove_wait_queue(&chan->txwaitq, &wait);
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
up(&chan->wsem);
spin_unlock_irqrestore(&cosa->lock, flags);
kfree(kbuf);
struct ath_common *common = ath9k_hw_common(sc->sc_ah);
struct ath_vif *avp = (void *)vif->drv_priv;
struct ath_buf *bf = avp->av_bcbuf;
+ struct ath_beacon_config *cur_conf = &sc->cur_chan->beacon;
ath_dbg(common, CONFIG, "Removing interface at beacon slot: %d\n",
avp->av_bslot);
tasklet_disable(&sc->bcon_tasklet);
+ cur_conf->enable_beacon &= ~BIT(avp->av_bslot);
+
if (bf && bf->bf_mpdu) {
struct sk_buff *skb = bf->bf_mpdu;
dma_unmap_single(sc->dev, bf->bf_buf_addr,
}
if (sc->sc_ah->opmode == NL80211_IFTYPE_AP) {
- if ((vif->type != NL80211_IFTYPE_AP) ||
- (sc->nbcnvifs > 1)) {
+ if (vif->type != NL80211_IFTYPE_AP) {
ath_dbg(common, CONFIG,
"An AP interface is already present !\n");
return false;
* enabling/disabling SWBA.
*/
if (changed & BSS_CHANGED_BEACON_ENABLED) {
- if (!bss_conf->enable_beacon &&
- (sc->nbcnvifs <= 1)) {
- cur_conf->enable_beacon = false;
- } else if (bss_conf->enable_beacon) {
- cur_conf->enable_beacon = true;
- ath9k_cache_beacon_config(sc, ctx, bss_conf);
+ bool enabled = cur_conf->enable_beacon;
+
+ if (!bss_conf->enable_beacon) {
+ cur_conf->enable_beacon &= ~BIT(avp->av_bslot);
+ } else {
+ cur_conf->enable_beacon |= BIT(avp->av_bslot);
+ if (!enabled)
+ ath9k_cache_beacon_config(sc, ctx, bss_conf);
}
}
u16 dtim_period;
u16 bmiss_timeout;
u8 dtim_count;
- bool enable_beacon;
+ u8 enable_beacon;
bool ibss_creator;
u32 nexttbtt;
u32 intval;
ah->power_mode = ATH9K_PM_UNDEFINED;
ah->htc_reset_init = true;
- ah->tpc_enabled = true;
+ ah->tpc_enabled = false;
ah->ani_function = ATH9K_ANI_ALL;
if (!AR_SREV_9300_20_OR_LATER(ah))
case 0x432a: /* BCM4321 */
case 0x432d: /* BCM4322 */
case 0x4352: /* BCM43222 */
+ case 0x435a: /* BCM43228 */
case 0x4333: /* BCM4331 */
case 0x43a2: /* BCM4360 */
case 0x43b3: /* BCM4352 */
brcmf_feat_iovar_int_get(ifp, BRCMF_FEAT_MCHAN, "mchan");
if (drvr->bus_if->wowl_supported)
brcmf_feat_iovar_int_get(ifp, BRCMF_FEAT_WOWL, "wowl");
- brcmf_feat_iovar_int_set(ifp, BRCMF_FEAT_MBSS, "mbss", 0);
+ if (drvr->bus_if->chip != BRCM_CC_43362_CHIP_ID)
+ brcmf_feat_iovar_int_set(ifp, BRCMF_FEAT_MBSS, "mbss", 0);
/* set chip related quirks */
switch (drvr->bus_if->chip) {
void *dcmd_buf = NULL, *wr_pointer;
u16 msglen, maxmsglen = PAGE_SIZE - 0x100;
- brcmf_dbg(TRACE, "cmd %x set %d len %d\n", cmdhdr->cmd, cmdhdr->set,
- cmdhdr->len);
+ if (len < sizeof(*cmdhdr)) {
+ brcmf_err("vendor command too short: %d\n", len);
+ return -EINVAL;
+ }
vif = container_of(wdev, struct brcmf_cfg80211_vif, wdev);
ifp = vif->ifp;
- len -= sizeof(struct brcmf_vndr_dcmd_hdr);
+ brcmf_dbg(TRACE, "ifidx=%d, cmd=%d\n", ifp->ifidx, cmdhdr->cmd);
+
+ if (cmdhdr->offset > len) {
+ brcmf_err("bad buffer offset %d > %d\n", cmdhdr->offset, len);
+ return -EINVAL;
+ }
+
+ len -= cmdhdr->offset;
ret_len = cmdhdr->len;
if (ret_len > 0 || len > 0) {
if (len > BRCMF_DCMD_MAXLEN) {
unsigned long reload_jiffies;
int reload_count;
bool ucode_loaded;
- bool init_ucode_run; /* Don't run init uCode again */
u8 plcp_delta_threshold;
scd_queues &= ~(BIT(IWL_IPAN_CMD_QUEUE_NUM) |
BIT(IWL_DEFAULT_CMD_QUEUE_NUM));
- if (vif)
- scd_queues &= ~BIT(vif->hw_queue[IEEE80211_AC_VO]);
-
- IWL_DEBUG_TX_QUEUES(priv, "Flushing SCD queues: 0x%x\n", scd_queues);
- if (iwlagn_txfifo_flush(priv, scd_queues)) {
- IWL_ERR(priv, "flush request fail\n");
- goto done;
+ if (drop) {
+ IWL_DEBUG_TX_QUEUES(priv, "Flushing SCD queues: 0x%x\n",
+ scd_queues);
+ if (iwlagn_txfifo_flush(priv, scd_queues)) {
+ IWL_ERR(priv, "flush request fail\n");
+ goto done;
+ }
}
+
IWL_DEBUG_TX_QUEUES(priv, "wait transmit/flush all frames\n");
- iwl_trans_wait_tx_queue_empty(priv->trans, 0xffffffff);
+ iwl_trans_wait_tx_queue_empty(priv->trans, scd_queues);
done:
mutex_unlock(&priv->mutex);
IWL_DEBUG_MAC80211(priv, "leave\n");
if (!priv->fw->img[IWL_UCODE_INIT].sec[0].len)
return 0;
- if (priv->init_ucode_run)
- return 0;
-
iwl_init_notification_wait(&priv->notif_wait, &calib_wait,
calib_complete, ARRAY_SIZE(calib_complete),
iwlagn_wait_calib, priv);
*/
ret = iwl_wait_notification(&priv->notif_wait, &calib_wait,
UCODE_CALIB_TIMEOUT);
- if (!ret)
- priv->init_ucode_run = true;
goto out;
.nvm_calib_ver = EEPROM_1000_TX_POWER_VERSION, \
.base_params = &iwl1000_base_params, \
.eeprom_params = &iwl1000_eeprom_params, \
- .led_mode = IWL_LED_BLINK
+ .led_mode = IWL_LED_BLINK, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl1000_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 1000 BGN",
.base_params = &iwl1000_base_params, \
.eeprom_params = &iwl1000_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
- .rx_with_siso_diversity = true
+ .rx_with_siso_diversity = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl100_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 100 BGN",
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
.base_params = &iwl2000_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
- .led_mode = IWL_LED_RF_STATE
+ .led_mode = IWL_LED_RF_STATE, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
+
const struct iwl_cfg iwl2000_2bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 2200 BGN",
.nvm_calib_ver = EEPROM_2000_TX_POWER_VERSION, \
.base_params = &iwl2030_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
- .led_mode = IWL_LED_RF_STATE
+ .led_mode = IWL_LED_RF_STATE, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl2030_2bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 2230 BGN",
.base_params = &iwl2000_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
- .rx_with_siso_diversity = true
+ .rx_with_siso_diversity = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl105_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 105 BGN",
.base_params = &iwl2030_base_params, \
.eeprom_params = &iwl20x0_eeprom_params, \
.led_mode = IWL_LED_RF_STATE, \
- .rx_with_siso_diversity = true
+ .rx_with_siso_diversity = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl135_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N 135 BGN",
.nvm_calib_ver = EEPROM_5000_TX_POWER_VERSION, \
.base_params = &iwl5000_base_params, \
.eeprom_params = &iwl5000_eeprom_params, \
- .led_mode = IWL_LED_BLINK
+ .led_mode = IWL_LED_BLINK, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl5300_agn_cfg = {
.name = "Intel(R) Ultimate N WiFi Link 5300 AGN",
.base_params = &iwl5000_base_params, \
.eeprom_params = &iwl5000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
- .internal_wimax_coex = true
+ .internal_wimax_coex = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl5150_agn_cfg = {
.name = "Intel(R) WiMAX/WiFi Link 5150 AGN",
.nvm_calib_ver = EEPROM_6005_TX_POWER_VERSION, \
.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
- .led_mode = IWL_LED_RF_STATE
+ .led_mode = IWL_LED_RF_STATE, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6005_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6205 AGN",
.nvm_calib_ver = EEPROM_6030_TX_POWER_VERSION, \
.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
- .led_mode = IWL_LED_RF_STATE
+ .led_mode = IWL_LED_RF_STATE, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6030_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6230 AGN",
.nvm_calib_ver = EEPROM_6030_TX_POWER_VERSION, \
.base_params = &iwl6000_g2_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
- .led_mode = IWL_LED_RF_STATE
+ .led_mode = IWL_LED_RF_STATE, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6035_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6235 AGN",
.nvm_calib_ver = EEPROM_6000_TX_POWER_VERSION, \
.base_params = &iwl6000_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
- .led_mode = IWL_LED_BLINK
+ .led_mode = IWL_LED_BLINK, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6000i_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N 6200 AGN",
.base_params = &iwl6050_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
- .internal_wimax_coex = true
+ .internal_wimax_coex = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6050_2agn_cfg = {
.name = "Intel(R) Centrino(R) Advanced-N + WiMAX 6250 AGN",
.base_params = &iwl6050_base_params, \
.eeprom_params = &iwl6000_eeprom_params, \
.led_mode = IWL_LED_BLINK, \
- .internal_wimax_coex = true
+ .internal_wimax_coex = true, \
+ .max_ht_ampdu_exponent = IEEE80211_HT_MAX_AMPDU_64K
const struct iwl_cfg iwl6150_bgn_cfg = {
.name = "Intel(R) Centrino(R) Wireless-N + WiMAX 6150 BGN",
op->name, err);
#endif
}
+ kfree(pieces);
return;
try_again:
if (!vif->bss_conf.assoc)
smps_mode = IEEE80211_SMPS_AUTOMATIC;
- if (IWL_COEX_IS_RRC_ON(mvm->last_bt_notif.ttc_rrc_status,
+ if (mvmvif->phy_ctxt &&
+ IWL_COEX_IS_RRC_ON(mvm->last_bt_notif.ttc_rrc_status,
mvmvif->phy_ctxt->id))
smps_mode = IEEE80211_SMPS_AUTOMATIC;
if (!vif->bss_conf.assoc)
smps_mode = IEEE80211_SMPS_AUTOMATIC;
- if (data->notif->rrc_enabled & BIT(mvmvif->phy_ctxt->id))
+ if (mvmvif->phy_ctxt &&
+ data->notif->rrc_enabled & BIT(mvmvif->phy_ctxt->id))
smps_mode = IEEE80211_SMPS_AUTOMATIC;
IWL_DEBUG_COEX(data->mvm,
hw->wiphy->bands[IEEE80211_BAND_5GHZ] =
&mvm->nvm_data->bands[IEEE80211_BAND_5GHZ];
- if (mvm->fw->ucode_capa.capa[0] & IWL_UCODE_TLV_CAPA_BEAMFORMER)
+ if ((mvm->fw->ucode_capa.capa[0] &
+ IWL_UCODE_TLV_CAPA_BEAMFORMER) &&
+ (mvm->fw->ucode_capa.api[0] &
+ IWL_UCODE_TLV_API_LQ_SS_PARAMS))
hw->wiphy->bands[IEEE80211_BAND_5GHZ]->vht_cap.cap |=
IEEE80211_VHT_CAP_SU_BEAMFORMER_CAPABLE;
}
mutex_lock(&mvm->mutex);
- iwl_mvm_cancel_scan(mvm);
+ /* Due to a race condition, it's possible that mac80211 asks
+ * us to stop a hw_scan when it's already stopped. This can
+ * happen, for instance, if we stopped the scan ourselves,
+ * called ieee80211_scan_completed() and the userspace called
+ * cancel scan scan before ieee80211_scan_work() could run.
+ * To handle that, simply return if the scan is not running.
+ */
+ /* FIXME: for now, we ignore this race for UMAC scans, since
+ * they don't set the scan_status.
+ */
+ if ((mvm->scan_status == IWL_MVM_SCAN_OS) ||
+ (mvm->fw->ucode_capa.capa[0] & IWL_UCODE_TLV_CAPA_UMAC_SCAN))
+ iwl_mvm_cancel_scan(mvm);
mutex_unlock(&mvm->mutex);
}
int ret;
mutex_lock(&mvm->mutex);
+
+ /* Due to a race condition, it's possible that mac80211 asks
+ * us to stop a sched_scan when it's already stopped. This
+ * can happen, for instance, if we stopped the scan ourselves,
+ * called ieee80211_sched_scan_stopped() and the userspace called
+ * stop sched scan scan before ieee80211_sched_scan_stopped_work()
+ * could run. To handle this, simply return if the scan is
+ * not running.
+ */
+ /* FIXME: for now, we ignore this race for UMAC scans, since
+ * they don't set the scan_status.
+ */
+ if (mvm->scan_status != IWL_MVM_SCAN_SCHED &&
+ !(mvm->fw->ucode_capa.capa[0] & IWL_UCODE_TLV_CAPA_UMAC_SCAN)) {
+ mutex_unlock(&mvm->mutex);
+ return 0;
+ }
+
ret = iwl_mvm_scan_offload_stop(mvm, false);
mutex_unlock(&mvm->mutex);
iwl_mvm_wait_for_async_handlers(mvm);
return ret;
-
}
static int iwl_mvm_mac_set_key(struct ieee80211_hw *hw,
#define MAX_NEXT_COLUMNS 7
#define MAX_COLUMN_CHECKS 3
+struct rs_tx_column;
+
typedef bool (*allow_column_func_t) (struct iwl_mvm *mvm,
struct ieee80211_sta *sta,
- struct iwl_scale_tbl_info *tbl);
+ struct iwl_scale_tbl_info *tbl,
+ const struct rs_tx_column *next_col);
struct rs_tx_column {
enum rs_column_mode mode;
};
static bool rs_ant_allow(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct iwl_scale_tbl_info *tbl)
+ struct iwl_scale_tbl_info *tbl,
+ const struct rs_tx_column *next_col)
{
- return iwl_mvm_bt_coex_is_ant_avail(mvm, tbl->rate.ant);
+ return iwl_mvm_bt_coex_is_ant_avail(mvm, next_col->ant);
}
static bool rs_mimo_allow(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct iwl_scale_tbl_info *tbl)
+ struct iwl_scale_tbl_info *tbl,
+ const struct rs_tx_column *next_col)
{
if (!sta->ht_cap.ht_supported)
return false;
}
static bool rs_siso_allow(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct iwl_scale_tbl_info *tbl)
+ struct iwl_scale_tbl_info *tbl,
+ const struct rs_tx_column *next_col)
{
if (!sta->ht_cap.ht_supported)
return false;
}
static bool rs_sgi_allow(struct iwl_mvm *mvm, struct ieee80211_sta *sta,
- struct iwl_scale_tbl_info *tbl)
+ struct iwl_scale_tbl_info *tbl,
+ const struct rs_tx_column *next_col)
{
struct rs_rate *rate = &tbl->rate;
struct ieee80211_sta_ht_cap *ht_cap = &sta->ht_cap;
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
+ if (!iwl_mvm_sta_from_mac80211(sta)->vif)
+ return;
+
if (!ieee80211_is_data(hdr->frame_control) ||
info->flags & IEEE80211_TX_CTL_NO_ACK)
return;
for (j = 0; j < MAX_COLUMN_CHECKS; j++) {
allow_func = next_col->checks[j];
- if (allow_func && !allow_func(mvm, sta, tbl))
+ if (allow_func && !allow_func(mvm, sta, tbl, next_col))
break;
}
struct ieee80211_tx_info *info = IEEE80211_SKB_CB(skb);
struct iwl_lq_sta *lq_sta = mvm_sta;
+ if (sta && !iwl_mvm_sta_from_mac80211(sta)->vif) {
+ /* if vif isn't initialized mvm doesn't know about
+ * this station, so don't do anything with the it
+ */
+ sta = NULL;
+ mvm_sta = NULL;
+ }
+
/* TODO: handle rate_idx_mask and rate_idx_mcs_mask */
/* Treat uninitialized rate scaling data same as non-existing. */
(struct iwl_op_mode *)mvm_r;
struct iwl_mvm *mvm = IWL_OP_MODE_GET_MVM(op_mode);
+ if (!iwl_mvm_sta_from_mac80211(sta)->vif)
+ return;
+
/* Stop any ongoing aggregations as rs starts off assuming no agg */
for (tid = 0; tid < IWL_MAX_TID_COUNT; tid++)
ieee80211_stop_tx_ba_session(sta, tid);
MVM_DEBUGFS_READ_WRITE_FILE_OPS(ss_force, 32);
-static void rs_add_debugfs(void *mvm, void *mvm_sta, struct dentry *dir)
+static void rs_add_debugfs(void *mvm, void *priv_sta, struct dentry *dir)
{
- struct iwl_lq_sta *lq_sta = mvm_sta;
+ struct iwl_lq_sta *lq_sta = priv_sta;
+ struct iwl_mvm_sta *mvmsta;
+
+ mvmsta = container_of(lq_sta, struct iwl_mvm_sta, lq_sta);
+
+ if (!mvmsta->vif)
+ return;
debugfs_create_file("rate_scale_table", S_IRUSR | S_IWUSR, dir,
lq_sta, &rs_sta_dbgfs_scale_table_ops);
if (mvm->scan_status == IWL_MVM_SCAN_NONE)
return 0;
- if (iwl_mvm_is_radio_killed(mvm))
+ if (iwl_mvm_is_radio_killed(mvm)) {
+ ret = 0;
goto out;
+ }
if (mvm->scan_status != IWL_MVM_SCAN_SCHED &&
(!(mvm->fw->ucode_capa.api[0] & IWL_UCODE_TLV_API_LMAC_SCAN) ||
IWL_DEBUG_SCAN(mvm, "Send stop %sscan failed %d\n",
sched ? "offloaded " : "", ret);
iwl_remove_notification(&mvm->notif_wait, &wait_scan_done);
- return ret;
+ goto out;
}
IWL_DEBUG_SCAN(mvm, "Successfully sent stop %sscan\n",
sched ? "offloaded " : "");
ret = iwl_wait_notification(&mvm->notif_wait, &wait_scan_done, 1 * HZ);
- if (ret)
- return ret;
-
+out:
/*
* Clear the scan status so the next scan requests will succeed. This
* also ensures the Rx handler doesn't do anything, as the scan was
if (mvm->scan_status == IWL_MVM_SCAN_OS)
iwl_mvm_unref(mvm, IWL_MVM_REF_SCAN);
-out:
mvm->scan_status = IWL_MVM_SCAN_NONE;
if (notify) {
ieee80211_scan_completed(mvm->hw, true);
}
- return 0;
+ return ret;
}
static void iwl_mvm_unified_scan_fill_tx_cmd(struct iwl_mvm *mvm,
struct iwl_time_event_notif *notif)
{
if (!le32_to_cpu(notif->status)) {
+ if (te_data->vif->type == NL80211_IFTYPE_STATION)
+ ieee80211_connection_loss(te_data->vif);
IWL_DEBUG_TE(mvm, "CSA time event failed to start\n");
iwl_mvm_te_clear_data(mvm, te_data);
return;
* request
*/
list_for_each_entry(te_data, &mvm->time_event_list, list) {
- if (te_data->vif->type == NL80211_IFTYPE_P2P_DEVICE &&
- te_data->running) {
+ if (te_data->vif->type == NL80211_IFTYPE_P2P_DEVICE) {
mvmvif = iwl_mvm_vif_from_mac80211(te_data->vif);
is_p2p = true;
goto remove_te;
* request
*/
list_for_each_entry(te_data, &mvm->aux_roc_te_list, list) {
- if (te_data->running) {
- mvmvif = iwl_mvm_vif_from_mac80211(te_data->vif);
- goto remove_te;
- }
+ mvmvif = iwl_mvm_vif_from_mac80211(te_data->vif);
+ goto remove_te;
}
remove_te:
mvmsta = iwl_mvm_sta_from_mac80211(sta);
tid_data = &mvmsta->tid_data[tid];
- if (WARN_ONCE(tid_data->txq_id != scd_flow, "Q %d, tid %d, flow %d",
- tid_data->txq_id, tid, scd_flow)) {
+ if (tid_data->txq_id != scd_flow) {
+ IWL_ERR(mvm,
+ "invalid BA notification: Q %d, tid %d, flow %d\n",
+ tid_data->txq_id, tid, scd_flow);
rcu_read_unlock();
return 0;
}
/* 3165 Series */
{IWL_PCI_DEVICE(0x3165, 0x4010, iwl3165_2ac_cfg)},
{IWL_PCI_DEVICE(0x3165, 0x4012, iwl3165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x3165, 0x4110, iwl3165_2ac_cfg)},
- {IWL_PCI_DEVICE(0x3165, 0x4210, iwl3165_2ac_cfg)},
{IWL_PCI_DEVICE(0x3165, 0x4410, iwl3165_2ac_cfg)},
{IWL_PCI_DEVICE(0x3165, 0x4510, iwl3165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x3165, 0x4110, iwl3165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x3166, 0x4310, iwl3165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x3166, 0x4210, iwl3165_2ac_cfg)},
+ {IWL_PCI_DEVICE(0x3165, 0x8010, iwl3165_2ac_cfg)},
/* 7265 Series */
{IWL_PCI_DEVICE(0x095A, 0x5010, iwl7265_2ac_cfg)},
goto nla_put_failure;
genlmsg_end(skb, msg_head);
- genlmsg_unicast(&init_net, skb, dst_portid);
+ if (genlmsg_unicast(&init_net, skb, dst_portid))
+ goto err_free_txskb;
/* Enqueue the packet */
skb_queue_tail(&data->pending, my_skb);
return;
nla_put_failure:
+ nlmsg_free(skb);
+err_free_txskb:
printk(KERN_DEBUG "mac80211_hwsim: error occurred in %s\n", __func__);
ieee80211_free_txskb(hw, my_skb);
data->tx_failed++;
}
return true;
- } else if (0x86DD == ether_type) {
- return true;
+ } else if (ETH_P_IPV6 == ether_type) {
+ /* TODO: Handle any IPv6 cases that need special handling.
+ * For now, always return false
+ */
+ goto end;
}
end:
/*This is for new trx flow*/
struct rtl_tx_buffer_desc *pbuffer_desc = NULL;
u8 temp_one = 1;
+ u8 *entry;
memset(&tcb_desc, 0, sizeof(struct rtl_tcb_desc));
ring = &rtlpci->tx_ring[BEACON_QUEUE];
pskb = __skb_dequeue(&ring->queue);
- if (pskb)
+ if (rtlpriv->use_new_trx_flow)
+ entry = (u8 *)(&ring->buffer_desc[ring->idx]);
+ else
+ entry = (u8 *)(&ring->desc[ring->idx]);
+ if (pskb) {
+ pci_unmap_single(rtlpci->pdev,
+ rtlpriv->cfg->ops->get_desc(
+ (u8 *)entry, true, HW_DESC_TXBUFF_ADDR),
+ pskb->len, PCI_DMA_TODEVICE);
kfree_skb(pskb);
+ }
/*NB: the beacon data buffer must be 32-bit aligned. */
pskb = ieee80211_beacon_get(hw, mac->vif);
unsigned int num_queues = vif->num_queues;
int i;
unsigned int queue_index;
- struct xenvif_stats *vif_stats;
for (i = 0; i < ARRAY_SIZE(xenvif_stats); i++) {
unsigned long accum = 0;
for (queue_index = 0; queue_index < num_queues; ++queue_index) {
- vif_stats = &vif->queues[queue_index].stats;
+ void *vif_stats = &vif->queues[queue_index].stats;
accum += *(unsigned long *)(vif_stats + xenvif_stats[i].offset);
}
data[i] = accum;
static void make_tx_response(struct xenvif_queue *queue,
struct xen_netif_tx_request *txp,
s8 st);
+static void push_tx_responses(struct xenvif_queue *queue);
static inline int tx_work_todo(struct xenvif_queue *queue);
do {
spin_lock_irqsave(&queue->response_lock, flags);
make_tx_response(queue, txp, XEN_NETIF_RSP_ERROR);
+ push_tx_responses(queue);
spin_unlock_irqrestore(&queue->response_lock, flags);
if (cons == end)
break;
{
unsigned int offset = skb_headlen(skb);
skb_frag_t frags[MAX_SKB_FRAGS];
- int i;
+ int i, f;
struct ubuf_info *uarg;
struct sk_buff *nskb = skb_shinfo(skb)->frag_list;
frags[i].page_offset = 0;
skb_frag_size_set(&frags[i], len);
}
- /* swap out with old one */
- memcpy(skb_shinfo(skb)->frags,
- frags,
- i * sizeof(skb_frag_t));
- skb_shinfo(skb)->nr_frags = i;
- skb->truesize += i * PAGE_SIZE;
- /* remove traces of mapped pages and frag_list */
+ /* Copied all the bits from the frag list -- free it. */
skb_frag_list_init(skb);
+ xenvif_skb_zerocopy_prepare(queue, nskb);
+ kfree_skb(nskb);
+
+ /* Release all the original (foreign) frags. */
+ for (f = 0; f < skb_shinfo(skb)->nr_frags; f++)
+ skb_frag_unref(skb, f);
uarg = skb_shinfo(skb)->destructor_arg;
/* increase inflight counter to offset decrement in callback */
atomic_inc(&queue->inflight_packets);
uarg->callback(uarg, true);
skb_shinfo(skb)->destructor_arg = NULL;
- xenvif_skb_zerocopy_prepare(queue, nskb);
- kfree_skb(nskb);
+ /* Fill the skb with the new (local) frags. */
+ memcpy(skb_shinfo(skb)->frags, frags, i * sizeof(skb_frag_t));
+ skb_shinfo(skb)->nr_frags = i;
+ skb->truesize += i * PAGE_SIZE;
return 0;
}
unsigned long flags;
pending_tx_info = &queue->pending_tx_info[pending_idx];
+
spin_lock_irqsave(&queue->response_lock, flags);
+
make_tx_response(queue, &pending_tx_info->req, status);
- index = pending_index(queue->pending_prod);
+
+ /* Release the pending index before pusing the Tx response so
+ * its available before a new Tx request is pushed by the
+ * frontend.
+ */
+ index = pending_index(queue->pending_prod++);
queue->pending_ring[index] = pending_idx;
- /* TX shouldn't use the index before we give it back here */
- mb();
- queue->pending_prod++;
+
+ push_tx_responses(queue);
+
spin_unlock_irqrestore(&queue->response_lock, flags);
}
{
RING_IDX i = queue->tx.rsp_prod_pvt;
struct xen_netif_tx_response *resp;
- int notify;
resp = RING_GET_RESPONSE(&queue->tx, i);
resp->id = txp->id;
RING_GET_RESPONSE(&queue->tx, ++i)->status = XEN_NETIF_RSP_NULL;
queue->tx.rsp_prod_pvt = ++i;
+}
+
+static void push_tx_responses(struct xenvif_queue *queue)
+{
+ int notify;
+
RING_PUSH_RESPONSES_AND_CHECK_NOTIFY(&queue->tx, notify);
if (notify)
notify_remote_via_irq(queue->tx_irq);
static int xennet_change_mtu(struct net_device *dev, int mtu)
{
- int max = xennet_can_sg(dev) ?
- XEN_NETIF_MAX_TX_SIZE - MAX_TCP_HEADER : ETH_DATA_LEN;
+ int max = xennet_can_sg(dev) ? XEN_NETIF_MAX_TX_SIZE : ETH_DATA_LEN;
if (mtu > max)
return -EINVAL;
netdev->ethtool_ops = &xennet_ethtool_ops;
SET_NETDEV_DEV(netdev, &dev->dev);
- netif_set_gso_max_size(netdev, XEN_NETIF_MAX_TX_SIZE - MAX_TCP_HEADER);
-
np->netdev = netdev;
netif_carrier_off(netdev);
bool
config OF_OVERLAY
- bool
- depends on OF
+ bool "Device Tree overlays"
select OF_DYNAMIC
select OF_RESOLVE
return NULL;
}
-static int of_empty_ranges_quirk(void)
+static int of_empty_ranges_quirk(struct device_node *np)
{
if (IS_ENABLED(CONFIG_PPC)) {
- /* To save cycles, we cache the result */
+ /* To save cycles, we cache the result for global "Mac" setting */
static int quirk_state = -1;
+ /* PA-SEMI sdc DT bug */
+ if (of_device_is_compatible(np, "1682m-sdc"))
+ return true;
+
+ /* Make quirk cached */
if (quirk_state < 0)
quirk_state =
of_machine_is_compatible("Power Macintosh") ||
* This code is only enabled on powerpc. --gcl
*/
ranges = of_get_property(parent, rprop, &rlen);
- if (ranges == NULL && !of_empty_ranges_quirk()) {
+ if (ranges == NULL && !of_empty_ranges_quirk(parent)) {
pr_debug("OF: no ranges; cannot translate\n");
return 1;
}
const char *path)
{
struct device_node *child;
- int len = strchrnul(path, '/') - path;
- int term;
+ int len;
+ len = strcspn(path, "/:");
if (!len)
return NULL;
- term = strchrnul(path, ':') - path;
- if (term < len)
- len = term;
-
__for_each_child_of_node(parent, child) {
const char *name = strrchr(child->full_name, '/');
if (WARN(!name, "malformed device_node %s\n", child->full_name))
/* The path could begin with an alias */
if (*path != '/') {
- char *p = strchrnul(path, '/');
- int len = separator ? separator - path : p - path;
+ int len;
+ const char *p = separator;
+
+ if (!p)
+ p = strchrnul(path, '/');
+ len = p - path;
/* of_aliases must not be NULL */
if (!of_aliases)
path++; /* Increment past '/' delimiter */
np = __of_find_node_by_path(np, path);
path = strchrnul(path, '/');
+ if (separator && separator < path)
+ break;
}
raw_spin_unlock_irqrestore(&devtree_lock, flags);
return np;
#include <linux/kernel.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/of_address.h>
+#include <linux/of_iommu.h>
+#include <linux/dma-mapping.h>
#include <linux/init.h>
#include <linux/module.h>
#include <linux/mod_devicetable.h>
return device_add(&ofdev->dev);
}
+/**
+ * of_dma_configure - Setup DMA configuration
+ * @dev: Device to apply DMA configuration
+ * @np: Pointer to OF node having DMA configuration
+ *
+ * Try to get devices's DMA configuration from DT and update it
+ * accordingly.
+ *
+ * If platform code needs to use its own special DMA configuration, it
+ * can use a platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE events
+ * to fix up DMA configuration.
+ */
+void of_dma_configure(struct device *dev, struct device_node *np)
+{
+ u64 dma_addr, paddr, size;
+ int ret;
+ bool coherent;
+ unsigned long offset;
+ struct iommu_ops *iommu;
+
+ /*
+ * Set default coherent_dma_mask to 32 bit. Drivers are expected to
+ * setup the correct supported mask.
+ */
+ if (!dev->coherent_dma_mask)
+ dev->coherent_dma_mask = DMA_BIT_MASK(32);
+
+ /*
+ * Set it to coherent_dma_mask by default if the architecture
+ * code has not set it.
+ */
+ if (!dev->dma_mask)
+ dev->dma_mask = &dev->coherent_dma_mask;
+
+ ret = of_dma_get_range(np, &dma_addr, &paddr, &size);
+ if (ret < 0) {
+ dma_addr = offset = 0;
+ size = dev->coherent_dma_mask + 1;
+ } else {
+ offset = PFN_DOWN(paddr - dma_addr);
+
+ /*
+ * Add a work around to treat the size as mask + 1 in case
+ * it is defined in DT as a mask.
+ */
+ if (size & 1) {
+ dev_warn(dev, "Invalid size 0x%llx for dma-range\n",
+ size);
+ size = size + 1;
+ }
+
+ if (!size) {
+ dev_err(dev, "Adjusted size 0x%llx invalid\n", size);
+ return;
+ }
+ dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
+ }
+
+ dev->dma_pfn_offset = offset;
+
+ /*
+ * Limit coherent and dma mask based on size and default mask
+ * set by the driver.
+ */
+ dev->coherent_dma_mask = min(dev->coherent_dma_mask,
+ DMA_BIT_MASK(ilog2(dma_addr + size)));
+ *dev->dma_mask = min((*dev->dma_mask),
+ DMA_BIT_MASK(ilog2(dma_addr + size)));
+
+ coherent = of_dma_is_coherent(np);
+ dev_dbg(dev, "device is%sdma coherent\n",
+ coherent ? " " : " not ");
+
+ iommu = of_iommu_configure(dev, np);
+ dev_dbg(dev, "device is%sbehind an iommu\n",
+ iommu ? " " : " not ");
+
+ arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
+}
+EXPORT_SYMBOL_GPL(of_dma_configure);
+
int of_device_register(struct platform_device *pdev)
{
device_initialize(&pdev->dev);
struct device_node *p;
const __be32 *intspec, *tmp, *addr;
u32 intsize, intlen;
- int i, res = -EINVAL;
+ int i, res;
pr_debug("of_irq_parse_one: dev=%s, index=%d\n", of_node_full_name(device), index);
/* Get size of interrupt specifier */
tmp = of_get_property(p, "#interrupt-cells", NULL);
- if (tmp == NULL)
+ if (tmp == NULL) {
+ res = -EINVAL;
goto out;
+ }
intsize = be32_to_cpu(*tmp);
pr_debug(" intsize=%d intlen=%d\n", intsize, intlen);
/* Check index */
- if ((index + 1) * intsize > intlen)
+ if ((index + 1) * intsize > intlen) {
+ res = -EINVAL;
goto out;
+ }
/* Copy intspec into irq structure */
intspec += index * intsize;
#include <linux/export.h>
#include <linux/of.h>
#include <linux/of_address.h>
+#include <linux/of_device.h>
#include <linux/of_pci.h>
#include <linux/slab.h>
}
EXPORT_SYMBOL_GPL(of_get_pci_domain_nr);
+/**
+ * of_pci_dma_configure - Setup DMA configuration
+ * @dev: ptr to pci_dev struct of the PCI device
+ *
+ * Function to update PCI devices's DMA configuration using the same
+ * info from the OF node of host bridge's parent (if any).
+ */
+void of_pci_dma_configure(struct pci_dev *pci_dev)
+{
+ struct device *dev = &pci_dev->dev;
+ struct device *bridge = pci_get_host_bridge_device(pci_dev);
+
+ if (!bridge->parent)
+ return;
+
+ of_dma_configure(dev, bridge->parent->of_node);
+ pci_put_host_bridge_device(bridge);
+}
+EXPORT_SYMBOL_GPL(of_pci_dma_configure);
+
#if defined(CONFIG_OF_ADDRESS)
/**
* of_pci_get_host_bridge_resources - Parse PCI host bridge resources from DT
#include <linux/string.h>
#include <linux/slab.h>
#include <linux/err.h>
+#include <linux/idr.h>
#include "of_private.h"
struct device_node *target, struct device_node *child)
{
const char *cname;
- struct device_node *tchild, *grandchild;
+ struct device_node *tchild;
int ret = 0;
cname = kbasename(child->full_name);
#include <linux/slab.h>
#include <linux/of_address.h>
#include <linux/of_device.h>
-#include <linux/of_iommu.h>
#include <linux/of_irq.h>
#include <linux/of_platform.h>
#include <linux/platform_device.h>
}
EXPORT_SYMBOL(of_device_alloc);
-/**
- * of_dma_configure - Setup DMA configuration
- * @dev: Device to apply DMA configuration
- *
- * Try to get devices's DMA configuration from DT and update it
- * accordingly.
- *
- * In case if platform code need to use own special DMA configuration,it
- * can use Platform bus notifier and handle BUS_NOTIFY_ADD_DEVICE event
- * to fix up DMA configuration.
- */
-static void of_dma_configure(struct device *dev)
-{
- u64 dma_addr, paddr, size;
- int ret;
- bool coherent;
- unsigned long offset;
- struct iommu_ops *iommu;
-
- /*
- * Set default dma-mask to 32 bit. Drivers are expected to setup
- * the correct supported dma_mask.
- */
- dev->coherent_dma_mask = DMA_BIT_MASK(32);
-
- /*
- * Set it to coherent_dma_mask by default if the architecture
- * code has not set it.
- */
- if (!dev->dma_mask)
- dev->dma_mask = &dev->coherent_dma_mask;
-
- ret = of_dma_get_range(dev->of_node, &dma_addr, &paddr, &size);
- if (ret < 0) {
- dma_addr = offset = 0;
- size = dev->coherent_dma_mask;
- } else {
- offset = PFN_DOWN(paddr - dma_addr);
- dev_dbg(dev, "dma_pfn_offset(%#08lx)\n", offset);
- }
- dev->dma_pfn_offset = offset;
-
- coherent = of_dma_is_coherent(dev->of_node);
- dev_dbg(dev, "device is%sdma coherent\n",
- coherent ? " " : " not ");
-
- iommu = of_iommu_configure(dev);
- dev_dbg(dev, "device is%sbehind an iommu\n",
- iommu ? " " : " not ");
-
- arch_setup_dma_ops(dev, dma_addr, size, iommu, coherent);
-}
-
static void of_dma_deconfigure(struct device *dev)
{
arch_teardown_dma_ops(dev);
dev->dev.bus = &platform_bus_type;
dev->dev.platform_data = platform_data;
- of_dma_configure(&dev->dev);
+ of_dma_configure(&dev->dev, dev->dev.of_node);
if (of_device_add(dev) != 0) {
of_dma_deconfigure(&dev->dev);
dev_set_name(&dev->dev, "%s", bus_id);
else
of_device_make_bus_id(&dev->dev);
- of_dma_configure(&dev->dev);
+ of_dma_configure(&dev->dev, dev->dev.of_node);
/* Allow the HW Peripheral ID to be overridden */
prop = of_get_property(node, "arm,primecell-periphid", NULL);
"option path test failed\n");
of_node_put(np);
+ np = of_find_node_opts_by_path("/testcase-data:test/option", &options);
+ selftest(np && !strcmp("test/option", options),
+ "option path test, subcase #1 failed\n");
+ of_node_put(np);
+
+ np = of_find_node_opts_by_path("/testcase-data/testcase-device1:test/option", &options);
+ selftest(np && !strcmp("test/option", options),
+ "option path test, subcase #2 failed\n");
+ of_node_put(np);
+
np = of_find_node_opts_by_path("/testcase-data:testoption", NULL);
selftest(np, "NULL option path test failed\n");
of_node_put(np);
"option alias path test failed\n");
of_node_put(np);
+ np = of_find_node_opts_by_path("testcase-alias:test/alias/option",
+ &options);
+ selftest(np && !strcmp("test/alias/option", options),
+ "option alias path test, subcase #1 failed\n");
+ of_node_put(np);
+
np = of_find_node_opts_by_path("testcase-alias:testaliasoption", NULL);
selftest(np, "NULL option alias path test failed\n");
of_node_put(np);
rc = of_property_match_string(np, "phandle-list-names", "first");
selftest(rc == 0, "first expected:0 got:%i\n", rc);
rc = of_property_match_string(np, "phandle-list-names", "second");
- selftest(rc == 1, "second expected:0 got:%i\n", rc);
+ selftest(rc == 1, "second expected:1 got:%i\n", rc);
rc = of_property_match_string(np, "phandle-list-names", "third");
- selftest(rc == 2, "third expected:0 got:%i\n", rc);
+ selftest(rc == 2, "third expected:2 got:%i\n", rc);
rc = of_property_match_string(np, "phandle-list-names", "fourth");
selftest(rc == -ENODATA, "unmatched string; rc=%i\n", rc);
rc = of_property_match_string(np, "missing-property", "blah");
struct device_node *n1, *n2, *n21, *nremove, *parent, *np;
struct of_changeset chgset;
- of_changeset_init(&chgset);
n1 = __of_node_dup(NULL, "/testcase-data/changeset/n1");
selftest(n1, "testcase setup failure\n");
n2 = __of_node_dup(NULL, "/testcase-data/changeset/n2");
return pdev != NULL;
}
-#if IS_ENABLED(CONFIG_I2C)
+#if IS_BUILTIN(CONFIG_I2C)
/* get the i2c client device instantiated at the path */
static struct i2c_client *of_path_to_i2c_client(const char *path)
return;
}
-#if IS_ENABLED(CONFIG_I2C) && IS_ENABLED(CONFIG_OF_OVERLAY)
+#if IS_BUILTIN(CONFIG_I2C) && IS_ENABLED(CONFIG_OF_OVERLAY)
struct selftest_i2c_bus_data {
struct platform_device *pdev;
.id_table = selftest_i2c_dev_id,
};
-#if IS_ENABLED(CONFIG_I2C_MUX)
+#if IS_BUILTIN(CONFIG_I2C_MUX)
struct selftest_i2c_mux_data {
int nchans;
"could not register selftest i2c bus driver\n"))
return ret;
-#if IS_ENABLED(CONFIG_I2C_MUX)
+#if IS_BUILTIN(CONFIG_I2C_MUX)
ret = i2c_add_driver(&selftest_i2c_mux_driver);
if (selftest(ret == 0,
"could not register selftest i2c mux driver\n"))
static void of_selftest_overlay_i2c_cleanup(void)
{
-#if IS_ENABLED(CONFIG_I2C_MUX)
+#if IS_BUILTIN(CONFIG_I2C_MUX)
i2c_del_driver(&selftest_i2c_mux_driver);
#endif
platform_driver_unregister(&selftest_i2c_bus_driver);
of_selftest_overlay_10();
of_selftest_overlay_11();
-#if IS_ENABLED(CONFIG_I2C)
+#if IS_BUILTIN(CONFIG_I2C)
if (selftest(of_selftest_overlay_i2c_init() == 0, "i2c init failed\n"))
goto out;
return bus;
}
-static struct pci_host_bridge *find_pci_host_bridge(struct pci_bus *bus)
+struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus)
{
struct pci_bus *root_bus = find_pci_root_bus(bus);
return to_pci_host_bridge(root_bus->bridge);
}
+struct device *pci_get_host_bridge_device(struct pci_dev *dev)
+{
+ struct pci_bus *root_bus = find_pci_root_bus(dev->bus);
+ struct device *bridge = root_bus->bridge;
+
+ kobject_get(&bridge->kobj);
+ return bridge;
+}
+
+void pci_put_host_bridge_device(struct device *dev)
+{
+ kobject_put(&dev->kobj);
+}
+
void pci_set_host_bridge_release(struct pci_host_bridge *bridge,
void (*release_fn)(struct pci_host_bridge *),
void *release_data)
void pcibios_resource_to_bus(struct pci_bus *bus, struct pci_bus_region *region,
struct resource *res)
{
- struct pci_host_bridge *bridge = find_pci_host_bridge(bus);
+ struct pci_host_bridge *bridge = pci_find_host_bridge(bus);
struct resource_entry *window;
resource_size_t offset = 0;
void pcibios_bus_to_resource(struct pci_bus *bus, struct resource *res,
struct pci_bus_region *region)
{
- struct pci_host_bridge *bridge = find_pci_host_bridge(bus);
+ struct pci_host_bridge *bridge = pci_find_host_bridge(bus);
struct resource_entry *window;
resource_size_t offset = 0;
bool "ARM Versatile PB PCI controller"
depends on ARCH_VERSATILE
+config PCIE_IPROC
+ tristate "Broadcom iProc PCIe controller"
+ depends on OF && ARM
+ default n
+ help
+ This enables the iProc PCIe core controller support for Broadcom's
+ iProc family of SoCs. An appropriate bus interface driver also needs
+ to be enabled
+
+config PCIE_IPROC_PLATFORM
+ tristate "Broadcom iProc PCIe platform bus driver"
+ depends on ARCH_BCM_IPROC || (ARM && COMPILE_TEST)
+ depends on OF
+ select PCIE_IPROC
+ default ARCH_BCM_IPROC
+ help
+ Say Y here if you want to use the Broadcom iProc PCIe controller
+ through the generic platform bus interface
+
endmenu
obj-$(CONFIG_PCI_XGENE) += pci-xgene.o
obj-$(CONFIG_PCI_LAYERSCAPE) += pci-layerscape.o
obj-$(CONFIG_PCI_VERSATILE) += pci-versatile.o
+obj-$(CONFIG_PCIE_IPROC) += pcie-iproc.o
+obj-$(CONFIG_PCIE_IPROC_PLATFORM) += pcie-iproc-platform.o
/* enable INTX interrupt */
val = IRQ_INTA_ASSERT | IRQ_INTB_ASSERT |
- IRQ_INTC_ASSERT | IRQ_INTD_ASSERT,
+ IRQ_INTC_ASSERT | IRQ_INTD_ASSERT;
exynos_elb_writel(exynos_pcie, val, PCIE_IRQ_EN_PULSE);
}
/* Index 1 is the application reg. space address */
res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- ks_pcie->app = *res;
ks_pcie->va_app_base = devm_ioremap_resource(pp->dev, res);
if (IS_ERR(ks_pcie->va_app_base))
return PTR_ERR(ks_pcie->va_app_base);
+ ks_pcie->app = *res;
+
/* Create legacy IRQ domain */
ks_pcie->legacy_irq_domain =
irq_domain_add_linear(ks_pcie->legacy_intc_np,
pcie->dev = &pdev->dev;
dbi_base = platform_get_resource_byname(pdev, IORESOURCE_MEM, "regs");
- if (!dbi_base) {
- dev_err(&pdev->dev, "missing *regs* space\n");
- return -ENODEV;
- }
-
pcie->dbi = devm_ioremap_resource(&pdev->dev, dbi_base);
- if (IS_ERR(pcie->dbi))
+ if (IS_ERR(pcie->dbi)) {
+ dev_err(&pdev->dev, "missing *regs* space\n");
return PTR_ERR(pcie->dbi);
+ }
pcie->scfg = syscon_regmap_lookup_by_phandle(pdev->dev.of_node,
"fsl,pcie-scfg");
size_t memwin_size;
phys_addr_t iowin_base;
size_t iowin_size;
+ u32 saved_pcie_stat;
};
static inline void mvebu_writel(struct mvebu_pcie_port *port, u32 val, u32 reg)
pcie->msi->dev = &pcie->pdev->dev;
}
+static int mvebu_pcie_suspend(struct device *dev)
+{
+ struct mvebu_pcie *pcie;
+ int i;
+
+ pcie = dev_get_drvdata(dev);
+ for (i = 0; i < pcie->nports; i++) {
+ struct mvebu_pcie_port *port = pcie->ports + i;
+ port->saved_pcie_stat = mvebu_readl(port, PCIE_STAT_OFF);
+ }
+
+ return 0;
+}
+
+static int mvebu_pcie_resume(struct device *dev)
+{
+ struct mvebu_pcie *pcie;
+ int i;
+
+ pcie = dev_get_drvdata(dev);
+ for (i = 0; i < pcie->nports; i++) {
+ struct mvebu_pcie_port *port = pcie->ports + i;
+ mvebu_writel(port, port->saved_pcie_stat, PCIE_STAT_OFF);
+ mvebu_pcie_setup_hw(port);
+ }
+
+ return 0;
+}
+
static int mvebu_pcie_probe(struct platform_device *pdev)
{
struct mvebu_pcie *pcie;
mvebu_pcie_msi_enable(pcie);
mvebu_pcie_enable(pcie);
+ platform_set_drvdata(pdev, pcie);
+
return 0;
}
};
MODULE_DEVICE_TABLE(of, mvebu_pcie_of_match_table);
+static struct dev_pm_ops mvebu_pcie_pm_ops = {
+ .suspend_noirq = mvebu_pcie_suspend,
+ .resume_noirq = mvebu_pcie_resume,
+};
+
static struct platform_driver mvebu_pcie_driver = {
.driver = {
.name = "mvebu-pcie",
.of_match_table = mvebu_pcie_of_match_table,
/* driver unloading/unbinding currently not supported */
.suppress_bind_attrs = true,
+ .pm = &mvebu_pcie_pm_ops,
},
.probe = mvebu_pcie_probe,
};
if (!mem_res || !mem_res->start)
return -ENODEV;
+ if (mem_res->start & 0xFFFF)
+ return -EINVAL;
+
priv = devm_kzalloc(&pdev->dev,
sizeof(struct rcar_pci_priv), GFP_KERNEL);
if (!priv)
if (err)
return err;
- resource_list_for_each_entry(win, res, list) {
+ resource_list_for_each_entry(win, res) {
struct resource *parent, *res = win->res;
switch (resource_type(res)) {
LIST_HEAD(pci_res);
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
- if (!res)
- return -ENODEV;
versatile_pci_base = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(versatile_pci_base))
+ return PTR_ERR(versatile_pci_base);
res = platform_get_resource(pdev, IORESOURCE_MEM, 1);
- if (!res)
- return -ENODEV;
versatile_cfg_base[0] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(versatile_cfg_base[0]))
+ return PTR_ERR(versatile_cfg_base[0]);
res = platform_get_resource(pdev, IORESOURCE_MEM, 2);
- if (!res)
- return -ENODEV;
versatile_cfg_base[1] = devm_ioremap_resource(&pdev->dev, res);
+ if (IS_ERR(versatile_cfg_base[1]))
+ return PTR_ERR(versatile_cfg_base[1]);
ret = versatile_pci_parse_request_of_pci_ranges(&pdev->dev, &pci_res);
if (ret)
pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
pci_assign_unassigned_bus_resources(bus);
+ pci_bus_add_devices(bus);
return 0;
}
return false;
}
-static int xgene_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
+static void __iomem *xgene_pcie_map_bus(struct pci_bus *bus, unsigned int devfn,
int offset)
{
struct xgene_pcie_port *port = bus->sysdata;
return NULL;
xgene_pcie_set_rtdid_reg(bus, devfn);
- return xgene_pcie_get_cfg_base(bus);
+ return xgene_pcie_get_cfg_base(bus) + offset;
}
static struct pci_ops xgene_pcie_ops = {
.map = dw_pcie_msi_map,
};
-int __init dw_pcie_host_init(struct pcie_port *pp)
+int dw_pcie_host_init(struct pcie_port *pp)
{
struct device_node *np = pp->dev->of_node;
struct platform_device *pdev = to_platform_device(pp->dev);
--- /dev/null
+/*
+ * Copyright (C) 2015 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/slab.h>
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/phy/phy.h>
+
+#include "pcie-iproc.h"
+
+static int iproc_pcie_pltfm_probe(struct platform_device *pdev)
+{
+ struct iproc_pcie *pcie;
+ struct device_node *np = pdev->dev.of_node;
+ struct resource reg;
+ resource_size_t iobase = 0;
+ LIST_HEAD(res);
+ int ret;
+
+ pcie = devm_kzalloc(&pdev->dev, sizeof(struct iproc_pcie), GFP_KERNEL);
+ if (!pcie)
+ return -ENOMEM;
+
+ pcie->dev = &pdev->dev;
+ platform_set_drvdata(pdev, pcie);
+
+ ret = of_address_to_resource(np, 0, ®);
+ if (ret < 0) {
+ dev_err(pcie->dev, "unable to obtain controller resources\n");
+ return ret;
+ }
+
+ pcie->base = devm_ioremap(pcie->dev, reg.start, resource_size(®));
+ if (!pcie->base) {
+ dev_err(pcie->dev, "unable to map controller registers\n");
+ return -ENOMEM;
+ }
+
+ /* PHY use is optional */
+ pcie->phy = devm_phy_get(&pdev->dev, "pcie-phy");
+ if (IS_ERR(pcie->phy)) {
+ if (PTR_ERR(pcie->phy) == -EPROBE_DEFER)
+ return -EPROBE_DEFER;
+ pcie->phy = NULL;
+ }
+
+ ret = of_pci_get_host_bridge_resources(np, 0, 0xff, &res, &iobase);
+ if (ret) {
+ dev_err(pcie->dev,
+ "unable to get PCI host bridge resources\n");
+ return ret;
+ }
+
+ pcie->resources = &res;
+
+ ret = iproc_pcie_setup(pcie);
+ if (ret) {
+ dev_err(pcie->dev, "PCIe controller setup failed\n");
+ return ret;
+ }
+
+ return 0;
+}
+
+static int iproc_pcie_pltfm_remove(struct platform_device *pdev)
+{
+ struct iproc_pcie *pcie = platform_get_drvdata(pdev);
+
+ return iproc_pcie_remove(pcie);
+}
+
+static const struct of_device_id iproc_pcie_of_match_table[] = {
+ { .compatible = "brcm,iproc-pcie", },
+ { /* sentinel */ }
+};
+MODULE_DEVICE_TABLE(of, iproc_pcie_of_match_table);
+
+static struct platform_driver iproc_pcie_pltfm_driver = {
+ .driver = {
+ .name = "iproc-pcie",
+ .of_match_table = of_match_ptr(iproc_pcie_of_match_table),
+ },
+ .probe = iproc_pcie_pltfm_probe,
+ .remove = iproc_pcie_pltfm_remove,
+};
+module_platform_driver(iproc_pcie_pltfm_driver);
+
+MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>");
+MODULE_DESCRIPTION("Broadcom iPROC PCIe platform driver");
+MODULE_LICENSE("GPL v2");
--- /dev/null
+/*
+ * Copyright (C) 2014 Hauke Mehrtens <hauke@hauke-m.de>
+ * Copyright (C) 2015 Broadcom Corporatcommon ion
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kernel.h>
+#include <linux/pci.h>
+#include <linux/msi.h>
+#include <linux/clk.h>
+#include <linux/module.h>
+#include <linux/mbus.h>
+#include <linux/slab.h>
+#include <linux/delay.h>
+#include <linux/interrupt.h>
+#include <linux/platform_device.h>
+#include <linux/of_address.h>
+#include <linux/of_pci.h>
+#include <linux/of_irq.h>
+#include <linux/of_platform.h>
+#include <linux/phy/phy.h>
+
+#include "pcie-iproc.h"
+
+#define CLK_CONTROL_OFFSET 0x000
+#define EP_MODE_SURVIVE_PERST_SHIFT 1
+#define EP_MODE_SURVIVE_PERST BIT(EP_MODE_SURVIVE_PERST_SHIFT)
+#define RC_PCIE_RST_OUTPUT_SHIFT 0
+#define RC_PCIE_RST_OUTPUT BIT(RC_PCIE_RST_OUTPUT_SHIFT)
+
+#define CFG_IND_ADDR_OFFSET 0x120
+#define CFG_IND_ADDR_MASK 0x00001ffc
+
+#define CFG_IND_DATA_OFFSET 0x124
+
+#define CFG_ADDR_OFFSET 0x1f8
+#define CFG_ADDR_BUS_NUM_SHIFT 20
+#define CFG_ADDR_BUS_NUM_MASK 0x0ff00000
+#define CFG_ADDR_DEV_NUM_SHIFT 15
+#define CFG_ADDR_DEV_NUM_MASK 0x000f8000
+#define CFG_ADDR_FUNC_NUM_SHIFT 12
+#define CFG_ADDR_FUNC_NUM_MASK 0x00007000
+#define CFG_ADDR_REG_NUM_SHIFT 2
+#define CFG_ADDR_REG_NUM_MASK 0x00000ffc
+#define CFG_ADDR_CFG_TYPE_SHIFT 0
+#define CFG_ADDR_CFG_TYPE_MASK 0x00000003
+
+#define CFG_DATA_OFFSET 0x1fc
+
+#define SYS_RC_INTX_EN 0x330
+#define SYS_RC_INTX_MASK 0xf
+
+static inline struct iproc_pcie *sys_to_pcie(struct pci_sys_data *sys)
+{
+ return sys->private_data;
+}
+
+/**
+ * Note access to the configuration registers are protected at the higher layer
+ * by 'pci_lock' in drivers/pci/access.c
+ */
+static void __iomem *iproc_pcie_map_cfg_bus(struct pci_bus *bus,
+ unsigned int devfn,
+ int where)
+{
+ struct pci_sys_data *sys = bus->sysdata;
+ struct iproc_pcie *pcie = sys_to_pcie(sys);
+ unsigned slot = PCI_SLOT(devfn);
+ unsigned fn = PCI_FUNC(devfn);
+ unsigned busno = bus->number;
+ u32 val;
+
+ /* root complex access */
+ if (busno == 0) {
+ if (slot >= 1)
+ return NULL;
+ writel(where & CFG_IND_ADDR_MASK,
+ pcie->base + CFG_IND_ADDR_OFFSET);
+ return (pcie->base + CFG_IND_DATA_OFFSET);
+ }
+
+ if (fn > 1)
+ return NULL;
+
+ /* EP device access */
+ val = (busno << CFG_ADDR_BUS_NUM_SHIFT) |
+ (slot << CFG_ADDR_DEV_NUM_SHIFT) |
+ (fn << CFG_ADDR_FUNC_NUM_SHIFT) |
+ (where & CFG_ADDR_REG_NUM_MASK) |
+ (1 & CFG_ADDR_CFG_TYPE_MASK);
+ writel(val, pcie->base + CFG_ADDR_OFFSET);
+
+ return (pcie->base + CFG_DATA_OFFSET);
+}
+
+static struct pci_ops iproc_pcie_ops = {
+ .map_bus = iproc_pcie_map_cfg_bus,
+ .read = pci_generic_config_read32,
+ .write = pci_generic_config_write32,
+};
+
+static void iproc_pcie_reset(struct iproc_pcie *pcie)
+{
+ u32 val;
+
+ /*
+ * Configure the PCIe controller as root complex and send a downstream
+ * reset
+ */
+ val = EP_MODE_SURVIVE_PERST | RC_PCIE_RST_OUTPUT;
+ writel(val, pcie->base + CLK_CONTROL_OFFSET);
+ udelay(250);
+ val &= ~EP_MODE_SURVIVE_PERST;
+ writel(val, pcie->base + CLK_CONTROL_OFFSET);
+ msleep(250);
+}
+
+static int iproc_pcie_check_link(struct iproc_pcie *pcie, struct pci_bus *bus)
+{
+ u8 hdr_type;
+ u32 link_ctrl;
+ u16 pos, link_status;
+ int link_is_active = 0;
+
+ /* make sure we are not in EP mode */
+ pci_bus_read_config_byte(bus, 0, PCI_HEADER_TYPE, &hdr_type);
+ if ((hdr_type & 0x7f) != PCI_HEADER_TYPE_BRIDGE) {
+ dev_err(pcie->dev, "in EP mode, hdr=%#02x\n", hdr_type);
+ return -EFAULT;
+ }
+
+ /* force class to PCI_CLASS_BRIDGE_PCI (0x0604) */
+ pci_bus_write_config_word(bus, 0, PCI_CLASS_DEVICE,
+ PCI_CLASS_BRIDGE_PCI);
+
+ /* check link status to see if link is active */
+ pos = pci_bus_find_capability(bus, 0, PCI_CAP_ID_EXP);
+ pci_bus_read_config_word(bus, 0, pos + PCI_EXP_LNKSTA, &link_status);
+ if (link_status & PCI_EXP_LNKSTA_NLW)
+ link_is_active = 1;
+
+ if (!link_is_active) {
+ /* try GEN 1 link speed */
+#define PCI_LINK_STATUS_CTRL_2_OFFSET 0x0dc
+#define PCI_TARGET_LINK_SPEED_MASK 0xf
+#define PCI_TARGET_LINK_SPEED_GEN2 0x2
+#define PCI_TARGET_LINK_SPEED_GEN1 0x1
+ pci_bus_read_config_dword(bus, 0,
+ PCI_LINK_STATUS_CTRL_2_OFFSET,
+ &link_ctrl);
+ if ((link_ctrl & PCI_TARGET_LINK_SPEED_MASK) ==
+ PCI_TARGET_LINK_SPEED_GEN2) {
+ link_ctrl &= ~PCI_TARGET_LINK_SPEED_MASK;
+ link_ctrl |= PCI_TARGET_LINK_SPEED_GEN1;
+ pci_bus_write_config_dword(bus, 0,
+ PCI_LINK_STATUS_CTRL_2_OFFSET,
+ link_ctrl);
+ msleep(100);
+
+ pos = pci_bus_find_capability(bus, 0, PCI_CAP_ID_EXP);
+ pci_bus_read_config_word(bus, 0, pos + PCI_EXP_LNKSTA,
+ &link_status);
+ if (link_status & PCI_EXP_LNKSTA_NLW)
+ link_is_active = 1;
+ }
+ }
+
+ dev_info(pcie->dev, "link: %s\n", link_is_active ? "UP" : "DOWN");
+
+ return link_is_active ? 0 : -ENODEV;
+}
+
+static void iproc_pcie_enable(struct iproc_pcie *pcie)
+{
+ writel(SYS_RC_INTX_MASK, pcie->base + SYS_RC_INTX_EN);
+}
+
+int iproc_pcie_setup(struct iproc_pcie *pcie)
+{
+ int ret;
+ struct pci_bus *bus;
+
+ if (!pcie || !pcie->dev || !pcie->base)
+ return -EINVAL;
+
+ if (pcie->phy) {
+ ret = phy_init(pcie->phy);
+ if (ret) {
+ dev_err(pcie->dev, "unable to initialize PCIe PHY\n");
+ return ret;
+ }
+
+ ret = phy_power_on(pcie->phy);
+ if (ret) {
+ dev_err(pcie->dev, "unable to power on PCIe PHY\n");
+ goto err_exit_phy;
+ }
+
+ }
+
+ iproc_pcie_reset(pcie);
+
+ pcie->sysdata.private_data = pcie;
+
+ bus = pci_create_root_bus(pcie->dev, 0, &iproc_pcie_ops,
+ &pcie->sysdata, pcie->resources);
+ if (!bus) {
+ dev_err(pcie->dev, "unable to create PCI root bus\n");
+ ret = -ENOMEM;
+ goto err_power_off_phy;
+ }
+ pcie->root_bus = bus;
+
+ ret = iproc_pcie_check_link(pcie, bus);
+ if (ret) {
+ dev_err(pcie->dev, "no PCIe EP device detected\n");
+ goto err_rm_root_bus;
+ }
+
+ iproc_pcie_enable(pcie);
+
+ pci_scan_child_bus(bus);
+ pci_assign_unassigned_bus_resources(bus);
+ pci_fixup_irqs(pci_common_swizzle, of_irq_parse_and_map_pci);
+ pci_bus_add_devices(bus);
+
+ return 0;
+
+err_rm_root_bus:
+ pci_stop_root_bus(bus);
+ pci_remove_root_bus(bus);
+
+err_power_off_phy:
+ if (pcie->phy)
+ phy_power_off(pcie->phy);
+err_exit_phy:
+ if (pcie->phy)
+ phy_exit(pcie->phy);
+
+ return ret;
+}
+EXPORT_SYMBOL(iproc_pcie_setup);
+
+int iproc_pcie_remove(struct iproc_pcie *pcie)
+{
+ pci_stop_root_bus(pcie->root_bus);
+ pci_remove_root_bus(pcie->root_bus);
+
+ if (pcie->phy) {
+ phy_power_off(pcie->phy);
+ phy_exit(pcie->phy);
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL(iproc_pcie_remove);
+
+MODULE_AUTHOR("Ray Jui <rjui@broadcom.com>");
+MODULE_DESCRIPTION("Broadcom iPROC PCIe common driver");
+MODULE_LICENSE("GPL v2");
--- /dev/null
+/*
+ * Copyright (C) 2014-2015 Broadcom Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation version 2.
+ *
+ * This program is distributed "as is" WITHOUT ANY WARRANTY of any
+ * kind, whether express or implied; without even the implied warranty
+ * of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _PCIE_IPROC_H
+#define _PCIE_IPROC_H
+
+#define IPROC_PCIE_MAX_NUM_IRQS 6
+
+/**
+ * iProc PCIe device
+ * @dev: pointer to device data structure
+ * @base: PCIe host controller I/O register base
+ * @resources: linked list of all PCI resources
+ * @sysdata: Per PCI controller data
+ * @root_bus: pointer to root bus
+ * @phy: optional PHY device that controls the Serdes
+ * @irqs: interrupt IDs
+ */
+struct iproc_pcie {
+ struct device *dev;
+ void __iomem *base;
+ struct list_head *resources;
+ struct pci_sys_data sysdata;
+ struct pci_bus *root_bus;
+ struct phy *phy;
+ int irqs[IPROC_PCIE_MAX_NUM_IRQS];
+};
+
+int iproc_pcie_setup(struct iproc_pcie *pcie);
+int iproc_pcie_remove(struct iproc_pcie *pcie);
+
+#endif /* _PCIE_IPROC_H */
#define LAR_ENABLE (1 << 1)
/* PCIe address reg & mask */
-#define PCIEPARL(x) (0x03400 + ((x) * 0x20))
-#define PCIEPARH(x) (0x03404 + ((x) * 0x20))
+#define PCIEPALR(x) (0x03400 + ((x) * 0x20))
+#define PCIEPAUR(x) (0x03404 + ((x) * 0x20))
#define PCIEPAMR(x) (0x03408 + ((x) * 0x20))
#define PCIEPTCTLR(x) (0x0340c + ((x) * 0x20))
#define PAR_ENABLE (1 << 31)
else
res_start = res->start;
- rcar_pci_write_reg(pcie, upper_32_bits(res_start), PCIEPARH(win));
- rcar_pci_write_reg(pcie, lower_32_bits(res_start), PCIEPARL(win));
+ rcar_pci_write_reg(pcie, upper_32_bits(res_start), PCIEPAUR(win));
+ rcar_pci_write_reg(pcie, lower_32_bits(res_start) & ~0x7F,
+ PCIEPALR(win));
/* First resource is for IO */
mask = PAR_ENABLE;
/* Enable MSI */
if (IS_ENABLED(CONFIG_PCI_MSI))
- rcar_pci_write_reg(pcie, 0x101f0000, PCIEMSITXR);
+ rcar_pci_write_reg(pcie, 0x801f0000, PCIEMSITXR);
/* Finish initialization - establish a PCI Express link */
rcar_pci_write_reg(pcie, CFINIT, PCIETCTLR);
.host_init = spear13xx_pcie_host_init,
};
-static int __init spear13xx_add_pcie_port(struct pcie_port *pp,
+static int spear13xx_add_pcie_port(struct pcie_port *pp,
struct platform_device *pdev)
{
struct device *dev = &pdev->dev;
return 0;
}
-static int __init spear13xx_pcie_probe(struct platform_device *pdev)
+static int spear13xx_pcie_probe(struct platform_device *pdev)
{
struct spear13xx_pcie *spear13xx_pcie;
struct pcie_port *pp;
};
MODULE_DEVICE_TABLE(of, spear13xx_pcie_of_match);
-static struct platform_driver spear13xx_pcie_driver __initdata = {
+static struct platform_driver spear13xx_pcie_driver = {
.probe = spear13xx_pcie_probe,
.driver = {
.name = "spear-pcie",
}
parent = slot->dev->bus;
- list_for_each_entry(dev, &parent->devices, bus_list)
+ list_for_each_entry(dev, &parent->devices, bus_list) {
if (PCI_SLOT(dev->devfn) != PCI_SLOT(slot->devfn))
continue;
if (pci_is_bridge(dev))
pci_hp_add_bridge(dev);
+ }
pci_assign_unassigned_bridge_resources(parent->self);
*/
static u8 bus_structure_fixup(u8 busno)
{
- struct pci_bus *bus;
+ struct pci_bus *bus, *b;
struct pci_dev *dev;
u16 l;
(l != 0x0000) && (l != 0xffff)) {
debug("%s - Inside bus_structure_fixup()\n",
__func__);
- pci_scan_bus(busno, ibmphp_pci_bus->ops, NULL);
+ b = pci_scan_bus(busno, ibmphp_pci_bus->ops, NULL);
+ if (!b)
+ continue;
+
+ pci_bus_add_devices(b);
break;
}
}
#include <linux/pm_qos.h>
#include "pci.h"
+/*
+ * The UUID is defined in the PCI Firmware Specification available here:
+ * https://www.pcisig.com/members/downloads/pcifw_r3_1_13Dec10.pdf
+ */
+const u8 pci_acpi_dsm_uuid[] = {
+ 0xd0, 0x37, 0xc9, 0xe5, 0x53, 0x35, 0x7a, 0x4d,
+ 0x91, 0x17, 0xea, 0x4d, 0x19, 0xc3, 0x43, 0x4d
+};
+
phys_addr_t acpi_pci_root_get_mcfg_addr(acpi_handle handle)
{
acpi_status status = AE_NOT_EXIST;
acpi_handle handle, phandle;
struct pci_bus *pbus;
+ if (acpi_pci_disabled)
+ return -ENODEV;
+
handle = NULL;
for (pbus = dev->bus; pbus; pbus = pbus->parent) {
handle = acpi_pci_get_bridge_handle(pbus);
void acpi_pci_add_bus(struct pci_bus *bus)
{
+ union acpi_object *obj;
+ struct pci_host_bridge *bridge;
+
if (acpi_pci_disabled || !bus->bridge)
return;
acpi_pci_slot_enumerate(bus);
acpiphp_enumerate_slots(bus);
+
+ /*
+ * For a host bridge, check its _DSM for function 8 and if
+ * that is available, mark it in pci_host_bridge.
+ */
+ if (!pci_is_root_bus(bus))
+ return;
+
+ obj = acpi_evaluate_dsm(ACPI_HANDLE(bus->bridge), pci_acpi_dsm_uuid, 3,
+ RESET_DELAY_DSM, NULL);
+ if (!obj)
+ return;
+
+ if (obj->type == ACPI_TYPE_INTEGER && obj->integer.value == 1) {
+ bridge = pci_find_host_bridge(bus);
+ bridge->ignore_reset_delay = 1;
+ }
+ ACPI_FREE(obj);
}
void acpi_pci_remove_bus(struct pci_bus *bus)
check_children);
}
+/**
+ * pci_acpi_optimize_delay - optimize PCI D3 and D3cold delay from ACPI
+ * @pdev: the PCI device whose delay is to be updated
+ * @adev: the companion ACPI device of this PCI device
+ *
+ * Update the d3_delay and d3cold_delay of a PCI device from the ACPI _DSM
+ * control method of either the device itself or the PCI host bridge.
+ *
+ * Function 8, "Reset Delay," applies to the entire hierarchy below a PCI
+ * host bridge. If it returns one, the OS may assume that all devices in
+ * the hierarchy have already completed power-on reset delays.
+ *
+ * Function 9, "Device Readiness Durations," applies only to the object
+ * where it is located. It returns delay durations required after various
+ * events if the device requires less time than the spec requires. Delays
+ * from this function take precedence over the Reset Delay function.
+ *
+ * These _DSM functions are defined by the draft ECN of January 28, 2014,
+ * titled "ACPI additions for FW latency optimizations."
+ */
+static void pci_acpi_optimize_delay(struct pci_dev *pdev,
+ acpi_handle handle)
+{
+ struct pci_host_bridge *bridge = pci_find_host_bridge(pdev->bus);
+ int value;
+ union acpi_object *obj, *elements;
+
+ if (bridge->ignore_reset_delay)
+ pdev->d3cold_delay = 0;
+
+ obj = acpi_evaluate_dsm(handle, pci_acpi_dsm_uuid, 3,
+ FUNCTION_DELAY_DSM, NULL);
+ if (!obj)
+ return;
+
+ if (obj->type == ACPI_TYPE_PACKAGE && obj->package.count == 5) {
+ elements = obj->package.elements;
+ if (elements[0].type == ACPI_TYPE_INTEGER) {
+ value = (int)elements[0].integer.value / 1000;
+ if (value < PCI_PM_D3COLD_WAIT)
+ pdev->d3cold_delay = value;
+ }
+ if (elements[3].type == ACPI_TYPE_INTEGER) {
+ value = (int)elements[3].integer.value / 1000;
+ if (value < PCI_PM_D3_WAIT)
+ pdev->d3_delay = value;
+ }
+ }
+ ACPI_FREE(obj);
+}
+
static void pci_acpi_setup(struct device *dev)
{
struct pci_dev *pci_dev = to_pci_dev(dev);
if (!adev)
return;
+ pci_acpi_optimize_delay(pci_dev, adev->handle);
+
pci_acpi_add_pm_notifier(adev, pci_dev);
if (!adev->wakeup.flags.valid)
return;
#include <linux/pci-acpi.h>
#include "pci.h"
-#define DEVICE_LABEL_DSM 0x07
-
#ifdef CONFIG_DMI
enum smbios_attr_enum {
SMBIOS_ATTR_NONE = 0,
#endif
#ifdef CONFIG_ACPI
-static const char device_label_dsm_uuid[] = {
- 0xD0, 0x37, 0xC9, 0xE5, 0x53, 0x35, 0x7A, 0x4D,
- 0x91, 0x17, 0xEA, 0x4D, 0x19, 0xC3, 0x43, 0x4D
-};
-
enum acpi_attr_enum {
ACPI_ATTR_LABEL_SHOW,
ACPI_ATTR_INDEX_SHOW,
if (!handle)
return -1;
- obj = acpi_evaluate_dsm(handle, device_label_dsm_uuid, 0x2,
+ obj = acpi_evaluate_dsm(handle, pci_acpi_dsm_uuid, 0x2,
DEVICE_LABEL_DSM, NULL);
if (!obj)
return -1;
if (!handle)
return false;
- return !!acpi_check_dsm(handle, device_label_dsm_uuid, 0x2,
+ return !!acpi_check_dsm(handle, pci_acpi_dsm_uuid, 0x2,
1 << DEVICE_LABEL_DSM);
}
struct pci_dev *pdev = to_pci_dev(dev);
char *driver_override, *old = pdev->driver_override, *cp;
- if (count > PATH_MAX)
+ /* We need to keep extra room for a newline */
+ if (count >= (PAGE_SIZE - 1))
return -EINVAL;
driver_override = kstrndup(buf, count, GFP_KERNEL);
{
struct pci_dev *pdev = to_pci_dev(dev);
- return sprintf(buf, "%s\n", pdev->driver_override);
+ return snprintf(buf, PAGE_SIZE, "%s\n", pdev->driver_override);
}
static DEVICE_ATTR_RW(driver_override);
#ifdef CONFIG_HAS_IOMEM
void __iomem *pci_ioremap_bar(struct pci_dev *pdev, int bar)
{
+ struct resource *res = &pdev->resource[bar];
+
/*
* Make sure the BAR is actually a memory resource, not an IO resource
*/
- if (!(pci_resource_flags(pdev, bar) & IORESOURCE_MEM)) {
- WARN_ON(1);
+ if (res->flags & IORESOURCE_UNSET || !(res->flags & IORESOURCE_MEM)) {
+ dev_warn(&pdev->dev, "can't ioremap BAR %d: %pR\n", bar, res);
return NULL;
}
- return ioremap_nocache(pci_resource_start(pdev, bar),
- pci_resource_len(pdev, bar));
+ return ioremap_nocache(res->start, resource_size(res));
}
EXPORT_SYMBOL_GPL(pci_ioremap_bar);
#endif
u8 pos, int cap, int *ttl)
{
u8 id;
+ u16 ent;
+
+ pci_bus_read_config_byte(bus, devfn, pos, &pos);
while ((*ttl)--) {
- pci_bus_read_config_byte(bus, devfn, pos, &pos);
if (pos < 0x40)
break;
pos &= ~3;
- pci_bus_read_config_byte(bus, devfn, pos + PCI_CAP_LIST_ID,
- &id);
+ pci_bus_read_config_word(bus, devfn, pos, &ent);
+
+ id = ent & 0xff;
if (id == 0xff)
break;
if (id == cap)
return pos;
- pos += PCI_CAP_LIST_NEXT;
+ pos = (ent >> 8);
}
return 0;
}
*pinp = pin;
return PCI_SLOT(dev->devfn);
}
+EXPORT_SYMBOL_GPL(pci_common_swizzle);
/**
* pci_release_region - Release a PCI bar
}
#endif
+struct pci_host_bridge *pci_find_host_bridge(struct pci_bus *bus);
+
#endif /* DRIVERS_PCI_H */
static void __print_tlp_header(struct pci_dev *dev,
struct aer_header_log_regs *t)
{
- unsigned char *tlp = (unsigned char *)&t;
-
- dev_err(&dev->dev, " TLP Header:"
- " %02x%02x%02x%02x %02x%02x%02x%02x"
- " %02x%02x%02x%02x %02x%02x%02x%02x\n",
- *(tlp + 3), *(tlp + 2), *(tlp + 1), *tlp,
- *(tlp + 7), *(tlp + 6), *(tlp + 5), *(tlp + 4),
- *(tlp + 11), *(tlp + 10), *(tlp + 9),
- *(tlp + 8), *(tlp + 15), *(tlp + 14),
- *(tlp + 13), *(tlp + 12));
+ dev_err(&dev->dev, " TLP Header: %08x %08x %08x %08x\n",
+ t->dw0, t->dw1, t->dw2, t->dw3);
}
static void __aer_print_error(struct pci_dev *dev,
}
EXPORT_SYMBOL(pci_disable_link_state);
-void pcie_clear_aspm(struct pci_bus *bus)
-{
- struct pci_dev *child;
-
- if (aspm_force)
- return;
-
- /*
- * Clear any ASPM setup that the firmware has carried out on this bus
- */
- list_for_each_entry(child, &bus->devices, bus_list) {
- __pci_disable_link_state(child, PCIE_LINK_STATE_L0S |
- PCIE_LINK_STATE_L1 |
- PCIE_LINK_STATE_CLKPM,
- false, true);
- }
-}
-
static int pcie_aspm_set_policy(const char *val, struct kernel_param *kp)
{
int i;
#include <linux/delay.h>
#include <linux/init.h>
#include <linux/pci.h>
+#include <linux/of_pci.h>
#include <linux/pci_hotplug.h>
#include <linux/slab.h>
#include <linux/module.h>
dev->dev.dma_mask = &dev->dma_mask;
dev->dev.dma_parms = &dev->dma_parms;
dev->dev.coherent_dma_mask = 0xffffffffull;
+ of_pci_dma_configure(dev);
pci_set_dma_max_seg_size(dev, 65536);
pci_set_dma_seg_boundary(dev, 0xffffffff);
kfree(b);
return NULL;
}
+EXPORT_SYMBOL_GPL(pci_create_root_bus);
int pci_bus_insert_busn_res(struct pci_bus *b, int bus, int bus_max)
{
if (!found)
pci_bus_update_busn_res_end(b, max);
- pci_bus_add_devices(b);
return b;
}
EXPORT_SYMBOL(pci_scan_root_bus);
b = pci_create_root_bus(NULL, bus, ops, sysdata, &resources);
if (b) {
pci_scan_child_bus(b);
- pci_bus_add_devices(b);
} else {
pci_free_resource_list(&resources);
}
|| nhi->subsystem_vendor != 0x2222
|| nhi->subsystem_device != 0x1111)
goto out;
- dev_info(&dev->dev, "quirk: wating for thunderbolt to reestablish pci tunnels...\n");
+ dev_info(&dev->dev, "quirk: waiting for thunderbolt to reestablish PCI tunnels...\n");
device_pm_wait_for_dev(&dev->dev, &nhi->dev);
out:
pci_dev_put(nhi);
{ PCI_VENDOR_ID_INTEL, 0x154F, pci_quirk_mf_endpoint_acs },
{ PCI_VENDOR_ID_INTEL, 0x1551, pci_quirk_mf_endpoint_acs },
{ PCI_VENDOR_ID_INTEL, 0x1558, pci_quirk_mf_endpoint_acs },
+ /* 82580 */
+ { PCI_VENDOR_ID_INTEL, 0x1509, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x150E, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x150F, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1510, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1511, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1516, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1527, pci_quirk_mf_endpoint_acs },
+ /* 82576 */
+ { PCI_VENDOR_ID_INTEL, 0x10C9, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10E6, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10E7, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10E8, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x150A, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x150D, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1518, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1526, pci_quirk_mf_endpoint_acs },
+ /* 82575 */
+ { PCI_VENDOR_ID_INTEL, 0x10A7, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10A9, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10D6, pci_quirk_mf_endpoint_acs },
+ /* I350 */
+ { PCI_VENDOR_ID_INTEL, 0x1521, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1522, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1523, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1524, pci_quirk_mf_endpoint_acs },
+ /* 82571 (Quads omitted due to non-ACS switch) */
+ { PCI_VENDOR_ID_INTEL, 0x105E, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x105F, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x1060, pci_quirk_mf_endpoint_acs },
+ { PCI_VENDOR_ID_INTEL, 0x10D9, pci_quirk_mf_endpoint_acs },
+ /* Intel PCH root ports */
{ PCI_VENDOR_ID_INTEL, PCI_ANY_ID, pci_quirk_intel_pch_acs },
{ 0x19a2, 0x710, pci_quirk_mf_endpoint_acs }, /* Emulex BE3-R */
{ 0x10df, 0x720, pci_quirk_mf_endpoint_acs }, /* Emulex Skyhawk-R */
/* stop the host bridge */
device_release_driver(&host_bridge->dev);
}
+EXPORT_SYMBOL_GPL(pci_stop_root_bus);
void pci_remove_root_bus(struct pci_bus *bus)
{
/* remove the host bridge */
device_unregister(&host_bridge->dev);
}
+EXPORT_SYMBOL_GPL(pci_remove_root_bus);
__pci_bus_assign_resources(bus, &add_list, NULL);
BUG_ON(!list_empty(&add_list));
}
+EXPORT_SYMBOL_GPL(pci_assign_unassigned_bus_resources);
for_each_pci_dev(dev)
pdev_fixup_irq(dev, swizzle, map_irq);
}
+EXPORT_SYMBOL_GPL(pci_fixup_irqs);
if (!root) {
dev_info(&dev->dev, "can't claim BAR %d %pR: no compatible bridge window\n",
resource, res);
+ res->flags |= IORESOURCE_UNSET;
return -EINVAL;
}
if (conflict) {
dev_info(&dev->dev, "can't claim BAR %d %pR: address conflict with %s %pR\n",
resource, res, conflict->name, conflict);
+ res->flags |= IORESOURCE_UNSET;
return -EBUSY;
}
tristate "CardBus yenta-compatible bridge support"
depends on PCI
select CARDBUS if !EXPERT
- select PCCARD_NONSTATIC if PCMCIA != n && ISA
- select PCCARD_PCI if PCMCIA !=n && !ISA
+ select PCCARD_NONSTATIC if PCMCIA != n
---help---
This option enables support for CardBus host bridges. Virtually
all modern PCMCIA bridges are CardBus compatible. A "bridge" is
config PD6729
tristate "Cirrus PD6729 compatible bridge support"
depends on PCMCIA && PCI
- select PCCARD_NONSTATIC if PCMCIA != n && ISA
- select PCCARD_PCI if PCMCIA !=n && !ISA
+ select PCCARD_NONSTATIC
help
This provides support for the Cirrus PD6729 PCI-to-PCMCIA bridge
device, found in some older laptops and PCMCIA card readers.
config I82092
tristate "i82092 compatible bridge support"
depends on PCMCIA && PCI
- select PCCARD_NONSTATIC if PCMCIA != n && ISA
- select PCCARD_PCI if PCMCIA !=n && !ISA
+ select PCCARD_NONSTATIC
help
This provides support for the Intel I82092AA PCI-to-PCMCIA bridge device,
found in some older laptops and more commonly in evaluation boards for the
Say Y here to support the CompactFlash controller on the
PA Semi Electra eval board.
-config PCCARD_PCI
- bool
-
config PCCARD_NONSTATIC
bool
pcmcia_rsrc-y += rsrc_mgr.o
pcmcia_rsrc-$(CONFIG_PCCARD_NONSTATIC) += rsrc_nonstatic.o
pcmcia_rsrc-$(CONFIG_PCCARD_IODYN) += rsrc_iodyn.o
-pcmcia_rsrc-$(CONFIG_PCCARD_PCI) += rsrc_pci.o
obj-$(CONFIG_PCCARD) += pcmcia_rsrc.o
+++ /dev/null
-#include <linux/slab.h>
-#include <linux/module.h>
-#include <linux/kernel.h>
-#include <linux/pci.h>
-
-#include <pcmcia/ss.h>
-#include <pcmcia/cistpl.h>
-#include "cs_internal.h"
-
-
-struct pcmcia_align_data {
- unsigned long mask;
- unsigned long offset;
-};
-
-static resource_size_t pcmcia_align(void *align_data,
- const struct resource *res,
- resource_size_t size, resource_size_t align)
-{
- struct pcmcia_align_data *data = align_data;
- resource_size_t start;
-
- start = (res->start & ~data->mask) + data->offset;
- if (start < res->start)
- start += data->mask + 1;
- return start;
-}
-
-static struct resource *find_io_region(struct pcmcia_socket *s,
- unsigned long base, int num,
- unsigned long align)
-{
- struct resource *res = pcmcia_make_resource(0, num, IORESOURCE_IO,
- dev_name(&s->dev));
- struct pcmcia_align_data data;
- int ret;
-
- data.mask = align - 1;
- data.offset = base & data.mask;
-
- ret = pci_bus_alloc_resource(s->cb_dev->bus, res, num, 1,
- base, 0, pcmcia_align, &data);
- if (ret != 0) {
- kfree(res);
- res = NULL;
- }
- return res;
-}
-
-static int res_pci_find_io(struct pcmcia_socket *s, unsigned int attr,
- unsigned int *base, unsigned int num,
- unsigned int align, struct resource **parent)
-{
- int i, ret = 0;
-
- /* Check for an already-allocated window that must conflict with
- * what was asked for. It is a hack because it does not catch all
- * potential conflicts, just the most obvious ones.
- */
- for (i = 0; i < MAX_IO_WIN; i++) {
- if (!s->io[i].res)
- continue;
-
- if (!*base)
- continue;
-
- if ((s->io[i].res->start & (align-1)) == *base)
- return -EBUSY;
- }
-
- for (i = 0; i < MAX_IO_WIN; i++) {
- struct resource *res = s->io[i].res;
- unsigned int try;
-
- if (res && (res->flags & IORESOURCE_BITS) !=
- (attr & IORESOURCE_BITS))
- continue;
-
- if (!res) {
- if (align == 0)
- align = 0x10000;
-
- res = s->io[i].res = find_io_region(s, *base, num,
- align);
- if (!res)
- return -EINVAL;
-
- *base = res->start;
- s->io[i].res->flags =
- ((res->flags & ~IORESOURCE_BITS) |
- (attr & IORESOURCE_BITS));
- s->io[i].InUse = num;
- *parent = res;
- return 0;
- }
-
- /* Try to extend top of window */
- try = res->end + 1;
- if ((*base == 0) || (*base == try)) {
- ret = adjust_resource(s->io[i].res, res->start,
- resource_size(res) + num);
- if (ret)
- continue;
- *base = try;
- s->io[i].InUse += num;
- *parent = res;
- return 0;
- }
-
- /* Try to extend bottom of window */
- try = res->start - num;
- if ((*base == 0) || (*base == try)) {
- ret = adjust_resource(s->io[i].res,
- res->start - num,
- resource_size(res) + num);
- if (ret)
- continue;
- *base = try;
- s->io[i].InUse += num;
- *parent = res;
- return 0;
- }
- }
- return -EINVAL;
-}
-
-static struct resource *res_pci_find_mem(u_long base, u_long num,
- u_long align, int low, struct pcmcia_socket *s)
-{
- struct resource *res = pcmcia_make_resource(0, num, IORESOURCE_MEM,
- dev_name(&s->dev));
- struct pcmcia_align_data data;
- unsigned long min;
- int ret;
-
- if (align < 0x20000)
- align = 0x20000;
- data.mask = align - 1;
- data.offset = base & data.mask;
-
- min = 0;
- if (!low)
- min = 0x100000UL;
-
- ret = pci_bus_alloc_resource(s->cb_dev->bus,
- res, num, 1, min, 0,
- pcmcia_align, &data);
-
- if (ret != 0) {
- kfree(res);
- res = NULL;
- }
- return res;
-}
-
-
-static int res_pci_init(struct pcmcia_socket *s)
-{
- if (!s->cb_dev || !(s->features & SS_CAP_PAGE_REGS)) {
- dev_err(&s->dev, "not supported by res_pci\n");
- return -EOPNOTSUPP;
- }
- return 0;
-}
-
-struct pccard_resource_ops pccard_nonstatic_ops = {
- .validate_mem = NULL,
- .find_io = res_pci_find_io,
- .find_mem = res_pci_find_mem,
- .init = res_pci_init,
- .exit = NULL,
-};
-EXPORT_SYMBOL(pccard_nonstatic_ops);
struct armada375_cluster_phy *cluster_phy;
u32 reg;
- cluster_phy = dev_get_drvdata(phy->dev.parent);
+ cluster_phy = phy_get_drvdata(phy);
if (!cluster_phy)
return -ENODEV;
cluster_phy->reg = usb_cluster_base;
dev_set_drvdata(dev, cluster_phy);
+ phy_set_drvdata(phy, cluster_phy);
phy_provider = devm_of_phy_provider_register(&pdev->dev,
armada375_usb_phy_xlate);
static int devm_phy_match(struct device *dev, void *res, void *match_data)
{
- return res == match_data;
+ struct phy **phy = res;
+
+ return *phy == match_data;
}
/**
ret = phy_pm_runtime_get_sync(phy);
if (ret < 0 && ret != -ENOTSUPP)
return ret;
+ ret = 0; /* Override possible ret == -ENOTSUPP */
mutex_lock(&phy->mutex);
if (phy->init_count == 0 && phy->ops->init) {
dev_err(&phy->dev, "phy init failed --> %d\n", ret);
goto out;
}
- } else {
- ret = 0; /* Override possible ret == -ENOTSUPP */
}
++phy->init_count;
ret = phy_pm_runtime_get_sync(phy);
if (ret < 0 && ret != -ENOTSUPP)
return ret;
+ ret = 0; /* Override possible ret == -ENOTSUPP */
mutex_lock(&phy->mutex);
if (phy->init_count == 1 && phy->ops->exit) {
ret = phy_pm_runtime_get_sync(phy);
if (ret < 0 && ret != -ENOTSUPP)
return ret;
+ ret = 0; /* Override possible ret == -ENOTSUPP */
mutex_lock(&phy->mutex);
if (phy->power_count == 0 && phy->ops->power_on) {
dev_err(&phy->dev, "phy poweron failed --> %d\n", ret);
goto out;
}
- } else {
- ret = 0; /* Override possible ret == -ENOTSUPP */
}
++phy->power_count;
mutex_unlock(&phy->mutex);
const struct exynos_dp_video_phy_drvdata *drvdata;
};
-static void exynos_dp_video_phy_pwr_isol(struct exynos_dp_video_phy *state,
- unsigned int on)
-{
- unsigned int val;
-
- if (IS_ERR(state->regs))
- return;
-
- val = on ? 0 : EXYNOS5_PHY_ENABLE;
-
- regmap_update_bits(state->regs, state->drvdata->phy_ctrl_offset,
- EXYNOS5_PHY_ENABLE, val);
-}
-
static int exynos_dp_video_phy_power_on(struct phy *phy)
{
struct exynos_dp_video_phy *state = phy_get_drvdata(phy);
/* Disable power isolation on DP-PHY */
- exynos_dp_video_phy_pwr_isol(state, 0);
-
- return 0;
+ return regmap_update_bits(state->regs, state->drvdata->phy_ctrl_offset,
+ EXYNOS5_PHY_ENABLE, EXYNOS5_PHY_ENABLE);
}
static int exynos_dp_video_phy_power_off(struct phy *phy)
struct exynos_dp_video_phy *state = phy_get_drvdata(phy);
/* Enable power isolation on DP-PHY */
- exynos_dp_video_phy_pwr_isol(state, 1);
-
- return 0;
+ return regmap_update_bits(state->regs, state->drvdata->phy_ctrl_offset,
+ EXYNOS5_PHY_ENABLE, 0);
}
static struct phy_ops exynos_dp_video_phy_ops = {
} phys[EXYNOS_MIPI_PHYS_NUM];
spinlock_t slock;
void __iomem *regs;
- struct mutex mutex;
struct regmap *regmap;
};
else
reset = EXYNOS4_MIPI_PHY_SRESETN;
- if (state->regmap) {
- mutex_lock(&state->mutex);
+ spin_lock(&state->slock);
+
+ if (!IS_ERR(state->regmap)) {
regmap_read(state->regmap, offset, &val);
if (on)
val |= reset;
else if (!(val & EXYNOS4_MIPI_PHY_RESET_MASK))
val &= ~EXYNOS4_MIPI_PHY_ENABLE;
regmap_write(state->regmap, offset, val);
- mutex_unlock(&state->mutex);
} else {
addr = state->regs + EXYNOS_MIPI_PHY_CONTROL(id / 2);
- spin_lock(&state->slock);
val = readl(addr);
if (on)
val |= reset;
val &= ~EXYNOS4_MIPI_PHY_ENABLE;
writel(val, addr);
- spin_unlock(&state->slock);
}
+ spin_unlock(&state->slock);
return 0;
}
dev_set_drvdata(dev, state);
spin_lock_init(&state->slock);
- mutex_init(&state->mutex);
for (i = 0; i < EXYNOS_MIPI_PHYS_NUM; i++) {
struct phy *phy = devm_phy_create(dev, NULL,
.power_on = exynos4210_power_on,
.power_off = exynos4210_power_off,
},
- {},
};
const struct samsung_usb2_phy_config exynos4210_usb2_phy_config = {
.power_on = exynos4x12_power_on,
.power_off = exynos4x12_power_off,
},
- {},
};
const struct samsung_usb2_phy_config exynos3250_usb2_phy_config = {
{
struct exynos5_usbdrd_phy *phy_drd = dev_get_drvdata(dev);
- if (WARN_ON(args->args[0] > EXYNOS5_DRDPHYS_NUM))
+ if (WARN_ON(args->args[0] >= EXYNOS5_DRDPHYS_NUM))
return ERR_PTR(-ENODEV);
return phy_drd->phys[args->args[0]].phy;
.power_on = exynos5250_power_on,
.power_off = exynos5250_power_off,
},
- {},
};
const struct samsung_usb2_phy_config exynos5250_usb2_phy_config = {
return -ENOMEM;
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
+ if (!res)
+ return -EINVAL;
+
priv->base = devm_ioremap(dev, res->start, resource_size(res));
if (!priv->base)
return -ENOMEM;
struct regmap *regmap;
struct mutex miphy_mutex;
struct miphy28lp_phy **phys;
+ int nphys;
};
struct miphy_initval {
return ERR_PTR(-EINVAL);
}
- for (index = 0; index < of_get_child_count(dev->of_node); index++)
+ for (index = 0; index < miphy_dev->nphys; index++)
if (phynode == miphy_dev->phys[index]->phy->dev.of_node) {
miphy_phy = miphy_dev->phys[index];
break;
static struct phy_ops miphy28lp_ops = {
.init = miphy28lp_init,
+ .owner = THIS_MODULE,
};
static int miphy28lp_probe_resets(struct device_node *node,
struct miphy28lp_dev *miphy_dev;
struct phy_provider *provider;
struct phy *phy;
- int chancount, port = 0;
- int ret;
+ int ret, port = 0;
miphy_dev = devm_kzalloc(&pdev->dev, sizeof(*miphy_dev), GFP_KERNEL);
if (!miphy_dev)
return -ENOMEM;
- chancount = of_get_child_count(np);
- miphy_dev->phys = devm_kzalloc(&pdev->dev, sizeof(phy) * chancount,
- GFP_KERNEL);
+ miphy_dev->nphys = of_get_child_count(np);
+ miphy_dev->phys = devm_kcalloc(&pdev->dev, miphy_dev->nphys,
+ sizeof(*miphy_dev->phys), GFP_KERNEL);
if (!miphy_dev->phys)
return -ENOMEM;
struct regmap *regmap;
struct mutex miphy_mutex;
struct miphy365x_phy **phys;
+ int nphys;
};
/*
return ERR_PTR(-EINVAL);
}
- for (index = 0; index < of_get_child_count(dev->of_node); index++)
+ for (index = 0; index < miphy_dev->nphys; index++)
if (phynode == miphy_dev->phys[index]->phy->dev.of_node) {
miphy_phy = miphy_dev->phys[index];
break;
struct miphy365x_dev *miphy_dev;
struct phy_provider *provider;
struct phy *phy;
- int chancount, port = 0;
- int ret;
+ int ret, port = 0;
miphy_dev = devm_kzalloc(&pdev->dev, sizeof(*miphy_dev), GFP_KERNEL);
if (!miphy_dev)
return -ENOMEM;
- chancount = of_get_child_count(np);
- miphy_dev->phys = devm_kzalloc(&pdev->dev, sizeof(phy) * chancount,
- GFP_KERNEL);
+ miphy_dev->nphys = of_get_child_count(np);
+ miphy_dev->phys = devm_kcalloc(&pdev->dev, miphy_dev->nphys,
+ sizeof(*miphy_dev->phys), GFP_KERNEL);
if (!miphy_dev->phys)
return -ENOMEM;
}
module_exit(omap_control_phy_exit);
-MODULE_ALIAS("platform: omap_control_phy");
+MODULE_ALIAS("platform:omap_control_phy");
MODULE_AUTHOR("Texas Instruments Inc.");
MODULE_DESCRIPTION("OMAP Control Module PHY Driver");
MODULE_LICENSE("GPL v2");
dev_warn(&pdev->dev,
"found usb_otg_ss_refclk960m, please fix DTS\n");
}
- } else {
- clk_prepare(phy->optclk);
}
+ if (!IS_ERR(phy->optclk))
+ clk_prepare(phy->optclk);
+
usb_add_phy_dev(&phy->phy);
return 0;
module_platform_driver(omap_usb2_driver);
-MODULE_ALIAS("platform: omap_usb2");
+MODULE_ALIAS("platform:omap_usb2");
MODULE_AUTHOR("Texas Instruments Inc.");
MODULE_DESCRIPTION("OMAP USB2 phy driver");
MODULE_LICENSE("GPL v2");
goto out;
}
uA_load = on ? vreg->max_uA : 0;
- ret = regulator_set_optimum_mode(reg, uA_load);
+ ret = regulator_set_load(reg, uA_load);
if (ret >= 0) {
/*
- * regulator_set_optimum_mode() returns new regulator
+ * regulator_set_load() returns new regulator
* mode upon success.
*/
ret = 0;
return ret;
clk_disable_unprepare(phy->clk);
- if (ret)
- return ret;
return 0;
}
/* Power up usb phy analog blocks by set siddq 0 */
ret = rockchip_usb_phy_power(phy, 0);
- if (ret)
+ if (ret) {
+ clk_disable_unprepare(phy->clk);
return ret;
+ }
return 0;
}
cpu_relax();
val = ti_pipe3_readl(phy->pll_ctrl_base, PLL_STATUS);
if (val & PLL_LOCK)
- break;
+ return 0;
} while (!time_after(jiffies, timeout));
- if (!(val & PLL_LOCK)) {
- dev_err(phy->dev, "DPLL failed to lock\n");
- return -EBUSY;
- }
-
- return 0;
+ dev_err(phy->dev, "DPLL failed to lock\n");
+ return -EBUSY;
}
static int ti_pipe3_dpll_program(struct ti_pipe3 *phy)
module_platform_driver(ti_pipe3_driver);
-MODULE_ALIAS("platform: ti_pipe3");
+MODULE_ALIAS("platform:ti_pipe3");
MODULE_AUTHOR("Texas Instruments Inc.");
MODULE_DESCRIPTION("TI PIPE3 phy driver");
MODULE_LICENSE("GPL v2");
twl->dev = &pdev->dev;
twl->irq = platform_get_irq(pdev, 0);
twl->vbus_supplied = false;
- twl->linkstat = -EINVAL;
twl->linkstat = OMAP_MUSB_UNKNOWN;
twl->phy.dev = twl->dev;
for (i = 0; i < MAX_LANE; i++)
ctx->sata_param.speed[i] = 2; /* Default to Gen3 */
- ctx->dev = &pdev->dev;
platform_set_drvdata(pdev, ctx);
ctx->phy = devm_phy_create(ctx->dev, NULL, &xgene_phy_ops);
#define BYT_DIR_MASK (BIT(1) | BIT(2))
#define BYT_TRIG_MASK (BIT(26) | BIT(25) | BIT(24))
+#define BYT_CONF0_RESTORE_MASK (BYT_DIRECT_IRQ_EN | BYT_TRIG_MASK | \
+ BYT_PIN_MUX)
+#define BYT_VAL_RESTORE_MASK (BYT_DIR_MASK | BYT_LEVEL)
+
#define BYT_NGPIO_SCORE 102
#define BYT_NGPIO_NCORE 28
#define BYT_NGPIO_SUS 44
},
};
+struct byt_gpio_pin_context {
+ u32 conf0;
+ u32 val;
+};
+
struct byt_gpio {
struct gpio_chip chip;
struct platform_device *pdev;
spinlock_t lock;
void __iomem *reg_base;
struct pinctrl_gpio_range *range;
+ struct byt_gpio_pin_context *saved_context;
};
#define to_byt_gpio(c) container_of(c, struct byt_gpio, chip)
return vg->reg_base + reg_offset + reg;
}
-static bool is_special_pin(struct byt_gpio *vg, unsigned offset)
+static void byt_gpio_clear_triggering(struct byt_gpio *vg, unsigned offset)
+{
+ void __iomem *reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
+ unsigned long flags;
+ u32 value;
+
+ spin_lock_irqsave(&vg->lock, flags);
+ value = readl(reg);
+ value &= ~(BYT_TRIG_POS | BYT_TRIG_NEG | BYT_TRIG_LVL);
+ writel(value, reg);
+ spin_unlock_irqrestore(&vg->lock, flags);
+}
+
+static u32 byt_get_gpio_mux(struct byt_gpio *vg, unsigned offset)
{
/* SCORE pin 92-93 */
if (!strcmp(vg->range->name, BYT_SCORE_ACPI_UID) &&
offset >= 92 && offset <= 93)
- return true;
+ return 1;
/* SUS pin 11-21 */
if (!strcmp(vg->range->name, BYT_SUS_ACPI_UID) &&
offset >= 11 && offset <= 21)
- return true;
+ return 1;
- return false;
+ return 0;
}
static int byt_gpio_request(struct gpio_chip *chip, unsigned offset)
{
struct byt_gpio *vg = to_byt_gpio(chip);
void __iomem *reg = byt_gpio_reg(chip, offset, BYT_CONF0_REG);
- u32 value;
- bool special;
+ u32 value, gpio_mux;
/*
* In most cases, func pin mux 000 means GPIO function.
* But, some pins may have func pin mux 001 represents
- * GPIO function. Only allow user to export pin with
- * func pin mux preset as GPIO function by BIOS/FW.
+ * GPIO function.
+ *
+ * Because there are devices out there where some pins were not
+ * configured correctly we allow changing the mux value from
+ * request (but print out warning about that).
*/
value = readl(reg) & BYT_PIN_MUX;
- special = is_special_pin(vg, offset);
- if ((special && value != 1) || (!special && value)) {
- dev_err(&vg->pdev->dev,
- "pin %u cannot be used as GPIO.\n", offset);
- return -EINVAL;
+ gpio_mux = byt_get_gpio_mux(vg, offset);
+ if (WARN_ON(gpio_mux != value)) {
+ unsigned long flags;
+
+ spin_lock_irqsave(&vg->lock, flags);
+ value = readl(reg) & ~BYT_PIN_MUX;
+ value |= gpio_mux;
+ writel(value, reg);
+ spin_unlock_irqrestore(&vg->lock, flags);
+
+ dev_warn(&vg->pdev->dev,
+ "pin %u forcibly re-configured as GPIO\n", offset);
}
pm_runtime_get(&vg->pdev->dev);
static void byt_gpio_free(struct gpio_chip *chip, unsigned offset)
{
struct byt_gpio *vg = to_byt_gpio(chip);
- void __iomem *reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
- u32 value;
-
- /* clear interrupt triggering */
- value = readl(reg);
- value &= ~(BYT_TRIG_POS | BYT_TRIG_NEG | BYT_TRIG_LVL);
- writel(value, reg);
+ byt_gpio_clear_triggering(vg, offset);
pm_runtime_put(&vg->pdev->dev);
}
value &= ~(BYT_DIRECT_IRQ_EN | BYT_TRIG_POS | BYT_TRIG_NEG |
BYT_TRIG_LVL);
- switch (type) {
- case IRQ_TYPE_LEVEL_HIGH:
- value |= BYT_TRIG_LVL;
- case IRQ_TYPE_EDGE_RISING:
- value |= BYT_TRIG_POS;
- break;
- case IRQ_TYPE_LEVEL_LOW:
- value |= BYT_TRIG_LVL;
- case IRQ_TYPE_EDGE_FALLING:
- value |= BYT_TRIG_NEG;
- break;
- case IRQ_TYPE_EDGE_BOTH:
- value |= (BYT_TRIG_NEG | BYT_TRIG_POS);
- break;
- }
writel(value, reg);
+ if (type & IRQ_TYPE_EDGE_BOTH)
+ __irq_set_handler_locked(d->irq, handle_edge_irq);
+ else if (type & IRQ_TYPE_LEVEL_MASK)
+ __irq_set_handler_locked(d->irq, handle_level_irq);
+
spin_unlock_irqrestore(&vg->lock, flags);
return 0;
struct irq_data *data = irq_desc_get_irq_data(desc);
struct byt_gpio *vg = to_byt_gpio(irq_desc_get_handler_data(desc));
struct irq_chip *chip = irq_data_get_irq_chip(data);
- u32 base, pin, mask;
+ u32 base, pin;
void __iomem *reg;
- u32 pending;
+ unsigned long pending;
unsigned virq;
- int looplimit = 0;
/* check from GPIO controller which pin triggered the interrupt */
for (base = 0; base < vg->chip.ngpio; base += 32) {
-
reg = byt_gpio_reg(&vg->chip, base, BYT_INT_STAT_REG);
-
- while ((pending = readl(reg))) {
- pin = __ffs(pending);
- mask = BIT(pin);
- /* Clear before handling so we can't lose an edge */
- writel(mask, reg);
-
+ pending = readl(reg);
+ for_each_set_bit(pin, &pending, 32) {
virq = irq_find_mapping(vg->chip.irqdomain, base + pin);
generic_handle_irq(virq);
-
- /* In case bios or user sets triggering incorretly a pin
- * might remain in "interrupt triggered" state.
- */
- if (looplimit++ > 32) {
- dev_err(&vg->pdev->dev,
- "Gpio %d interrupt flood, disabling\n",
- base + pin);
-
- reg = byt_gpio_reg(&vg->chip, base + pin,
- BYT_CONF0_REG);
- mask = readl(reg);
- mask &= ~(BYT_TRIG_NEG | BYT_TRIG_POS |
- BYT_TRIG_LVL);
- writel(mask, reg);
- mask = readl(reg); /* flush */
- break;
- }
}
}
chip->irq_eoi(data);
}
+static void byt_irq_ack(struct irq_data *d)
+{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct byt_gpio *vg = to_byt_gpio(gc);
+ unsigned offset = irqd_to_hwirq(d);
+ void __iomem *reg;
+
+ reg = byt_gpio_reg(&vg->chip, offset, BYT_INT_STAT_REG);
+ writel(BIT(offset % 32), reg);
+}
+
static void byt_irq_unmask(struct irq_data *d)
{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct byt_gpio *vg = to_byt_gpio(gc);
+ unsigned offset = irqd_to_hwirq(d);
+ unsigned long flags;
+ void __iomem *reg;
+ u32 value;
+
+ spin_lock_irqsave(&vg->lock, flags);
+
+ reg = byt_gpio_reg(&vg->chip, offset, BYT_CONF0_REG);
+ value = readl(reg);
+
+ switch (irqd_get_trigger_type(d)) {
+ case IRQ_TYPE_LEVEL_HIGH:
+ value |= BYT_TRIG_LVL;
+ case IRQ_TYPE_EDGE_RISING:
+ value |= BYT_TRIG_POS;
+ break;
+ case IRQ_TYPE_LEVEL_LOW:
+ value |= BYT_TRIG_LVL;
+ case IRQ_TYPE_EDGE_FALLING:
+ value |= BYT_TRIG_NEG;
+ break;
+ case IRQ_TYPE_EDGE_BOTH:
+ value |= (BYT_TRIG_NEG | BYT_TRIG_POS);
+ break;
+ }
+
+ writel(value, reg);
+
+ spin_unlock_irqrestore(&vg->lock, flags);
}
static void byt_irq_mask(struct irq_data *d)
{
+ struct gpio_chip *gc = irq_data_get_irq_chip_data(d);
+ struct byt_gpio *vg = to_byt_gpio(gc);
+
+ byt_gpio_clear_triggering(vg, irqd_to_hwirq(d));
}
static struct irq_chip byt_irqchip = {
.name = "BYT-GPIO",
+ .irq_ack = byt_irq_ack,
.irq_mask = byt_irq_mask,
.irq_unmask = byt_irq_unmask,
.irq_set_type = byt_irq_type,
{
void __iomem *reg;
u32 base, value;
+ int i;
+
+ /*
+ * Clear interrupt triggers for all pins that are GPIOs and
+ * do not use direct IRQ mode. This will prevent spurious
+ * interrupts from misconfigured pins.
+ */
+ for (i = 0; i < vg->chip.ngpio; i++) {
+ value = readl(byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG));
+ if ((value & BYT_PIN_MUX) == byt_get_gpio_mux(vg, i) &&
+ !(value & BYT_DIRECT_IRQ_EN)) {
+ byt_gpio_clear_triggering(vg, i);
+ dev_dbg(&vg->pdev->dev, "disabling GPIO %d\n", i);
+ }
+ }
/* clear interrupt status trigger registers */
for (base = 0; base < vg->chip.ngpio; base += 32) {
gc->can_sleep = false;
gc->dev = dev;
+#ifdef CONFIG_PM_SLEEP
+ vg->saved_context = devm_kcalloc(&pdev->dev, gc->ngpio,
+ sizeof(*vg->saved_context), GFP_KERNEL);
+#endif
+
ret = gpiochip_add(gc);
if (ret) {
dev_err(&pdev->dev, "failed adding byt-gpio chip\n");
return 0;
}
+#ifdef CONFIG_PM_SLEEP
+static int byt_gpio_suspend(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct byt_gpio *vg = platform_get_drvdata(pdev);
+ int i;
+
+ for (i = 0; i < vg->chip.ngpio; i++) {
+ void __iomem *reg;
+ u32 value;
+
+ reg = byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG);
+ value = readl(reg) & BYT_CONF0_RESTORE_MASK;
+ vg->saved_context[i].conf0 = value;
+
+ reg = byt_gpio_reg(&vg->chip, i, BYT_VAL_REG);
+ value = readl(reg) & BYT_VAL_RESTORE_MASK;
+ vg->saved_context[i].val = value;
+ }
+
+ return 0;
+}
+
+static int byt_gpio_resume(struct device *dev)
+{
+ struct platform_device *pdev = to_platform_device(dev);
+ struct byt_gpio *vg = platform_get_drvdata(pdev);
+ int i;
+
+ for (i = 0; i < vg->chip.ngpio; i++) {
+ void __iomem *reg;
+ u32 value;
+
+ reg = byt_gpio_reg(&vg->chip, i, BYT_CONF0_REG);
+ value = readl(reg);
+ if ((value & BYT_CONF0_RESTORE_MASK) !=
+ vg->saved_context[i].conf0) {
+ value &= ~BYT_CONF0_RESTORE_MASK;
+ value |= vg->saved_context[i].conf0;
+ writel(value, reg);
+ dev_info(dev, "restored pin %d conf0 %#08x", i, value);
+ }
+
+ reg = byt_gpio_reg(&vg->chip, i, BYT_VAL_REG);
+ value = readl(reg);
+ if ((value & BYT_VAL_RESTORE_MASK) !=
+ vg->saved_context[i].val) {
+ u32 v;
+
+ v = value & ~BYT_VAL_RESTORE_MASK;
+ v |= vg->saved_context[i].val;
+ if (v != value) {
+ writel(v, reg);
+ dev_dbg(dev, "restored pin %d val %#08x\n",
+ i, v);
+ }
+ }
+ }
+
+ return 0;
+}
+#endif
+
static int byt_gpio_runtime_suspend(struct device *dev)
{
return 0;
}
static const struct dev_pm_ops byt_gpio_pm_ops = {
- .runtime_suspend = byt_gpio_runtime_suspend,
- .runtime_resume = byt_gpio_runtime_resume,
+ SET_LATE_SYSTEM_SLEEP_PM_OPS(byt_gpio_suspend, byt_gpio_resume)
+ SET_RUNTIME_PM_OPS(byt_gpio_runtime_suspend, byt_gpio_runtime_resume,
+ NULL)
};
static const struct acpi_device_id byt_gpio_acpi_match[] = {
static int chv_gpio_direction_output(struct gpio_chip *chip, unsigned offset,
int value)
{
+ chv_gpio_set(chip, offset, value);
return pinctrl_gpio_direction_output(chip->base + offset);
}
/* the interrupt is already cleared before by reading ISR */
}
-static unsigned int gpio_irq_startup(struct irq_data *d)
+static int gpio_irq_request_res(struct irq_data *d)
{
struct at91_gpio_chip *at91_gpio = irq_data_get_irq_chip_data(d);
unsigned pin = d->hwirq;
int ret;
ret = gpiochip_lock_as_irq(&at91_gpio->chip, pin);
- if (ret) {
+ if (ret)
dev_err(at91_gpio->chip.dev, "unable to lock pind %lu IRQ\n",
d->hwirq);
- return ret;
- }
- gpio_irq_unmask(d);
- return 0;
+
+ return ret;
}
-static void gpio_irq_shutdown(struct irq_data *d)
+static void gpio_irq_release_res(struct irq_data *d)
{
struct at91_gpio_chip *at91_gpio = irq_data_get_irq_chip_data(d);
unsigned pin = d->hwirq;
- gpio_irq_mask(d);
gpiochip_unlock_as_irq(&at91_gpio->chip, pin);
}
static struct irq_chip gpio_irqchip = {
.name = "GPIO",
.irq_ack = gpio_irq_ack,
- .irq_startup = gpio_irq_startup,
- .irq_shutdown = gpio_irq_shutdown,
+ .irq_request_resources = gpio_irq_request_res,
+ .irq_release_resources = gpio_irq_release_res,
.irq_disable = gpio_irq_mask,
.irq_mask = gpio_irq_mask,
.irq_unmask = gpio_irq_unmask,
.pins = sun4i_a10_pins,
.npins = ARRAY_SIZE(sun4i_a10_pins),
.irq_banks = 1,
+ .irq_read_needs_mux = true,
};
static int sun4i_a10_pinctrl_probe(struct platform_device *pdev)
#include <linux/slab.h>
#include "../core.h"
+#include "../../gpio/gpiolib.h"
#include "pinctrl-sunxi.h"
static struct irq_chip sunxi_pinctrl_edge_irq_chip;
static int sunxi_pinctrl_gpio_get(struct gpio_chip *chip, unsigned offset)
{
struct sunxi_pinctrl *pctl = dev_get_drvdata(chip->dev);
-
u32 reg = sunxi_data_reg(offset);
u8 index = sunxi_data_offset(offset);
- u32 val = (readl(pctl->membase + reg) >> index) & DATA_PINS_MASK;
+ u32 set_mux = pctl->desc->irq_read_needs_mux &&
+ test_bit(FLAG_USED_AS_IRQ, &chip->desc[offset].flags);
+ u32 val;
+
+ if (set_mux)
+ sunxi_pmx_set(pctl->pctl_dev, offset, SUN4I_FUNC_INPUT);
+
+ val = (readl(pctl->membase + reg) >> index) & DATA_PINS_MASK;
+
+ if (set_mux)
+ sunxi_pmx_set(pctl->pctl_dev, offset, SUN4I_FUNC_IRQ);
return val;
}
#define IRQ_LEVEL_LOW 0x03
#define IRQ_EDGE_BOTH 0x04
+#define SUN4I_FUNC_INPUT 0
+#define SUN4I_FUNC_IRQ 6
+
struct sunxi_desc_function {
const char *name;
u8 muxval;
int npins;
unsigned pin_base;
unsigned irq_banks;
+ bool irq_read_needs_mux;
};
struct sunxi_pinctrl_function {
unsigned char curr_pwm;
/* Power supply */
- struct power_supply psy;
+ struct power_supply *psy;
struct power_supply_info psy_info;
char bat_model_name[BAT_MODEL_NAME_LEN + 1];
char bat_manufacturer_name[BAT_MANUFACTURER_NAME_LEN + 1];
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct compal_data *data;
- data = container_of(psy, struct compal_data, psy);
+ struct compal_data *data = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
};
MODULE_DEVICE_TABLE(dmi, compal_dmi_table);
+static const struct power_supply_desc psy_bat_desc = {
+ .name = DRIVER_NAME,
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = compal_bat_properties,
+ .num_properties = ARRAY_SIZE(compal_bat_properties),
+ .get_property = bat_get_property,
+};
+
static void initialize_power_supply_data(struct compal_data *data)
{
- data->psy.name = DRIVER_NAME;
- data->psy.type = POWER_SUPPLY_TYPE_BATTERY;
- data->psy.properties = compal_bat_properties;
- data->psy.num_properties = ARRAY_SIZE(compal_bat_properties);
- data->psy.get_property = bat_get_property;
ec_read_sequence(BAT_MANUFACTURER_NAME_ADDR,
data->bat_manufacturer_name,
int err;
struct compal_data *data;
struct device *hwmon_dev;
+ struct power_supply_config psy_cfg = {};
if (!extra_features)
return 0;
if (err)
return err;
- hwmon_dev = hwmon_device_register_with_groups(&pdev->dev,
- "compal", data,
- compal_hwmon_groups);
+ hwmon_dev = devm_hwmon_device_register_with_groups(&pdev->dev,
+ "compal", data,
+ compal_hwmon_groups);
if (IS_ERR(hwmon_dev)) {
err = PTR_ERR(hwmon_dev);
goto remove;
/* Power supply */
initialize_power_supply_data(data);
- power_supply_register(&compal_device->dev, &data->psy);
+ psy_cfg.drv_data = data;
+ data->psy = power_supply_register(&compal_device->dev, &psy_bat_desc,
+ &psy_cfg);
+ if (IS_ERR(data->psy)) {
+ err = PTR_ERR(data->psy);
+ goto remove;
+ }
platform_set_drvdata(pdev, data);
pwm_disable_control();
data = platform_get_drvdata(pdev);
- power_supply_unregister(&data->psy);
+ power_supply_unregister(data->psy);
sysfs_remove_group(&pdev->dev.kobj, &compal_platform_attr_group);
*/
for_each_pci_dev(pdev) {
for (i = 0; i < DEVICE_COUNT_RESOURCE; i++) {
- unsigned long type;
+ unsigned long flags, type;
- type = pci_resource_flags(pdev, i) &
- (IORESOURCE_IO | IORESOURCE_MEM);
+ flags = pci_resource_flags(pdev, i);
+ type = flags & (IORESOURCE_IO | IORESOURCE_MEM);
if (!type || pci_resource_len(pdev, i) == 0)
continue;
+ if (flags & IORESOURCE_UNSET)
+ continue;
+
pci_start = pci_resource_start(pdev, i);
pci_end = pci_resource_end(pdev, i);
for (j = 0;
struct i2c_client *i2c;
struct device *dev;
- struct power_supply battery;
+ struct power_supply *battery;
struct mutex lock;
int status;
int irq_cc;
static void pm860x_external_power_changed(struct power_supply *psy)
{
- struct pm860x_battery_info *info;
+ struct pm860x_battery_info *info = dev_get_drvdata(psy->dev.parent);
- info = container_of(psy, struct pm860x_battery_info, battery);
calc_resistor(info);
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pm860x_battery_info *info = dev_get_drvdata(psy->dev->parent);
+ struct pm860x_battery_info *info = dev_get_drvdata(psy->dev.parent);
int data;
int ret;
enum power_supply_property psp,
const union power_supply_propval *val)
{
- struct pm860x_battery_info *info = dev_get_drvdata(psy->dev->parent);
+ struct pm860x_battery_info *info = dev_get_drvdata(psy->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_CHARGE_FULL:
POWER_SUPPLY_PROP_TEMP,
};
+static const struct power_supply_desc pm860x_battery_desc = {
+ .name = "battery-monitor",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = pm860x_batt_props,
+ .num_properties = ARRAY_SIZE(pm860x_batt_props),
+ .get_property = pm860x_batt_get_prop,
+ .set_property = pm860x_batt_set_prop,
+ .external_power_changed = pm860x_external_power_changed,
+};
+
static int pm860x_battery_probe(struct platform_device *pdev)
{
struct pm860x_chip *chip = dev_get_drvdata(pdev->dev.parent);
pm860x_init_battery(info);
- info->battery.name = "battery-monitor";
- info->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- info->battery.properties = pm860x_batt_props;
- info->battery.num_properties = ARRAY_SIZE(pm860x_batt_props);
- info->battery.get_property = pm860x_batt_get_prop;
- info->battery.set_property = pm860x_batt_set_prop;
- info->battery.external_power_changed = pm860x_external_power_changed;
-
if (pdata && pdata->max_capacity)
info->max_capacity = pdata->max_capacity;
else
else
info->resistor = 300; /* set default internal resistor */
- ret = power_supply_register(&pdev->dev, &info->battery);
- if (ret)
- return ret;
- info->battery.dev->parent = &pdev->dev;
+ info->battery = power_supply_register(&pdev->dev, &pm860x_battery_desc,
+ NULL);
+ if (IS_ERR(info->battery))
+ return PTR_ERR(info->battery);
+ info->battery->dev.parent = &pdev->dev;
ret = request_threaded_irq(info->irq_cc, NULL,
pm860x_coulomb_handler, IRQF_ONESHOT,
out_coulomb:
free_irq(info->irq_cc, info);
out_reg:
- power_supply_unregister(&info->battery);
+ power_supply_unregister(info->battery);
return ret;
}
free_irq(info->irq_batt, info);
free_irq(info->irq_cc, info);
- power_supply_unregister(&info->battery);
+ power_supply_unregister(info->battery);
return 0;
}
struct i2c_client *i2c_8606;
struct device *dev;
- struct power_supply usb;
+ struct power_supply *usb;
struct mutex lock;
int irq_nums;
int irq[7];
psy = power_supply_get_by_name(pm860x_supplied_to[0]);
if (!psy)
return -EINVAL;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_VOLTAGE_NOW, &data);
- if (ret)
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ &data);
+ if (ret) {
+ power_supply_put(psy);
return ret;
+ }
vbatt = data.intval / 1000;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_PRESENT, &data);
- if (ret)
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_PRESENT, &data);
+ if (ret) {
+ power_supply_put(psy);
return ret;
+ }
+ power_supply_put(psy);
mutex_lock(&info->lock);
info->present = data.intval;
set_charging_fsm(info);
- power_supply_changed(&info->usb);
+ power_supply_changed(info->usb);
out:
return IRQ_HANDLED;
}
psy = power_supply_get_by_name(pm860x_supplied_to[0]);
if (!psy)
goto out;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_TEMP, &temp);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_TEMP, &temp);
if (ret)
goto out;
value = temp.intval / 10;
set_charging_fsm(info);
out:
+ power_supply_put(psy);
return IRQ_HANDLED;
}
psy = power_supply_get_by_name(pm860x_supplied_to[0]);
if (!psy)
goto out;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_VOLTAGE_NOW, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ &val);
if (ret)
- goto out;
+ goto out_psy_put;
vbatt = val.intval / 1000;
/*
* CHG_DONE interrupt is faster than CHG_DET interrupt when
*/
ret = pm860x_reg_read(info->i2c, PM8607_STATUS_2);
if (ret < 0)
- goto out;
+ goto out_psy_put;
if (vbatt > CHARGE_THRESHOLD && ret & STATUS2_CHG)
- psy->set_property(psy, POWER_SUPPLY_PROP_CHARGE_FULL, &val);
+ power_supply_set_property(psy, POWER_SUPPLY_PROP_CHARGE_FULL,
+ &val);
+out_psy_put:
+ power_supply_put(psy);
out:
mutex_unlock(&info->lock);
dev_dbg(info->dev, "%s, Allowed: %d\n", __func__, info->allowed);
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pm860x_charger_info *info =
- dev_get_drvdata(psy->dev->parent);
+ struct pm860x_charger_info *info = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
{ "vchg", pm860x_vchg_handler },
};
+static const struct power_supply_desc pm860x_charger_desc = {
+ .name = "usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = pm860x_usb_props,
+ .num_properties = ARRAY_SIZE(pm860x_usb_props),
+ .get_property = pm860x_usb_get_prop,
+};
+
static int pm860x_charger_probe(struct platform_device *pdev)
{
struct pm860x_chip *chip = dev_get_drvdata(pdev->dev.parent);
+ struct power_supply_config psy_cfg = {};
struct pm860x_charger_info *info;
int ret;
int count;
mutex_init(&info->lock);
platform_set_drvdata(pdev, info);
- info->usb.name = "usb";
- info->usb.type = POWER_SUPPLY_TYPE_USB;
- info->usb.supplied_to = pm860x_supplied_to;
- info->usb.num_supplicants = ARRAY_SIZE(pm860x_supplied_to);
- info->usb.properties = pm860x_usb_props;
- info->usb.num_properties = ARRAY_SIZE(pm860x_usb_props);
- info->usb.get_property = pm860x_usb_get_prop;
- ret = power_supply_register(&pdev->dev, &info->usb);
- if (ret)
+ psy_cfg.drv_data = info;
+ psy_cfg.supplied_to = pm860x_supplied_to;
+ psy_cfg.num_supplicants = ARRAY_SIZE(pm860x_supplied_to);
+ info->usb = power_supply_register(&pdev->dev, &pm860x_charger_desc,
+ &psy_cfg);
+ if (IS_ERR(info->usb)) {
+ ret = PTR_ERR(info->usb);
goto out;
+ }
pm860x_init_charger(info);
return 0;
out_irq:
- power_supply_unregister(&info->usb);
+ power_supply_unregister(info->usb);
while (--i >= 0)
free_irq(info->irq[i], info);
out:
struct pm860x_charger_info *info = platform_get_drvdata(pdev);
int i;
- power_supply_unregister(&info->usb);
+ power_supply_unregister(info->usb);
free_irq(info->irq[0], info);
for (i = 0; i < info->irq_nums; i++)
free_irq(info->irq[i], info);
Say Y here to enable support for batteries charger integrated into
DA9052 PMIC.
+config CHARGER_DA9150
+ tristate "Dialog Semiconductor DA9150 Charger support"
+ depends on MFD_DA9150
+ depends on DA9150_GPADC
+ depends on IIO
+ help
+ Say Y here to enable support for charger unit of the DA9150
+ Integrated Charger & Fuel-Gauge IC.
+
+ This driver can also be built as a module. If so, the module will be
+ called da9150-charger.
+
+config AXP288_FUEL_GAUGE
+ tristate "X-Powers AXP288 Fuel Gauge"
+ depends on MFD_AXP20X && IIO
+ help
+ Say yes here to have support for X-Power power management IC (PMIC)
+ Fuel Gauge. The device provides battery statistics and status
+ monitoring as well as alerts for battery over/under voltage and
+ over/under temperature.
+
config BATTERY_MAX17040
tristate "Maxim MAX17040 Fuel Gauge"
depends on I2C
-ccflags-$(CONFIG_POWER_SUPPLY_DEBUG) := -DDEBUG
+subdir-ccflags-$(CONFIG_POWER_SUPPLY_DEBUG) := -DDEBUG
power_supply-y := power_supply_core.o
power_supply-$(CONFIG_SYSFS) += power_supply_sysfs.o
obj-$(CONFIG_BATTERY_BQ27x00) += bq27x00_battery.o
obj-$(CONFIG_BATTERY_DA9030) += da9030_battery.o
obj-$(CONFIG_BATTERY_DA9052) += da9052-battery.o
+obj-$(CONFIG_CHARGER_DA9150) += da9150-charger.o
obj-$(CONFIG_BATTERY_MAX17040) += max17040_battery.o
obj-$(CONFIG_BATTERY_MAX17042) += max17042_battery.o
obj-$(CONFIG_BATTERY_Z2) += z2_battery.o
obj-$(CONFIG_CHARGER_SMB347) += smb347-charger.o
obj-$(CONFIG_CHARGER_TPS65090) += tps65090-charger.o
obj-$(CONFIG_POWER_RESET) += reset/
+obj-$(CONFIG_AXP288_FUEL_GAUGE) += axp288_fuel_gauge.o
#define BTEMP_BATCTRL_CURR_SRC_60UA 60
#define BTEMP_BATCTRL_CURR_SRC_120UA 120
-#define to_ab8500_btemp_device_info(x) container_of((x), \
- struct ab8500_btemp, btemp_psy);
-
/**
* struct ab8500_btemp_interrupts - ab8500 interrupts
* @name: name of the interrupt
struct ab8500_gpadc *gpadc;
struct ab8500_fg *fg;
struct abx500_bm_data *bm;
- struct power_supply btemp_psy;
+ struct power_supply *btemp_psy;
struct ab8500_btemp_events events;
struct ab8500_btemp_ranges btemp_ranges;
struct workqueue_struct *btemp_wq;
if ((di->bat_temp != di->prev_bat_temp) || !di->initialized) {
di->initialized = true;
di->bat_temp = bat_temp;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
}
} else if (bat_temp < di->prev_bat_temp) {
di->bat_temp--;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
} else if (bat_temp > di->prev_bat_temp) {
di->bat_temp++;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
}
di->prev_bat_temp = bat_temp;
dev_err(di->dev, "Battery removal detected!\n");
di->events.batt_rem = true;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
return IRQ_HANDLED;
}
di->events.btemp_high = false;
di->events.btemp_medhigh = false;
di->events.btemp_lowmed = false;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
}
return IRQ_HANDLED;
di->events.btemp_medhigh = false;
di->events.btemp_lowmed = false;
di->events.btemp_low = false;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
return IRQ_HANDLED;
}
di->events.btemp_medhigh = false;
di->events.btemp_high = false;
di->events.btemp_low = false;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
return IRQ_HANDLED;
}
di->events.btemp_lowmed = false;
di->events.btemp_high = false;
di->events.btemp_low = false;
- power_supply_changed(&di->btemp_psy);
+ power_supply_changed(di->btemp_psy);
return IRQ_HANDLED;
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct ab8500_btemp *di;
-
- di = to_ab8500_btemp_device_info(psy);
+ struct ab8500_btemp *di = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_PRESENT:
psy = (struct power_supply *)data;
ext = dev_get_drvdata(dev);
- di = to_ab8500_btemp_device_info(psy);
+ di = power_supply_get_drvdata(psy);
/*
* For all psy where the name of your driver
* appears in any supplied_to
*/
for (i = 0; i < ext->num_supplicants; i++) {
- if (!strcmp(ext->supplied_to[i], psy->name))
+ if (!strcmp(ext->supplied_to[i], psy->desc->name))
psy_found = true;
}
return 0;
/* Go through all properties for the psy */
- for (j = 0; j < ext->num_properties; j++) {
+ for (j = 0; j < ext->desc->num_properties; j++) {
enum power_supply_property prop;
- prop = ext->properties[j];
+ prop = ext->desc->properties[j];
- if (ext->get_property(ext, prop, &ret))
+ if (power_supply_get_property(ext, prop, &ret))
continue;
switch (prop) {
case POWER_SUPPLY_PROP_PRESENT:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_MAINS:
/* AC disconnected */
if (!ret.intval && di->events.ac_conn) {
*/
static void ab8500_btemp_external_power_changed(struct power_supply *psy)
{
- struct ab8500_btemp *di = to_ab8500_btemp_device_info(psy);
+ struct ab8500_btemp *di = power_supply_get_drvdata(psy);
class_for_each_device(power_supply_class, NULL,
- &di->btemp_psy, ab8500_btemp_get_ext_psy_data);
+ di->btemp_psy, ab8500_btemp_get_ext_psy_data);
}
/* ab8500 btemp driver interrupts and their respective isr */
destroy_workqueue(di->btemp_wq);
flush_scheduled_work();
- power_supply_unregister(&di->btemp_psy);
+ power_supply_unregister(di->btemp_psy);
return 0;
}
"ab8500_fg",
};
+static const struct power_supply_desc ab8500_btemp_desc = {
+ .name = "ab8500_btemp",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = ab8500_btemp_props,
+ .num_properties = ARRAY_SIZE(ab8500_btemp_props),
+ .get_property = ab8500_btemp_get_property,
+ .external_power_changed = ab8500_btemp_external_power_changed,
+};
+
static int ab8500_btemp_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct abx500_bm_data *plat = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct ab8500_btemp *di;
int irq, i, ret = 0;
u8 val;
di->initialized = false;
- /* BTEMP supply */
- di->btemp_psy.name = "ab8500_btemp";
- di->btemp_psy.type = POWER_SUPPLY_TYPE_BATTERY;
- di->btemp_psy.properties = ab8500_btemp_props;
- di->btemp_psy.num_properties = ARRAY_SIZE(ab8500_btemp_props);
- di->btemp_psy.get_property = ab8500_btemp_get_property;
- di->btemp_psy.supplied_to = supply_interface;
- di->btemp_psy.num_supplicants = ARRAY_SIZE(supply_interface);
- di->btemp_psy.external_power_changed =
- ab8500_btemp_external_power_changed;
-
+ psy_cfg.supplied_to = supply_interface;
+ psy_cfg.num_supplicants = ARRAY_SIZE(supply_interface);
+ psy_cfg.drv_data = di;
/* Create a work queue for the btemp */
di->btemp_wq =
}
/* Register BTEMP power supply class */
- ret = power_supply_register(di->dev, &di->btemp_psy);
- if (ret) {
+ di->btemp_psy = power_supply_register(di->dev, &ab8500_btemp_desc,
+ &psy_cfg);
+ if (IS_ERR(di->btemp_psy)) {
dev_err(di->dev, "failed to register BTEMP psy\n");
+ ret = PTR_ERR(di->btemp_psy);
goto free_btemp_wq;
}
return ret;
free_irq:
- power_supply_unregister(&di->btemp_psy);
+ power_supply_unregister(di->btemp_psy);
/* We also have to free all successfully registered irqs */
for (i = i - 1; i >= 0; i--) {
if (!connected)
di->flags.vbus_drop_end = false;
- sysfs_notify(&di->usb_chg.psy.dev->kobj, NULL, "present");
+ sysfs_notify(&di->usb_chg.psy->dev.kobj, NULL, "present");
if (connected) {
mutex_lock(&di->charger_attached_mutex);
dev_dbg(di->dev, "%s Disabled AC charging\n", __func__);
}
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
return ret;
}
cancel_delayed_work(&di->check_vbat_work);
}
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
return ret;
}
int ret;
struct ab8500_charger *di;
- if (charger->psy.type == POWER_SUPPLY_TYPE_MAINS)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_MAINS)
di = to_ab8500_charger_ac_device_info(charger);
- else if (charger->psy.type == POWER_SUPPLY_TYPE_USB)
+ else if (charger->psy->desc->type == POWER_SUPPLY_TYPE_USB)
di = to_ab8500_charger_usb_device_info(charger);
else
return -ENXIO;
int ret;
struct ab8500_charger *di;
- if (charger->psy.type == POWER_SUPPLY_TYPE_MAINS)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_MAINS)
di = to_ab8500_charger_ac_device_info(charger);
- else if (charger->psy.type == POWER_SUPPLY_TYPE_USB)
+ else if (charger->psy->desc->type == POWER_SUPPLY_TYPE_USB)
di = to_ab8500_charger_usb_device_info(charger);
else
return -ENXIO;
int ret;
struct ab8500_charger *di;
- if (charger->psy.type == POWER_SUPPLY_TYPE_USB)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_USB)
di = to_ab8500_charger_usb_device_info(charger);
else
return -ENXIO;
int ret;
struct ab8500_charger *di;
- if (charger->psy.type == POWER_SUPPLY_TYPE_USB)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_USB)
di = to_ab8500_charger_usb_device_info(charger);
else
return -ENXIO;
struct ux500_charger *usb_chg;
usb_chg = (struct ux500_charger *)data;
- psy = &usb_chg->psy;
+ psy = usb_chg->psy;
di = to_ab8500_charger_usb_device_info(usb_chg);
/* For all psy where the driver name appears in any supplied_to */
for (i = 0; i < ext->num_supplicants; i++) {
- if (!strcmp(ext->supplied_to[i], psy->name))
+ if (!strcmp(ext->supplied_to[i], psy->desc->name))
psy_found = true;
}
return 0;
/* Go through all properties for the psy */
- for (j = 0; j < ext->num_properties; j++) {
+ for (j = 0; j < ext->desc->num_properties; j++) {
enum power_supply_property prop;
- prop = ext->properties[j];
+ prop = ext->desc->properties[j];
- if (ext->get_property(ext, prop, &ret))
+ if (power_supply_get_property(ext, prop, &ret))
continue;
switch (prop) {
case POWER_SUPPLY_PROP_VOLTAGE_NOW:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
di->vbat = ret.intval / 1000;
break;
struct ab8500_charger, check_vbat_work.work);
class_for_each_device(power_supply_class, NULL,
- &di->usb_chg.psy, ab8500_charger_get_ext_psy_data);
+ di->usb_chg.psy, ab8500_charger_get_ext_psy_data);
/* First run old_vbat is 0. */
if (di->old_vbat == 0)
di->vbat, di->old_vbat);
ab8500_charger_set_vbus_in_curr(di,
di->max_usb_in_curr.usb_type_max);
- power_supply_changed(&di->usb_chg.psy);
+ power_supply_changed(di->usb_chg.psy);
}
di->old_vbat = di->vbat;
}
if (!(reg_value & MAIN_CH_NOK)) {
di->flags.mainextchnotok = false;
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
}
}
if (di->flags.vbus_ovv) {
}
if (!(reg_value & VBUS_OVV_TH)) {
di->flags.vbus_ovv = false;
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
}
/* If we still have a failure, schedule a new check */
di->ac.charger_connected = 0;
}
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
- sysfs_notify(&di->ac_chg.psy.dev->kobj, NULL, "present");
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
+ sysfs_notify(&di->ac_chg.psy->dev.kobj, NULL, "present");
}
static void ab8500_charger_usb_attached_work(struct work_struct *work)
dev_dbg(di->dev, "%s di->vbus_detected = false\n", __func__);
di->vbus_detected = false;
ab8500_charger_set_usb_connected(di, false);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
} else {
dev_dbg(di->dev, "%s di->vbus_detected = true\n", __func__);
di->vbus_detected = true;
if (!ret) {
ab8500_charger_set_usb_connected(di, true);
ab8500_power_supply_changed(di,
- &di->usb_chg.psy);
+ di->usb_chg.psy);
}
} else {
/*
ab8500_charger_set_usb_connected(di,
true);
ab8500_power_supply_changed(di,
- &di->usb_chg.psy);
+ di->usb_chg.psy);
}
}
}
}
ab8500_charger_set_usb_connected(di, true);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
/**
if (!(detected_chargers & USB_PW_CONN)) {
di->vbus_detected = false;
ab8500_charger_set_usb_connected(di, false);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
return;
}
if (ret == -ENXIO) {
/* No valid charger type detected */
ab8500_charger_set_usb_connected(di, false);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
return;
}
case AB8500_BM_USB_STATE_SUSPEND:
case AB8500_BM_USB_STATE_MAX:
ab8500_charger_set_usb_connected(di, false);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
break;
case AB8500_BM_USB_STATE_RESUME:
return;
ab8500_charger_set_usb_connected(di, true);
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
break;
}
if (prev_status != di->flags.usbchargernotok)
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
/**
else
di->flags.main_thermal_prot = false;
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
}
/**
else
di->flags.usb_thermal_prot = false;
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
/**
dev_dbg(di->dev, "Main charger not ok\n");
di->flags.mainextchnotok = true;
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
/* Schedule a new HW failure check */
queue_delayed_work(di->charger_wq, &di->check_hw_failure_work, 0);
*/
if (di->ac.charger_online) {
di->ac.wd_expired = true;
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
}
if (di->usb.charger_online) {
di->usb.wd_expired = true;
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
}
return IRQ_HANDLED;
dev_dbg(di->dev, "VBUS overvoltage detected\n");
di->flags.vbus_ovv = true;
- ab8500_power_supply_changed(di, &di->usb_chg.psy);
+ ab8500_power_supply_changed(di, di->usb_chg.psy);
/* Schedule a new HW failure check */
queue_delayed_work(di->charger_wq, &di->check_hw_failure_work, 0);
flush_scheduled_work();
if (di->usb_chg.enabled)
- power_supply_unregister(&di->usb_chg.psy);
+ power_supply_unregister(di->usb_chg.psy);
if (di->ac_chg.enabled && !di->ac_chg.external)
- power_supply_unregister(&di->ac_chg.psy);
+ power_supply_unregister(di->ac_chg.psy);
return 0;
}
"ab8500_btemp",
};
+static const struct power_supply_desc ab8500_ac_chg_desc = {
+ .name = "ab8500_ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = ab8500_charger_ac_props,
+ .num_properties = ARRAY_SIZE(ab8500_charger_ac_props),
+ .get_property = ab8500_charger_ac_get_property,
+};
+
+static const struct power_supply_desc ab8500_usb_chg_desc = {
+ .name = "ab8500_usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = ab8500_charger_usb_props,
+ .num_properties = ARRAY_SIZE(ab8500_charger_usb_props),
+ .get_property = ab8500_charger_usb_get_property,
+};
+
static int ab8500_charger_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct abx500_bm_data *plat = pdev->dev.platform_data;
+ struct power_supply_config ac_psy_cfg = {}, usb_psy_cfg = {};
struct ab8500_charger *di;
int irq, i, charger_status, ret = 0, ch_stat;
di->autopower = false;
di->invalid_charger_detect_state = 0;
+ /* AC and USB supply config */
+ ac_psy_cfg.supplied_to = supply_interface;
+ ac_psy_cfg.num_supplicants = ARRAY_SIZE(supply_interface);
+ ac_psy_cfg.drv_data = &di->ac_chg;
+ usb_psy_cfg.supplied_to = supply_interface;
+ usb_psy_cfg.num_supplicants = ARRAY_SIZE(supply_interface);
+ usb_psy_cfg.drv_data = &di->usb_chg;
+
/* AC supply */
- /* power_supply base class */
- di->ac_chg.psy.name = "ab8500_ac";
- di->ac_chg.psy.type = POWER_SUPPLY_TYPE_MAINS;
- di->ac_chg.psy.properties = ab8500_charger_ac_props;
- di->ac_chg.psy.num_properties = ARRAY_SIZE(ab8500_charger_ac_props);
- di->ac_chg.psy.get_property = ab8500_charger_ac_get_property;
- di->ac_chg.psy.supplied_to = supply_interface;
- di->ac_chg.psy.num_supplicants = ARRAY_SIZE(supply_interface),
/* ux500_charger sub-class */
di->ac_chg.ops.enable = &ab8500_charger_ac_en;
di->ac_chg.ops.check_enable = &ab8500_charger_ac_check_enable;
&charger_notifier_list, &charger_nb);
/* USB supply */
- /* power_supply base class */
- di->usb_chg.psy.name = "ab8500_usb";
- di->usb_chg.psy.type = POWER_SUPPLY_TYPE_USB;
- di->usb_chg.psy.properties = ab8500_charger_usb_props;
- di->usb_chg.psy.num_properties = ARRAY_SIZE(ab8500_charger_usb_props);
- di->usb_chg.psy.get_property = ab8500_charger_usb_get_property;
- di->usb_chg.psy.supplied_to = supply_interface;
- di->usb_chg.psy.num_supplicants = ARRAY_SIZE(supply_interface),
/* ux500_charger sub-class */
di->usb_chg.ops.enable = &ab8500_charger_usb_en;
di->usb_chg.ops.check_enable = &ab8500_charger_usb_check_enable;
/* Register AC charger class */
if (di->ac_chg.enabled) {
- ret = power_supply_register(di->dev, &di->ac_chg.psy);
- if (ret) {
+ di->ac_chg.psy = power_supply_register(di->dev,
+ &ab8500_ac_chg_desc,
+ &ac_psy_cfg);
+ if (IS_ERR(di->ac_chg.psy)) {
dev_err(di->dev, "failed to register AC charger\n");
+ ret = PTR_ERR(di->ac_chg.psy);
goto free_charger_wq;
}
}
/* Register USB charger class */
if (di->usb_chg.enabled) {
- ret = power_supply_register(di->dev, &di->usb_chg.psy);
- if (ret) {
+ di->usb_chg.psy = power_supply_register(di->dev,
+ &ab8500_usb_chg_desc,
+ &usb_psy_cfg);
+ if (IS_ERR(di->usb_chg.psy)) {
dev_err(di->dev, "failed to register USB charger\n");
+ ret = PTR_ERR(di->usb_chg.psy);
goto free_ac;
}
}
if (charger_status & AC_PW_CONN) {
di->ac.charger_connected = 1;
di->ac_conn = true;
- ab8500_power_supply_changed(di, &di->ac_chg.psy);
- sysfs_notify(&di->ac_chg.psy.dev->kobj, NULL, "present");
+ ab8500_power_supply_changed(di, di->ac_chg.psy);
+ sysfs_notify(&di->ac_chg.psy->dev.kobj, NULL, "present");
}
if (charger_status & USB_PW_CONN) {
usb_put_phy(di->usb_phy);
free_usb:
if (di->usb_chg.enabled)
- power_supply_unregister(&di->usb_chg.psy);
+ power_supply_unregister(di->usb_chg.psy);
free_ac:
if (di->ac_chg.enabled)
- power_supply_unregister(&di->ac_chg.psy);
+ power_supply_unregister(di->ac_chg.psy);
free_charger_wq:
destroy_workqueue(di->charger_wq);
return ret;
#define interpolate(x, x1, y1, x2, y2) \
((y1) + ((((y2) - (y1)) * ((x) - (x1))) / ((x2) - (x1))));
-#define to_ab8500_fg_device_info(x) container_of((x), \
- struct ab8500_fg, fg_psy);
-
/**
* struct ab8500_fg_interrupts - ab8500 fg interupts
* @name: name of the interrupt
struct ab8500 *parent;
struct ab8500_gpadc *gpadc;
struct abx500_bm_data *bm;
- struct power_supply fg_psy;
+ struct power_supply *fg_psy;
struct workqueue_struct *fg_wq;
struct delayed_work fg_periodic_work;
struct delayed_work fg_low_bat_work;
u8 low, high;
int val;
int ret;
- int timeout;
+ unsigned long timeout;
if (!completion_done(&di->ab8500_fg_complete)) {
timeout = wait_for_completion_timeout(
&di->ab8500_fg_complete,
INS_CURR_TIMEOUT);
dev_dbg(di->dev, "Finalize time: %d ms\n",
- ((INS_CURR_TIMEOUT - timeout) * 1000) / HZ);
+ jiffies_to_msecs(INS_CURR_TIMEOUT - timeout));
if (!timeout) {
ret = -ETIME;
disable_irq(di->irq);
int ab8500_fg_inst_curr_blocking(struct ab8500_fg *di)
{
int ret;
- int timeout;
+ unsigned long timeout;
int res = 0;
ret = ab8500_fg_inst_curr_start(di);
&di->ab8500_fg_started,
INS_CURR_TIMEOUT);
dev_dbg(di->dev, "Start time: %d ms\n",
- ((INS_CURR_TIMEOUT - timeout) * 1000) / HZ);
+ jiffies_to_msecs(INS_CURR_TIMEOUT - timeout));
if (!timeout) {
ret = -ETIME;
dev_err(di->dev, "completion timed out [%d]\n",
di->bat_cap.prev_percent,
di->bat_cap.cap_scale.scaled_cap);
}
- power_supply_changed(&di->fg_psy);
+ power_supply_changed(di->fg_psy);
if (di->flags.fully_charged && di->flags.force_full) {
dev_dbg(di->dev, "Battery full, notifying.\n");
di->flags.force_full = false;
if (!di->flags.bat_ovv) {
dev_dbg(di->dev, "Battery OVV\n");
di->flags.bat_ovv = true;
- power_supply_changed(&di->fg_psy);
+ power_supply_changed(di->fg_psy);
}
/* Not yet recovered from ovv, reschedule this test */
queue_delayed_work(di->fg_wq, &di->fg_check_hw_failure_work,
} else {
dev_dbg(di->dev, "Battery recovered from OVV\n");
di->flags.bat_ovv = false;
- power_supply_changed(&di->fg_psy);
+ power_supply_changed(di->fg_psy);
}
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
/*
* If battery is identified as unknown and charging of unknown
psy = (struct power_supply *)data;
ext = dev_get_drvdata(dev);
- di = to_ab8500_fg_device_info(psy);
+ di = power_supply_get_drvdata(psy);
/*
* For all psy where the name of your driver
* appears in any supplied_to
*/
for (i = 0; i < ext->num_supplicants; i++) {
- if (!strcmp(ext->supplied_to[i], psy->name))
+ if (!strcmp(ext->supplied_to[i], psy->desc->name))
psy_found = true;
}
return 0;
/* Go through all properties for the psy */
- for (j = 0; j < ext->num_properties; j++) {
+ for (j = 0; j < ext->desc->num_properties; j++) {
enum power_supply_property prop;
- prop = ext->properties[j];
+ prop = ext->desc->properties[j];
- if (ext->get_property(ext, prop, &ret))
+ if (power_supply_get_property(ext, prop, &ret))
continue;
switch (prop) {
case POWER_SUPPLY_PROP_STATUS:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
switch (ret.intval) {
case POWER_SUPPLY_STATUS_UNKNOWN:
};
break;
case POWER_SUPPLY_PROP_TECHNOLOGY:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
if (!di->flags.batt_id_received &&
di->bm->batt_id != BATTERY_UNKNOWN) {
}
break;
case POWER_SUPPLY_PROP_TEMP:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
if (di->flags.batt_id_received)
di->bat_temp = ret.intval;
*/
static void ab8500_fg_external_power_changed(struct power_supply *psy)
{
- struct ab8500_fg *di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
class_for_each_device(power_supply_class, NULL,
- &di->fg_psy, ab8500_fg_get_ext_psy_data);
+ di->fg_psy, ab8500_fg_get_ext_psy_data);
}
/**
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_FLAG_TIME_REG, ®_value);
int ret;
long unsigned reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
reg_value = simple_strtoul(buf, NULL, 10);
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_MAX_TIME_REG, ®_value);
int ret;
int reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
reg_value = simple_strtoul(buf, NULL, 10);
if (reg_value > 0x7F) {
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_RESTART_REG, ®_value);
int ret;
int reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
reg_value = simple_strtoul(buf, NULL, 10);
if (reg_value > 0xF) {
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_TIME_REG, ®_value);
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_RESTART_REG, ®_value);
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_CTL_STATUS_REG, ®_value);
int ret;
int reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
reg_value = simple_strtoul(buf, NULL, 10);
if (reg_value > 0x1) {
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_CTL_STATUS_REG, ®_value);
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_DEBOUNCE_REG, ®_value);
int ret;
int reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
reg_value = simple_strtoul(buf, NULL, 10);
if (reg_value > 0x7) {
int ret;
u8 reg_value;
struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
+ struct ab8500_fg *di = power_supply_get_drvdata(psy);
ret = abx500_get_register_interruptible(di->dev, AB8500_RTC,
AB8505_RTC_PCUT_CTL_STATUS_REG, ®_value);
ab8505_powercut_enable_status_read, NULL),
};
-static int ab8500_fg_sysfs_psy_create_attrs(struct device *dev)
+static int ab8500_fg_sysfs_psy_create_attrs(struct ab8500_fg *di)
{
unsigned int i;
- struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
if (((is_ab8505(di->parent) || is_ab9540(di->parent)) &&
- abx500_get_chip_id(dev->parent) >= AB8500_CUT2P0)
+ abx500_get_chip_id(di->dev) >= AB8500_CUT2P0)
|| is_ab8540(di->parent)) {
for (i = 0; i < ARRAY_SIZE(ab8505_fg_sysfs_psy_attrs); i++)
- if (device_create_file(dev,
+ if (device_create_file(&di->fg_psy->dev,
&ab8505_fg_sysfs_psy_attrs[i]))
goto sysfs_psy_create_attrs_failed_ab8505;
}
return 0;
sysfs_psy_create_attrs_failed_ab8505:
- dev_err(dev, "Failed creating sysfs psy attrs for ab8505.\n");
+ dev_err(&di->fg_psy->dev, "Failed creating sysfs psy attrs for ab8505.\n");
while (i--)
- device_remove_file(dev, &ab8505_fg_sysfs_psy_attrs[i]);
+ device_remove_file(&di->fg_psy->dev,
+ &ab8505_fg_sysfs_psy_attrs[i]);
return -EIO;
}
-static void ab8500_fg_sysfs_psy_remove_attrs(struct device *dev)
+static void ab8500_fg_sysfs_psy_remove_attrs(struct ab8500_fg *di)
{
unsigned int i;
- struct power_supply *psy = dev_get_drvdata(dev);
- struct ab8500_fg *di;
-
- di = to_ab8500_fg_device_info(psy);
if (((is_ab8505(di->parent) || is_ab9540(di->parent)) &&
- abx500_get_chip_id(dev->parent) >= AB8500_CUT2P0)
+ abx500_get_chip_id(di->dev) >= AB8500_CUT2P0)
|| is_ab8540(di->parent)) {
for (i = 0; i < ARRAY_SIZE(ab8505_fg_sysfs_psy_attrs); i++)
- (void)device_remove_file(dev, &ab8505_fg_sysfs_psy_attrs[i]);
+ (void)device_remove_file(&di->fg_psy->dev,
+ &ab8505_fg_sysfs_psy_attrs[i]);
}
}
ab8500_fg_sysfs_exit(di);
flush_scheduled_work();
- ab8500_fg_sysfs_psy_remove_attrs(di->fg_psy.dev);
- power_supply_unregister(&di->fg_psy);
+ ab8500_fg_sysfs_psy_remove_attrs(di);
+ power_supply_unregister(di->fg_psy);
return ret;
}
/* ab8500 fg driver interrupts and their respective isr */
-static struct ab8500_fg_interrupts ab8500_fg_irq[] = {
+static struct ab8500_fg_interrupts ab8500_fg_irq_th[] = {
{"NCONV_ACCU", ab8500_fg_cc_convend_handler},
{"BATT_OVV", ab8500_fg_batt_ovv_handler},
{"LOW_BAT_F", ab8500_fg_lowbatf_handler},
{"CC_INT_CALIB", ab8500_fg_cc_int_calib_handler},
+};
+
+static struct ab8500_fg_interrupts ab8500_fg_irq_bh[] = {
{"CCEOC", ab8500_fg_cc_data_end_handler},
};
"ab8500_usb",
};
+static const struct power_supply_desc ab8500_fg_desc = {
+ .name = "ab8500_fg",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = ab8500_fg_props,
+ .num_properties = ARRAY_SIZE(ab8500_fg_props),
+ .get_property = ab8500_fg_get_property,
+ .external_power_changed = ab8500_fg_external_power_changed,
+};
+
static int ab8500_fg_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct abx500_bm_data *plat = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct ab8500_fg *di;
int i, irq;
int ret = 0;
di->parent = dev_get_drvdata(pdev->dev.parent);
di->gpadc = ab8500_gpadc_get("ab8500-gpadc.0");
- di->fg_psy.name = "ab8500_fg";
- di->fg_psy.type = POWER_SUPPLY_TYPE_BATTERY;
- di->fg_psy.properties = ab8500_fg_props;
- di->fg_psy.num_properties = ARRAY_SIZE(ab8500_fg_props);
- di->fg_psy.get_property = ab8500_fg_get_property;
- di->fg_psy.supplied_to = supply_interface;
- di->fg_psy.num_supplicants = ARRAY_SIZE(supply_interface),
- di->fg_psy.external_power_changed = ab8500_fg_external_power_changed;
+ psy_cfg.supplied_to = supply_interface;
+ psy_cfg.num_supplicants = ARRAY_SIZE(supply_interface);
+ psy_cfg.drv_data = di;
di->bat_cap.max_mah_design = MILLI_TO_MICRO *
di->bm->bat_type[di->bm->batt_id].charge_full_design;
di->flags.batt_id_received = false;
/* Register FG power supply class */
- ret = power_supply_register(di->dev, &di->fg_psy);
- if (ret) {
+ di->fg_psy = power_supply_register(di->dev, &ab8500_fg_desc, &psy_cfg);
+ if (IS_ERR(di->fg_psy)) {
dev_err(di->dev, "failed to register FG psy\n");
+ ret = PTR_ERR(di->fg_psy);
goto free_inst_curr_wq;
}
init_completion(&di->ab8500_fg_started);
init_completion(&di->ab8500_fg_complete);
- /* Register interrupts */
- for (i = 0; i < ARRAY_SIZE(ab8500_fg_irq); i++) {
- irq = platform_get_irq_byname(pdev, ab8500_fg_irq[i].name);
- ret = request_threaded_irq(irq, NULL, ab8500_fg_irq[i].isr,
- IRQF_SHARED | IRQF_NO_SUSPEND,
- ab8500_fg_irq[i].name, di);
+ /* Register primary interrupt handlers */
+ for (i = 0; i < ARRAY_SIZE(ab8500_fg_irq_th); i++) {
+ irq = platform_get_irq_byname(pdev, ab8500_fg_irq_th[i].name);
+ ret = request_irq(irq, ab8500_fg_irq_th[i].isr,
+ IRQF_SHARED | IRQF_NO_SUSPEND,
+ ab8500_fg_irq_th[i].name, di);
if (ret != 0) {
- dev_err(di->dev, "failed to request %s IRQ %d: %d\n"
- , ab8500_fg_irq[i].name, irq, ret);
+ dev_err(di->dev, "failed to request %s IRQ %d: %d\n",
+ ab8500_fg_irq_th[i].name, irq, ret);
goto free_irq;
}
dev_dbg(di->dev, "Requested %s IRQ %d: %d\n",
- ab8500_fg_irq[i].name, irq, ret);
+ ab8500_fg_irq_th[i].name, irq, ret);
}
+
+ /* Register threaded interrupt handler */
+ irq = platform_get_irq_byname(pdev, ab8500_fg_irq_bh[0].name);
+ ret = request_threaded_irq(irq, NULL, ab8500_fg_irq_bh[0].isr,
+ IRQF_SHARED | IRQF_NO_SUSPEND | IRQF_ONESHOT,
+ ab8500_fg_irq_bh[0].name, di);
+
+ if (ret != 0) {
+ dev_err(di->dev, "failed to request %s IRQ %d: %d\n",
+ ab8500_fg_irq_bh[0].name, irq, ret);
+ goto free_irq;
+ }
+ dev_dbg(di->dev, "Requested %s IRQ %d: %d\n",
+ ab8500_fg_irq_bh[0].name, irq, ret);
+
di->irq = platform_get_irq_byname(pdev, "CCEOC");
disable_irq(di->irq);
di->nbr_cceoc_irq_cnt = 0;
goto free_irq;
}
- ret = ab8500_fg_sysfs_psy_create_attrs(di->fg_psy.dev);
+ ret = ab8500_fg_sysfs_psy_create_attrs(di);
if (ret) {
dev_err(di->dev, "failed to create FG psy\n");
ab8500_fg_sysfs_exit(di);
return ret;
free_irq:
- power_supply_unregister(&di->fg_psy);
+ power_supply_unregister(di->fg_psy);
- /* We also have to free all successfully registered irqs */
- for (i = i - 1; i >= 0; i--) {
- irq = platform_get_irq_byname(pdev, ab8500_fg_irq[i].name);
+ /* We also have to free all registered irqs */
+ for (i = 0; i < ARRAY_SIZE(ab8500_fg_irq_th); i++) {
+ irq = platform_get_irq_byname(pdev, ab8500_fg_irq_th[i].name);
free_irq(irq, di);
}
+ irq = platform_get_irq_byname(pdev, ab8500_fg_irq_bh[0].name);
+ free_irq(irq, di);
free_inst_curr_wq:
destroy_workqueue(di->fg_wq);
return ret;
#define CHARGALG_CURR_STEP_LOW 0
#define CHARGALG_CURR_STEP_HIGH 100
-#define to_abx500_chargalg_device_info(x) container_of((x), \
- struct abx500_chargalg, chargalg_psy);
-
enum abx500_chargers {
NO_CHG,
AC_CHG,
struct ab8500 *parent;
struct abx500_chargalg_current_step_status curr_status;
struct abx500_bm_data *bm;
- struct power_supply chargalg_psy;
+ struct power_supply *chargalg_psy;
struct ux500_charger *ac_chg;
struct ux500_charger *usb_chg;
struct abx500_chargalg_events events;
di->charge_status = POWER_SUPPLY_STATUS_NOT_CHARGING;
di->maintenance_chg = false;
cancel_delayed_work(&di->chargalg_wd_work);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
}
/**
di->charge_status = POWER_SUPPLY_STATUS_CHARGING;
di->maintenance_chg = false;
cancel_delayed_work(&di->chargalg_wd_work);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
}
/**
di->charge_status = POWER_SUPPLY_STATUS_FULL;
di->maintenance_chg = true;
dev_dbg(di->dev, "EOC reached!\n");
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
} else {
dev_dbg(di->dev,
" EOC limit reached for the %d"
psy = (struct power_supply *)data;
ext = dev_get_drvdata(dev);
- di = to_abx500_chargalg_device_info(psy);
+ di = power_supply_get_drvdata(psy);
/* For all psy where the driver name appears in any supplied_to */
for (i = 0; i < ext->num_supplicants; i++) {
- if (!strcmp(ext->supplied_to[i], psy->name))
+ if (!strcmp(ext->supplied_to[i], psy->desc->name))
psy_found = true;
}
if (!psy_found)
* property because of handling that sysfs entry on its own, this is
* the place to get the battery capacity.
*/
- if (!ext->get_property(ext, POWER_SUPPLY_PROP_CAPACITY, &ret)) {
+ if (!power_supply_get_property(ext, POWER_SUPPLY_PROP_CAPACITY, &ret)) {
di->batt_data.percent = ret.intval;
capacity_updated = true;
}
/* Go through all properties for the psy */
- for (j = 0; j < ext->num_properties; j++) {
+ for (j = 0; j < ext->desc->num_properties; j++) {
enum power_supply_property prop;
- prop = ext->properties[j];
+ prop = ext->desc->properties[j];
- /* Initialize chargers if not already done */
+ /*
+ * Initialize chargers if not already done.
+ * The ab8500_charger*/
if (!di->ac_chg &&
- ext->type == POWER_SUPPLY_TYPE_MAINS)
+ ext->desc->type == POWER_SUPPLY_TYPE_MAINS)
di->ac_chg = psy_to_ux500_charger(ext);
else if (!di->usb_chg &&
- ext->type == POWER_SUPPLY_TYPE_USB)
+ ext->desc->type == POWER_SUPPLY_TYPE_USB)
di->usb_chg = psy_to_ux500_charger(ext);
- if (ext->get_property(ext, prop, &ret))
+ if (power_supply_get_property(ext, prop, &ret))
continue;
switch (prop) {
case POWER_SUPPLY_PROP_PRESENT:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
/* Battery present */
if (ret.intval)
break;
case POWER_SUPPLY_PROP_ONLINE:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
break;
case POWER_SUPPLY_TYPE_MAINS:
break;
case POWER_SUPPLY_PROP_HEALTH:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
break;
case POWER_SUPPLY_TYPE_MAINS:
break;
case POWER_SUPPLY_PROP_VOLTAGE_NOW:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
di->batt_data.volt = ret.intval / 1000;
break;
break;
case POWER_SUPPLY_PROP_VOLTAGE_AVG:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_MAINS:
/* AVG is used to indicate when we are
* in CV mode */
break;
case POWER_SUPPLY_PROP_TECHNOLOGY:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
if (ret.intval)
di->events.batt_unknown = false;
break;
case POWER_SUPPLY_PROP_CURRENT_NOW:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_MAINS:
di->chg_info.ac_curr =
ret.intval / 1000;
break;
case POWER_SUPPLY_PROP_CURRENT_AVG:
- switch (ext->type) {
+ switch (ext->desc->type) {
case POWER_SUPPLY_TYPE_BATTERY:
di->batt_data.avg_curr = ret.intval / 1000;
break;
*/
static void abx500_chargalg_external_power_changed(struct power_supply *psy)
{
- struct abx500_chargalg *di = to_abx500_chargalg_device_info(psy);
+ struct abx500_chargalg *di = power_supply_get_drvdata(psy);
/*
* Trigger execution of the algorithm instantly and read
/* Collect data from all power_supply class devices */
class_for_each_device(power_supply_class, NULL,
- &di->chargalg_psy, abx500_chargalg_get_ext_psy_data);
+ di->chargalg_psy, abx500_chargalg_get_ext_psy_data);
abx500_chargalg_end_of_charge(di);
abx500_chargalg_check_temp(di);
di->charge_status = POWER_SUPPLY_STATUS_NOT_CHARGING;
di->maintenance_chg = false;
abx500_chargalg_state_to(di, STATE_SUSPENDED);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
/* Intentional fallthrough */
case STATE_SUSPENDED:
di->charge_status = POWER_SUPPLY_STATUS_CHARGING;
di->eoc_cnt = 0;
di->maintenance_chg = false;
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
break;
di->bm->bat_type[
di->bm->batt_id].maint_a_cur_lvl);
abx500_chargalg_state_to(di, STATE_MAINTENANCE_A);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
/* Intentional fallthrough*/
case STATE_MAINTENANCE_A:
di->bm->bat_type[
di->bm->batt_id].maint_b_cur_lvl);
abx500_chargalg_state_to(di, STATE_MAINTENANCE_B);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
/* Intentional fallthrough*/
case STATE_MAINTENANCE_B:
abx500_chargalg_stop_maintenance_timer(di);
di->charge_status = POWER_SUPPLY_STATUS_CHARGING;
abx500_chargalg_state_to(di, STATE_TEMP_LOWHIGH);
- power_supply_changed(&di->chargalg_psy);
+ power_supply_changed(di->chargalg_psy);
/* Intentional fallthrough */
case STATE_TEMP_LOWHIGH:
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct abx500_chargalg *di;
-
- di = to_abx500_chargalg_device_info(psy);
+ struct abx500_chargalg *di = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
/* Delete the work queue */
destroy_workqueue(di->chargalg_wq);
- power_supply_unregister(&di->chargalg_psy);
+ power_supply_unregister(di->chargalg_psy);
return 0;
}
"ab8500_fg",
};
+static const struct power_supply_desc abx500_chargalg_desc = {
+ .name = "abx500_chargalg",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = abx500_chargalg_props,
+ .num_properties = ARRAY_SIZE(abx500_chargalg_props),
+ .get_property = abx500_chargalg_get_property,
+ .external_power_changed = abx500_chargalg_external_power_changed,
+};
+
static int abx500_chargalg_probe(struct platform_device *pdev)
{
struct device_node *np = pdev->dev.of_node;
struct abx500_bm_data *plat = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct abx500_chargalg *di;
int ret = 0;
di->dev = &pdev->dev;
di->parent = dev_get_drvdata(pdev->dev.parent);
- /* chargalg supply */
- di->chargalg_psy.name = "abx500_chargalg";
- di->chargalg_psy.type = POWER_SUPPLY_TYPE_BATTERY;
- di->chargalg_psy.properties = abx500_chargalg_props;
- di->chargalg_psy.num_properties = ARRAY_SIZE(abx500_chargalg_props);
- di->chargalg_psy.get_property = abx500_chargalg_get_property;
- di->chargalg_psy.supplied_to = supply_interface;
- di->chargalg_psy.num_supplicants = ARRAY_SIZE(supply_interface),
- di->chargalg_psy.external_power_changed =
- abx500_chargalg_external_power_changed;
+ psy_cfg.supplied_to = supply_interface;
+ psy_cfg.num_supplicants = ARRAY_SIZE(supply_interface);
+ psy_cfg.drv_data = di;
/* Initilialize safety timer */
hrtimer_init(&di->safety_timer, CLOCK_REALTIME, HRTIMER_MODE_ABS);
di->chg_info.prev_conn_chg = -1;
/* Register chargalg power supply class */
- ret = power_supply_register(di->dev, &di->chargalg_psy);
- if (ret) {
+ di->chargalg_psy = power_supply_register(di->dev, &abx500_chargalg_desc,
+ &psy_cfg);
+ if (IS_ERR(di->chargalg_psy)) {
dev_err(di->dev, "failed to register chargalg psy\n");
+ ret = PTR_ERR(di->chargalg_psy);
goto free_chargalg_wq;
}
return ret;
free_psy:
- power_supply_unregister(&di->chargalg_psy);
+ power_supply_unregister(di->chargalg_psy);
free_chargalg_wq:
destroy_workqueue(di->chargalg_wq);
return ret;
#include <linux/apm-emulation.h>
-#define PSY_PROP(psy, prop, val) (psy->get_property(psy, \
+#define PSY_PROP(psy, prop, val) (power_supply_get_property(psy, \
POWER_SUPPLY_PROP_##prop, val))
-#define _MPSY_PROP(prop, val) (main_battery->get_property(main_battery, \
+#define _MPSY_PROP(prop, val) (power_supply_get_property(main_battery, \
prop, val))
#define MPSY_PROP(prop, val) _MPSY_PROP(POWER_SUPPLY_PROP_##prop, val)
bp->bat = dev_get_drvdata(dev);
- if (bp->bat->use_for_apm) {
+ if (bp->bat->desc->use_for_apm) {
/* nice, we explicitly asked to report this battery. */
bp->main = bp->bat;
return 1;
--- /dev/null
+/*
+ * axp288_fuel_gauge.c - Xpower AXP288 PMIC Fuel Gauge Driver
+ *
+ * Copyright (C) 2014 Intel Corporation
+ *
+ * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; version 2 of the License.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ */
+
+#include <linux/module.h>
+#include <linux/kernel.h>
+#include <linux/device.h>
+#include <linux/regmap.h>
+#include <linux/jiffies.h>
+#include <linux/interrupt.h>
+#include <linux/device.h>
+#include <linux/workqueue.h>
+#include <linux/mfd/axp20x.h>
+#include <linux/platform_device.h>
+#include <linux/power_supply.h>
+#include <linux/iio/consumer.h>
+#include <linux/debugfs.h>
+#include <linux/seq_file.h>
+
+#define CHRG_STAT_BAT_SAFE_MODE (1 << 3)
+#define CHRG_STAT_BAT_VALID (1 << 4)
+#define CHRG_STAT_BAT_PRESENT (1 << 5)
+#define CHRG_STAT_CHARGING (1 << 6)
+#define CHRG_STAT_PMIC_OTP (1 << 7)
+
+#define CHRG_CCCV_CC_MASK 0xf /* 4 bits */
+#define CHRG_CCCV_CC_BIT_POS 0
+#define CHRG_CCCV_CC_OFFSET 200 /* 200mA */
+#define CHRG_CCCV_CC_LSB_RES 200 /* 200mA */
+#define CHRG_CCCV_ITERM_20P (1 << 4) /* 20% of CC */
+#define CHRG_CCCV_CV_MASK 0x60 /* 2 bits */
+#define CHRG_CCCV_CV_BIT_POS 5
+#define CHRG_CCCV_CV_4100MV 0x0 /* 4.10V */
+#define CHRG_CCCV_CV_4150MV 0x1 /* 4.15V */
+#define CHRG_CCCV_CV_4200MV 0x2 /* 4.20V */
+#define CHRG_CCCV_CV_4350MV 0x3 /* 4.35V */
+#define CHRG_CCCV_CHG_EN (1 << 7)
+
+#define CV_4100 4100 /* 4100mV */
+#define CV_4150 4150 /* 4150mV */
+#define CV_4200 4200 /* 4200mV */
+#define CV_4350 4350 /* 4350mV */
+
+#define TEMP_IRQ_CFG_QWBTU (1 << 0)
+#define TEMP_IRQ_CFG_WBTU (1 << 1)
+#define TEMP_IRQ_CFG_QWBTO (1 << 2)
+#define TEMP_IRQ_CFG_WBTO (1 << 3)
+#define TEMP_IRQ_CFG_MASK 0xf
+
+#define FG_IRQ_CFG_LOWBATT_WL2 (1 << 0)
+#define FG_IRQ_CFG_LOWBATT_WL1 (1 << 1)
+#define FG_IRQ_CFG_LOWBATT_MASK 0x3
+#define LOWBAT_IRQ_STAT_LOWBATT_WL2 (1 << 0)
+#define LOWBAT_IRQ_STAT_LOWBATT_WL1 (1 << 1)
+
+#define FG_CNTL_OCV_ADJ_STAT (1 << 2)
+#define FG_CNTL_OCV_ADJ_EN (1 << 3)
+#define FG_CNTL_CAP_ADJ_STAT (1 << 4)
+#define FG_CNTL_CAP_ADJ_EN (1 << 5)
+#define FG_CNTL_CC_EN (1 << 6)
+#define FG_CNTL_GAUGE_EN (1 << 7)
+
+#define FG_REP_CAP_VALID (1 << 7)
+#define FG_REP_CAP_VAL_MASK 0x7F
+
+#define FG_DES_CAP1_VALID (1 << 7)
+#define FG_DES_CAP1_VAL_MASK 0x7F
+#define FG_DES_CAP0_VAL_MASK 0xFF
+#define FG_DES_CAP_RES_LSB 1456 /* 1.456mAhr */
+
+#define FG_CC_MTR1_VALID (1 << 7)
+#define FG_CC_MTR1_VAL_MASK 0x7F
+#define FG_CC_MTR0_VAL_MASK 0xFF
+#define FG_DES_CC_RES_LSB 1456 /* 1.456mAhr */
+
+#define FG_OCV_CAP_VALID (1 << 7)
+#define FG_OCV_CAP_VAL_MASK 0x7F
+#define FG_CC_CAP_VALID (1 << 7)
+#define FG_CC_CAP_VAL_MASK 0x7F
+
+#define FG_LOW_CAP_THR1_MASK 0xf0 /* 5% tp 20% */
+#define FG_LOW_CAP_THR1_VAL 0xa0 /* 15 perc */
+#define FG_LOW_CAP_THR2_MASK 0x0f /* 0% to 15% */
+#define FG_LOW_CAP_WARN_THR 14 /* 14 perc */
+#define FG_LOW_CAP_CRIT_THR 4 /* 4 perc */
+#define FG_LOW_CAP_SHDN_THR 0 /* 0 perc */
+
+#define STATUS_MON_DELAY_JIFFIES (HZ * 60) /*60 sec */
+#define NR_RETRY_CNT 3
+#define DEV_NAME "axp288_fuel_gauge"
+
+/* 1.1mV per LSB expressed in uV */
+#define VOLTAGE_FROM_ADC(a) ((a * 11) / 10)
+/* properties converted to tenths of degrees, uV, uA, uW */
+#define PROP_TEMP(a) ((a) * 10)
+#define UNPROP_TEMP(a) ((a) / 10)
+#define PROP_VOLT(a) ((a) * 1000)
+#define PROP_CURR(a) ((a) * 1000)
+
+#define AXP288_FG_INTR_NUM 6
+enum {
+ QWBTU_IRQ = 0,
+ WBTU_IRQ,
+ QWBTO_IRQ,
+ WBTO_IRQ,
+ WL2_IRQ,
+ WL1_IRQ,
+};
+
+struct axp288_fg_info {
+ struct platform_device *pdev;
+ struct axp20x_fg_pdata *pdata;
+ struct regmap *regmap;
+ struct regmap_irq_chip_data *regmap_irqc;
+ int irq[AXP288_FG_INTR_NUM];
+ struct power_supply *bat;
+ struct mutex lock;
+ int status;
+ struct delayed_work status_monitor;
+ struct dentry *debug_file;
+};
+
+static enum power_supply_property fuel_gauge_props[] = {
+ POWER_SUPPLY_PROP_STATUS,
+ POWER_SUPPLY_PROP_PRESENT,
+ POWER_SUPPLY_PROP_HEALTH,
+ POWER_SUPPLY_PROP_VOLTAGE_MAX_DESIGN,
+ POWER_SUPPLY_PROP_VOLTAGE_MIN_DESIGN,
+ POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ POWER_SUPPLY_PROP_VOLTAGE_OCV,
+ POWER_SUPPLY_PROP_CURRENT_NOW,
+ POWER_SUPPLY_PROP_CAPACITY,
+ POWER_SUPPLY_PROP_CAPACITY_ALERT_MIN,
+ POWER_SUPPLY_PROP_TEMP,
+ POWER_SUPPLY_PROP_TEMP_MAX,
+ POWER_SUPPLY_PROP_TEMP_MIN,
+ POWER_SUPPLY_PROP_TEMP_ALERT_MIN,
+ POWER_SUPPLY_PROP_TEMP_ALERT_MAX,
+ POWER_SUPPLY_PROP_TECHNOLOGY,
+ POWER_SUPPLY_PROP_CHARGE_FULL,
+ POWER_SUPPLY_PROP_CHARGE_NOW,
+ POWER_SUPPLY_PROP_CHARGE_FULL_DESIGN,
+ POWER_SUPPLY_PROP_MODEL_NAME,
+};
+
+static int fuel_gauge_reg_readb(struct axp288_fg_info *info, int reg)
+{
+ int ret, i;
+ unsigned int val;
+
+ for (i = 0; i < NR_RETRY_CNT; i++) {
+ ret = regmap_read(info->regmap, reg, &val);
+ if (ret == -EBUSY)
+ continue;
+ else
+ break;
+ }
+
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "axp288 reg read err:%d\n", ret);
+
+ return val;
+}
+
+static int fuel_gauge_reg_writeb(struct axp288_fg_info *info, int reg, u8 val)
+{
+ int ret;
+
+ ret = regmap_write(info->regmap, reg, (unsigned int)val);
+
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "axp288 reg write err:%d\n", ret);
+
+ return ret;
+}
+
+static int pmic_read_adc_val(const char *name, int *raw_val,
+ struct axp288_fg_info *info)
+{
+ int ret, val = 0;
+ struct iio_channel *indio_chan;
+
+ indio_chan = iio_channel_get(NULL, name);
+ if (IS_ERR_OR_NULL(indio_chan)) {
+ ret = PTR_ERR(indio_chan);
+ goto exit;
+ }
+ ret = iio_read_channel_raw(indio_chan, &val);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev,
+ "IIO channel read error: %x, %x\n", ret, val);
+ goto err_exit;
+ }
+
+ dev_dbg(&info->pdev->dev, "adc raw val=%x\n", val);
+ *raw_val = val;
+
+err_exit:
+ iio_channel_release(indio_chan);
+exit:
+ return ret;
+}
+
+#ifdef CONFIG_DEBUG_FS
+static int fuel_gauge_debug_show(struct seq_file *s, void *data)
+{
+ struct axp288_fg_info *info = s->private;
+ int raw_val, ret;
+
+ seq_printf(s, " PWR_STATUS[%02x] : %02x\n",
+ AXP20X_PWR_INPUT_STATUS,
+ fuel_gauge_reg_readb(info, AXP20X_PWR_INPUT_STATUS));
+ seq_printf(s, "PWR_OP_MODE[%02x] : %02x\n",
+ AXP20X_PWR_OP_MODE,
+ fuel_gauge_reg_readb(info, AXP20X_PWR_OP_MODE));
+ seq_printf(s, " CHRG_CTRL1[%02x] : %02x\n",
+ AXP20X_CHRG_CTRL1,
+ fuel_gauge_reg_readb(info, AXP20X_CHRG_CTRL1));
+ seq_printf(s, " VLTF[%02x] : %02x\n",
+ AXP20X_V_LTF_DISCHRG,
+ fuel_gauge_reg_readb(info, AXP20X_V_LTF_DISCHRG));
+ seq_printf(s, " VHTF[%02x] : %02x\n",
+ AXP20X_V_HTF_DISCHRG,
+ fuel_gauge_reg_readb(info, AXP20X_V_HTF_DISCHRG));
+ seq_printf(s, " CC_CTRL[%02x] : %02x\n",
+ AXP20X_CC_CTRL,
+ fuel_gauge_reg_readb(info, AXP20X_CC_CTRL));
+ seq_printf(s, "BATTERY CAP[%02x] : %02x\n",
+ AXP20X_FG_RES,
+ fuel_gauge_reg_readb(info, AXP20X_FG_RES));
+ seq_printf(s, " FG_RDC1[%02x] : %02x\n",
+ AXP288_FG_RDC1_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_RDC1_REG));
+ seq_printf(s, " FG_RDC0[%02x] : %02x\n",
+ AXP288_FG_RDC0_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_RDC0_REG));
+ seq_printf(s, " FG_OCVH[%02x] : %02x\n",
+ AXP288_FG_OCVH_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_OCVH_REG));
+ seq_printf(s, " FG_OCVL[%02x] : %02x\n",
+ AXP288_FG_OCVL_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_OCVL_REG));
+ seq_printf(s, "FG_DES_CAP1[%02x] : %02x\n",
+ AXP288_FG_DES_CAP1_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_DES_CAP1_REG));
+ seq_printf(s, "FG_DES_CAP0[%02x] : %02x\n",
+ AXP288_FG_DES_CAP0_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_DES_CAP0_REG));
+ seq_printf(s, " FG_CC_MTR1[%02x] : %02x\n",
+ AXP288_FG_CC_MTR1_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_CC_MTR1_REG));
+ seq_printf(s, " FG_CC_MTR0[%02x] : %02x\n",
+ AXP288_FG_CC_MTR0_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_CC_MTR0_REG));
+ seq_printf(s, " FG_OCV_CAP[%02x] : %02x\n",
+ AXP288_FG_OCV_CAP_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_OCV_CAP_REG));
+ seq_printf(s, " FG_CC_CAP[%02x] : %02x\n",
+ AXP288_FG_CC_CAP_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_CC_CAP_REG));
+ seq_printf(s, " FG_LOW_CAP[%02x] : %02x\n",
+ AXP288_FG_LOW_CAP_REG,
+ fuel_gauge_reg_readb(info, AXP288_FG_LOW_CAP_REG));
+ seq_printf(s, "TUNING_CTL0[%02x] : %02x\n",
+ AXP288_FG_TUNE0,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE0));
+ seq_printf(s, "TUNING_CTL1[%02x] : %02x\n",
+ AXP288_FG_TUNE1,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE1));
+ seq_printf(s, "TUNING_CTL2[%02x] : %02x\n",
+ AXP288_FG_TUNE2,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE2));
+ seq_printf(s, "TUNING_CTL3[%02x] : %02x\n",
+ AXP288_FG_TUNE3,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE3));
+ seq_printf(s, "TUNING_CTL4[%02x] : %02x\n",
+ AXP288_FG_TUNE4,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE4));
+ seq_printf(s, "TUNING_CTL5[%02x] : %02x\n",
+ AXP288_FG_TUNE5,
+ fuel_gauge_reg_readb(info, AXP288_FG_TUNE5));
+
+ ret = pmic_read_adc_val("axp288-batt-temp", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-batttemp : %d\n", raw_val);
+ ret = pmic_read_adc_val("axp288-pmic-temp", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-pmictemp : %d\n", raw_val);
+ ret = pmic_read_adc_val("axp288-system-temp", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-systtemp : %d\n", raw_val);
+ ret = pmic_read_adc_val("axp288-chrg-curr", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-chrgcurr : %d\n", raw_val);
+ ret = pmic_read_adc_val("axp288-chrg-d-curr", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-dchrgcur : %d\n", raw_val);
+ ret = pmic_read_adc_val("axp288-batt-volt", &raw_val, info);
+ if (ret >= 0)
+ seq_printf(s, "axp288-battvolt : %d\n", raw_val);
+
+ return 0;
+}
+
+static int debug_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, fuel_gauge_debug_show, inode->i_private);
+}
+
+static const struct file_operations fg_debug_fops = {
+ .open = debug_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+};
+
+static void fuel_gauge_create_debugfs(struct axp288_fg_info *info)
+{
+ info->debug_file = debugfs_create_file("fuelgauge", 0666, NULL,
+ info, &fg_debug_fops);
+}
+
+static void fuel_gauge_remove_debugfs(struct axp288_fg_info *info)
+{
+ debugfs_remove(info->debug_file);
+}
+#else
+static inline void fuel_gauge_create_debugfs(struct axp288_fg_info *info)
+{
+}
+static inline void fuel_gauge_remove_debugfs(struct axp288_fg_info *info)
+{
+}
+#endif
+
+static void fuel_gauge_get_status(struct axp288_fg_info *info)
+{
+ int pwr_stat, ret;
+ int charge, discharge;
+
+ pwr_stat = fuel_gauge_reg_readb(info, AXP20X_PWR_INPUT_STATUS);
+ if (pwr_stat < 0) {
+ dev_err(&info->pdev->dev,
+ "PWR STAT read failed:%d\n", pwr_stat);
+ return;
+ }
+ ret = pmic_read_adc_val("axp288-chrg-curr", &charge, info);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev,
+ "ADC charge current read failed:%d\n", ret);
+ return;
+ }
+ ret = pmic_read_adc_val("axp288-chrg-d-curr", &discharge, info);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev,
+ "ADC discharge current read failed:%d\n", ret);
+ return;
+ }
+
+ if (charge > 0)
+ info->status = POWER_SUPPLY_STATUS_CHARGING;
+ else if (discharge > 0)
+ info->status = POWER_SUPPLY_STATUS_DISCHARGING;
+ else {
+ if (pwr_stat & CHRG_STAT_BAT_PRESENT)
+ info->status = POWER_SUPPLY_STATUS_FULL;
+ else
+ info->status = POWER_SUPPLY_STATUS_NOT_CHARGING;
+ }
+}
+
+static int fuel_gauge_get_vbatt(struct axp288_fg_info *info, int *vbatt)
+{
+ int ret = 0, raw_val;
+
+ ret = pmic_read_adc_val("axp288-batt-volt", &raw_val, info);
+ if (ret < 0)
+ goto vbatt_read_fail;
+
+ *vbatt = VOLTAGE_FROM_ADC(raw_val);
+vbatt_read_fail:
+ return ret;
+}
+
+static int fuel_gauge_get_current(struct axp288_fg_info *info, int *cur)
+{
+ int ret, value = 0;
+ int charge, discharge;
+
+ ret = pmic_read_adc_val("axp288-chrg-curr", &charge, info);
+ if (ret < 0)
+ goto current_read_fail;
+ ret = pmic_read_adc_val("axp288-chrg-d-curr", &discharge, info);
+ if (ret < 0)
+ goto current_read_fail;
+
+ if (charge > 0)
+ value = charge;
+ else if (discharge > 0)
+ value = -1 * discharge;
+
+ *cur = value;
+current_read_fail:
+ return ret;
+}
+
+static int temp_to_adc(struct axp288_fg_info *info, int tval)
+{
+ int rntc = 0, i, ret, adc_val;
+ int rmin, rmax, tmin, tmax;
+ int tcsz = info->pdata->tcsz;
+
+ /* get the Rntc resitance value for this temp */
+ if (tval > info->pdata->thermistor_curve[0][1]) {
+ rntc = info->pdata->thermistor_curve[0][0];
+ } else if (tval <= info->pdata->thermistor_curve[tcsz-1][1]) {
+ rntc = info->pdata->thermistor_curve[tcsz-1][0];
+ } else {
+ for (i = 1; i < tcsz; i++) {
+ if (tval > info->pdata->thermistor_curve[i][1]) {
+ rmin = info->pdata->thermistor_curve[i-1][0];
+ rmax = info->pdata->thermistor_curve[i][0];
+ tmin = info->pdata->thermistor_curve[i-1][1];
+ tmax = info->pdata->thermistor_curve[i][1];
+ rntc = rmin + ((rmax - rmin) *
+ (tval - tmin) / (tmax - tmin));
+ break;
+ }
+ }
+ }
+
+ /* we need the current to calculate the proper adc voltage */
+ ret = fuel_gauge_reg_readb(info, AXP20X_ADC_RATE);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev, "%s:read err:%d\n", __func__, ret);
+ ret = 0x30;
+ }
+
+ /*
+ * temperature is proportional to NTS thermistor resistance
+ * ADC_RATE[5-4] determines current, 00=20uA,01=40uA,10=60uA,11=80uA
+ * [12-bit ADC VAL] = R_NTC(Ω) * current / 800
+ */
+ adc_val = rntc * (20 + (20 * ((ret >> 4) & 0x3))) / 800;
+
+ return adc_val;
+}
+
+static int adc_to_temp(struct axp288_fg_info *info, int adc_val)
+{
+ int ret, r, i, tval = 0;
+ int rmin, rmax, tmin, tmax;
+ int tcsz = info->pdata->tcsz;
+
+ ret = fuel_gauge_reg_readb(info, AXP20X_ADC_RATE);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev, "%s:read err:%d\n", __func__, ret);
+ ret = 0x30;
+ }
+
+ /*
+ * temperature is proportional to NTS thermistor resistance
+ * ADC_RATE[5-4] determines current, 00=20uA,01=40uA,10=60uA,11=80uA
+ * R_NTC(Ω) = [12-bit ADC VAL] * 800 / current
+ */
+ r = adc_val * 800 / (20 + (20 * ((ret >> 4) & 0x3)));
+
+ if (r < info->pdata->thermistor_curve[0][0]) {
+ tval = info->pdata->thermistor_curve[0][1];
+ } else if (r >= info->pdata->thermistor_curve[tcsz-1][0]) {
+ tval = info->pdata->thermistor_curve[tcsz-1][1];
+ } else {
+ for (i = 1; i < tcsz; i++) {
+ if (r < info->pdata->thermistor_curve[i][0]) {
+ rmin = info->pdata->thermistor_curve[i-1][0];
+ rmax = info->pdata->thermistor_curve[i][0];
+ tmin = info->pdata->thermistor_curve[i-1][1];
+ tmax = info->pdata->thermistor_curve[i][1];
+ tval = tmin + ((tmax - tmin) *
+ (r - rmin) / (rmax - rmin));
+ break;
+ }
+ }
+ }
+
+ return tval;
+}
+
+static int fuel_gauge_get_btemp(struct axp288_fg_info *info, int *btemp)
+{
+ int ret, raw_val = 0;
+
+ ret = pmic_read_adc_val("axp288-batt-temp", &raw_val, info);
+ if (ret < 0)
+ goto temp_read_fail;
+
+ *btemp = adc_to_temp(info, raw_val);
+
+temp_read_fail:
+ return ret;
+}
+
+static int fuel_gauge_get_vocv(struct axp288_fg_info *info, int *vocv)
+{
+ int ret, value;
+
+ /* 12-bit data value, upper 8 in OCVH, lower 4 in OCVL */
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_OCVH_REG);
+ if (ret < 0)
+ goto vocv_read_fail;
+ value = ret << 4;
+
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_OCVL_REG);
+ if (ret < 0)
+ goto vocv_read_fail;
+ value |= (ret & 0xf);
+
+ *vocv = VOLTAGE_FROM_ADC(value);
+vocv_read_fail:
+ return ret;
+}
+
+static int fuel_gauge_battery_health(struct axp288_fg_info *info)
+{
+ int temp, vocv;
+ int ret, health = POWER_SUPPLY_HEALTH_UNKNOWN;
+
+ ret = fuel_gauge_get_btemp(info, &temp);
+ if (ret < 0)
+ goto health_read_fail;
+
+ ret = fuel_gauge_get_vocv(info, &vocv);
+ if (ret < 0)
+ goto health_read_fail;
+
+ if (vocv > info->pdata->max_volt)
+ health = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
+ else if (temp > info->pdata->max_temp)
+ health = POWER_SUPPLY_HEALTH_OVERHEAT;
+ else if (temp < info->pdata->min_temp)
+ health = POWER_SUPPLY_HEALTH_COLD;
+ else if (vocv < info->pdata->min_volt)
+ health = POWER_SUPPLY_HEALTH_DEAD;
+ else
+ health = POWER_SUPPLY_HEALTH_GOOD;
+
+health_read_fail:
+ return health;
+}
+
+static int fuel_gauge_set_high_btemp_alert(struct axp288_fg_info *info)
+{
+ int ret, adc_val;
+
+ /* program temperature threshold as 1/16 ADC value */
+ adc_val = temp_to_adc(info, info->pdata->max_temp);
+ ret = fuel_gauge_reg_writeb(info, AXP20X_V_HTF_DISCHRG, adc_val >> 4);
+
+ return ret;
+}
+
+static int fuel_gauge_set_low_btemp_alert(struct axp288_fg_info *info)
+{
+ int ret, adc_val;
+
+ /* program temperature threshold as 1/16 ADC value */
+ adc_val = temp_to_adc(info, info->pdata->min_temp);
+ ret = fuel_gauge_reg_writeb(info, AXP20X_V_LTF_DISCHRG, adc_val >> 4);
+
+ return ret;
+}
+
+static int fuel_gauge_get_property(struct power_supply *ps,
+ enum power_supply_property prop,
+ union power_supply_propval *val)
+{
+ struct axp288_fg_info *info = power_supply_get_drvdata(ps);
+ int ret = 0, value;
+
+ mutex_lock(&info->lock);
+ switch (prop) {
+ case POWER_SUPPLY_PROP_STATUS:
+ fuel_gauge_get_status(info);
+ val->intval = info->status;
+ break;
+ case POWER_SUPPLY_PROP_HEALTH:
+ val->intval = fuel_gauge_battery_health(info);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_NOW:
+ ret = fuel_gauge_get_vbatt(info, &value);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ val->intval = PROP_VOLT(value);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_OCV:
+ ret = fuel_gauge_get_vocv(info, &value);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ val->intval = PROP_VOLT(value);
+ break;
+ case POWER_SUPPLY_PROP_CURRENT_NOW:
+ ret = fuel_gauge_get_current(info, &value);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ val->intval = PROP_CURR(value);
+ break;
+ case POWER_SUPPLY_PROP_PRESENT:
+ ret = fuel_gauge_reg_readb(info, AXP20X_PWR_OP_MODE);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+
+ if (ret & CHRG_STAT_BAT_PRESENT)
+ val->intval = 1;
+ else
+ val->intval = 0;
+ break;
+ case POWER_SUPPLY_PROP_CAPACITY:
+ ret = fuel_gauge_reg_readb(info, AXP20X_FG_RES);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+
+ if (!(ret & FG_REP_CAP_VALID))
+ dev_err(&info->pdev->dev,
+ "capacity measurement not valid\n");
+ val->intval = (ret & FG_REP_CAP_VAL_MASK);
+ break;
+ case POWER_SUPPLY_PROP_CAPACITY_ALERT_MIN:
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_LOW_CAP_REG);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ val->intval = (ret & 0x0f);
+ break;
+ case POWER_SUPPLY_PROP_TEMP:
+ ret = fuel_gauge_get_btemp(info, &value);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ val->intval = PROP_TEMP(value);
+ break;
+ case POWER_SUPPLY_PROP_TEMP_MAX:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MAX:
+ val->intval = PROP_TEMP(info->pdata->max_temp);
+ break;
+ case POWER_SUPPLY_PROP_TEMP_MIN:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MIN:
+ val->intval = PROP_TEMP(info->pdata->min_temp);
+ break;
+ case POWER_SUPPLY_PROP_TECHNOLOGY:
+ val->intval = POWER_SUPPLY_TECHNOLOGY_LION;
+ break;
+ case POWER_SUPPLY_PROP_CHARGE_NOW:
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_CC_MTR1_REG);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+
+ value = (ret & FG_CC_MTR1_VAL_MASK) << 8;
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_CC_MTR0_REG);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ value |= (ret & FG_CC_MTR0_VAL_MASK);
+ val->intval = value * FG_DES_CAP_RES_LSB;
+ break;
+ case POWER_SUPPLY_PROP_CHARGE_FULL:
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_DES_CAP1_REG);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+
+ value = (ret & FG_DES_CAP1_VAL_MASK) << 8;
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_DES_CAP0_REG);
+ if (ret < 0)
+ goto fuel_gauge_read_err;
+ value |= (ret & FG_DES_CAP0_VAL_MASK);
+ val->intval = value * FG_DES_CAP_RES_LSB;
+ break;
+ case POWER_SUPPLY_PROP_CHARGE_FULL_DESIGN:
+ val->intval = PROP_CURR(info->pdata->design_cap);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_MAX_DESIGN:
+ val->intval = PROP_VOLT(info->pdata->max_volt);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_MIN_DESIGN:
+ val->intval = PROP_VOLT(info->pdata->min_volt);
+ break;
+ case POWER_SUPPLY_PROP_MODEL_NAME:
+ val->strval = info->pdata->battid;
+ break;
+ default:
+ mutex_unlock(&info->lock);
+ return -EINVAL;
+ }
+
+ mutex_unlock(&info->lock);
+ return 0;
+
+fuel_gauge_read_err:
+ mutex_unlock(&info->lock);
+ return ret;
+}
+
+static int fuel_gauge_set_property(struct power_supply *ps,
+ enum power_supply_property prop,
+ const union power_supply_propval *val)
+{
+ struct axp288_fg_info *info = power_supply_get_drvdata(ps);
+ int ret = 0;
+
+ mutex_lock(&info->lock);
+ switch (prop) {
+ case POWER_SUPPLY_PROP_STATUS:
+ info->status = val->intval;
+ break;
+ case POWER_SUPPLY_PROP_TEMP_MIN:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MIN:
+ if ((val->intval < PD_DEF_MIN_TEMP) ||
+ (val->intval > PD_DEF_MAX_TEMP)) {
+ ret = -EINVAL;
+ break;
+ }
+ info->pdata->min_temp = UNPROP_TEMP(val->intval);
+ ret = fuel_gauge_set_low_btemp_alert(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev,
+ "temp alert min set fail:%d\n", ret);
+ break;
+ case POWER_SUPPLY_PROP_TEMP_MAX:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MAX:
+ if ((val->intval < PD_DEF_MIN_TEMP) ||
+ (val->intval > PD_DEF_MAX_TEMP)) {
+ ret = -EINVAL;
+ break;
+ }
+ info->pdata->max_temp = UNPROP_TEMP(val->intval);
+ ret = fuel_gauge_set_high_btemp_alert(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev,
+ "temp alert max set fail:%d\n", ret);
+ break;
+ case POWER_SUPPLY_PROP_CAPACITY_ALERT_MIN:
+ if ((val->intval < 0) || (val->intval > 15)) {
+ ret = -EINVAL;
+ break;
+ }
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_LOW_CAP_REG);
+ if (ret < 0)
+ break;
+ ret &= 0xf0;
+ ret |= (val->intval & 0xf);
+ ret = fuel_gauge_reg_writeb(info, AXP288_FG_LOW_CAP_REG, ret);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ mutex_unlock(&info->lock);
+ return ret;
+}
+
+static int fuel_gauge_property_is_writeable(struct power_supply *psy,
+ enum power_supply_property psp)
+{
+ int ret;
+
+ switch (psp) {
+ case POWER_SUPPLY_PROP_STATUS:
+ case POWER_SUPPLY_PROP_TEMP_MIN:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MIN:
+ case POWER_SUPPLY_PROP_TEMP_MAX:
+ case POWER_SUPPLY_PROP_TEMP_ALERT_MAX:
+ case POWER_SUPPLY_PROP_CAPACITY_ALERT_MIN:
+ ret = 1;
+ break;
+ default:
+ ret = 0;
+ }
+
+ return ret;
+}
+
+static void fuel_gauge_status_monitor(struct work_struct *work)
+{
+ struct axp288_fg_info *info = container_of(work,
+ struct axp288_fg_info, status_monitor.work);
+
+ fuel_gauge_get_status(info);
+ power_supply_changed(info->bat);
+ schedule_delayed_work(&info->status_monitor, STATUS_MON_DELAY_JIFFIES);
+}
+
+static irqreturn_t fuel_gauge_thread_handler(int irq, void *dev)
+{
+ struct axp288_fg_info *info = dev;
+ int i;
+
+ for (i = 0; i < AXP288_FG_INTR_NUM; i++) {
+ if (info->irq[i] == irq)
+ break;
+ }
+
+ if (i >= AXP288_FG_INTR_NUM) {
+ dev_warn(&info->pdev->dev, "spurious interrupt!!\n");
+ return IRQ_NONE;
+ }
+
+ switch (i) {
+ case QWBTU_IRQ:
+ dev_info(&info->pdev->dev,
+ "Quit Battery under temperature in work mode IRQ (QWBTU)\n");
+ break;
+ case WBTU_IRQ:
+ dev_info(&info->pdev->dev,
+ "Battery under temperature in work mode IRQ (WBTU)\n");
+ break;
+ case QWBTO_IRQ:
+ dev_info(&info->pdev->dev,
+ "Quit Battery over temperature in work mode IRQ (QWBTO)\n");
+ break;
+ case WBTO_IRQ:
+ dev_info(&info->pdev->dev,
+ "Battery over temperature in work mode IRQ (WBTO)\n");
+ break;
+ case WL2_IRQ:
+ dev_info(&info->pdev->dev, "Low Batt Warning(2) INTR\n");
+ break;
+ case WL1_IRQ:
+ dev_info(&info->pdev->dev, "Low Batt Warning(1) INTR\n");
+ break;
+ default:
+ dev_warn(&info->pdev->dev, "Spurious Interrupt!!!\n");
+ }
+
+ power_supply_changed(info->bat);
+ return IRQ_HANDLED;
+}
+
+static void fuel_gauge_external_power_changed(struct power_supply *psy)
+{
+ struct axp288_fg_info *info = power_supply_get_drvdata(psy);
+
+ power_supply_changed(info->bat);
+}
+
+static const struct power_supply_desc fuel_gauge_desc = {
+ .name = DEV_NAME,
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = fuel_gauge_props,
+ .num_properties = ARRAY_SIZE(fuel_gauge_props),
+ .get_property = fuel_gauge_get_property,
+ .set_property = fuel_gauge_set_property,
+ .property_is_writeable = fuel_gauge_property_is_writeable,
+ .external_power_changed = fuel_gauge_external_power_changed,
+};
+
+static int fuel_gauge_set_lowbatt_thresholds(struct axp288_fg_info *info)
+{
+ int ret;
+ u8 reg_val;
+
+ ret = fuel_gauge_reg_readb(info, AXP20X_FG_RES);
+ if (ret < 0) {
+ dev_err(&info->pdev->dev, "%s:read err:%d\n", __func__, ret);
+ return ret;
+ }
+ ret = (ret & FG_REP_CAP_VAL_MASK);
+
+ if (ret > FG_LOW_CAP_WARN_THR)
+ reg_val = FG_LOW_CAP_WARN_THR;
+ else if (ret > FG_LOW_CAP_CRIT_THR)
+ reg_val = FG_LOW_CAP_CRIT_THR;
+ else
+ reg_val = FG_LOW_CAP_SHDN_THR;
+
+ reg_val |= FG_LOW_CAP_THR1_VAL;
+ ret = fuel_gauge_reg_writeb(info, AXP288_FG_LOW_CAP_REG, reg_val);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "%s:write err:%d\n", __func__, ret);
+
+ return ret;
+}
+
+static int fuel_gauge_program_vbatt_full(struct axp288_fg_info *info)
+{
+ int ret;
+ u8 val;
+
+ ret = fuel_gauge_reg_readb(info, AXP20X_CHRG_CTRL1);
+ if (ret < 0)
+ goto fg_prog_ocv_fail;
+ else
+ val = (ret & ~CHRG_CCCV_CV_MASK);
+
+ switch (info->pdata->max_volt) {
+ case CV_4100:
+ val |= (CHRG_CCCV_CV_4100MV << CHRG_CCCV_CV_BIT_POS);
+ break;
+ case CV_4150:
+ val |= (CHRG_CCCV_CV_4150MV << CHRG_CCCV_CV_BIT_POS);
+ break;
+ case CV_4200:
+ val |= (CHRG_CCCV_CV_4200MV << CHRG_CCCV_CV_BIT_POS);
+ break;
+ case CV_4350:
+ val |= (CHRG_CCCV_CV_4350MV << CHRG_CCCV_CV_BIT_POS);
+ break;
+ default:
+ val |= (CHRG_CCCV_CV_4200MV << CHRG_CCCV_CV_BIT_POS);
+ break;
+ }
+
+ ret = fuel_gauge_reg_writeb(info, AXP20X_CHRG_CTRL1, val);
+fg_prog_ocv_fail:
+ return ret;
+}
+
+static int fuel_gauge_program_design_cap(struct axp288_fg_info *info)
+{
+ int ret;
+
+ ret = fuel_gauge_reg_writeb(info,
+ AXP288_FG_DES_CAP1_REG, info->pdata->cap1);
+ if (ret < 0)
+ goto fg_prog_descap_fail;
+
+ ret = fuel_gauge_reg_writeb(info,
+ AXP288_FG_DES_CAP0_REG, info->pdata->cap0);
+
+fg_prog_descap_fail:
+ return ret;
+}
+
+static int fuel_gauge_program_ocv_curve(struct axp288_fg_info *info)
+{
+ int ret = 0, i;
+
+ for (i = 0; i < OCV_CURVE_SIZE; i++) {
+ ret = fuel_gauge_reg_writeb(info,
+ AXP288_FG_OCV_CURVE_REG + i, info->pdata->ocv_curve[i]);
+ if (ret < 0)
+ goto fg_prog_ocv_fail;
+ }
+
+fg_prog_ocv_fail:
+ return ret;
+}
+
+static int fuel_gauge_program_rdc_vals(struct axp288_fg_info *info)
+{
+ int ret;
+
+ ret = fuel_gauge_reg_writeb(info,
+ AXP288_FG_RDC1_REG, info->pdata->rdc1);
+ if (ret < 0)
+ goto fg_prog_ocv_fail;
+
+ ret = fuel_gauge_reg_writeb(info,
+ AXP288_FG_RDC0_REG, info->pdata->rdc0);
+
+fg_prog_ocv_fail:
+ return ret;
+}
+
+static void fuel_gauge_init_config_regs(struct axp288_fg_info *info)
+{
+ int ret;
+
+ /*
+ * check if the config data is already
+ * programmed and if so just return.
+ */
+
+ ret = fuel_gauge_reg_readb(info, AXP288_FG_DES_CAP1_REG);
+ if (ret < 0) {
+ dev_warn(&info->pdev->dev, "CAP1 reg read err!!\n");
+ } else if (!(ret & FG_DES_CAP1_VALID)) {
+ dev_info(&info->pdev->dev, "FG data needs to be initialized\n");
+ } else {
+ dev_info(&info->pdev->dev, "FG data is already initialized\n");
+ return;
+ }
+
+ ret = fuel_gauge_program_vbatt_full(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "set vbatt full fail:%d\n", ret);
+
+ ret = fuel_gauge_program_design_cap(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "set design cap fail:%d\n", ret);
+
+ ret = fuel_gauge_program_rdc_vals(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "set rdc fail:%d\n", ret);
+
+ ret = fuel_gauge_program_ocv_curve(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "set ocv curve fail:%d\n", ret);
+
+ ret = fuel_gauge_set_lowbatt_thresholds(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "lowbatt thr set fail:%d\n", ret);
+
+ ret = fuel_gauge_reg_writeb(info, AXP20X_CC_CTRL, 0xef);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "gauge cntl set fail:%d\n", ret);
+}
+
+static void fuel_gauge_init_irq(struct axp288_fg_info *info)
+{
+ int ret, i, pirq;
+
+ for (i = 0; i < AXP288_FG_INTR_NUM; i++) {
+ pirq = platform_get_irq(info->pdev, i);
+ info->irq[i] = regmap_irq_get_virq(info->regmap_irqc, pirq);
+ if (info->irq[i] < 0) {
+ dev_warn(&info->pdev->dev,
+ "regmap_irq get virq failed for IRQ %d: %d\n",
+ pirq, info->irq[i]);
+ info->irq[i] = -1;
+ goto intr_failed;
+ }
+ ret = request_threaded_irq(info->irq[i],
+ NULL, fuel_gauge_thread_handler,
+ IRQF_ONESHOT, DEV_NAME, info);
+ if (ret) {
+ dev_warn(&info->pdev->dev,
+ "request irq failed for IRQ %d: %d\n",
+ pirq, info->irq[i]);
+ info->irq[i] = -1;
+ goto intr_failed;
+ } else {
+ dev_info(&info->pdev->dev, "HW IRQ %d -> VIRQ %d\n",
+ pirq, info->irq[i]);
+ }
+ }
+ return;
+
+intr_failed:
+ for (; i > 0; i--) {
+ free_irq(info->irq[i - 1], info);
+ info->irq[i - 1] = -1;
+ }
+}
+
+static void fuel_gauge_init_hw_regs(struct axp288_fg_info *info)
+{
+ int ret;
+ unsigned int val;
+
+ ret = fuel_gauge_set_high_btemp_alert(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "high batt temp set fail:%d\n", ret);
+
+ ret = fuel_gauge_set_low_btemp_alert(info);
+ if (ret < 0)
+ dev_err(&info->pdev->dev, "low batt temp set fail:%d\n", ret);
+
+ /* enable interrupts */
+ val = fuel_gauge_reg_readb(info, AXP20X_IRQ3_EN);
+ val |= TEMP_IRQ_CFG_MASK;
+ fuel_gauge_reg_writeb(info, AXP20X_IRQ3_EN, val);
+
+ val = fuel_gauge_reg_readb(info, AXP20X_IRQ4_EN);
+ val |= FG_IRQ_CFG_LOWBATT_MASK;
+ val = fuel_gauge_reg_writeb(info, AXP20X_IRQ4_EN, val);
+}
+
+static int axp288_fuel_gauge_probe(struct platform_device *pdev)
+{
+ int ret = 0;
+ struct axp288_fg_info *info;
+ struct axp20x_dev *axp20x = dev_get_drvdata(pdev->dev.parent);
+ struct power_supply_config psy_cfg = {};
+
+ info = devm_kzalloc(&pdev->dev, sizeof(*info), GFP_KERNEL);
+ if (!info)
+ return -ENOMEM;
+
+ info->pdev = pdev;
+ info->regmap = axp20x->regmap;
+ info->regmap_irqc = axp20x->regmap_irqc;
+ info->status = POWER_SUPPLY_STATUS_UNKNOWN;
+ info->pdata = pdev->dev.platform_data;
+ if (!info->pdata)
+ return -ENODEV;
+
+ platform_set_drvdata(pdev, info);
+
+ mutex_init(&info->lock);
+ INIT_DELAYED_WORK(&info->status_monitor, fuel_gauge_status_monitor);
+
+ psy_cfg.drv_data = info;
+ info->bat = power_supply_register(&pdev->dev, &fuel_gauge_desc, &psy_cfg);
+ if (IS_ERR(info->bat)) {
+ ret = PTR_ERR(info->bat);
+ dev_err(&pdev->dev, "failed to register battery: %d\n", ret);
+ return ret;
+ }
+
+ fuel_gauge_create_debugfs(info);
+ fuel_gauge_init_config_regs(info);
+ fuel_gauge_init_irq(info);
+ fuel_gauge_init_hw_regs(info);
+ schedule_delayed_work(&info->status_monitor, STATUS_MON_DELAY_JIFFIES);
+
+ return ret;
+}
+
+static struct platform_device_id axp288_fg_id_table[] = {
+ { .name = DEV_NAME },
+ {},
+};
+
+static int axp288_fuel_gauge_remove(struct platform_device *pdev)
+{
+ struct axp288_fg_info *info = platform_get_drvdata(pdev);
+ int i;
+
+ cancel_delayed_work_sync(&info->status_monitor);
+ power_supply_unregister(info->bat);
+ fuel_gauge_remove_debugfs(info);
+
+ for (i = 0; i < AXP288_FG_INTR_NUM; i++)
+ if (info->irq[i] >= 0)
+ free_irq(info->irq[i], info);
+
+ return 0;
+}
+
+static struct platform_driver axp288_fuel_gauge_driver = {
+ .probe = axp288_fuel_gauge_probe,
+ .remove = axp288_fuel_gauge_remove,
+ .id_table = axp288_fg_id_table,
+ .driver = {
+ .name = DEV_NAME,
+ },
+};
+
+module_platform_driver(axp288_fuel_gauge_driver);
+
+MODULE_AUTHOR("Todd Brandt <todd.e.brandt@linux.intel.com>");
+MODULE_DESCRIPTION("Xpower AXP288 Fuel Gauge Driver");
+MODULE_LICENSE("GPL");
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
* GNU General Public License for more details.
*
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
- */
-
-/*
* Datasheets:
* http://www.ti.com/product/bq24150
* http://www.ti.com/product/bq24150a
* http://www.ti.com/product/bq24153
* http://www.ti.com/product/bq24153a
* http://www.ti.com/product/bq24155
+ * http://www.ti.com/product/bq24157s
+ * http://www.ti.com/product/bq24158
*/
#include <linux/kernel.h>
BQ24155,
BQ24156,
BQ24156A,
+ BQ24157S,
BQ24158,
};
"bq24155",
"bq24156",
"bq24156a",
+ "bq24157s",
"bq24158",
};
struct bq2415x_device {
struct device *dev;
struct bq2415x_platform_data init_data;
- struct power_supply charger;
+ struct power_supply *charger;
+ struct power_supply_desc charger_desc;
struct delayed_work work;
struct power_supply *notify_psy;
struct notifier_block nb;
enum bq2415x_mode reported_mode;/* mode reported by hook function */
- enum bq2415x_mode mode; /* current configured mode */
+ enum bq2415x_mode mode; /* currently configured mode */
enum bq2415x_chip chip;
const char *timer_error;
char *model;
BQ2415X_BIT_CE);
if (ret < 0)
return ret;
- else
- return ret > 0 ? 0 : 1;
+ return ret > 0 ? 0 : 1;
case BQ2415X_CHARGER_ENABLE:
return bq2415x_i2c_write_bit(bq, BQ2415X_REG_CONTROL,
0, BQ2415X_BIT_CE);
case 0:
if (bq->chip == BQ24151A)
return bq->chip;
- else
- return BQ24151;
+ return BQ24151;
case 1:
if (bq->chip == BQ24150A ||
bq->chip == BQ24152 ||
bq->chip == BQ24155)
return bq->chip;
- else
- return BQ24150;
+ return BQ24150;
case 2:
if (bq->chip == BQ24153A)
return bq->chip;
- else
- return BQ24153;
+ return BQ24153;
default:
return BQUNKNOWN;
}
case 0:
if (bq->chip == BQ24156A)
return bq->chip;
- else
- return BQ24156;
+ return BQ24156;
case 2:
+ if (bq->chip == BQ24157S)
+ return bq->chip;
return BQ24158;
default:
return BQUNKNOWN;
case BQ24152:
if (ret >= 0 && ret <= 3)
return ret;
- else
- return -1;
+ return -1;
case BQ24153:
case BQ24153A:
case BQ24156:
case BQ24156A:
+ case BQ24157S:
case BQ24158:
if (ret == 3)
return 0;
else if (ret == 1)
return 1;
- else
- return -1;
+ return -1;
case BQ24155:
if (ret == 3)
return 3;
- else
- return -1;
+ return -1;
case BQUNKNOWN:
return -1;
}
bq2415x_set_default_value(bq, battery_regulation_voltage);
bq->mode = mode;
- sysfs_notify(&bq->charger.dev->kobj, NULL, "mode");
+ sysfs_notify(&bq->charger->dev.kobj, NULL, "mode");
return 0;
dev_dbg(bq->dev, "notifier call was called\n");
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_CURRENT_MAX, &prop);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_CURRENT_MAX,
+ &prop);
if (ret != 0)
return NOTIFY_OK;
static void bq2415x_timer_error(struct bq2415x_device *bq, const char *msg)
{
bq->timer_error = msg;
- sysfs_notify(&bq->charger.dev->kobj, NULL, "timer");
+ sysfs_notify(&bq->charger->dev.kobj, NULL, "timer");
dev_err(bq->dev, "%s\n", msg);
if (bq->automode > 0)
bq->automode = 0;
int boost;
if (bq->automode > 0 && (bq->reported_mode != bq->mode)) {
- sysfs_notify(&bq->charger.dev->kobj, NULL, "reported_mode");
+ sysfs_notify(&bq->charger->dev.kobj, NULL, "reported_mode");
bq2415x_set_mode(bq, bq->reported_mode);
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
int ret;
switch (psp) {
int ret;
int chip;
char revstr[8];
+ struct power_supply_config psy_cfg = { .drv_data = bq, };
- bq->charger.name = bq->name;
- bq->charger.type = POWER_SUPPLY_TYPE_USB;
- bq->charger.properties = bq2415x_power_supply_props;
- bq->charger.num_properties = ARRAY_SIZE(bq2415x_power_supply_props);
- bq->charger.get_property = bq2415x_power_supply_get_property;
+ bq->charger_desc.name = bq->name;
+ bq->charger_desc.type = POWER_SUPPLY_TYPE_USB;
+ bq->charger_desc.properties = bq2415x_power_supply_props;
+ bq->charger_desc.num_properties =
+ ARRAY_SIZE(bq2415x_power_supply_props);
+ bq->charger_desc.get_property = bq2415x_power_supply_get_property;
ret = bq2415x_detect_chip(bq);
if (ret < 0)
return -ENOMEM;
}
- ret = power_supply_register(bq->dev, &bq->charger);
- if (ret) {
+ bq->charger = power_supply_register(bq->dev, &bq->charger_desc,
+ &psy_cfg);
+ if (IS_ERR(bq->charger)) {
kfree(bq->model);
- return ret;
+ return PTR_ERR(bq->charger);
}
return 0;
if (bq->automode > 0)
bq->automode = 0;
cancel_delayed_work_sync(&bq->work);
- power_supply_unregister(&bq->charger);
+ power_supply_unregister(bq->charger);
kfree(bq->model);
}
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
enum bq2415x_command command;
int ret;
size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
int ret = 0;
if (strncmp(buf, "auto", 4) == 0)
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
if (bq->timer_error)
return sprintf(buf, "%s\n", bq->timer_error);
size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
enum bq2415x_mode mode;
int ret = 0;
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
ssize_t ret = 0;
if (bq->automode > 0)
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
if (bq->automode < 0)
return -EINVAL;
size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
ssize_t ret = 0;
unsigned int reg;
unsigned int val;
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
ssize_t ret = 0;
ret += bq2415x_sysfs_print_reg(bq, BQ2415X_REG_STATUS, buf+ret);
size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
long val;
int ret;
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
int ret;
if (strcmp(attr->attr.name, "current_limit") == 0)
size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
enum bq2415x_command command;
long val;
int ret;
char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq2415x_device *bq = container_of(psy, struct bq2415x_device,
- charger);
+ struct bq2415x_device *bq = power_supply_get_drvdata(psy);
enum bq2415x_command command;
int ret;
static int bq2415x_sysfs_init(struct bq2415x_device *bq)
{
- return sysfs_create_group(&bq->charger.dev->kobj,
+ return sysfs_create_group(&bq->charger->dev.kobj,
&bq2415x_sysfs_attr_group);
}
static void bq2415x_sysfs_exit(struct bq2415x_device *bq)
{
- sysfs_remove_group(&bq->charger.dev->kobj, &bq2415x_sysfs_attr_group);
+ sysfs_remove_group(&bq->charger->dev.kobj, &bq2415x_sysfs_attr_group);
}
/* main bq2415x probe function */
ret = of_property_read_u32(np, "ti,current-limit",
&bq->init_data.current_limit);
if (ret)
- goto error_2;
+ goto error_3;
ret = of_property_read_u32(np, "ti,weak-battery-voltage",
&bq->init_data.weak_battery_voltage);
if (ret)
- goto error_2;
+ goto error_3;
ret = of_property_read_u32(np, "ti,battery-regulation-voltage",
&bq->init_data.battery_regulation_voltage);
if (ret)
- goto error_2;
+ goto error_3;
ret = of_property_read_u32(np, "ti,charge-current",
&bq->init_data.charge_current);
if (ret)
- goto error_2;
+ goto error_3;
ret = of_property_read_u32(np, "ti,termination-current",
&bq->init_data.termination_current);
if (ret)
- goto error_2;
+ goto error_3;
ret = of_property_read_u32(np, "ti,resistor-sense",
&bq->init_data.resistor_sense);
if (ret)
- goto error_2;
+ goto error_3;
} else {
memcpy(&bq->init_data, pdata, sizeof(bq->init_data));
}
ret = bq2415x_power_supply_init(bq);
if (ret) {
dev_err(bq->dev, "failed to register power supply: %d\n", ret);
- goto error_2;
+ goto error_3;
}
ret = bq2415x_sysfs_init(bq);
if (ret) {
dev_err(bq->dev, "failed to create sysfs entries: %d\n", ret);
- goto error_3;
+ goto error_4;
}
ret = bq2415x_set_defaults(bq);
if (ret) {
dev_err(bq->dev, "failed to set default values: %d\n", ret);
- goto error_4;
+ goto error_5;
}
if (bq->notify_psy) {
ret = power_supply_reg_notifier(&bq->nb);
if (ret) {
dev_err(bq->dev, "failed to reg notifier: %d\n", ret);
- goto error_5;
+ goto error_6;
}
/* Query for initial reported_mode and set it */
dev_info(bq->dev, "driver registered\n");
return 0;
+error_6:
error_5:
-error_4:
bq2415x_sysfs_exit(bq);
-error_3:
+error_4:
bq2415x_power_supply_exit(bq);
+error_3:
+ if (bq->notify_psy)
+ power_supply_put(bq->notify_psy);
error_2:
kfree(name);
error_1:
{
struct bq2415x_device *bq = i2c_get_clientdata(client);
- if (bq->notify_psy)
+ if (bq->notify_psy) {
power_supply_unreg_notifier(&bq->nb);
+ power_supply_put(bq->notify_psy);
+ }
bq2415x_sysfs_exit(bq);
bq2415x_power_supply_exit(bq);
{ "bq24155", BQ24155 },
{ "bq24156", BQ24156 },
{ "bq24156a", BQ24156A },
+ { "bq24157s", BQ24157S },
{ "bq24158", BQ24158 },
{},
};
struct bq24190_dev_info {
struct i2c_client *client;
struct device *dev;
- struct power_supply charger;
- struct power_supply battery;
+ struct power_supply *charger;
+ struct power_supply *battery;
char model_name[I2C_NAME_SIZE];
kernel_ulong_t model;
unsigned int gpio_int;
struct device_attribute *attr, char *buf)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, charger);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
struct bq24190_sysfs_field_info *info;
int ret;
u8 v;
struct device_attribute *attr, const char *buf, size_t count)
{
struct power_supply *psy = dev_get_drvdata(dev);
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, charger);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
struct bq24190_sysfs_field_info *info;
int ret;
u8 v;
{
bq24190_sysfs_init_attrs();
- return sysfs_create_group(&bdi->charger.dev->kobj,
+ return sysfs_create_group(&bdi->charger->dev.kobj,
&bq24190_sysfs_attr_group);
}
static void bq24190_sysfs_remove_group(struct bq24190_dev_info *bdi)
{
- sysfs_remove_group(&bdi->charger.dev->kobj, &bq24190_sysfs_attr_group);
+ sysfs_remove_group(&bdi->charger->dev.kobj, &bq24190_sysfs_attr_group);
}
#else
static int bq24190_sysfs_create_group(struct bq24190_dev_info *bdi)
static int bq24190_charger_get_property(struct power_supply *psy,
enum power_supply_property psp, union power_supply_propval *val)
{
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, charger);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
int ret;
dev_dbg(bdi->dev, "prop: %d\n", psp);
enum power_supply_property psp,
const union power_supply_propval *val)
{
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, charger);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
int ret;
dev_dbg(bdi->dev, "prop: %d\n", psp);
"main-battery",
};
-static void bq24190_charger_init(struct power_supply *charger)
-{
- charger->name = "bq24190-charger";
- charger->type = POWER_SUPPLY_TYPE_USB;
- charger->properties = bq24190_charger_properties;
- charger->num_properties = ARRAY_SIZE(bq24190_charger_properties);
- charger->supplied_to = bq24190_charger_supplied_to;
- charger->num_supplicants = ARRAY_SIZE(bq24190_charger_supplied_to);
- charger->get_property = bq24190_charger_get_property;
- charger->set_property = bq24190_charger_set_property;
- charger->property_is_writeable = bq24190_charger_property_is_writeable;
-}
+static const struct power_supply_desc bq24190_charger_desc = {
+ .name = "bq24190-charger",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = bq24190_charger_properties,
+ .num_properties = ARRAY_SIZE(bq24190_charger_properties),
+ .get_property = bq24190_charger_get_property,
+ .set_property = bq24190_charger_set_property,
+ .property_is_writeable = bq24190_charger_property_is_writeable,
+};
/* Battery power supply property routines */
static int bq24190_battery_get_property(struct power_supply *psy,
enum power_supply_property psp, union power_supply_propval *val)
{
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, battery);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
int ret;
dev_dbg(bdi->dev, "prop: %d\n", psp);
enum power_supply_property psp,
const union power_supply_propval *val)
{
- struct bq24190_dev_info *bdi =
- container_of(psy, struct bq24190_dev_info, battery);
+ struct bq24190_dev_info *bdi = power_supply_get_drvdata(psy);
int ret;
dev_dbg(bdi->dev, "prop: %d\n", psp);
POWER_SUPPLY_PROP_SCOPE,
};
-static void bq24190_battery_init(struct power_supply *battery)
-{
- battery->name = "bq24190-battery";
- battery->type = POWER_SUPPLY_TYPE_BATTERY;
- battery->properties = bq24190_battery_properties;
- battery->num_properties = ARRAY_SIZE(bq24190_battery_properties);
- battery->get_property = bq24190_battery_get_property;
- battery->set_property = bq24190_battery_set_property;
- battery->property_is_writeable = bq24190_battery_property_is_writeable;
-}
+static const struct power_supply_desc bq24190_battery_desc = {
+ .name = "bq24190-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = bq24190_battery_properties,
+ .num_properties = ARRAY_SIZE(bq24190_battery_properties),
+ .get_property = bq24190_battery_get_property,
+ .set_property = bq24190_battery_set_property,
+ .property_is_writeable = bq24190_battery_property_is_writeable,
+};
static irqreturn_t bq24190_irq_handler_thread(int irq, void *data)
{
* interrupt received).
*/
if (alert_userspace && !bdi->first_time) {
- power_supply_changed(&bdi->charger);
- power_supply_changed(&bdi->battery);
+ power_supply_changed(bdi->charger);
+ power_supply_changed(bdi->battery);
bdi->first_time = false;
}
struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
struct device *dev = &client->dev;
struct bq24190_platform_data *pdata = client->dev.platform_data;
+ struct power_supply_config charger_cfg = {}, battery_cfg = {};
struct bq24190_dev_info *bdi;
int ret;
goto out2;
}
- bq24190_charger_init(&bdi->charger);
-
- ret = power_supply_register(dev, &bdi->charger);
- if (ret) {
+ charger_cfg.drv_data = bdi;
+ charger_cfg.supplied_to = bq24190_charger_supplied_to;
+ charger_cfg.num_supplicants = ARRAY_SIZE(bq24190_charger_supplied_to),
+ bdi->charger = power_supply_register(dev, &bq24190_charger_desc,
+ &charger_cfg);
+ if (IS_ERR(bdi->charger)) {
dev_err(dev, "Can't register charger\n");
+ ret = PTR_ERR(bdi->charger);
goto out2;
}
- bq24190_battery_init(&bdi->battery);
-
- ret = power_supply_register(dev, &bdi->battery);
- if (ret) {
+ battery_cfg.drv_data = bdi;
+ bdi->battery = power_supply_register(dev, &bq24190_battery_desc,
+ &battery_cfg);
+ if (IS_ERR(bdi->battery)) {
dev_err(dev, "Can't register battery\n");
+ ret = PTR_ERR(bdi->battery);
goto out3;
}
return 0;
out4:
- power_supply_unregister(&bdi->battery);
+ power_supply_unregister(bdi->battery);
out3:
- power_supply_unregister(&bdi->charger);
+ power_supply_unregister(bdi->charger);
out2:
pm_runtime_disable(dev);
out1:
pm_runtime_put_sync(bdi->dev);
bq24190_sysfs_remove_group(bdi);
- power_supply_unregister(&bdi->battery);
- power_supply_unregister(&bdi->charger);
+ power_supply_unregister(bdi->battery);
+ power_supply_unregister(bdi->charger);
pm_runtime_disable(bdi->dev);
if (bdi->gpio_int)
pm_runtime_put_sync(bdi->dev);
/* Things may have changed while suspended so alert upper layer */
- power_supply_changed(&bdi->charger);
- power_supply_changed(&bdi->battery);
+ power_supply_changed(bdi->charger);
+ power_supply_changed(bdi->battery);
return 0;
}
#define BQ24735_DEVICE_ID 0xff
struct bq24735 {
- struct power_supply charger;
- struct i2c_client *client;
- struct bq24735_platform *pdata;
+ struct power_supply *charger;
+ struct power_supply_desc charger_desc;
+ struct i2c_client *client;
+ struct bq24735_platform *pdata;
};
static inline struct bq24735 *to_bq24735(struct power_supply *psy)
{
- return container_of(psy, struct bq24735, charger);
+ return power_supply_get_drvdata(psy);
}
static enum power_supply_property bq24735_charger_properties[] = {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct bq24735 *charger;
-
- charger = container_of(psy, struct bq24735, charger);
+ struct bq24735 *charger = to_bq24735(psy);
switch (psp) {
case POWER_SUPPLY_PROP_ONLINE:
{
int ret;
struct bq24735 *charger;
- struct power_supply *supply;
+ struct power_supply_desc *supply_desc;
+ struct power_supply_config psy_cfg = {};
char *name;
charger = devm_kzalloc(&client->dev, sizeof(*charger), GFP_KERNEL);
charger->client = client;
- supply = &charger->charger;
+ supply_desc = &charger->charger_desc;
- supply->name = name;
- supply->type = POWER_SUPPLY_TYPE_MAINS;
- supply->properties = bq24735_charger_properties;
- supply->num_properties = ARRAY_SIZE(bq24735_charger_properties);
- supply->get_property = bq24735_charger_get_property;
- supply->supplied_to = charger->pdata->supplied_to;
- supply->num_supplicants = charger->pdata->num_supplicants;
- supply->of_node = client->dev.of_node;
+ supply_desc->name = name;
+ supply_desc->type = POWER_SUPPLY_TYPE_MAINS;
+ supply_desc->properties = bq24735_charger_properties;
+ supply_desc->num_properties = ARRAY_SIZE(bq24735_charger_properties);
+ supply_desc->get_property = bq24735_charger_get_property;
+
+ psy_cfg.supplied_to = charger->pdata->supplied_to;
+ psy_cfg.num_supplicants = charger->pdata->num_supplicants;
+ psy_cfg.of_node = client->dev.of_node;
+ psy_cfg.drv_data = charger;
i2c_set_clientdata(client, charger);
}
}
- ret = power_supply_register(&client->dev, supply);
- if (ret < 0) {
+ charger->charger = power_supply_register(&client->dev, supply_desc,
+ &psy_cfg);
+ if (IS_ERR(charger->charger)) {
+ ret = PTR_ERR(charger->charger);
dev_err(&client->dev, "Failed to register power supply: %d\n",
ret);
goto err_free_name;
IRQF_TRIGGER_RISING |
IRQF_TRIGGER_FALLING |
IRQF_ONESHOT,
- supply->name, supply);
+ supply_desc->name,
+ charger->charger);
if (ret) {
dev_err(&client->dev,
"Unable to register IRQ %d err %d\n",
return 0;
err_unregister_supply:
- power_supply_unregister(supply);
+ power_supply_unregister(charger->charger);
err_free_name:
if (name != charger->pdata->name)
kfree(name);
devm_free_irq(&charger->client->dev, charger->client->irq,
&charger->charger);
- power_supply_unregister(&charger->charger);
+ power_supply_unregister(charger->charger);
- if (charger->charger.name != charger->pdata->name)
- kfree(charger->charger.name);
+ if (charger->charger_desc.name != charger->pdata->name)
+ kfree(charger->charger_desc.name);
return 0;
}
* IMPLIED WARRANTIES, INCLUDING, WITHOUT LIMITATION, THE IMPLIED
* WARRANTIES OF MERCHANTIBILITY AND FITNESS FOR A PARTICULAR PURPOSE.
*
- */
-
-/*
* Datasheets:
* http://focus.ti.com/docs/prod/folders/print/bq27000.html
* http://focus.ti.com/docs/prod/folders/print/bq27500.html
* http://www.ti.com/product/bq27425-g1
* http://www.ti.com/product/BQ27742-G1
+ * http://www.ti.com/product/BQ27510-G3
*/
#include <linux/device.h>
#define BQ27742_POWER_AVG 0x76
+#define BQ27510_REG_SOC 0x20
+#define BQ27510_REG_DCAP 0x2E /* Design capacity */
+#define BQ27510_REG_CYCT 0x1E /* Cycle count total */
+
/* bq27425 register addresses are same as bq27x00 addresses minus 4 */
#define BQ27425_REG_OFFSET 0x04
#define BQ27425_REG_SOC (0x1C + BQ27425_REG_OFFSET)
int (*read)(struct bq27x00_device_info *di, u8 reg, bool single);
};
-enum bq27x00_chip { BQ27000, BQ27500, BQ27425, BQ27742};
+enum bq27x00_chip { BQ27000, BQ27500, BQ27425, BQ27742, BQ27510};
struct bq27x00_reg_cache {
int temperature;
unsigned long last_update;
struct delayed_work work;
- struct power_supply bat;
+ struct power_supply *bat;
struct bq27x00_access_methods bus;
POWER_SUPPLY_PROP_HEALTH,
};
+static enum power_supply_property bq27510_battery_props[] = {
+ POWER_SUPPLY_PROP_STATUS,
+ POWER_SUPPLY_PROP_PRESENT,
+ POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ POWER_SUPPLY_PROP_CURRENT_NOW,
+ POWER_SUPPLY_PROP_CAPACITY,
+ POWER_SUPPLY_PROP_CAPACITY_LEVEL,
+ POWER_SUPPLY_PROP_TEMP,
+ POWER_SUPPLY_PROP_TIME_TO_EMPTY_NOW,
+ POWER_SUPPLY_PROP_TECHNOLOGY,
+ POWER_SUPPLY_PROP_CHARGE_FULL,
+ POWER_SUPPLY_PROP_CHARGE_NOW,
+ POWER_SUPPLY_PROP_CHARGE_FULL_DESIGN,
+ POWER_SUPPLY_PROP_CYCLE_COUNT,
+ POWER_SUPPLY_PROP_POWER_AVG,
+ POWER_SUPPLY_PROP_HEALTH,
+};
+
static unsigned int poll_interval = 360;
module_param(poll_interval, uint, 0644);
MODULE_PARM_DESC(poll_interval, "battery poll interval in seconds - " \
*/
static bool bq27xxx_is_chip_version_higher(struct bq27x00_device_info *di)
{
- if (di->chip == BQ27425 || di->chip == BQ27500 || di->chip == BQ27742)
+ if (di->chip == BQ27425 || di->chip == BQ27500 || di->chip == BQ27742
+ || di->chip == BQ27510)
return true;
return false;
}
if (di->chip == BQ27500 || di->chip == BQ27742)
rsoc = bq27x00_read(di, BQ27500_REG_SOC, false);
+ else if (di->chip == BQ27510)
+ rsoc = bq27x00_read(di, BQ27510_REG_SOC, false);
else if (di->chip == BQ27425)
rsoc = bq27x00_read(di, BQ27425_REG_SOC, false);
else
if (bq27xxx_is_chip_version_higher(di)) {
if (di->chip == BQ27425)
ilmd = bq27x00_read(di, BQ27425_REG_DCAP, false);
+ else if (di->chip == BQ27510)
+ ilmd = bq27x00_read(di, BQ27510_REG_DCAP, false);
else
ilmd = bq27x00_read(di, BQ27500_REG_DCAP, false);
} else
{
int cyct;
- cyct = bq27x00_read(di, BQ27x00_REG_CYCT, false);
+ if (di->chip == BQ27510)
+ cyct = bq27x00_read(di, BQ27510_REG_CYCT, false);
+ else
+ cyct = bq27x00_read(di, BQ27x00_REG_CYCT, false);
if (cyct < 0)
dev_err(di->dev, "error reading cycle count total\n");
else
tval = POWER_SUPPLY_HEALTH_GOOD;
return tval;
+ } else if (di->chip == BQ27510) {
+ if (tval & BQ27500_FLAG_OTC)
+ return POWER_SUPPLY_HEALTH_OVERHEAT;
+ return POWER_SUPPLY_HEALTH_GOOD;
} else {
if (tval & BQ27000_FLAG_EDV1)
tval = POWER_SUPPLY_HEALTH_DEAD;
{
struct bq27x00_reg_cache cache = {0, };
bool is_bq27500 = di->chip == BQ27500;
+ bool is_bq27510 = di->chip == BQ27510;
bool is_bq27425 = di->chip == BQ27425;
bool is_bq27742 = di->chip == BQ27742;
bool flags_1b = !(is_bq27500 || is_bq27742);
/* read error */
cache.flags = -1;
if (cache.flags >= 0) {
- if (!is_bq27500 && !is_bq27425 && !is_bq27742
+ if (!is_bq27500 && !is_bq27425 && !is_bq27742 && !is_bq27510
&& (cache.flags & BQ27000_FLAG_CI)) {
dev_info(di->dev, "battery is not calibrated! ignoring capacity values\n");
cache.capacity = -ENODATA;
cache.health = -ENODATA;
} else {
cache.capacity = bq27x00_battery_read_rsoc(di);
- if (is_bq27742)
+ if (is_bq27742 || is_bq27510)
cache.time_to_empty =
bq27x00_battery_read_time(di,
BQ27x00_REG_TTE);
}
if (di->cache.capacity != cache.capacity)
- power_supply_changed(&di->bat);
+ power_supply_changed(di->bat);
if (memcmp(&di->cache, &cache, sizeof(cache)) != 0)
di->cache = cache;
status = POWER_SUPPLY_STATUS_FULL;
else if (di->cache.flags & BQ27000_FLAG_CHGS)
status = POWER_SUPPLY_STATUS_CHARGING;
- else if (power_supply_am_i_supplied(&di->bat))
+ else if (power_supply_am_i_supplied(di->bat))
status = POWER_SUPPLY_STATUS_NOT_CHARGING;
else
status = POWER_SUPPLY_STATUS_DISCHARGING;
return 0;
}
-#define to_bq27x00_device_info(x) container_of((x), \
- struct bq27x00_device_info, bat);
-
static int bq27x00_battery_get_property(struct power_supply *psy,
enum power_supply_property psp,
union power_supply_propval *val)
{
int ret = 0;
- struct bq27x00_device_info *di = to_bq27x00_device_info(psy);
+ struct bq27x00_device_info *di = power_supply_get_drvdata(psy);
mutex_lock(&di->lock);
if (time_is_before_jiffies(di->last_update + 5 * HZ)) {
static void bq27x00_external_power_changed(struct power_supply *psy)
{
- struct bq27x00_device_info *di = to_bq27x00_device_info(psy);
+ struct bq27x00_device_info *di = power_supply_get_drvdata(psy);
cancel_delayed_work_sync(&di->work);
schedule_delayed_work(&di->work, 0);
}
-static int bq27x00_powersupply_init(struct bq27x00_device_info *di)
+static int bq27x00_powersupply_init(struct bq27x00_device_info *di,
+ const char *name)
{
int ret;
+ struct power_supply_desc *psy_desc;
+ struct power_supply_config psy_cfg = { .drv_data = di, };
- di->bat.type = POWER_SUPPLY_TYPE_BATTERY;
+ psy_desc = devm_kzalloc(di->dev, sizeof(*psy_desc), GFP_KERNEL);
+ if (!psy_desc)
+ return -ENOMEM;
+
+ psy_desc->name = name;
+ psy_desc->type = POWER_SUPPLY_TYPE_BATTERY;
if (di->chip == BQ27425) {
- di->bat.properties = bq27425_battery_props;
- di->bat.num_properties = ARRAY_SIZE(bq27425_battery_props);
+ psy_desc->properties = bq27425_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(bq27425_battery_props);
} else if (di->chip == BQ27742) {
- di->bat.properties = bq27742_battery_props;
- di->bat.num_properties = ARRAY_SIZE(bq27742_battery_props);
+ psy_desc->properties = bq27742_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(bq27742_battery_props);
+ } else if (di->chip == BQ27510) {
+ psy_desc->properties = bq27510_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(bq27510_battery_props);
} else {
- di->bat.properties = bq27x00_battery_props;
- di->bat.num_properties = ARRAY_SIZE(bq27x00_battery_props);
+ psy_desc->properties = bq27x00_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(bq27x00_battery_props);
}
- di->bat.get_property = bq27x00_battery_get_property;
- di->bat.external_power_changed = bq27x00_external_power_changed;
+ psy_desc->get_property = bq27x00_battery_get_property;
+ psy_desc->external_power_changed = bq27x00_external_power_changed;
INIT_DELAYED_WORK(&di->work, bq27x00_battery_poll);
mutex_init(&di->lock);
- ret = power_supply_register(di->dev, &di->bat);
- if (ret) {
+ di->bat = power_supply_register_no_ws(di->dev, psy_desc, &psy_cfg);
+ if (IS_ERR(di->bat)) {
+ ret = PTR_ERR(di->bat);
dev_err(di->dev, "failed to register battery: %d\n", ret);
return ret;
}
cancel_delayed_work_sync(&di->work);
- power_supply_unregister(&di->bat);
+ power_supply_unregister(di->bat);
mutex_destroy(&di->lock);
}
if (num < 0)
return num;
- name = kasprintf(GFP_KERNEL, "%s-%d", id->name, num);
+ name = devm_kasprintf(&client->dev, GFP_KERNEL, "%s-%d", id->name, num);
if (!name) {
dev_err(&client->dev, "failed to allocate device name\n");
retval = -ENOMEM;
- goto batt_failed_1;
+ goto batt_failed;
}
di = devm_kzalloc(&client->dev, sizeof(*di), GFP_KERNEL);
if (!di) {
dev_err(&client->dev, "failed to allocate device info data\n");
retval = -ENOMEM;
- goto batt_failed_2;
+ goto batt_failed;
}
di->id = num;
di->dev = &client->dev;
di->chip = id->driver_data;
- di->bat.name = name;
di->bus.read = &bq27x00_read_i2c;
- retval = bq27x00_powersupply_init(di);
+ retval = bq27x00_powersupply_init(di, name);
if (retval)
- goto batt_failed_2;
+ goto batt_failed;
i2c_set_clientdata(client, di);
return 0;
-batt_failed_2:
- kfree(name);
-batt_failed_1:
+batt_failed:
mutex_lock(&battery_mutex);
idr_remove(&battery_id, num);
mutex_unlock(&battery_mutex);
bq27x00_powersupply_unregister(di);
- kfree(di->bat.name);
-
mutex_lock(&battery_mutex);
idr_remove(&battery_id, di->id);
mutex_unlock(&battery_mutex);
{ "bq27500", BQ27500 },
{ "bq27425", BQ27425 },
{ "bq27742", BQ27742 },
+ { "bq27510", BQ27510 },
{},
};
MODULE_DEVICE_TABLE(i2c, bq27x00_id);
{
struct bq27x00_device_info *di;
struct bq27000_platform_data *pdata = pdev->dev.platform_data;
+ const char *name;
if (!pdata) {
dev_err(&pdev->dev, "no platform_data supplied\n");
di->dev = &pdev->dev;
di->chip = BQ27000;
- di->bat.name = pdata->name ?: dev_name(&pdev->dev);
+ name = pdata->name ?: dev_name(&pdev->dev);
di->bus.read = &bq27000_read_platform;
- return bq27x00_powersupply_init(di);
+ return bq27x00_powersupply_init(di, name);
}
static int bq27000_battery_remove(struct platform_device *pdev)
if (!psy)
break;
- ret = psy->get_property(psy,
- POWER_SUPPLY_PROP_PRESENT, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_PRESENT,
+ &val);
if (ret == 0 && val.intval)
present = true;
+ power_supply_put(psy);
break;
case CM_CHARGER_STAT:
for (i = 0; cm->desc->psy_charger_stat[i]; i++) {
continue;
}
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_PRESENT,
- &val);
+ ret = power_supply_get_property(psy,
+ POWER_SUPPLY_PROP_PRESENT, &val);
+ power_supply_put(psy);
if (ret == 0 && val.intval) {
present = true;
break;
continue;
}
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_ONLINE,
+ &val);
+ power_supply_put(psy);
if (ret == 0 && val.intval) {
online = true;
break;
if (!fuel_gauge)
return -ENODEV;
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_VOLTAGE_NOW, &val);
+ power_supply_put(fuel_gauge);
if (ret)
return ret;
}
/* 2. The charger should be online (ext-power) */
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_ONLINE,
+ &val);
if (ret) {
dev_warn(cm->dev, "Cannot read ONLINE value from %s\n",
cm->desc->psy_charger_stat[i]);
+ power_supply_put(psy);
continue;
}
- if (val.intval == 0)
+ if (val.intval == 0) {
+ power_supply_put(psy);
continue;
+ }
/*
* 3. The charger should not be FULL, DISCHARGING,
* or NOT_CHARGING.
*/
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_STATUS, &val);
+ ret = power_supply_get_property(psy, POWER_SUPPLY_PROP_STATUS,
+ &val);
+ power_supply_put(psy);
if (ret) {
dev_warn(cm->dev, "Cannot read STATUS value from %s\n",
cm->desc->psy_charger_stat[i]);
struct charger_desc *desc = cm->desc;
union power_supply_propval val;
struct power_supply *fuel_gauge;
+ bool is_full = false;
int ret = 0;
int uV;
val.intval = 0;
/* Not full if capacity of fuel gauge isn't full */
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CHARGE_FULL, &val);
- if (!ret && val.intval > desc->fullbatt_full_capacity)
- return true;
+ if (!ret && val.intval > desc->fullbatt_full_capacity) {
+ is_full = true;
+ goto out;
+ }
}
/* Full, if it's over the fullbatt voltage */
if (desc->fullbatt_uV > 0) {
ret = get_batt_uV(cm, &uV);
- if (!ret && uV >= desc->fullbatt_uV)
- return true;
+ if (!ret && uV >= desc->fullbatt_uV) {
+ is_full = true;
+ goto out;
+ }
}
/* Full, if the capacity is more than fullbatt_soc */
if (desc->fullbatt_soc > 0) {
val.intval = 0;
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CAPACITY, &val);
- if (!ret && val.intval >= desc->fullbatt_soc)
- return true;
+ if (!ret && val.intval >= desc->fullbatt_soc) {
+ is_full = true;
+ goto out;
+ }
}
- return false;
+out:
+ power_supply_put(fuel_gauge);
+ return is_full;
}
/**
int *temp)
{
struct power_supply *fuel_gauge;
+ int ret;
fuel_gauge = power_supply_get_by_name(cm->desc->psy_fuel_gauge);
if (!fuel_gauge)
return -ENODEV;
- return fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_TEMP,
(union power_supply_propval *)temp);
+ power_supply_put(fuel_gauge);
+
+ return ret;
}
static int cm_get_battery_temperature(struct charger_manager *cm,
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct charger_manager *cm = container_of(psy,
- struct charger_manager, charger_psy);
+ struct charger_manager *cm = power_supply_get_drvdata(psy);
struct charger_desc *desc = cm->desc;
- struct power_supply *fuel_gauge;
+ struct power_supply *fuel_gauge = NULL;
int ret = 0;
int uV;
ret = -ENODEV;
break;
}
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CURRENT_NOW, val);
break;
case POWER_SUPPLY_PROP_TEMP:
case POWER_SUPPLY_PROP_TEMP_AMBIENT:
return cm_get_battery_temperature(cm, &val->intval);
case POWER_SUPPLY_PROP_CAPACITY:
- fuel_gauge = power_supply_get_by_name(cm->desc->psy_fuel_gauge);
- if (!fuel_gauge) {
- ret = -ENODEV;
- break;
- }
-
if (!is_batt_present(cm)) {
/* There is no battery. Assume 100% */
val->intval = 100;
break;
}
- ret = fuel_gauge->get_property(fuel_gauge,
+ fuel_gauge = power_supply_get_by_name(cm->desc->psy_fuel_gauge);
+ if (!fuel_gauge) {
+ ret = -ENODEV;
+ break;
+ }
+
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CAPACITY, val);
if (ret)
break;
break;
}
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CHARGE_NOW,
val);
if (ret) {
default:
return -EINVAL;
}
+ if (fuel_gauge)
+ power_supply_put(fuel_gauge);
return ret;
}
*/
};
-static struct power_supply psy_default = {
+static const struct power_supply_desc psy_default = {
.name = "battery",
.type = POWER_SUPPLY_TYPE_BATTERY,
.properties = default_charger_props,
dev_info(cm->dev, "'%s' regulator's externally_control is %d\n",
charger->regulator_name, charger->externally_control);
- ret = sysfs_create_group(&cm->charger_psy.dev->kobj,
+ ret = sysfs_create_group(&cm->charger_psy->dev.kobj,
&charger->attr_g);
if (ret < 0) {
dev_err(cm->dev, "Cannot create sysfs entry of %s regulator\n",
int ret;
/* Verify whether fuel gauge provides battery temperature */
- ret = fuel_gauge->get_property(fuel_gauge,
+ ret = power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_TEMP, &val);
if (!ret) {
- cm->charger_psy.properties[cm->charger_psy.num_properties] =
+ cm->charger_psy_desc.properties[cm->charger_psy_desc.num_properties] =
POWER_SUPPLY_PROP_TEMP;
- cm->charger_psy.num_properties++;
+ cm->charger_psy_desc.num_properties++;
cm->desc->measure_battery_temp = true;
}
#ifdef CONFIG_THERMAL
return PTR_ERR(cm->tzd_batt);
/* Use external thermometer */
- cm->charger_psy.properties[cm->charger_psy.num_properties] =
+ cm->charger_psy_desc.properties[cm->charger_psy_desc.num_properties] =
POWER_SUPPLY_PROP_TEMP_AMBIENT;
- cm->charger_psy.num_properties++;
+ cm->charger_psy_desc.num_properties++;
cm->desc->measure_battery_temp = true;
ret = 0;
}
return ret;
}
-static struct of_device_id charger_manager_match[] = {
+static const struct of_device_id charger_manager_match[] = {
{
.compatible = "charger-manager",
},
int j = 0;
union power_supply_propval val;
struct power_supply *fuel_gauge;
+ struct power_supply_config psy_cfg = {};
if (IS_ERR(desc)) {
dev_err(&pdev->dev, "No platform data (desc) found\n");
/* Basic Values. Unspecified are Null or 0 */
cm->dev = &pdev->dev;
cm->desc = desc;
+ psy_cfg.drv_data = cm;
/* Initialize alarm timer */
if (alarmtimer_get_rtcdev()) {
desc->psy_charger_stat[i]);
return -ENODEV;
}
- }
-
- fuel_gauge = power_supply_get_by_name(desc->psy_fuel_gauge);
- if (!fuel_gauge) {
- dev_err(&pdev->dev, "Cannot find power supply \"%s\"\n",
- desc->psy_fuel_gauge);
- return -ENODEV;
+ power_supply_put(psy);
}
if (desc->polling_interval_ms == 0 ||
platform_set_drvdata(pdev, cm);
- memcpy(&cm->charger_psy, &psy_default, sizeof(psy_default));
+ memcpy(&cm->charger_psy_desc, &psy_default, sizeof(psy_default));
if (!desc->psy_name)
strncpy(cm->psy_name_buf, psy_default.name, PSY_NAME_MAX);
else
strncpy(cm->psy_name_buf, desc->psy_name, PSY_NAME_MAX);
- cm->charger_psy.name = cm->psy_name_buf;
+ cm->charger_psy_desc.name = cm->psy_name_buf;
/* Allocate for psy properties because they may vary */
- cm->charger_psy.properties = devm_kzalloc(&pdev->dev,
+ cm->charger_psy_desc.properties = devm_kzalloc(&pdev->dev,
sizeof(enum power_supply_property)
* (ARRAY_SIZE(default_charger_props) +
NUM_CHARGER_PSY_OPTIONAL), GFP_KERNEL);
- if (!cm->charger_psy.properties)
+ if (!cm->charger_psy_desc.properties)
return -ENOMEM;
- memcpy(cm->charger_psy.properties, default_charger_props,
+ memcpy(cm->charger_psy_desc.properties, default_charger_props,
sizeof(enum power_supply_property) *
ARRAY_SIZE(default_charger_props));
- cm->charger_psy.num_properties = psy_default.num_properties;
+ cm->charger_psy_desc.num_properties = psy_default.num_properties;
/* Find which optional psy-properties are available */
- if (!fuel_gauge->get_property(fuel_gauge,
+ fuel_gauge = power_supply_get_by_name(desc->psy_fuel_gauge);
+ if (!fuel_gauge) {
+ dev_err(&pdev->dev, "Cannot find power supply \"%s\"\n",
+ desc->psy_fuel_gauge);
+ return -ENODEV;
+ }
+ if (!power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CHARGE_NOW, &val)) {
- cm->charger_psy.properties[cm->charger_psy.num_properties] =
+ cm->charger_psy_desc.properties[cm->charger_psy_desc.num_properties] =
POWER_SUPPLY_PROP_CHARGE_NOW;
- cm->charger_psy.num_properties++;
+ cm->charger_psy_desc.num_properties++;
}
- if (!fuel_gauge->get_property(fuel_gauge,
+ if (!power_supply_get_property(fuel_gauge,
POWER_SUPPLY_PROP_CURRENT_NOW,
&val)) {
- cm->charger_psy.properties[cm->charger_psy.num_properties] =
+ cm->charger_psy_desc.properties[cm->charger_psy_desc.num_properties] =
POWER_SUPPLY_PROP_CURRENT_NOW;
- cm->charger_psy.num_properties++;
+ cm->charger_psy_desc.num_properties++;
}
ret = cm_init_thermal_data(cm, fuel_gauge);
dev_err(&pdev->dev, "Failed to initialize thermal data\n");
cm->desc->measure_battery_temp = false;
}
+ power_supply_put(fuel_gauge);
INIT_DELAYED_WORK(&cm->fullbatt_vchk_work, fullbatt_vchk);
- ret = power_supply_register(NULL, &cm->charger_psy);
- if (ret) {
+ cm->charger_psy = power_supply_register(NULL, &cm->charger_psy_desc,
+ &psy_cfg);
+ if (IS_ERR(cm->charger_psy)) {
dev_err(&pdev->dev, "Cannot register charger-manager with name \"%s\"\n",
- cm->charger_psy.name);
- return ret;
+ cm->charger_psy_desc.name);
+ return PTR_ERR(cm->charger_psy);
}
/* Register extcon device for charger cable */
struct charger_regulator *charger;
charger = &desc->charger_regulators[i];
- sysfs_remove_group(&cm->charger_psy.dev->kobj,
+ sysfs_remove_group(&cm->charger_psy->dev.kobj,
&charger->attr_g);
}
err_reg_extcon:
regulator_put(desc->charger_regulators[i].consumer);
}
- power_supply_unregister(&cm->charger_psy);
+ power_supply_unregister(cm->charger_psy);
return ret;
}
for (i = 0 ; i < desc->num_charger_regulators ; i++)
regulator_put(desc->charger_regulators[i].consumer);
- power_supply_unregister(&cm->charger_psy);
+ power_supply_unregister(cm->charger_psy);
try_charger_enable(cm, false);
bool found = false;
for (i = 0; cm->desc->psy_charger_stat[i]; i++) {
- if (!strcmp(psy->name, cm->desc->psy_charger_stat[i])) {
+ if (!strcmp(psy->desc->name, cm->desc->psy_charger_stat[i])) {
found = true;
break;
}
struct collie_bat {
int status;
- struct power_supply psy;
+ struct power_supply *psy;
int full_chrg;
struct mutex work_lock; /* protects data */
union power_supply_propval *val)
{
int ret = 0;
- struct collie_bat *bat = container_of(psy, struct collie_bat, psy);
+ struct collie_bat *bat = power_supply_get_drvdata(psy);
if (bat->is_present && !bat->is_present(bat)
&& psp != POWER_SUPPLY_PROP_PRESENT) {
static void collie_bat_update(struct collie_bat *bat)
{
int old;
- struct power_supply *psy = &bat->psy;
+ struct power_supply *psy = bat->psy;
mutex_lock(&bat->work_lock);
old = bat->status;
if (bat->is_present && !bat->is_present(bat)) {
- printk(KERN_NOTICE "%s not present\n", psy->name);
+ printk(KERN_NOTICE "%s not present\n", psy->desc->name);
bat->status = POWER_SUPPLY_STATUS_UNKNOWN;
bat->full_chrg = -1;
} else if (power_supply_am_i_supplied(psy)) {
POWER_SUPPLY_PROP_PRESENT,
};
+static const struct power_supply_desc collie_bat_main_desc = {
+ .name = "main-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = collie_bat_main_props,
+ .num_properties = ARRAY_SIZE(collie_bat_main_props),
+ .get_property = collie_bat_get_property,
+ .external_power_changed = collie_bat_external_power_changed,
+ .use_for_apm = 1,
+};
+
static struct collie_bat collie_bat_main = {
.status = POWER_SUPPLY_STATUS_DISCHARGING,
.full_chrg = -1,
- .psy = {
- .name = "main-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = collie_bat_main_props,
- .num_properties = ARRAY_SIZE(collie_bat_main_props),
- .get_property = collie_bat_get_property,
- .external_power_changed = collie_bat_external_power_changed,
- .use_for_apm = 1,
- },
+ .psy = NULL,
.gpio_full = COLLIE_GPIO_CO,
.gpio_charge_on = COLLIE_GPIO_CHARGE_ON,
.adc_temp_divider = 10000,
};
+static const struct power_supply_desc collie_bat_bu_desc = {
+ .name = "backup-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = collie_bat_bu_props,
+ .num_properties = ARRAY_SIZE(collie_bat_bu_props),
+ .get_property = collie_bat_get_property,
+ .external_power_changed = collie_bat_external_power_changed,
+};
+
static struct collie_bat collie_bat_bu = {
.status = POWER_SUPPLY_STATUS_UNKNOWN,
.full_chrg = -1,
-
- .psy = {
- .name = "backup-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = collie_bat_bu_props,
- .num_properties = ARRAY_SIZE(collie_bat_bu_props),
- .get_property = collie_bat_get_property,
- .external_power_changed = collie_bat_external_power_changed,
- },
+ .psy = NULL,
.gpio_full = -1,
.gpio_charge_on = -1,
static int collie_bat_probe(struct ucb1x00_dev *dev)
{
int ret;
+ struct power_supply_config psy_main_cfg = {}, psy_bu_cfg = {};
if (!machine_is_collie())
return -ENODEV;
INIT_WORK(&bat_work, collie_bat_work);
- ret = power_supply_register(&dev->ucb->dev, &collie_bat_main.psy);
- if (ret)
+ psy_main_cfg.drv_data = &collie_bat_main;
+ collie_bat_main.psy = power_supply_register(&dev->ucb->dev,
+ &collie_bat_main_desc,
+ &psy_main_cfg);
+ if (IS_ERR(collie_bat_main.psy)) {
+ ret = PTR_ERR(collie_bat_main.psy);
goto err_psy_reg_main;
- ret = power_supply_register(&dev->ucb->dev, &collie_bat_bu.psy);
- if (ret)
+ }
+
+ psy_main_cfg.drv_data = &collie_bat_bu;
+ collie_bat_bu.psy = power_supply_register(&dev->ucb->dev,
+ &collie_bat_bu_desc,
+ &psy_bu_cfg);
+ if (IS_ERR(collie_bat_bu.psy)) {
+ ret = PTR_ERR(collie_bat_bu.psy);
goto err_psy_reg_bu;
+ }
ret = request_irq(gpio_to_irq(COLLIE_GPIO_CO),
collie_bat_gpio_isr,
return 0;
err_irq:
- power_supply_unregister(&collie_bat_bu.psy);
+ power_supply_unregister(collie_bat_bu.psy);
err_psy_reg_bu:
- power_supply_unregister(&collie_bat_main.psy);
+ power_supply_unregister(collie_bat_main.psy);
err_psy_reg_main:
/* see comment in collie_bat_remove */
{
free_irq(gpio_to_irq(COLLIE_GPIO_CO), &collie_bat_main);
- power_supply_unregister(&collie_bat_bu.psy);
- power_supply_unregister(&collie_bat_main.psy);
+ power_supply_unregister(collie_bat_bu.psy);
+ power_supply_unregister(collie_bat_main.psy);
/*
* Now cancel the bat_work. We won't get any more schedules,
};
struct da9030_charger {
- struct power_supply psy;
+ struct power_supply *psy;
+ struct power_supply_desc psy_desc;
struct device *master;
da903x_write(charger->master, DA9030_CHARGE_CONTROL, val);
- power_supply_changed(&charger->psy);
+ power_supply_changed(charger->psy);
}
static void da9030_charger_check_state(struct da9030_charger *charger)
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct da9030_charger *charger;
- charger = container_of(psy, struct da9030_charger, psy);
+ struct da9030_charger *charger = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
static void da9030_battery_setup_psy(struct da9030_charger *charger)
{
- struct power_supply *psy = &charger->psy;
+ struct power_supply_desc *psy_desc = &charger->psy_desc;
struct power_supply_info *info = charger->battery_info;
- psy->name = info->name;
- psy->use_for_apm = info->use_for_apm;
- psy->type = POWER_SUPPLY_TYPE_BATTERY;
- psy->get_property = da9030_battery_get_property;
+ psy_desc->name = info->name;
+ psy_desc->use_for_apm = info->use_for_apm;
+ psy_desc->type = POWER_SUPPLY_TYPE_BATTERY;
+ psy_desc->get_property = da9030_battery_get_property;
- psy->properties = da9030_battery_props;
- psy->num_properties = ARRAY_SIZE(da9030_battery_props);
+ psy_desc->properties = da9030_battery_props;
+ psy_desc->num_properties = ARRAY_SIZE(da9030_battery_props);
};
static int da9030_battery_charger_init(struct da9030_charger *charger)
static int da9030_battery_probe(struct platform_device *pdev)
{
struct da9030_charger *charger;
+ struct power_supply_config psy_cfg = {};
struct da9030_battery_info *pdata = pdev->dev.platform_data;
int ret;
goto err_notifier;
da9030_battery_setup_psy(charger);
- ret = power_supply_register(&pdev->dev, &charger->psy);
- if (ret)
+ psy_cfg.drv_data = charger;
+ charger->psy = power_supply_register(&pdev->dev, &charger->psy_desc,
+ &psy_cfg);
+ if (IS_ERR(charger->psy)) {
+ ret = PTR_ERR(charger->psy);
goto err_ps_register;
+ }
charger->debug_file = da9030_bat_create_debugfs(charger);
platform_set_drvdata(pdev, charger);
DA9030_EVENT_CHIOVER | DA9030_EVENT_TBAT);
cancel_delayed_work_sync(&charger->work);
da9030_set_charge(charger, 0);
- power_supply_unregister(&charger->psy);
+ power_supply_unregister(charger->psy);
return 0;
}
struct da9052_battery {
struct da9052 *da9052;
- struct power_supply psy;
+ struct power_supply *psy;
struct notifier_block nb;
int charger_type;
int status;
if (irq == DA9052_IRQ_CHGEND || irq == DA9052_IRQ_DCIN ||
irq == DA9052_IRQ_VBUS || irq == DA9052_IRQ_TBAT) {
- power_supply_changed(&bat->psy);
+ power_supply_changed(bat->psy);
}
return IRQ_HANDLED;
{
int ret;
int illegal;
- struct da9052_battery *bat = container_of(psy, struct da9052_battery,
- psy);
+ struct da9052_battery *bat = power_supply_get_drvdata(psy);
ret = da9052_bat_check_presence(bat, &illegal);
if (ret < 0)
POWER_SUPPLY_PROP_TECHNOLOGY,
};
-static struct power_supply template_battery = {
+static struct power_supply_desc psy_desc = {
.name = "da9052-bat",
.type = POWER_SUPPLY_TYPE_BATTERY,
.properties = da9052_bat_props,
{
struct da9052_pdata *pdata;
struct da9052_battery *bat;
+ struct power_supply_config psy_cfg = {};
int ret;
int i;
if (!bat)
return -ENOMEM;
+ psy_cfg.drv_data = bat;
+
bat->da9052 = dev_get_drvdata(pdev->dev.parent);
- bat->psy = template_battery;
bat->charger_type = DA9052_NOCHARGER;
bat->status = POWER_SUPPLY_STATUS_UNKNOWN;
bat->health = POWER_SUPPLY_HEALTH_UNKNOWN;
pdata = bat->da9052->dev->platform_data;
if (pdata != NULL && pdata->use_for_apm)
- bat->psy.use_for_apm = pdata->use_for_apm;
+ psy_desc.use_for_apm = pdata->use_for_apm;
else
- bat->psy.use_for_apm = 1;
+ psy_desc.use_for_apm = 1;
for (i = 0; i < ARRAY_SIZE(da9052_bat_irqs); i++) {
ret = da9052_request_irq(bat->da9052,
}
}
- ret = power_supply_register(&pdev->dev, &bat->psy);
- if (ret)
+ bat->psy = power_supply_register(&pdev->dev, &psy_desc, &psy_cfg);
+ if (IS_ERR(bat->psy)) {
+ ret = PTR_ERR(bat->psy);
goto err;
+ }
platform_set_drvdata(pdev, bat);
return 0;
for (i = 0; i < ARRAY_SIZE(da9052_bat_irqs); i++)
da9052_free_irq(bat->da9052, da9052_bat_irq_bits[i], bat);
- power_supply_unregister(&bat->psy);
+ power_supply_unregister(bat->psy);
return 0;
}
--- /dev/null
+/*
+ * DA9150 Charger Driver
+ *
+ * Copyright (c) 2014 Dialog Semiconductor
+ *
+ * Author: Adam Thomson <Adam.Thomson.Opensource@diasemi.com>
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms of the GNU General Public License as published by the
+ * Free Software Foundation; either version 2 of the License, or (at your
+ * option) any later version.
+ */
+
+#include <linux/kernel.h>
+#include <linux/slab.h>
+#include <linux/module.h>
+#include <linux/platform_device.h>
+#include <linux/of.h>
+#include <linux/of_platform.h>
+#include <linux/interrupt.h>
+#include <linux/power_supply.h>
+#include <linux/notifier.h>
+#include <linux/usb/phy.h>
+#include <linux/iio/consumer.h>
+#include <linux/mfd/da9150/core.h>
+#include <linux/mfd/da9150/registers.h>
+
+/* Private data */
+struct da9150_charger {
+ struct da9150 *da9150;
+ struct device *dev;
+
+ struct power_supply *usb;
+ struct power_supply *battery;
+ struct power_supply *supply_online;
+
+ struct usb_phy *usb_phy;
+ struct notifier_block otg_nb;
+ struct work_struct otg_work;
+ unsigned long usb_event;
+
+ struct iio_channel *ibus_chan;
+ struct iio_channel *vbus_chan;
+ struct iio_channel *tjunc_chan;
+ struct iio_channel *vbat_chan;
+};
+
+static inline int da9150_charger_supply_online(struct da9150_charger *charger,
+ struct power_supply *psy,
+ union power_supply_propval *val)
+{
+ val->intval = (psy == charger->supply_online) ? 1 : 0;
+
+ return 0;
+}
+
+/* Charger Properties */
+static int da9150_charger_vbus_voltage_now(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ int v_val, ret;
+
+ /* Read processed value - mV units */
+ ret = iio_read_channel_processed(charger->vbus_chan, &v_val);
+ if (ret < 0)
+ return ret;
+
+ /* Convert voltage to expected uV units */
+ val->intval = v_val * 1000;
+
+ return 0;
+}
+
+static int da9150_charger_ibus_current_avg(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ int i_val, ret;
+
+ /* Read processed value - mA units */
+ ret = iio_read_channel_processed(charger->ibus_chan, &i_val);
+ if (ret < 0)
+ return ret;
+
+ /* Convert current to expected uA units */
+ val->intval = i_val * 1000;
+
+ return 0;
+}
+
+static int da9150_charger_tjunc_temp(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ int t_val, ret;
+
+ /* Read processed value - 0.001 degrees C units */
+ ret = iio_read_channel_processed(charger->tjunc_chan, &t_val);
+ if (ret < 0)
+ return ret;
+
+ /* Convert temp to expect 0.1 degrees C units */
+ val->intval = t_val / 100;
+
+ return 0;
+}
+
+static enum power_supply_property da9150_charger_props[] = {
+ POWER_SUPPLY_PROP_ONLINE,
+ POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ POWER_SUPPLY_PROP_CURRENT_AVG,
+ POWER_SUPPLY_PROP_TEMP,
+};
+
+static int da9150_charger_get_prop(struct power_supply *psy,
+ enum power_supply_property psp,
+ union power_supply_propval *val)
+{
+ struct da9150_charger *charger = dev_get_drvdata(psy->dev.parent);
+ int ret;
+
+ switch (psp) {
+ case POWER_SUPPLY_PROP_ONLINE:
+ ret = da9150_charger_supply_online(charger, psy, val);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_NOW:
+ ret = da9150_charger_vbus_voltage_now(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_CURRENT_AVG:
+ ret = da9150_charger_ibus_current_avg(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_TEMP:
+ ret = da9150_charger_tjunc_temp(charger, val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+/* Battery Properties */
+static int da9150_charger_battery_status(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ /* Check to see if battery is discharging */
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_H);
+
+ if (((reg & DA9150_VBUS_STAT_MASK) == DA9150_VBUS_STAT_OFF) ||
+ ((reg & DA9150_VBUS_STAT_MASK) == DA9150_VBUS_STAT_WAIT)) {
+ val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
+
+ return 0;
+ }
+
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_J);
+
+ /* Now check for other states */
+ switch (reg & DA9150_CHG_STAT_MASK) {
+ case DA9150_CHG_STAT_ACT:
+ case DA9150_CHG_STAT_PRE:
+ case DA9150_CHG_STAT_CC:
+ case DA9150_CHG_STAT_CV:
+ val->intval = POWER_SUPPLY_STATUS_CHARGING;
+ break;
+ case DA9150_CHG_STAT_OFF:
+ case DA9150_CHG_STAT_SUSP:
+ case DA9150_CHG_STAT_TEMP:
+ case DA9150_CHG_STAT_TIME:
+ case DA9150_CHG_STAT_BAT:
+ val->intval = POWER_SUPPLY_STATUS_NOT_CHARGING;
+ break;
+ case DA9150_CHG_STAT_FULL:
+ val->intval = POWER_SUPPLY_STATUS_FULL;
+ break;
+ default:
+ val->intval = POWER_SUPPLY_STATUS_UNKNOWN;
+ break;
+ }
+
+ return 0;
+}
+
+static int da9150_charger_battery_health(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_J);
+
+ /* Check if temperature limit reached */
+ switch (reg & DA9150_CHG_TEMP_MASK) {
+ case DA9150_CHG_TEMP_UNDER:
+ val->intval = POWER_SUPPLY_HEALTH_COLD;
+ return 0;
+ case DA9150_CHG_TEMP_OVER:
+ val->intval = POWER_SUPPLY_HEALTH_OVERHEAT;
+ return 0;
+ default:
+ break;
+ }
+
+ /* Check for other health states */
+ switch (reg & DA9150_CHG_STAT_MASK) {
+ case DA9150_CHG_STAT_ACT:
+ case DA9150_CHG_STAT_PRE:
+ val->intval = POWER_SUPPLY_HEALTH_DEAD;
+ break;
+ case DA9150_CHG_STAT_TIME:
+ val->intval = POWER_SUPPLY_HEALTH_UNSPEC_FAILURE;
+ break;
+ default:
+ val->intval = POWER_SUPPLY_HEALTH_GOOD;
+ break;
+ }
+
+ return 0;
+}
+
+static int da9150_charger_battery_present(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ /* Check if battery present or removed */
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_J);
+ if ((reg & DA9150_CHG_STAT_MASK) == DA9150_CHG_STAT_BAT)
+ val->intval = 0;
+ else
+ val->intval = 1;
+
+ return 0;
+}
+
+static int da9150_charger_battery_charge_type(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_J);
+
+ switch (reg & DA9150_CHG_STAT_MASK) {
+ case DA9150_CHG_STAT_CC:
+ val->intval = POWER_SUPPLY_CHARGE_TYPE_FAST;
+ break;
+ case DA9150_CHG_STAT_ACT:
+ case DA9150_CHG_STAT_PRE:
+ case DA9150_CHG_STAT_CV:
+ val->intval = POWER_SUPPLY_CHARGE_TYPE_TRICKLE;
+ break;
+ default:
+ val->intval = POWER_SUPPLY_CHARGE_TYPE_NONE;
+ break;
+ }
+
+ return 0;
+}
+
+static int da9150_charger_battery_voltage_min(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_PPR_CHGCTRL_C);
+
+ /* Value starts at 2500 mV, 50 mV increments, presented in uV */
+ val->intval = ((reg & DA9150_CHG_VFAULT_MASK) * 50000) + 2500000;
+
+ return 0;
+}
+
+static int da9150_charger_battery_voltage_now(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ int v_val, ret;
+
+ /* Read processed value - mV units */
+ ret = iio_read_channel_processed(charger->vbat_chan, &v_val);
+ if (ret < 0)
+ return ret;
+
+ val->intval = v_val * 1000;
+
+ return 0;
+}
+
+static int da9150_charger_battery_current_max(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ int reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_PPR_CHGCTRL_D);
+
+ /* 25mA increments */
+ val->intval = reg * 25000;
+
+ return 0;
+}
+
+static int da9150_charger_battery_voltage_max(struct da9150_charger *charger,
+ union power_supply_propval *val)
+{
+ u8 reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_PPR_CHGCTRL_B);
+
+ /* Value starts at 3650 mV, 25 mV increments, presented in uV */
+ val->intval = ((reg & DA9150_CHG_VBAT_MASK) * 25000) + 3650000;
+ return 0;
+}
+
+static enum power_supply_property da9150_charger_bat_props[] = {
+ POWER_SUPPLY_PROP_STATUS,
+ POWER_SUPPLY_PROP_ONLINE,
+ POWER_SUPPLY_PROP_HEALTH,
+ POWER_SUPPLY_PROP_PRESENT,
+ POWER_SUPPLY_PROP_CHARGE_TYPE,
+ POWER_SUPPLY_PROP_VOLTAGE_MIN_DESIGN,
+ POWER_SUPPLY_PROP_VOLTAGE_NOW,
+ POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX,
+ POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX,
+};
+
+static int da9150_charger_battery_get_prop(struct power_supply *psy,
+ enum power_supply_property psp,
+ union power_supply_propval *val)
+{
+ struct da9150_charger *charger = dev_get_drvdata(psy->dev.parent);
+ int ret;
+
+ switch (psp) {
+ case POWER_SUPPLY_PROP_STATUS:
+ ret = da9150_charger_battery_status(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_ONLINE:
+ ret = da9150_charger_supply_online(charger, psy, val);
+ break;
+ case POWER_SUPPLY_PROP_HEALTH:
+ ret = da9150_charger_battery_health(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_PRESENT:
+ ret = da9150_charger_battery_present(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_CHARGE_TYPE:
+ ret = da9150_charger_battery_charge_type(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_MIN_DESIGN:
+ ret = da9150_charger_battery_voltage_min(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_VOLTAGE_NOW:
+ ret = da9150_charger_battery_voltage_now(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_CURRENT_MAX:
+ ret = da9150_charger_battery_current_max(charger, val);
+ break;
+ case POWER_SUPPLY_PROP_CONSTANT_CHARGE_VOLTAGE_MAX:
+ ret = da9150_charger_battery_voltage_max(charger, val);
+ break;
+ default:
+ ret = -EINVAL;
+ break;
+ }
+
+ return ret;
+}
+
+static irqreturn_t da9150_charger_chg_irq(int irq, void *data)
+{
+ struct da9150_charger *charger = data;
+
+ power_supply_changed(charger->battery);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t da9150_charger_tjunc_irq(int irq, void *data)
+{
+ struct da9150_charger *charger = data;
+
+ /* Nothing we can really do except report this. */
+ dev_crit(charger->dev, "TJunc over temperature!!!\n");
+ power_supply_changed(charger->usb);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t da9150_charger_vfault_irq(int irq, void *data)
+{
+ struct da9150_charger *charger = data;
+
+ /* Nothing we can really do except report this. */
+ dev_crit(charger->dev, "VSYS under voltage!!!\n");
+ power_supply_changed(charger->usb);
+ power_supply_changed(charger->battery);
+
+ return IRQ_HANDLED;
+}
+
+static irqreturn_t da9150_charger_vbus_irq(int irq, void *data)
+{
+ struct da9150_charger *charger = data;
+ u8 reg;
+
+ reg = da9150_reg_read(charger->da9150, DA9150_STATUS_H);
+
+ /* Charger plugged in or battery only */
+ switch (reg & DA9150_VBUS_STAT_MASK) {
+ case DA9150_VBUS_STAT_OFF:
+ case DA9150_VBUS_STAT_WAIT:
+ charger->supply_online = charger->battery;
+ break;
+ case DA9150_VBUS_STAT_CHG:
+ charger->supply_online = charger->usb;
+ break;
+ default:
+ dev_warn(charger->dev, "Unknown VBUS state - reg = 0x%x\n",
+ reg);
+ charger->supply_online = NULL;
+ break;
+ }
+
+ power_supply_changed(charger->usb);
+ power_supply_changed(charger->battery);
+
+ return IRQ_HANDLED;
+}
+
+static void da9150_charger_otg_work(struct work_struct *data)
+{
+ struct da9150_charger *charger =
+ container_of(data, struct da9150_charger, otg_work);
+
+ switch (charger->usb_event) {
+ case USB_EVENT_ID:
+ /* Enable OTG Boost */
+ da9150_set_bits(charger->da9150, DA9150_PPR_BKCTRL_A,
+ DA9150_VBUS_MODE_MASK, DA9150_VBUS_MODE_OTG);
+ break;
+ case USB_EVENT_NONE:
+ /* Revert to charge mode */
+ power_supply_changed(charger->usb);
+ power_supply_changed(charger->battery);
+ da9150_set_bits(charger->da9150, DA9150_PPR_BKCTRL_A,
+ DA9150_VBUS_MODE_MASK, DA9150_VBUS_MODE_CHG);
+ break;
+ }
+}
+
+static int da9150_charger_otg_ncb(struct notifier_block *nb, unsigned long val,
+ void *priv)
+{
+ struct da9150_charger *charger =
+ container_of(nb, struct da9150_charger, otg_nb);
+
+ dev_dbg(charger->dev, "DA9150 OTG notify %lu\n", val);
+
+ charger->usb_event = val;
+ schedule_work(&charger->otg_work);
+
+ return NOTIFY_OK;
+}
+
+static int da9150_charger_register_irq(struct platform_device *pdev,
+ irq_handler_t handler,
+ const char *irq_name)
+{
+ struct device *dev = &pdev->dev;
+ struct da9150_charger *charger = platform_get_drvdata(pdev);
+ int irq, ret;
+
+ irq = platform_get_irq_byname(pdev, irq_name);
+ if (irq < 0) {
+ dev_err(dev, "Failed to get IRQ CHG_STATUS: %d\n", irq);
+ return irq;
+ }
+
+ ret = request_threaded_irq(irq, NULL, handler, IRQF_ONESHOT, irq_name,
+ charger);
+ if (ret)
+ dev_err(dev, "Failed to request IRQ %d: %d\n", irq, ret);
+
+ return ret;
+}
+
+static void da9150_charger_unregister_irq(struct platform_device *pdev,
+ const char *irq_name)
+{
+ struct device *dev = &pdev->dev;
+ struct da9150_charger *charger = platform_get_drvdata(pdev);
+ int irq;
+
+ irq = platform_get_irq_byname(pdev, irq_name);
+ if (irq < 0) {
+ dev_err(dev, "Failed to get IRQ CHG_STATUS: %d\n", irq);
+ return;
+ }
+
+ free_irq(irq, charger);
+}
+
+static const struct power_supply_desc usb_desc = {
+ .name = "da9150-usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = da9150_charger_props,
+ .num_properties = ARRAY_SIZE(da9150_charger_props),
+ .get_property = da9150_charger_get_prop,
+};
+
+static const struct power_supply_desc battery_desc = {
+ .name = "da9150-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = da9150_charger_bat_props,
+ .num_properties = ARRAY_SIZE(da9150_charger_bat_props),
+ .get_property = da9150_charger_battery_get_prop,
+};
+
+static int da9150_charger_probe(struct platform_device *pdev)
+{
+ struct device *dev = &pdev->dev;
+ struct da9150 *da9150 = dev_get_drvdata(dev->parent);
+ struct da9150_charger *charger;
+ u8 reg;
+ int ret;
+
+ charger = devm_kzalloc(dev, sizeof(struct da9150_charger), GFP_KERNEL);
+ if (!charger)
+ return -ENOMEM;
+
+ platform_set_drvdata(pdev, charger);
+ charger->da9150 = da9150;
+ charger->dev = dev;
+
+ /* Acquire ADC channels */
+ charger->ibus_chan = iio_channel_get(dev, "CHAN_IBUS");
+ if (IS_ERR(charger->ibus_chan)) {
+ ret = PTR_ERR(charger->ibus_chan);
+ goto ibus_chan_fail;
+ }
+
+ charger->vbus_chan = iio_channel_get(dev, "CHAN_VBUS");
+ if (IS_ERR(charger->vbus_chan)) {
+ ret = PTR_ERR(charger->vbus_chan);
+ goto vbus_chan_fail;
+ }
+
+ charger->tjunc_chan = iio_channel_get(dev, "CHAN_TJUNC");
+ if (IS_ERR(charger->tjunc_chan)) {
+ ret = PTR_ERR(charger->tjunc_chan);
+ goto tjunc_chan_fail;
+ }
+
+ charger->vbat_chan = iio_channel_get(dev, "CHAN_VBAT");
+ if (IS_ERR(charger->vbat_chan)) {
+ ret = PTR_ERR(charger->vbat_chan);
+ goto vbat_chan_fail;
+ }
+
+ /* Register power supplies */
+ charger->usb = power_supply_register(dev, &usb_desc, NULL);
+ if (IS_ERR(charger->usb)) {
+ ret = PTR_ERR(charger->usb);
+ goto usb_fail;
+ }
+
+ charger->battery = power_supply_register(dev, &battery_desc, NULL);
+ if (IS_ERR(charger->battery)) {
+ ret = PTR_ERR(charger->battery);
+ goto battery_fail;
+ }
+
+ /* Get initial online supply */
+ reg = da9150_reg_read(da9150, DA9150_STATUS_H);
+
+ switch (reg & DA9150_VBUS_STAT_MASK) {
+ case DA9150_VBUS_STAT_OFF:
+ case DA9150_VBUS_STAT_WAIT:
+ charger->supply_online = charger->battery;
+ break;
+ case DA9150_VBUS_STAT_CHG:
+ charger->supply_online = charger->usb;
+ break;
+ default:
+ dev_warn(dev, "Unknown VBUS state - reg = 0x%x\n", reg);
+ charger->supply_online = NULL;
+ break;
+ }
+
+ /* Setup OTG reporting & configuration */
+ charger->usb_phy = devm_usb_get_phy(dev, USB_PHY_TYPE_USB2);
+ if (!IS_ERR_OR_NULL(charger->usb_phy)) {
+ INIT_WORK(&charger->otg_work, da9150_charger_otg_work);
+ charger->otg_nb.notifier_call = da9150_charger_otg_ncb;
+ usb_register_notifier(charger->usb_phy, &charger->otg_nb);
+ }
+
+ /* Register IRQs */
+ ret = da9150_charger_register_irq(pdev, da9150_charger_chg_irq,
+ "CHG_STATUS");
+ if (ret < 0)
+ goto chg_irq_fail;
+
+ ret = da9150_charger_register_irq(pdev, da9150_charger_tjunc_irq,
+ "CHG_TJUNC");
+ if (ret < 0)
+ goto tjunc_irq_fail;
+
+ ret = da9150_charger_register_irq(pdev, da9150_charger_vfault_irq,
+ "CHG_VFAULT");
+ if (ret < 0)
+ goto vfault_irq_fail;
+
+ ret = da9150_charger_register_irq(pdev, da9150_charger_vbus_irq,
+ "CHG_VBUS");
+ if (ret < 0)
+ goto vbus_irq_fail;
+
+ return 0;
+
+
+vbus_irq_fail:
+ da9150_charger_unregister_irq(pdev, "CHG_VFAULT");
+vfault_irq_fail:
+ da9150_charger_unregister_irq(pdev, "CHG_TJUNC");
+tjunc_irq_fail:
+ da9150_charger_unregister_irq(pdev, "CHG_STATUS");
+chg_irq_fail:
+ if (!IS_ERR_OR_NULL(charger->usb_phy))
+ usb_unregister_notifier(charger->usb_phy, &charger->otg_nb);
+battery_fail:
+ power_supply_unregister(charger->usb);
+
+usb_fail:
+ iio_channel_release(charger->vbat_chan);
+
+vbat_chan_fail:
+ iio_channel_release(charger->tjunc_chan);
+
+tjunc_chan_fail:
+ iio_channel_release(charger->vbus_chan);
+
+vbus_chan_fail:
+ iio_channel_release(charger->ibus_chan);
+
+ibus_chan_fail:
+ return ret;
+}
+
+static int da9150_charger_remove(struct platform_device *pdev)
+{
+ struct da9150_charger *charger = platform_get_drvdata(pdev);
+ int irq;
+
+ /* Make sure IRQs are released before unregistering power supplies */
+ irq = platform_get_irq_byname(pdev, "CHG_VBUS");
+ free_irq(irq, charger);
+
+ irq = platform_get_irq_byname(pdev, "CHG_VFAULT");
+ free_irq(irq, charger);
+
+ irq = platform_get_irq_byname(pdev, "CHG_TJUNC");
+ free_irq(irq, charger);
+
+ irq = platform_get_irq_byname(pdev, "CHG_STATUS");
+ free_irq(irq, charger);
+
+ if (!IS_ERR_OR_NULL(charger->usb_phy))
+ usb_unregister_notifier(charger->usb_phy, &charger->otg_nb);
+
+ power_supply_unregister(charger->battery);
+ power_supply_unregister(charger->usb);
+
+ /* Release ADC channels */
+ iio_channel_release(charger->ibus_chan);
+ iio_channel_release(charger->vbus_chan);
+ iio_channel_release(charger->tjunc_chan);
+ iio_channel_release(charger->vbat_chan);
+
+ return 0;
+}
+
+static struct platform_driver da9150_charger_driver = {
+ .driver = {
+ .name = "da9150-charger",
+ },
+ .probe = da9150_charger_probe,
+ .remove = da9150_charger_remove,
+};
+
+module_platform_driver(da9150_charger_driver);
+
+MODULE_DESCRIPTION("Charger Driver for DA9150");
+MODULE_AUTHOR("Adam Thomson <Adam.Thomson.Opensource@diasemi.com>");
+MODULE_LICENSE("GPL");
int charge_status; /* POWER_SUPPLY_STATUS_* */
int full_counter;
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct device *w1_dev;
struct workqueue_struct *monitor_wqueue;
struct delayed_work monitor_work;
if (di->charge_status == POWER_SUPPLY_STATUS_UNKNOWN)
di->full_counter = 0;
- if (power_supply_am_i_supplied(&di->bat)) {
+ if (power_supply_am_i_supplied(di->bat)) {
if (di->current_uA > 10000) {
di->charge_status = POWER_SUPPLY_STATUS_CHARGING;
di->full_counter = 0;
}
if (di->charge_status != old_charge_status)
- power_supply_changed(&di->bat);
+ power_supply_changed(di->bat);
}
static void ds2760_battery_write_status(struct ds2760_device_info *di,
queue_delayed_work(di->monitor_wqueue, &di->monitor_work, interval);
}
-#define to_ds2760_device_info(x) container_of((x), struct ds2760_device_info, \
- bat);
-
static void ds2760_battery_external_power_changed(struct power_supply *psy)
{
- struct ds2760_device_info *di = to_ds2760_device_info(psy);
+ struct ds2760_device_info *di = power_supply_get_drvdata(psy);
dev_dbg(di->dev, "%s\n", __func__);
* that error.
*/
- if (!power_supply_am_i_supplied(&di->bat))
+ if (!power_supply_am_i_supplied(di->bat))
return;
bias = (signed char) di->current_raw +
static void ds2760_battery_set_charged(struct power_supply *psy)
{
- struct ds2760_device_info *di = to_ds2760_device_info(psy);
+ struct ds2760_device_info *di = power_supply_get_drvdata(psy);
/* postpone the actual work by 20 secs. This is for debouncing GPIO
* signals and to let the current value settle. See AN4188. */
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct ds2760_device_info *di = to_ds2760_device_info(psy);
+ struct ds2760_device_info *di = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
enum power_supply_property psp,
const union power_supply_propval *val)
{
- struct ds2760_device_info *di = to_ds2760_device_info(psy);
+ struct ds2760_device_info *di = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_CHARGE_FULL:
static int ds2760_battery_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
char status;
int retval = 0;
struct ds2760_device_info *di;
platform_set_drvdata(pdev, di);
- di->dev = &pdev->dev;
- di->w1_dev = pdev->dev.parent;
- di->bat.name = dev_name(&pdev->dev);
- di->bat.type = POWER_SUPPLY_TYPE_BATTERY;
- di->bat.properties = ds2760_battery_props;
- di->bat.num_properties = ARRAY_SIZE(ds2760_battery_props);
- di->bat.get_property = ds2760_battery_get_property;
- di->bat.set_property = ds2760_battery_set_property;
- di->bat.property_is_writeable =
+ di->dev = &pdev->dev;
+ di->w1_dev = pdev->dev.parent;
+ di->bat_desc.name = dev_name(&pdev->dev);
+ di->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ di->bat_desc.properties = ds2760_battery_props;
+ di->bat_desc.num_properties = ARRAY_SIZE(ds2760_battery_props);
+ di->bat_desc.get_property = ds2760_battery_get_property;
+ di->bat_desc.set_property = ds2760_battery_set_property;
+ di->bat_desc.property_is_writeable =
ds2760_battery_property_is_writeable;
- di->bat.set_charged = ds2760_battery_set_charged;
- di->bat.external_power_changed =
+ di->bat_desc.set_charged = ds2760_battery_set_charged;
+ di->bat_desc.external_power_changed =
ds2760_battery_external_power_changed;
+ psy_cfg.drv_data = di;
+
di->charge_status = POWER_SUPPLY_STATUS_UNKNOWN;
/* enable sleep mode feature */
if (current_accum)
ds2760_battery_set_current_accum(di, current_accum);
- retval = power_supply_register(&pdev->dev, &di->bat);
- if (retval) {
+ di->bat = power_supply_register(&pdev->dev, &di->bat_desc, &psy_cfg);
+ if (IS_ERR(di->bat)) {
dev_err(di->dev, "failed to register battery\n");
+ retval = PTR_ERR(di->bat);
goto batt_failed;
}
goto success;
workqueue_failed:
- power_supply_unregister(&di->bat);
+ power_supply_unregister(di->bat);
batt_failed:
di_alloc_failed:
success:
cancel_delayed_work_sync(&di->monitor_work);
cancel_delayed_work_sync(&di->set_charged_work);
destroy_workqueue(di->monitor_wqueue);
- power_supply_unregister(&di->bat);
+ power_supply_unregister(di->bat);
return 0;
}
struct ds2760_device_info *di = platform_get_drvdata(pdev);
di->charge_status = POWER_SUPPLY_STATUS_UNKNOWN;
- power_supply_changed(&di->bat);
+ power_supply_changed(di->bat);
mod_delayed_work(di->monitor_wqueue, &di->monitor_work, HZ);
struct ds2780_device_info {
struct device *dev;
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct device *w1_dev;
};
static inline struct ds2780_device_info *
to_ds2780_device_info(struct power_supply *psy)
{
- return container_of(psy, struct ds2780_device_info, bat);
+ return power_supply_get_drvdata(psy);
}
static inline struct power_supply *to_power_supply(struct device *dev)
static int ds2780_battery_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
int ret = 0;
struct ds2780_device_info *dev_info;
dev_info->dev = &pdev->dev;
dev_info->w1_dev = pdev->dev.parent;
- dev_info->bat.name = dev_name(&pdev->dev);
- dev_info->bat.type = POWER_SUPPLY_TYPE_BATTERY;
- dev_info->bat.properties = ds2780_battery_props;
- dev_info->bat.num_properties = ARRAY_SIZE(ds2780_battery_props);
- dev_info->bat.get_property = ds2780_battery_get_property;
+ dev_info->bat_desc.name = dev_name(&pdev->dev);
+ dev_info->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ dev_info->bat_desc.properties = ds2780_battery_props;
+ dev_info->bat_desc.num_properties = ARRAY_SIZE(ds2780_battery_props);
+ dev_info->bat_desc.get_property = ds2780_battery_get_property;
- ret = power_supply_register(&pdev->dev, &dev_info->bat);
- if (ret) {
+ psy_cfg.drv_data = dev_info;
+
+ dev_info->bat = power_supply_register(&pdev->dev, &dev_info->bat_desc,
+ &psy_cfg);
+ if (IS_ERR(dev_info->bat)) {
dev_err(dev_info->dev, "failed to register battery\n");
+ ret = PTR_ERR(dev_info->bat);
goto fail;
}
- ret = sysfs_create_group(&dev_info->bat.dev->kobj, &ds2780_attr_group);
+ ret = sysfs_create_group(&dev_info->bat->dev.kobj, &ds2780_attr_group);
if (ret) {
dev_err(dev_info->dev, "failed to create sysfs group\n");
goto fail_unregister;
}
- ret = sysfs_create_bin_file(&dev_info->bat.dev->kobj,
+ ret = sysfs_create_bin_file(&dev_info->bat->dev.kobj,
&ds2780_param_eeprom_bin_attr);
if (ret) {
dev_err(dev_info->dev,
goto fail_remove_group;
}
- ret = sysfs_create_bin_file(&dev_info->bat.dev->kobj,
+ ret = sysfs_create_bin_file(&dev_info->bat->dev.kobj,
&ds2780_user_eeprom_bin_attr);
if (ret) {
dev_err(dev_info->dev,
return 0;
fail_remove_bin_file:
- sysfs_remove_bin_file(&dev_info->bat.dev->kobj,
+ sysfs_remove_bin_file(&dev_info->bat->dev.kobj,
&ds2780_param_eeprom_bin_attr);
fail_remove_group:
- sysfs_remove_group(&dev_info->bat.dev->kobj, &ds2780_attr_group);
+ sysfs_remove_group(&dev_info->bat->dev.kobj, &ds2780_attr_group);
fail_unregister:
- power_supply_unregister(&dev_info->bat);
+ power_supply_unregister(dev_info->bat);
fail:
return ret;
}
{
struct ds2780_device_info *dev_info = platform_get_drvdata(pdev);
- /* remove attributes */
- sysfs_remove_group(&dev_info->bat.dev->kobj, &ds2780_attr_group);
+ /*
+ * Remove attributes before unregistering power supply
+ * because 'bat' will be freed on power_supply_unregister() call.
+ */
+ sysfs_remove_group(&dev_info->bat->dev.kobj, &ds2780_attr_group);
- power_supply_unregister(&dev_info->bat);
+ power_supply_unregister(dev_info->bat);
return 0;
}
struct ds2781_device_info {
struct device *dev;
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct device *w1_dev;
};
static inline struct ds2781_device_info *
to_ds2781_device_info(struct power_supply *psy)
{
- return container_of(psy, struct ds2781_device_info, bat);
+ return power_supply_get_drvdata(psy);
}
static inline struct power_supply *to_power_supply(struct device *dev)
if (ret < 0)
return ret;
- if (power_supply_am_i_supplied(&dev_info->bat)) {
+ if (power_supply_am_i_supplied(dev_info->bat)) {
if (capacity == 100)
*status = POWER_SUPPLY_STATUS_FULL;
else if (current_uA > 50000)
static int ds2781_battery_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
int ret = 0;
struct ds2781_device_info *dev_info;
dev_info->dev = &pdev->dev;
dev_info->w1_dev = pdev->dev.parent;
- dev_info->bat.name = dev_name(&pdev->dev);
- dev_info->bat.type = POWER_SUPPLY_TYPE_BATTERY;
- dev_info->bat.properties = ds2781_battery_props;
- dev_info->bat.num_properties = ARRAY_SIZE(ds2781_battery_props);
- dev_info->bat.get_property = ds2781_battery_get_property;
+ dev_info->bat_desc.name = dev_name(&pdev->dev);
+ dev_info->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ dev_info->bat_desc.properties = ds2781_battery_props;
+ dev_info->bat_desc.num_properties = ARRAY_SIZE(ds2781_battery_props);
+ dev_info->bat_desc.get_property = ds2781_battery_get_property;
- ret = power_supply_register(&pdev->dev, &dev_info->bat);
- if (ret) {
+ psy_cfg.drv_data = dev_info;
+
+ dev_info->bat = power_supply_register(&pdev->dev, &dev_info->bat_desc,
+ &psy_cfg);
+ if (IS_ERR(dev_info->bat)) {
dev_err(dev_info->dev, "failed to register battery\n");
+ ret = PTR_ERR(dev_info->bat);
goto fail;
}
- ret = sysfs_create_group(&dev_info->bat.dev->kobj, &ds2781_attr_group);
+ ret = sysfs_create_group(&dev_info->bat->dev.kobj, &ds2781_attr_group);
if (ret) {
dev_err(dev_info->dev, "failed to create sysfs group\n");
goto fail_unregister;
}
- ret = sysfs_create_bin_file(&dev_info->bat.dev->kobj,
+ ret = sysfs_create_bin_file(&dev_info->bat->dev.kobj,
&ds2781_param_eeprom_bin_attr);
if (ret) {
dev_err(dev_info->dev,
goto fail_remove_group;
}
- ret = sysfs_create_bin_file(&dev_info->bat.dev->kobj,
+ ret = sysfs_create_bin_file(&dev_info->bat->dev.kobj,
&ds2781_user_eeprom_bin_attr);
if (ret) {
dev_err(dev_info->dev,
return 0;
fail_remove_bin_file:
- sysfs_remove_bin_file(&dev_info->bat.dev->kobj,
+ sysfs_remove_bin_file(&dev_info->bat->dev.kobj,
&ds2781_param_eeprom_bin_attr);
fail_remove_group:
- sysfs_remove_group(&dev_info->bat.dev->kobj, &ds2781_attr_group);
+ sysfs_remove_group(&dev_info->bat->dev.kobj, &ds2781_attr_group);
fail_unregister:
- power_supply_unregister(&dev_info->bat);
+ power_supply_unregister(dev_info->bat);
fail:
return ret;
}
{
struct ds2781_device_info *dev_info = platform_get_drvdata(pdev);
- /* remove attributes */
- sysfs_remove_group(&dev_info->bat.dev->kobj, &ds2781_attr_group);
+ /*
+ * Remove attributes before unregistering power supply
+ * because 'bat' will be freed on power_supply_unregister() call.
+ */
+ sysfs_remove_group(&dev_info->bat->dev.kobj, &ds2781_attr_group);
- power_supply_unregister(&dev_info->bat);
+ power_supply_unregister(dev_info->bat);
return 0;
}
int (*get_battery_capacity)(struct ds278x_info *info, int *capacity);
};
-#define to_ds278x_info(x) container_of(x, struct ds278x_info, battery)
+#define to_ds278x_info(x) power_supply_get_drvdata(x)
struct ds278x_info {
struct i2c_client *client;
- struct power_supply battery;
+ struct power_supply *battery;
+ struct power_supply_desc battery_desc;
struct ds278x_battery_ops *ops;
struct delayed_work bat_work;
int id;
ds278x_get_status(info, &info->status);
if ((old_status != info->status) || (old_capacity != info->capacity))
- power_supply_changed(&info->battery);
+ power_supply_changed(info->battery);
}
static void ds278x_bat_work(struct work_struct *work)
POWER_SUPPLY_PROP_TEMP,
};
-static void ds278x_power_supply_init(struct power_supply *battery)
+static void ds278x_power_supply_init(struct power_supply_desc *battery)
{
battery->type = POWER_SUPPLY_TYPE_BATTERY;
battery->properties = ds278x_battery_props;
{
struct ds278x_info *info = i2c_get_clientdata(client);
- power_supply_unregister(&info->battery);
- kfree(info->battery.name);
+ power_supply_unregister(info->battery);
+ kfree(info->battery_desc.name);
mutex_lock(&battery_lock);
idr_remove(&battery_id, info->id);
const struct i2c_device_id *id)
{
struct ds278x_platform_data *pdata = client->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct ds278x_info *info;
int ret;
int num;
goto fail_info;
}
- info->battery.name = kasprintf(GFP_KERNEL, "%s-%d", client->name, num);
- if (!info->battery.name) {
+ info->battery_desc.name = kasprintf(GFP_KERNEL, "%s-%d",
+ client->name, num);
+ if (!info->battery_desc.name) {
ret = -ENOMEM;
goto fail_name;
}
info->client = client;
info->id = num;
info->ops = &ds278x_ops[id->driver_data];
- ds278x_power_supply_init(&info->battery);
+ ds278x_power_supply_init(&info->battery_desc);
+ psy_cfg.drv_data = info;
info->capacity = 100;
info->status = POWER_SUPPLY_STATUS_FULL;
INIT_DELAYED_WORK(&info->bat_work, ds278x_bat_work);
- ret = power_supply_register(&client->dev, &info->battery);
- if (ret) {
+ info->battery = power_supply_register(&client->dev,
+ &info->battery_desc, &psy_cfg);
+ if (IS_ERR(info->battery)) {
dev_err(&client->dev, "failed to register battery\n");
+ ret = PTR_ERR(info->battery);
goto fail_register;
} else {
schedule_delayed_work(&info->bat_work, DS278x_DELAY);
return 0;
fail_register:
- kfree(info->battery.name);
+ kfree(info->battery_desc.name);
fail_name:
kfree(info);
fail_info:
};
struct gab {
- struct power_supply psy;
+ struct power_supply *psy;
+ struct power_supply_desc psy_desc;
struct iio_channel *channel[GAB_MAX_CHAN_TYPE];
struct gab_platform_data *pdata;
struct delayed_work bat_work;
static struct gab *to_generic_bat(struct power_supply *psy)
{
- return container_of(psy, struct gab, psy);
+ return power_supply_get_drvdata(psy);
}
static void gab_ext_power_changed(struct power_supply *psy)
adc_bat = to_generic_bat(psy);
if (!adc_bat) {
- dev_err(psy->dev, "no battery infos ?!\n");
+ dev_err(&psy->dev, "no battery infos ?!\n");
return -EINVAL;
}
pdata = adc_bat->pdata;
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
- gab_get_status(adc_bat);
+ val->intval = gab_get_status(adc_bat);
break;
case POWER_SUPPLY_PROP_CHARGE_EMPTY_DESIGN:
val->intval = 0;
pdata = adc_bat->pdata;
status = adc_bat->status;
- is_plugged = power_supply_am_i_supplied(&adc_bat->psy);
+ is_plugged = power_supply_am_i_supplied(adc_bat->psy);
adc_bat->cable_plugged = is_plugged;
if (!is_plugged)
adc_bat->status = POWER_SUPPLY_STATUS_CHARGING;
if (status != adc_bat->status)
- power_supply_changed(&adc_bat->psy);
+ power_supply_changed(adc_bat->psy);
}
static irqreturn_t gab_charged(int irq, void *dev_id)
static int gab_probe(struct platform_device *pdev)
{
struct gab *adc_bat;
- struct power_supply *psy;
+ struct power_supply_desc *psy_desc;
+ struct power_supply_config psy_cfg = {};
struct gab_platform_data *pdata = pdev->dev.platform_data;
enum power_supply_property *properties;
int ret = 0;
return -ENOMEM;
}
- psy = &adc_bat->psy;
- psy->name = pdata->battery_info.name;
+ psy_cfg.drv_data = adc_bat;
+ psy_desc = &adc_bat->psy_desc;
+ psy_desc->name = pdata->battery_info.name;
/* bootup default values for the battery */
adc_bat->cable_plugged = false;
adc_bat->status = POWER_SUPPLY_STATUS_DISCHARGING;
- psy->type = POWER_SUPPLY_TYPE_BATTERY;
- psy->get_property = gab_get_property;
- psy->external_power_changed = gab_ext_power_changed;
+ psy_desc->type = POWER_SUPPLY_TYPE_BATTERY;
+ psy_desc->get_property = gab_get_property;
+ psy_desc->external_power_changed = gab_ext_power_changed;
adc_bat->pdata = pdata;
/*
* copying the static properties and allocating extra memory for holding
* the extra configurable properties received from platform data.
*/
- psy->properties = kcalloc(ARRAY_SIZE(gab_props) +
+ psy_desc->properties = kcalloc(ARRAY_SIZE(gab_props) +
ARRAY_SIZE(gab_chan_name),
- sizeof(*psy->properties), GFP_KERNEL);
- if (!psy->properties) {
+ sizeof(*psy_desc->properties),
+ GFP_KERNEL);
+ if (!psy_desc->properties) {
ret = -ENOMEM;
goto first_mem_fail;
}
- memcpy(psy->properties, gab_props, sizeof(gab_props));
+ memcpy(psy_desc->properties, gab_props, sizeof(gab_props));
properties = (enum power_supply_property *)
- ((char *)psy->properties + sizeof(gab_props));
+ ((char *)psy_desc->properties + sizeof(gab_props));
/*
* getting channel from iio and copying the battery properties
adc_bat->channel[chan] = NULL;
} else {
/* copying properties for supported channels only */
- memcpy(properties + sizeof(*(psy->properties)) * index,
+ memcpy(properties + sizeof(*(psy_desc->properties)) * index,
&gab_dyn_props[chan],
sizeof(gab_dyn_props[chan]));
index++;
* as come channels may be not be supported by the device.So
* we need to take care of that.
*/
- psy->num_properties = ARRAY_SIZE(gab_props) + index;
+ psy_desc->num_properties = ARRAY_SIZE(gab_props) + index;
- ret = power_supply_register(&pdev->dev, psy);
- if (ret)
+ adc_bat->psy = power_supply_register(&pdev->dev, psy_desc, &psy_cfg);
+ if (IS_ERR(adc_bat->psy)) {
+ ret = PTR_ERR(adc_bat->psy);
goto err_reg_fail;
+ }
INIT_DELAYED_WORK(&adc_bat->bat_work, gab_work);
err_gpio:
gpio_free(pdata->gpio_charge_finished);
gpio_req_fail:
- power_supply_unregister(psy);
+ power_supply_unregister(adc_bat->psy);
err_reg_fail:
for (chan = 0; chan < ARRAY_SIZE(gab_chan_name); chan++) {
if (adc_bat->channel[chan])
iio_channel_release(adc_bat->channel[chan]);
}
second_mem_fail:
- kfree(psy->properties);
+ kfree(psy_desc->properties);
first_mem_fail:
return ret;
}
struct gab *adc_bat = platform_get_drvdata(pdev);
struct gab_platform_data *pdata = adc_bat->pdata;
- power_supply_unregister(&adc_bat->psy);
+ power_supply_unregister(adc_bat->psy);
if (gpio_is_valid(pdata->gpio_charge_finished)) {
free_irq(gpio_to_irq(pdata->gpio_charge_finished), adc_bat);
iio_channel_release(adc_bat->channel[chan]);
}
- kfree(adc_bat->psy.properties);
+ kfree(adc_bat->psy_desc.properties);
cancel_delayed_work(&adc_bat->bat_work);
return 0;
}
int irq;
spinlock_t lock;
- struct power_supply battery;
- struct power_supply ac;
+ struct power_supply *battery;
+ struct power_supply *ac;
};
#define GOLDFISH_BATTERY_READ(data, addr) \
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct goldfish_battery_data *data = container_of(psy,
- struct goldfish_battery_data, ac);
+ struct goldfish_battery_data *data = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct goldfish_battery_data *data = container_of(psy,
- struct goldfish_battery_data, battery);
+ struct goldfish_battery_data *data = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
status &= BATTERY_INT_MASK;
if (status & BATTERY_STATUS_CHANGED)
- power_supply_changed(&data->battery);
+ power_supply_changed(data->battery);
if (status & AC_STATUS_CHANGED)
- power_supply_changed(&data->ac);
+ power_supply_changed(data->ac);
spin_unlock_irqrestore(&data->lock, irq_flags);
return status ? IRQ_HANDLED : IRQ_NONE;
}
+static const struct power_supply_desc battery_desc = {
+ .properties = goldfish_battery_props,
+ .num_properties = ARRAY_SIZE(goldfish_battery_props),
+ .get_property = goldfish_battery_get_property,
+ .name = "battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+};
+
+static const struct power_supply_desc ac_desc = {
+ .properties = goldfish_ac_props,
+ .num_properties = ARRAY_SIZE(goldfish_ac_props),
+ .get_property = goldfish_ac_get_property,
+ .name = "ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+};
static int goldfish_battery_probe(struct platform_device *pdev)
{
int ret;
struct resource *r;
struct goldfish_battery_data *data;
+ struct power_supply_config psy_cfg = {};
data = devm_kzalloc(&pdev->dev, sizeof(*data), GFP_KERNEL);
if (data == NULL)
spin_lock_init(&data->lock);
- data->battery.properties = goldfish_battery_props;
- data->battery.num_properties = ARRAY_SIZE(goldfish_battery_props);
- data->battery.get_property = goldfish_battery_get_property;
- data->battery.name = "battery";
- data->battery.type = POWER_SUPPLY_TYPE_BATTERY;
-
- data->ac.properties = goldfish_ac_props;
- data->ac.num_properties = ARRAY_SIZE(goldfish_ac_props);
- data->ac.get_property = goldfish_ac_get_property;
- data->ac.name = "ac";
- data->ac.type = POWER_SUPPLY_TYPE_MAINS;
-
r = platform_get_resource(pdev, IORESOURCE_MEM, 0);
if (r == NULL) {
dev_err(&pdev->dev, "platform_get_resource failed\n");
if (ret)
return ret;
- ret = power_supply_register(&pdev->dev, &data->ac);
- if (ret)
- return ret;
+ psy_cfg.drv_data = data;
- ret = power_supply_register(&pdev->dev, &data->battery);
- if (ret) {
- power_supply_unregister(&data->ac);
- return ret;
+ data->ac = power_supply_register(&pdev->dev, &ac_desc, &psy_cfg);
+ if (IS_ERR(data->ac))
+ return PTR_ERR(data->ac);
+
+ data->battery = power_supply_register(&pdev->dev, &battery_desc,
+ &psy_cfg);
+ if (IS_ERR(data->battery)) {
+ power_supply_unregister(data->ac);
+ return PTR_ERR(data->battery);
}
platform_set_drvdata(pdev, data);
{
struct goldfish_battery_data *data = platform_get_drvdata(pdev);
- power_supply_unregister(&data->battery);
- power_supply_unregister(&data->ac);
+ power_supply_unregister(data->battery);
+ power_supply_unregister(data->ac);
battery_data = NULL;
return 0;
}
unsigned int irq;
bool wakeup_enabled;
- struct power_supply charger;
+ struct power_supply *charger;
+ struct power_supply_desc charger_desc;
};
static irqreturn_t gpio_charger_irq(int irq, void *devid)
static inline struct gpio_charger *psy_to_gpio_charger(struct power_supply *psy)
{
- return container_of(psy, struct gpio_charger, charger);
+ return power_supply_get_drvdata(psy);
}
static int gpio_charger_get_property(struct power_supply *psy,
static int gpio_charger_probe(struct platform_device *pdev)
{
const struct gpio_charger_platform_data *pdata = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct gpio_charger *gpio_charger;
- struct power_supply *charger;
+ struct power_supply_desc *charger_desc;
int ret;
int irq;
return -ENOMEM;
}
- charger = &gpio_charger->charger;
+ charger_desc = &gpio_charger->charger_desc;
+
+ charger_desc->name = pdata->name ? pdata->name : "gpio-charger";
+ charger_desc->type = pdata->type;
+ charger_desc->properties = gpio_charger_properties;
+ charger_desc->num_properties = ARRAY_SIZE(gpio_charger_properties);
+ charger_desc->get_property = gpio_charger_get_property;
- charger->name = pdata->name ? pdata->name : "gpio-charger";
- charger->type = pdata->type;
- charger->properties = gpio_charger_properties;
- charger->num_properties = ARRAY_SIZE(gpio_charger_properties);
- charger->get_property = gpio_charger_get_property;
- charger->supplied_to = pdata->supplied_to;
- charger->num_supplicants = pdata->num_supplicants;
- charger->of_node = pdev->dev.of_node;
+ psy_cfg.supplied_to = pdata->supplied_to;
+ psy_cfg.num_supplicants = pdata->num_supplicants;
+ psy_cfg.of_node = pdev->dev.of_node;
+ psy_cfg.drv_data = gpio_charger;
ret = gpio_request(pdata->gpio, dev_name(&pdev->dev));
if (ret) {
gpio_charger->pdata = pdata;
- ret = power_supply_register(&pdev->dev, charger);
- if (ret < 0) {
+ gpio_charger->charger = power_supply_register(&pdev->dev,
+ charger_desc, &psy_cfg);
+ if (IS_ERR(gpio_charger->charger)) {
+ ret = PTR_ERR(gpio_charger->charger);
dev_err(&pdev->dev, "Failed to register power supply: %d\n",
ret);
goto err_gpio_free;
if (irq > 0) {
ret = request_any_context_irq(irq, gpio_charger_irq,
IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING,
- dev_name(&pdev->dev), charger);
+ dev_name(&pdev->dev), gpio_charger->charger);
if (ret < 0)
dev_warn(&pdev->dev, "Failed to request irq: %d\n", ret);
else
struct gpio_charger *gpio_charger = platform_get_drvdata(pdev);
if (gpio_charger->irq)
- free_irq(gpio_charger->irq, &gpio_charger->charger);
+ free_irq(gpio_charger->irq, gpio_charger->charger);
- power_supply_unregister(&gpio_charger->charger);
+ power_supply_unregister(gpio_charger->charger);
gpio_free(gpio_charger->pdata->gpio);
if (device_may_wakeup(dev) && gpio_charger->wakeup_enabled)
disable_irq_wake(gpio_charger->irq);
- power_supply_changed(&gpio_charger->charger);
+ power_supply_changed(gpio_charger->charger);
return 0;
}
unsigned int batt_prev_charge_full; /* in mAS */
unsigned int batt_charge_rate; /* in units per second */
- struct power_supply usb;
- struct power_supply batt;
+ struct power_supply *usb;
+ struct power_supply *batt;
int irq; /* GPE_ID or IRQ# */
struct workqueue_struct *monitor_wqueue;
struct delayed_work monitor_battery;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pmic_power_module_info *pbi = container_of(psy,
- struct pmic_power_module_info, usb);
+ struct pmic_power_module_info *pbi = power_supply_get_drvdata(psy);
/* update pmic_power_module_info members */
pmic_battery_read_status(pbi);
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pmic_power_module_info *pbi = container_of(psy,
- struct pmic_power_module_info, batt);
+ struct pmic_power_module_info *pbi = power_supply_get_drvdata(psy);
/* update pmic_power_module_info members */
pmic_battery_read_status(pbi);
__func__);
}
+/*
+ * Description of power supplies
+ */
+static const struct power_supply_desc pmic_usb_desc = {
+ .name = "pmic-usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = pmic_usb_props,
+ .num_properties = ARRAY_SIZE(pmic_usb_props),
+ .get_property = pmic_usb_get_property,
+};
+
+static const struct power_supply_desc pmic_batt_desc = {
+ .name = "pmic-batt",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = pmic_battery_props,
+ .num_properties = ARRAY_SIZE(pmic_battery_props),
+ .get_property = pmic_battery_get_property,
+};
+
/**
* pmic_battery_probe - pmic battery initialize
* @irq: pmic battery device irq
{
int retval = 0;
struct pmic_power_module_info *pbi;
+ struct power_supply_config psy_cfg = {};
dev_dbg(dev, "pmic-battery: found pmic battery device\n");
pbi->dev = dev;
pbi->irq = irq;
dev_set_drvdata(dev, pbi);
+ psy_cfg.drv_data = pbi;
/* initialize all required framework before enabling interrupts */
INIT_WORK(&pbi->handler, pmic_battery_handle_intrpt);
}
/* register pmic-batt with power supply subsystem */
- pbi->batt.name = "pmic-batt";
- pbi->batt.type = POWER_SUPPLY_TYPE_BATTERY;
- pbi->batt.properties = pmic_battery_props;
- pbi->batt.num_properties = ARRAY_SIZE(pmic_battery_props);
- pbi->batt.get_property = pmic_battery_get_property;
- retval = power_supply_register(dev, &pbi->batt);
- if (retval) {
+ pbi->batt = power_supply_register(dev, &pmic_usb_desc, &psy_cfg);
+ if (IS_ERR(pbi->batt)) {
dev_err(dev,
"%s(): failed to register pmic battery device with power supply subsystem\n",
__func__);
+ retval = PTR_ERR(pbi->batt);
goto power_reg_failed;
}
queue_delayed_work(pbi->monitor_wqueue, &pbi->monitor_battery, HZ * 1);
/* register pmic-usb with power supply subsystem */
- pbi->usb.name = "pmic-usb";
- pbi->usb.type = POWER_SUPPLY_TYPE_USB;
- pbi->usb.properties = pmic_usb_props;
- pbi->usb.num_properties = ARRAY_SIZE(pmic_usb_props);
- pbi->usb.get_property = pmic_usb_get_property;
- retval = power_supply_register(dev, &pbi->usb);
- if (retval) {
+ pbi->usb = power_supply_register(dev, &pmic_batt_desc, &psy_cfg);
+ if (IS_ERR(pbi->usb)) {
dev_err(dev,
"%s(): failed to register pmic usb device with power supply subsystem\n",
__func__);
+ retval = PTR_ERR(pbi->usb);
goto power_reg_failed_1;
}
return retval;
power_reg_failed_1:
- power_supply_unregister(&pbi->batt);
+ power_supply_unregister(pbi->batt);
power_reg_failed:
cancel_delayed_work_sync(&pbi->monitor_battery);
requestirq_failed:
cancel_delayed_work_sync(&pbi->monitor_battery);
destroy_workqueue(pbi->monitor_wqueue);
- power_supply_unregister(&pbi->usb);
- power_supply_unregister(&pbi->batt);
+ power_supply_unregister(pbi->usb);
+ power_supply_unregister(pbi->batt);
cancel_work_sync(&pbi->handler);
kfree(pbi);
static int get_capacity(struct power_supply *b)
{
- struct micro_battery *mb = dev_get_drvdata(b->dev->parent);
+ struct micro_battery *mb = dev_get_drvdata(b->dev.parent);
switch (mb->flag & 0x07) {
case MICRO_BATT_STATUS_HIGH:
static int get_status(struct power_supply *b)
{
- struct micro_battery *mb = dev_get_drvdata(b->dev->parent);
+ struct micro_battery *mb = dev_get_drvdata(b->dev.parent);
if (mb->flag == MICRO_BATT_STATUS_UNKNOWN)
return POWER_SUPPLY_STATUS_UNKNOWN;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct micro_battery *mb = dev_get_drvdata(b->dev->parent);
+ struct micro_battery *mb = dev_get_drvdata(b->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_TECHNOLOGY:
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct micro_battery *mb = dev_get_drvdata(b->dev->parent);
+ struct micro_battery *mb = dev_get_drvdata(b->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_ONLINE:
POWER_SUPPLY_PROP_VOLTAGE_NOW,
};
-static struct power_supply micro_batt_power = {
+static const struct power_supply_desc micro_batt_power_desc = {
.name = "main-battery",
.type = POWER_SUPPLY_TYPE_BATTERY,
.properties = micro_batt_power_props,
POWER_SUPPLY_PROP_ONLINE,
};
-static struct power_supply micro_ac_power = {
+static const struct power_supply_desc micro_ac_power_desc = {
.name = "ac",
.type = POWER_SUPPLY_TYPE_MAINS,
.properties = micro_ac_power_props,
.get_property = micro_ac_get_property,
};
+static struct power_supply *micro_batt_power, *micro_ac_power;
+
static int micro_batt_probe(struct platform_device *pdev)
{
struct micro_battery *mb;
+ int ret;
mb = devm_kzalloc(&pdev->dev, sizeof(*mb), GFP_KERNEL);
if (!mb)
mb->micro = dev_get_drvdata(pdev->dev.parent);
mb->wq = create_singlethread_workqueue("ipaq-battery-wq");
+ if (!mb->wq)
+ return -ENOMEM;
+
INIT_DELAYED_WORK(&mb->update, micro_battery_work);
platform_set_drvdata(pdev, mb);
queue_delayed_work(mb->wq, &mb->update, 1);
- power_supply_register(&pdev->dev, µ_batt_power);
- power_supply_register(&pdev->dev, µ_ac_power);
+
+ micro_batt_power = power_supply_register(&pdev->dev,
+ µ_batt_power_desc, NULL);
+ if (IS_ERR(micro_batt_power)) {
+ ret = PTR_ERR(micro_batt_power);
+ goto batt_err;
+ }
+
+ micro_ac_power = power_supply_register(&pdev->dev,
+ µ_ac_power_desc, NULL);
+ if (IS_ERR(micro_ac_power)) {
+ ret = PTR_ERR(micro_ac_power);
+ goto ac_err;
+ }
dev_info(&pdev->dev, "iPAQ micro battery driver\n");
return 0;
+
+ac_err:
+ power_supply_unregister(micro_ac_power);
+batt_err:
+ cancel_delayed_work_sync(&mb->update);
+ destroy_workqueue(mb->wq);
+ return ret;
}
static int micro_batt_remove(struct platform_device *pdev)
{
struct micro_battery *mb = platform_get_drvdata(pdev);
- power_supply_unregister(µ_ac_power);
- power_supply_unregister(µ_batt_power);
+ power_supply_unregister(micro_ac_power);
+ power_supply_unregister(micro_batt_power);
cancel_delayed_work_sync(&mb->update);
+ destroy_workqueue(mb->wq);
return 0;
}
};
struct isp1704_charger {
- struct device *dev;
- struct power_supply psy;
- struct usb_phy *phy;
- struct notifier_block nb;
- struct work_struct work;
+ struct device *dev;
+ struct power_supply *psy;
+ struct power_supply_desc psy_desc;
+ struct usb_phy *phy;
+ struct notifier_block nb;
+ struct work_struct work;
/* properties */
char model[8];
/* detect wall charger */
if (isp1704_charger_detect_dcp(isp)) {
- isp->psy.type = POWER_SUPPLY_TYPE_USB_DCP;
+ isp->psy_desc.type = POWER_SUPPLY_TYPE_USB_DCP;
isp->current_max = 1800;
} else {
- isp->psy.type = POWER_SUPPLY_TYPE_USB;
+ isp->psy_desc.type = POWER_SUPPLY_TYPE_USB;
isp->current_max = 500;
}
usb_gadget_connect(isp->phy->otg->gadget);
}
- if (isp->psy.type != POWER_SUPPLY_TYPE_USB_DCP) {
+ if (isp->psy_desc.type != POWER_SUPPLY_TYPE_USB_DCP) {
/*
* Only 500mA here or high speed chirp
* handshaking may break
isp->current_max = 500;
if (isp->current_max > 100)
- isp->psy.type = POWER_SUPPLY_TYPE_USB_CDP;
+ isp->psy_desc.type = POWER_SUPPLY_TYPE_USB_CDP;
}
break;
case USB_EVENT_NONE:
isp->online = false;
isp->present = 0;
isp->current_max = 0;
- isp->psy.type = POWER_SUPPLY_TYPE_USB;
+ isp->psy_desc.type = POWER_SUPPLY_TYPE_USB;
/*
* Disable data pullups. We need to prevent the controller from
goto out;
}
- power_supply_changed(&isp->psy);
+ power_supply_changed(isp->psy);
out:
mutex_unlock(&lock);
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct isp1704_charger *isp =
- container_of(psy, struct isp1704_charger, psy);
+ struct isp1704_charger *isp = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_PRESENT:
{
struct isp1704_charger *isp;
int ret = -ENODEV;
+ struct power_supply_config psy_cfg = {};
struct isp1704_charger_data *pdata = dev_get_platdata(&pdev->dev);
struct device_node *np = pdev->dev.of_node;
if (ret < 0)
goto fail1;
- isp->psy.name = "isp1704";
- isp->psy.type = POWER_SUPPLY_TYPE_USB;
- isp->psy.properties = power_props;
- isp->psy.num_properties = ARRAY_SIZE(power_props);
- isp->psy.get_property = isp1704_charger_get_property;
+ isp->psy_desc.name = "isp1704";
+ isp->psy_desc.type = POWER_SUPPLY_TYPE_USB;
+ isp->psy_desc.properties = power_props;
+ isp->psy_desc.num_properties = ARRAY_SIZE(power_props);
+ isp->psy_desc.get_property = isp1704_charger_get_property;
- ret = power_supply_register(isp->dev, &isp->psy);
- if (ret)
+ psy_cfg.drv_data = isp;
+
+ isp->psy = power_supply_register(isp->dev, &isp->psy_desc, &psy_cfg);
+ if (IS_ERR(isp->psy)) {
+ ret = PTR_ERR(isp->psy);
goto fail1;
+ }
/*
* REVISIT: using work in order to allow the usb notifications to be
return 0;
fail2:
- power_supply_unregister(&isp->psy);
+ power_supply_unregister(isp->psy);
fail1:
isp1704_charger_set_power(isp, 0);
fail0:
struct isp1704_charger *isp = platform_get_drvdata(pdev);
usb_unregister_notifier(isp->phy, &isp->nb);
- power_supply_unregister(&isp->psy);
+ power_supply_unregister(isp->psy);
isp1704_charger_set_power(isp, 0);
return 0;
struct completion read_completion;
- struct power_supply battery;
+ struct power_supply *battery;
+ struct power_supply_desc battery_desc;
struct delayed_work work;
struct mutex lock;
static inline struct jz_battery *psy_to_jz_battery(struct power_supply *psy)
{
- return container_of(psy, struct jz_battery, battery);
+ return power_supply_get_drvdata(psy);
}
static irqreturn_t jz_battery_irq_handler(int irq, void *devid)
}
if (has_changed)
- power_supply_changed(&jz_battery->battery);
+ power_supply_changed(jz_battery->battery);
}
static enum power_supply_property jz_battery_properties[] = {
{
int ret = 0;
struct jz_battery_platform_data *pdata = pdev->dev.parent->platform_data;
+ struct power_supply_config psy_cfg = {};
struct jz_battery *jz_battery;
- struct power_supply *battery;
+ struct power_supply_desc *battery_desc;
struct resource *mem;
if (!pdata) {
if (IS_ERR(jz_battery->base))
return PTR_ERR(jz_battery->base);
- battery = &jz_battery->battery;
- battery->name = pdata->info.name;
- battery->type = POWER_SUPPLY_TYPE_BATTERY;
- battery->properties = jz_battery_properties;
- battery->num_properties = ARRAY_SIZE(jz_battery_properties);
- battery->get_property = jz_battery_get_property;
- battery->external_power_changed = jz_battery_external_power_changed;
- battery->use_for_apm = 1;
+ battery_desc = &jz_battery->battery_desc;
+ battery_desc->name = pdata->info.name;
+ battery_desc->type = POWER_SUPPLY_TYPE_BATTERY;
+ battery_desc->properties = jz_battery_properties;
+ battery_desc->num_properties = ARRAY_SIZE(jz_battery_properties);
+ battery_desc->get_property = jz_battery_get_property;
+ battery_desc->external_power_changed =
+ jz_battery_external_power_changed;
+ battery_desc->use_for_apm = 1;
+
+ psy_cfg.drv_data = jz_battery;
jz_battery->pdata = pdata;
jz_battery->pdev = pdev;
else
jz4740_adc_set_config(pdev->dev.parent, JZ_ADC_CONFIG_BAT_MB, 0);
- ret = power_supply_register(&pdev->dev, &jz_battery->battery);
- if (ret) {
+ jz_battery->battery = power_supply_register(&pdev->dev, battery_desc,
+ &psy_cfg);
+ if (IS_ERR(jz_battery->battery)) {
dev_err(&pdev->dev, "power supply battery register failed.\n");
+ ret = PTR_ERR(jz_battery->battery);
goto err_free_charge_irq;
}
gpio_free(jz_battery->pdata->gpio_charge);
}
- power_supply_unregister(&jz_battery->battery);
+ power_supply_unregister(jz_battery->battery);
free_irq(jz_battery->irq, jz_battery);
};
struct lp8727_psy {
- struct power_supply ac;
- struct power_supply usb;
- struct power_supply batt;
+ struct power_supply *ac;
+ struct power_supply *usb;
+ struct power_supply *batt;
};
struct lp8727_chg {
lp8727_id_detection(pchg, idno, vbus);
lp8727_enable_chgdet(pchg);
- power_supply_changed(&pchg->psy->ac);
- power_supply_changed(&pchg->psy->usb);
- power_supply_changed(&pchg->psy->batt);
+ power_supply_changed(pchg->psy->ac);
+ power_supply_changed(pchg->psy->usb);
+ power_supply_changed(pchg->psy->batt);
}
static irqreturn_t lp8727_isr_func(int irq, void *ptr)
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct lp8727_chg *pchg = dev_get_drvdata(psy->dev->parent);
+ struct lp8727_chg *pchg = dev_get_drvdata(psy->dev.parent);
if (psp != POWER_SUPPLY_PROP_ONLINE)
return -EINVAL;
- val->intval = lp8727_is_charger_attached(psy->name, pchg->devid);
+ val->intval = lp8727_is_charger_attached(psy->desc->name, pchg->devid);
return 0;
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct lp8727_chg *pchg = dev_get_drvdata(psy->dev->parent);
+ struct lp8727_chg *pchg = dev_get_drvdata(psy->dev.parent);
struct lp8727_platform_data *pdata = pchg->pdata;
enum lp8727_die_temp temp;
u8 read;
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
- if (!lp8727_is_charger_attached(psy->name, pchg->devid)) {
+ if (!lp8727_is_charger_attached(psy->desc->name, pchg->devid)) {
val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
return 0;
}
static void lp8727_charger_changed(struct power_supply *psy)
{
- struct lp8727_chg *pchg = dev_get_drvdata(psy->dev->parent);
+ struct lp8727_chg *pchg = dev_get_drvdata(psy->dev.parent);
u8 eoc_level;
u8 ichg;
u8 val;
/* skip if no charger exists */
- if (!lp8727_is_charger_attached(psy->name, pchg->devid))
+ if (!lp8727_is_charger_attached(psy->desc->name, pchg->devid))
return;
/* update charging parameters */
}
}
+static const struct power_supply_desc lp8727_ac_desc = {
+ .name = "ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = lp8727_charger_prop,
+ .num_properties = ARRAY_SIZE(lp8727_charger_prop),
+ .get_property = lp8727_charger_get_property,
+};
+
+static const struct power_supply_desc lp8727_usb_desc = {
+ .name = "usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = lp8727_charger_prop,
+ .num_properties = ARRAY_SIZE(lp8727_charger_prop),
+ .get_property = lp8727_charger_get_property,
+};
+
+static const struct power_supply_desc lp8727_batt_desc = {
+ .name = "main_batt",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = lp8727_battery_prop,
+ .num_properties = ARRAY_SIZE(lp8727_battery_prop),
+ .get_property = lp8727_battery_get_property,
+ .external_power_changed = lp8727_charger_changed,
+};
+
static int lp8727_register_psy(struct lp8727_chg *pchg)
{
+ struct power_supply_config psy_cfg = {}; /* Only for ac and usb */
struct lp8727_psy *psy;
psy = devm_kzalloc(pchg->dev, sizeof(*psy), GFP_KERNEL);
pchg->psy = psy;
- psy->ac.name = "ac";
- psy->ac.type = POWER_SUPPLY_TYPE_MAINS;
- psy->ac.properties = lp8727_charger_prop;
- psy->ac.num_properties = ARRAY_SIZE(lp8727_charger_prop);
- psy->ac.get_property = lp8727_charger_get_property;
- psy->ac.supplied_to = battery_supplied_to;
- psy->ac.num_supplicants = ARRAY_SIZE(battery_supplied_to);
+ psy_cfg.supplied_to = battery_supplied_to;
+ psy_cfg.num_supplicants = ARRAY_SIZE(battery_supplied_to);
- if (power_supply_register(pchg->dev, &psy->ac))
+ psy->ac = power_supply_register(pchg->dev, &lp8727_ac_desc, &psy_cfg);
+ if (IS_ERR(psy->ac))
goto err_psy_ac;
- psy->usb.name = "usb";
- psy->usb.type = POWER_SUPPLY_TYPE_USB;
- psy->usb.properties = lp8727_charger_prop;
- psy->usb.num_properties = ARRAY_SIZE(lp8727_charger_prop);
- psy->usb.get_property = lp8727_charger_get_property;
- psy->usb.supplied_to = battery_supplied_to;
- psy->usb.num_supplicants = ARRAY_SIZE(battery_supplied_to);
-
- if (power_supply_register(pchg->dev, &psy->usb))
+ psy->usb = power_supply_register(pchg->dev, &lp8727_usb_desc,
+ &psy_cfg);
+ if (IS_ERR(psy->usb))
goto err_psy_usb;
- psy->batt.name = "main_batt";
- psy->batt.type = POWER_SUPPLY_TYPE_BATTERY;
- psy->batt.properties = lp8727_battery_prop;
- psy->batt.num_properties = ARRAY_SIZE(lp8727_battery_prop);
- psy->batt.get_property = lp8727_battery_get_property;
- psy->batt.external_power_changed = lp8727_charger_changed;
-
- if (power_supply_register(pchg->dev, &psy->batt))
+ psy->batt = power_supply_register(pchg->dev, &lp8727_batt_desc, NULL);
+ if (IS_ERR(psy->batt))
goto err_psy_batt;
return 0;
err_psy_batt:
- power_supply_unregister(&psy->usb);
+ power_supply_unregister(psy->usb);
err_psy_usb:
- power_supply_unregister(&psy->ac);
+ power_supply_unregister(psy->ac);
err_psy_ac:
return -EPERM;
}
if (!psy)
return;
- power_supply_unregister(&psy->ac);
- power_supply_unregister(&psy->usb);
- power_supply_unregister(&psy->batt);
+ power_supply_unregister(psy->ac);
+ power_supply_unregister(psy->usb);
+ power_supply_unregister(psy->batt);
}
#ifdef CONFIG_OF
*/
struct lp8788_charger {
struct lp8788 *lp;
- struct power_supply charger;
- struct power_supply battery;
+ struct power_supply *charger;
+ struct power_supply *battery;
struct work_struct charger_work;
struct iio_channel *chan[LP8788_NUM_CHG_ADC];
struct lp8788_chg_irq irqs[LP8788_MAX_CHG_IRQS];
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct lp8788_charger *pchg = dev_get_drvdata(psy->dev->parent);
+ struct lp8788_charger *pchg = dev_get_drvdata(psy->dev.parent);
u8 read;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct lp8788_charger *pchg = dev_get_drvdata(psy->dev->parent);
+ struct lp8788_charger *pchg = dev_get_drvdata(psy->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
return 0;
}
+static const struct power_supply_desc lp8788_psy_charger_desc = {
+ .name = LP8788_CHARGER_NAME,
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = lp8788_charger_prop,
+ .num_properties = ARRAY_SIZE(lp8788_charger_prop),
+ .get_property = lp8788_charger_get_property,
+};
+
+static const struct power_supply_desc lp8788_psy_battery_desc = {
+ .name = LP8788_BATTERY_NAME,
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = lp8788_battery_prop,
+ .num_properties = ARRAY_SIZE(lp8788_battery_prop),
+ .get_property = lp8788_battery_get_property,
+};
+
static int lp8788_psy_register(struct platform_device *pdev,
struct lp8788_charger *pchg)
{
- pchg->charger.name = LP8788_CHARGER_NAME;
- pchg->charger.type = POWER_SUPPLY_TYPE_MAINS;
- pchg->charger.properties = lp8788_charger_prop;
- pchg->charger.num_properties = ARRAY_SIZE(lp8788_charger_prop);
- pchg->charger.get_property = lp8788_charger_get_property;
- pchg->charger.supplied_to = battery_supplied_to;
- pchg->charger.num_supplicants = ARRAY_SIZE(battery_supplied_to);
-
- if (power_supply_register(&pdev->dev, &pchg->charger))
- return -EPERM;
+ struct power_supply_config charger_cfg = {};
+
+ charger_cfg.supplied_to = battery_supplied_to;
+ charger_cfg.num_supplicants = ARRAY_SIZE(battery_supplied_to);
- pchg->battery.name = LP8788_BATTERY_NAME;
- pchg->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- pchg->battery.properties = lp8788_battery_prop;
- pchg->battery.num_properties = ARRAY_SIZE(lp8788_battery_prop);
- pchg->battery.get_property = lp8788_battery_get_property;
+ pchg->charger = power_supply_register(&pdev->dev,
+ &lp8788_psy_charger_desc,
+ &charger_cfg);
+ if (IS_ERR(pchg->charger))
+ return -EPERM;
- if (power_supply_register(&pdev->dev, &pchg->battery))
+ pchg->battery = power_supply_register(&pdev->dev,
+ &lp8788_psy_battery_desc, NULL);
+ if (IS_ERR(pchg->battery)) {
+ power_supply_unregister(pchg->charger);
return -EPERM;
+ }
return 0;
}
static void lp8788_psy_unregister(struct lp8788_charger *pchg)
{
- power_supply_unregister(&pchg->battery);
- power_supply_unregister(&pchg->charger);
+ power_supply_unregister(pchg->battery);
+ power_supply_unregister(pchg->charger);
}
static void lp8788_charger_event(struct work_struct *work)
case LP8788_INT_EOC:
case LP8788_INT_BATT_LOW:
case LP8788_INT_NO_BATT:
- power_supply_changed(&pchg->charger);
- power_supply_changed(&pchg->battery);
+ power_supply_changed(pchg->charger);
+ power_supply_changed(pchg->battery);
break;
default:
break;
struct ltc294x_info {
struct i2c_client *client; /* I2C Client pointer */
- struct power_supply supply; /* Supply pointer */
+ struct power_supply *supply; /* Supply pointer */
+ struct power_supply_desc supply_desc; /* Supply description */
struct delayed_work work; /* Work scheduler */
int num_regs; /* Number of registers (chip type) */
int id; /* Identifier of ltc294x chip */
enum power_supply_property prop,
union power_supply_propval *val)
{
- struct ltc294x_info *info =
- container_of(psy, struct ltc294x_info, supply);
+ struct ltc294x_info *info = power_supply_get_drvdata(psy);
switch (prop) {
case POWER_SUPPLY_PROP_CHARGE_NOW:
enum power_supply_property psp,
const union power_supply_propval *val)
{
- struct ltc294x_info *info =
- container_of(psy, struct ltc294x_info, supply);
+ struct ltc294x_info *info = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_CHARGE_NOW:
if (charge != info->charge) {
info->charge = charge;
- power_supply_changed(&info->supply);
+ power_supply_changed(info->supply);
}
}
struct ltc294x_info *info = i2c_get_clientdata(client);
cancel_delayed_work(&info->work);
- power_supply_unregister(&info->supply);
- kfree(info->supply.name);
+ power_supply_unregister(info->supply);
+ kfree(info->supply_desc.name);
mutex_lock(<c294x_lock);
idr_remove(<c294x_id, info->id);
mutex_unlock(<c294x_lock);
static int ltc294x_i2c_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
+ struct power_supply_config psy_cfg = {};
struct ltc294x_info *info;
int ret;
int num;
i2c_set_clientdata(client, info);
info->num_regs = id->driver_data;
- info->supply.name = kasprintf(GFP_KERNEL, "%s-%d", client->name, num);
- if (!info->supply.name) {
+ info->supply_desc.name = kasprintf(GFP_KERNEL, "%s-%d", client->name,
+ num);
+ if (!info->supply_desc.name) {
ret = -ENOMEM;
goto fail_name;
}
} else {
if (prescaler_exp > LTC2941_MAX_PRESCALER_EXP)
prescaler_exp = LTC2941_MAX_PRESCALER_EXP;
- info->Qlsb = ((58 * 50000) / r_sense) /
+ info->Qlsb = ((85 * 50000) / r_sense) /
(128 / (1 << prescaler_exp));
}
info->client = client;
info->id = num;
- info->supply.type = POWER_SUPPLY_TYPE_BATTERY;
- info->supply.properties = ltc294x_properties;
+ info->supply_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ info->supply_desc.properties = ltc294x_properties;
if (info->num_regs >= LTC294X_REG_TEMPERATURE_LSB)
- info->supply.num_properties =
+ info->supply_desc.num_properties =
ARRAY_SIZE(ltc294x_properties);
else if (info->num_regs >= LTC294X_REG_CURRENT_LSB)
- info->supply.num_properties =
+ info->supply_desc.num_properties =
ARRAY_SIZE(ltc294x_properties) - 1;
else if (info->num_regs >= LTC294X_REG_VOLTAGE_LSB)
- info->supply.num_properties =
+ info->supply_desc.num_properties =
ARRAY_SIZE(ltc294x_properties) - 2;
else
- info->supply.num_properties =
+ info->supply_desc.num_properties =
ARRAY_SIZE(ltc294x_properties) - 3;
- info->supply.get_property = ltc294x_get_property;
- info->supply.set_property = ltc294x_set_property;
- info->supply.property_is_writeable = ltc294x_property_is_writeable;
- info->supply.external_power_changed = NULL;
+ info->supply_desc.get_property = ltc294x_get_property;
+ info->supply_desc.set_property = ltc294x_set_property;
+ info->supply_desc.property_is_writeable = ltc294x_property_is_writeable;
+ info->supply_desc.external_power_changed = NULL;
+
+ psy_cfg.drv_data = info;
INIT_DELAYED_WORK(&info->work, ltc294x_work);
goto fail_comm;
}
- ret = power_supply_register(&client->dev, &info->supply);
- if (ret) {
+ info->supply = power_supply_register(&client->dev, &info->supply_desc,
+ &psy_cfg);
+ if (IS_ERR(info->supply)) {
dev_err(&client->dev, "failed to register ltc2941\n");
+ ret = PTR_ERR(info->supply);
goto fail_register;
} else {
schedule_delayed_work(&info->work, LTC294X_WORK_DELAY * HZ);
return 0;
fail_register:
- kfree(info->supply.name);
+ kfree(info->supply_desc.name);
fail_comm:
fail_name:
fail_info:
#include <linux/mfd/max14577.h>
struct max14577_charger {
- struct device *dev;
- struct max14577 *max14577;
- struct power_supply charger;
-
- unsigned int charging_state;
- unsigned int battery_state;
+ struct device *dev;
+ struct max14577 *max14577;
+ struct power_supply *charger;
struct max14577_charger_platform_data *pdata;
};
}
}
-static int max14577_get_charger_state(struct max14577_charger *chg)
+static int max14577_get_charger_state(struct max14577_charger *chg, int *val)
{
struct regmap *rmap = chg->max14577->regmap;
- int state = POWER_SUPPLY_STATUS_DISCHARGING;
+ int ret;
u8 reg_data;
/*
* - handle properly dead-battery charging (respect timer)
* - handle timers (fast-charge and prequal) /MBCCHGERR/
*/
- max14577_read_reg(rmap, MAX14577_CHG_REG_CHG_CTRL2, ®_data);
- if ((reg_data & CHGCTRL2_MBCHOSTEN_MASK) == 0)
- goto state_set;
+ ret = max14577_read_reg(rmap, MAX14577_CHG_REG_CHG_CTRL2, ®_data);
+ if (ret < 0)
+ goto out;
+
+ if ((reg_data & CHGCTRL2_MBCHOSTEN_MASK) == 0) {
+ *val = POWER_SUPPLY_STATUS_DISCHARGING;
+ goto out;
+ }
+
+ ret = max14577_read_reg(rmap, MAX14577_CHG_REG_STATUS3, ®_data);
+ if (ret < 0)
+ goto out;
- max14577_read_reg(rmap, MAX14577_CHG_REG_STATUS3, ®_data);
if (reg_data & STATUS3_CGMBC_MASK) {
/* Charger or USB-cable is connected */
if (reg_data & STATUS3_EOC_MASK)
- state = POWER_SUPPLY_STATUS_FULL;
+ *val = POWER_SUPPLY_STATUS_FULL;
else
- state = POWER_SUPPLY_STATUS_CHARGING;
- goto state_set;
+ *val = POWER_SUPPLY_STATUS_CHARGING;
+ goto out;
}
-state_set:
- chg->charging_state = state;
- return state;
+ *val = POWER_SUPPLY_STATUS_DISCHARGING;
+
+out:
+ return ret;
}
/*
* - POWER_SUPPLY_CHARGE_TYPE_NONE
* - POWER_SUPPLY_CHARGE_TYPE_FAST
*/
-static int max14577_get_charge_type(struct max14577_charger *chg)
+static int max14577_get_charge_type(struct max14577_charger *chg, int *val)
{
+ int ret, charging;
+
/*
* TODO: CHARGE_TYPE_TRICKLE (VCHGR_RC or EOC)?
* As spec says:
* top-off timer starts. The device continues to trickle
* charge the battery until the top-off timer runs out."
*/
- if (max14577_get_charger_state(chg) == POWER_SUPPLY_STATUS_CHARGING)
- return POWER_SUPPLY_CHARGE_TYPE_FAST;
- return POWER_SUPPLY_CHARGE_TYPE_NONE;
+ ret = max14577_get_charger_state(chg, &charging);
+ if (ret < 0)
+ return ret;
+
+ if (charging == POWER_SUPPLY_STATUS_CHARGING)
+ *val = POWER_SUPPLY_CHARGE_TYPE_FAST;
+ else
+ *val = POWER_SUPPLY_CHARGE_TYPE_NONE;
+
+ return 0;
}
-static int max14577_get_online(struct max14577_charger *chg)
+static int max14577_get_online(struct max14577_charger *chg, int *val)
{
struct regmap *rmap = chg->max14577->regmap;
u8 reg_data;
+ int ret;
enum max14577_muic_charger_type chg_type;
- max14577_read_reg(rmap, MAX14577_MUIC_REG_STATUS2, ®_data);
+ ret = max14577_read_reg(rmap, MAX14577_MUIC_REG_STATUS2, ®_data);
+ if (ret < 0)
+ return ret;
+
reg_data = ((reg_data & STATUS2_CHGTYP_MASK) >> STATUS2_CHGTYP_SHIFT);
chg_type = maxim_get_charger_type(chg->max14577->dev_type, reg_data);
switch (chg_type) {
case MAX14577_CHARGER_TYPE_SPECIAL_1A:
case MAX14577_CHARGER_TYPE_DEAD_BATTERY:
case MAX77836_CHARGER_TYPE_SPECIAL_BIAS:
- return 1;
+ *val = 1;
+ break;
case MAX14577_CHARGER_TYPE_NONE:
case MAX14577_CHARGER_TYPE_DOWNSTREAM_PORT:
case MAX14577_CHARGER_TYPE_RESERVED:
case MAX77836_CHARGER_TYPE_RESERVED:
default:
- return 0;
+ *val = 0;
}
+
+ return 0;
}
/*
* - POWER_SUPPLY_HEALTH_OVERVOLTAGE
* - POWER_SUPPLY_HEALTH_GOOD
*/
-static int max14577_get_battery_health(struct max14577_charger *chg)
+static int max14577_get_battery_health(struct max14577_charger *chg, int *val)
{
struct regmap *rmap = chg->max14577->regmap;
- int state = POWER_SUPPLY_HEALTH_GOOD;
+ int ret;
u8 reg_data;
enum max14577_muic_charger_type chg_type;
- max14577_read_reg(rmap, MAX14577_MUIC_REG_STATUS2, ®_data);
+ ret = max14577_read_reg(rmap, MAX14577_MUIC_REG_STATUS2, ®_data);
+ if (ret < 0)
+ goto out;
+
reg_data = ((reg_data & STATUS2_CHGTYP_MASK) >> STATUS2_CHGTYP_SHIFT);
chg_type = maxim_get_charger_type(chg->max14577->dev_type, reg_data);
if (chg_type == MAX14577_CHARGER_TYPE_DEAD_BATTERY) {
- state = POWER_SUPPLY_HEALTH_DEAD;
- goto state_set;
+ *val = POWER_SUPPLY_HEALTH_DEAD;
+ goto out;
}
- max14577_read_reg(rmap, MAX14577_CHG_REG_STATUS3, ®_data);
+ ret = max14577_read_reg(rmap, MAX14577_CHG_REG_STATUS3, ®_data);
+ if (ret < 0)
+ goto out;
+
if (reg_data & STATUS3_OVP_MASK) {
- state = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
- goto state_set;
+ *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
+ goto out;
}
-state_set:
- chg->battery_state = state;
- return state;
+ /* Not dead, not overvoltage */
+ *val = POWER_SUPPLY_HEALTH_GOOD;
+
+out:
+ return ret;
}
/*
* The max14577 chip doesn't report any status of battery presence.
* Lets assume that it will always be used with some battery.
*/
-static int max14577_get_present(struct max14577_charger *chg)
+static int max14577_get_present(struct max14577_charger *chg, int *val)
{
- return 1;
+ *val = 1;
+
+ return 0;
}
static int max14577_set_fast_charge_timer(struct max14577_charger *chg,
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max14577_charger *chg = container_of(psy,
- struct max14577_charger,
- charger);
+ struct max14577_charger *chg = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
- val->intval = max14577_get_charger_state(chg);
+ ret = max14577_get_charger_state(chg, &val->intval);
break;
case POWER_SUPPLY_PROP_CHARGE_TYPE:
- val->intval = max14577_get_charge_type(chg);
+ ret = max14577_get_charge_type(chg, &val->intval);
break;
case POWER_SUPPLY_PROP_HEALTH:
- val->intval = max14577_get_battery_health(chg);
+ ret = max14577_get_battery_health(chg, &val->intval);
break;
case POWER_SUPPLY_PROP_PRESENT:
- val->intval = max14577_get_present(chg);
+ ret = max14577_get_present(chg, &val->intval);
break;
case POWER_SUPPLY_PROP_ONLINE:
- val->intval = max14577_get_online(chg);
+ ret = max14577_get_online(chg, &val->intval);
break;
case POWER_SUPPLY_PROP_MODEL_NAME:
BUILD_BUG_ON(ARRAY_SIZE(model_names) != MAXIM_DEVICE_TYPE_NUM);
return ret;
}
+static const struct power_supply_desc max14577_charger_desc = {
+ .name = "max14577-charger",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = max14577_charger_props,
+ .num_properties = ARRAY_SIZE(max14577_charger_props),
+ .get_property = max14577_charger_get_property,
+};
+
#ifdef CONFIG_OF
static struct max14577_charger_platform_data *max14577_charger_dt_init(
struct platform_device *pdev)
static int max14577_charger_probe(struct platform_device *pdev)
{
struct max14577_charger *chg;
+ struct power_supply_config psy_cfg = {};
struct max14577 *max14577 = dev_get_drvdata(pdev->dev.parent);
int ret;
if (ret)
return ret;
- chg->charger.name = "max14577-charger",
- chg->charger.type = POWER_SUPPLY_TYPE_BATTERY,
- chg->charger.properties = max14577_charger_props,
- chg->charger.num_properties = ARRAY_SIZE(max14577_charger_props),
- chg->charger.get_property = max14577_charger_get_property,
-
ret = device_create_file(&pdev->dev, &dev_attr_fast_charge_timer);
if (ret) {
dev_err(&pdev->dev, "failed: create sysfs entry\n");
return ret;
}
- ret = power_supply_register(&pdev->dev, &chg->charger);
- if (ret) {
+ psy_cfg.drv_data = chg;
+ chg->charger = power_supply_register(&pdev->dev, &max14577_charger_desc,
+ &psy_cfg);
+ if (IS_ERR(chg->charger)) {
dev_err(&pdev->dev, "failed: power supply register\n");
+ ret = PTR_ERR(chg->charger);
goto err;
}
struct max14577_charger *chg = platform_get_drvdata(pdev);
device_remove_file(&pdev->dev, &dev_attr_fast_charge_timer);
- power_supply_unregister(&chg->charger);
+ power_supply_unregister(chg->charger);
return 0;
}
struct max17040_chip {
struct i2c_client *client;
struct delayed_work work;
- struct power_supply battery;
+ struct power_supply *battery;
struct max17040_platform_data *pdata;
/* State Of Connect */
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max17040_chip *chip = container_of(psy,
- struct max17040_chip, battery);
+ struct max17040_chip *chip = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
max17040_get_online(chip->client);
max17040_get_status(chip->client);
- schedule_delayed_work(&chip->work, MAX17040_DELAY);
+ queue_delayed_work(system_power_efficient_wq, &chip->work,
+ MAX17040_DELAY);
}
static enum power_supply_property max17040_battery_props[] = {
POWER_SUPPLY_PROP_CAPACITY,
};
+static const struct power_supply_desc max17040_battery_desc = {
+ .name = "battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = max17040_get_property,
+ .properties = max17040_battery_props,
+ .num_properties = ARRAY_SIZE(max17040_battery_props),
+};
+
static int max17040_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct power_supply_config psy_cfg = {};
struct max17040_chip *chip;
- int ret;
if (!i2c_check_functionality(adapter, I2C_FUNC_SMBUS_BYTE))
return -EIO;
chip->pdata = client->dev.platform_data;
i2c_set_clientdata(client, chip);
+ psy_cfg.drv_data = chip;
- chip->battery.name = "battery";
- chip->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- chip->battery.get_property = max17040_get_property;
- chip->battery.properties = max17040_battery_props;
- chip->battery.num_properties = ARRAY_SIZE(max17040_battery_props);
-
- ret = power_supply_register(&client->dev, &chip->battery);
- if (ret) {
+ chip->battery = power_supply_register(&client->dev,
+ &max17040_battery_desc, &psy_cfg);
+ if (IS_ERR(chip->battery)) {
dev_err(&client->dev, "failed: power supply register\n");
- return ret;
+ return PTR_ERR(chip->battery);
}
max17040_reset(client);
max17040_get_version(client);
INIT_DEFERRABLE_WORK(&chip->work, max17040_work);
- schedule_delayed_work(&chip->work, MAX17040_DELAY);
+ queue_delayed_work(system_power_efficient_wq, &chip->work,
+ MAX17040_DELAY);
return 0;
}
{
struct max17040_chip *chip = i2c_get_clientdata(client);
- power_supply_unregister(&chip->battery);
+ power_supply_unregister(chip->battery);
cancel_delayed_work(&chip->work);
return 0;
}
struct i2c_client *client = to_i2c_client(dev);
struct max17040_chip *chip = i2c_get_clientdata(client);
- schedule_delayed_work(&chip->work, MAX17040_DELAY);
+ queue_delayed_work(system_power_efficient_wq, &chip->work,
+ MAX17040_DELAY);
return 0;
}
#define dP_ACC_100 0x1900
#define dP_ACC_200 0x3200
-#define MAX17042_IC_VERSION 0x0092
-#define MAX17047_IC_VERSION 0x00AC /* same for max17050 */
-
struct max17042_chip {
struct i2c_client *client;
struct regmap *regmap;
- struct power_supply battery;
+ struct power_supply *battery;
enum max170xx_chip_type chip_type;
struct max17042_platform_data *pdata;
struct work_struct work;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max17042_chip *chip = container_of(psy,
- struct max17042_chip, battery);
+ struct max17042_chip *chip = power_supply_get_drvdata(psy);
struct regmap *map = chip->regmap;
int ret;
u32 data;
val->intval *= 20000; /* Units of LSB = 20mV */
break;
case POWER_SUPPLY_PROP_VOLTAGE_MIN_DESIGN:
- if (chip->chip_type == MAX17042)
+ if (chip->chip_type == MAXIM_DEVICE_TYPE_MAX17042)
ret = regmap_read(map, MAX17042_V_empty, &data);
else
ret = regmap_read(map, MAX17047_V_empty, &data);
static inline void max10742_unlock_model(struct max17042_chip *chip)
{
struct regmap *map = chip->regmap;
+
regmap_write(map, MAX17042_MLOCKReg1, MODEL_UNLOCK1);
regmap_write(map, MAX17042_MLOCKReg2, MODEL_UNLOCK2);
}
{
struct regmap *map = chip->regmap;
int i;
+
for (i = 0; i < size; i++)
regmap_write(map, addr + i,
chip->pdata->config_data->cell_char_tbl[i]);
regmap_write(map, MAX17042_FilterCFG,
config->filter_cfg);
regmap_write(map, MAX17042_RelaxCFG, config->relax_cfg);
- if (chip->chip_type == MAX17047)
+ if (chip->chip_type == MAXIM_DEVICE_TYPE_MAX17047 ||
+ chip->chip_type == MAXIM_DEVICE_TYPE_MAX17050)
regmap_write(map, MAX17047_FullSOCThr,
config->full_soc_thresh);
}
max17042_write_verify_reg(map, MAX17042_RCOMP0, config->rcomp0);
max17042_write_verify_reg(map, MAX17042_TempCo, config->tcompc0);
max17042_write_verify_reg(map, MAX17042_ICHGTerm, config->ichgt_term);
- if (chip->chip_type == MAX17042) {
+ if (chip->chip_type == MAXIM_DEVICE_TYPE_MAX17042) {
regmap_write(map, MAX17042_EmptyTempCo, config->empty_tempco);
max17042_write_verify_reg(map, MAX17042_K_empty0,
config->kempty0);
max17042_override_por(map, MAX17042_FullCAP, config->fullcap);
max17042_override_por(map, MAX17042_FullCAPNom, config->fullcapnom);
- if (chip->chip_type == MAX17042)
+ if (chip->chip_type == MAXIM_DEVICE_TYPE_MAX17042)
max17042_override_por(map, MAX17042_SOC_empty,
config->socempty);
max17042_override_por(map, MAX17042_LAvg_empty, config->lavg_empty);
max17042_override_por(map, MAX17042_dQacc, config->dqacc);
max17042_override_por(map, MAX17042_dPacc, config->dpacc);
- if (chip->chip_type == MAX17042)
+ if (chip->chip_type == MAXIM_DEVICE_TYPE_MAX17042)
max17042_override_por(map, MAX17042_V_empty, config->vempty);
else
max17042_override_por(map, MAX17047_V_empty, config->vempty);
{
struct regmap *map = chip->regmap;
int ret;
- int val;
max17042_override_por_values(chip);
/* After Power up, the MAX17042 requires 500mS in order
max17042_load_new_capacity_params(chip);
/* Init complete, Clear the POR bit */
- regmap_read(map, MAX17042_STATUS, &val);
- regmap_write(map, MAX17042_STATUS, val & (~STATUS_POR_BIT));
+ regmap_update_bits(map, MAX17042_STATUS, STATUS_POR_BIT, 0x0);
return 0;
}
max17042_set_soc_threshold(chip, 1);
}
- power_supply_changed(&chip->battery);
+ power_supply_changed(chip->battery);
return IRQ_HANDLED;
}
.val_format_endian = REGMAP_ENDIAN_NATIVE,
};
+static const struct power_supply_desc max17042_psy_desc = {
+ .name = "max170xx_battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = max17042_get_property,
+ .properties = max17042_battery_props,
+ .num_properties = ARRAY_SIZE(max17042_battery_props),
+};
+
+static const struct power_supply_desc max17042_no_current_sense_psy_desc = {
+ .name = "max170xx_battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = max17042_get_property,
+ .properties = max17042_battery_props,
+ .num_properties = ARRAY_SIZE(max17042_battery_props) - 2,
+};
+
static int max17042_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ const struct power_supply_desc *max17042_desc = &max17042_psy_desc;
+ struct power_supply_config psy_cfg = {};
struct max17042_chip *chip;
int ret;
int i;
}
i2c_set_clientdata(client, chip);
-
- regmap_read(chip->regmap, MAX17042_DevName, &val);
- if (val == MAX17042_IC_VERSION) {
- dev_dbg(&client->dev, "chip type max17042 detected\n");
- chip->chip_type = MAX17042;
- } else if (val == MAX17047_IC_VERSION) {
- dev_dbg(&client->dev, "chip type max17047/50 detected\n");
- chip->chip_type = MAX17047;
- } else {
- dev_err(&client->dev, "device version mismatch: %x\n", val);
- return -EIO;
- }
-
- chip->battery.name = "max170xx_battery";
- chip->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- chip->battery.get_property = max17042_get_property;
- chip->battery.properties = max17042_battery_props;
- chip->battery.num_properties = ARRAY_SIZE(max17042_battery_props);
+ chip->chip_type = id->driver_data;
+ psy_cfg.drv_data = chip;
/* When current is not measured,
* CURRENT_NOW and CURRENT_AVG properties should be invisible. */
if (!chip->pdata->enable_current_sense)
- chip->battery.num_properties -= 2;
+ max17042_desc = &max17042_no_current_sense_psy_desc;
if (chip->pdata->r_sns == 0)
chip->pdata->r_sns = MAX17042_DEFAULT_SNS_RESISTOR;
regmap_write(chip->regmap, MAX17042_LearnCFG, 0x0007);
}
- ret = power_supply_register(&client->dev, &chip->battery);
- if (ret) {
+ chip->battery = power_supply_register(&client->dev, max17042_desc,
+ &psy_cfg);
+ if (IS_ERR(chip->battery)) {
dev_err(&client->dev, "failed: power supply register\n");
- return ret;
+ return PTR_ERR(chip->battery);
}
if (client->irq) {
ret = request_threaded_irq(client->irq, NULL,
max17042_thread_handler,
IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
- chip->battery.name, chip);
+ chip->battery->desc->name, chip);
if (!ret) {
- regmap_read(chip->regmap, MAX17042_CONFIG, &val);
- val |= CONFIG_ALRT_BIT_ENBL;
- regmap_write(chip->regmap, MAX17042_CONFIG, val);
+ regmap_update_bits(chip->regmap, MAX17042_CONFIG,
+ CONFIG_ALRT_BIT_ENBL,
+ CONFIG_ALRT_BIT_ENBL);
max17042_set_soc_threshold(chip, 1);
} else {
client->irq = 0;
if (client->irq)
free_irq(client->irq, chip);
- power_supply_unregister(&chip->battery);
+ power_supply_unregister(chip->battery);
return 0;
}
#endif
static const struct i2c_device_id max17042_id[] = {
- { "max17042", 0 },
- { "max17047", 1 },
- { "max17050", 2 },
+ { "max17042", MAXIM_DEVICE_TYPE_MAX17042 },
+ { "max17047", MAXIM_DEVICE_TYPE_MAX17047 },
+ { "max17050", MAXIM_DEVICE_TYPE_MAX17050 },
{ }
};
MODULE_DEVICE_TABLE(i2c, max17042_id);
#include <linux/mfd/max77693.h>
#include <linux/mfd/max77693-private.h>
-static const char *max77693_charger_name = "max77693-charger";
+#define MAX77693_CHARGER_NAME "max77693-charger"
static const char *max77693_charger_model = "MAX77693";
static const char *max77693_charger_manufacturer = "Maxim Integrated";
struct max77693_charger {
struct device *dev;
struct max77693_dev *max77693;
- struct power_supply charger;
+ struct power_supply *charger;
u32 constant_volt;
u32 min_system_volt;
u32 charge_input_threshold_volt;
};
-static int max77693_get_charger_state(struct regmap *regmap)
+static int max77693_get_charger_state(struct regmap *regmap, int *val)
{
- int state;
+ int ret;
unsigned int data;
- if (regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data) < 0)
- return POWER_SUPPLY_STATUS_UNKNOWN;
+ ret = regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data);
+ if (ret < 0)
+ return ret;
data &= CHG_DETAILS_01_CHG_MASK;
data >>= CHG_DETAILS_01_CHG_SHIFT;
case MAX77693_CHARGING_TOP_OFF:
/* In high temp the charging current is reduced, but still charging */
case MAX77693_CHARGING_HIGH_TEMP:
- state = POWER_SUPPLY_STATUS_CHARGING;
+ *val = POWER_SUPPLY_STATUS_CHARGING;
break;
case MAX77693_CHARGING_DONE:
- state = POWER_SUPPLY_STATUS_FULL;
+ *val = POWER_SUPPLY_STATUS_FULL;
break;
case MAX77693_CHARGING_TIMER_EXPIRED:
case MAX77693_CHARGING_THERMISTOR_SUSPEND:
- state = POWER_SUPPLY_STATUS_NOT_CHARGING;
+ *val = POWER_SUPPLY_STATUS_NOT_CHARGING;
break;
case MAX77693_CHARGING_OFF:
case MAX77693_CHARGING_OVER_TEMP:
case MAX77693_CHARGING_WATCHDOG_EXPIRED:
- state = POWER_SUPPLY_STATUS_DISCHARGING;
+ *val = POWER_SUPPLY_STATUS_DISCHARGING;
break;
case MAX77693_CHARGING_RESERVED:
default:
- state = POWER_SUPPLY_STATUS_UNKNOWN;
+ *val = POWER_SUPPLY_STATUS_UNKNOWN;
}
- return state;
+ return 0;
}
-static int max77693_get_charge_type(struct regmap *regmap)
+static int max77693_get_charge_type(struct regmap *regmap, int *val)
{
- int state;
+ int ret;
unsigned int data;
- if (regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data) < 0)
- return POWER_SUPPLY_CHARGE_TYPE_UNKNOWN;
+ ret = regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data);
+ if (ret < 0)
+ return ret;
data &= CHG_DETAILS_01_CHG_MASK;
data >>= CHG_DETAILS_01_CHG_SHIFT;
* 100 and 250 mA. It is higher than prequalification current.
*/
case MAX77693_CHARGING_TOP_OFF:
- state = POWER_SUPPLY_CHARGE_TYPE_TRICKLE;
+ *val = POWER_SUPPLY_CHARGE_TYPE_TRICKLE;
break;
case MAX77693_CHARGING_FAST_CONST_CURRENT:
case MAX77693_CHARGING_FAST_CONST_VOLTAGE:
/* In high temp the charging current is reduced, but still charging */
case MAX77693_CHARGING_HIGH_TEMP:
- state = POWER_SUPPLY_CHARGE_TYPE_FAST;
+ *val = POWER_SUPPLY_CHARGE_TYPE_FAST;
break;
case MAX77693_CHARGING_DONE:
case MAX77693_CHARGING_TIMER_EXPIRED:
case MAX77693_CHARGING_OFF:
case MAX77693_CHARGING_OVER_TEMP:
case MAX77693_CHARGING_WATCHDOG_EXPIRED:
- state = POWER_SUPPLY_CHARGE_TYPE_NONE;
+ *val = POWER_SUPPLY_CHARGE_TYPE_NONE;
break;
case MAX77693_CHARGING_RESERVED:
default:
- state = POWER_SUPPLY_CHARGE_TYPE_UNKNOWN;
+ *val = POWER_SUPPLY_CHARGE_TYPE_UNKNOWN;
}
- return state;
+ return 0;
}
/*
* - POWER_SUPPLY_HEALTH_UNKNOWN
* - POWER_SUPPLY_HEALTH_UNSPEC_FAILURE
*/
-static int max77693_get_battery_health(struct regmap *regmap)
+static int max77693_get_battery_health(struct regmap *regmap, int *val)
{
- int state;
+ int ret;
unsigned int data;
- if (regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data) < 0)
- return POWER_SUPPLY_HEALTH_UNKNOWN;
+ ret = regmap_read(regmap, MAX77693_CHG_REG_CHG_DETAILS_01, &data);
+ if (ret < 0)
+ return ret;
data &= CHG_DETAILS_01_BAT_MASK;
data >>= CHG_DETAILS_01_BAT_SHIFT;
switch (data) {
case MAX77693_BATTERY_NOBAT:
- state = POWER_SUPPLY_HEALTH_DEAD;
+ *val = POWER_SUPPLY_HEALTH_DEAD;
break;
case MAX77693_BATTERY_PREQUALIFICATION:
case MAX77693_BATTERY_GOOD:
case MAX77693_BATTERY_LOWVOLTAGE:
- state = POWER_SUPPLY_HEALTH_GOOD;
+ *val = POWER_SUPPLY_HEALTH_GOOD;
break;
case MAX77693_BATTERY_TIMER_EXPIRED:
/*
* Took longer to charge than expected, charging suspended.
* Damaged battery?
*/
- state = POWER_SUPPLY_HEALTH_SAFETY_TIMER_EXPIRE;
+ *val = POWER_SUPPLY_HEALTH_SAFETY_TIMER_EXPIRE;
break;
case MAX77693_BATTERY_OVERVOLTAGE:
- state = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
+ *val = POWER_SUPPLY_HEALTH_OVERVOLTAGE;
break;
case MAX77693_BATTERY_OVERCURRENT:
- state = POWER_SUPPLY_HEALTH_UNSPEC_FAILURE;
+ *val = POWER_SUPPLY_HEALTH_UNSPEC_FAILURE;
break;
case MAX77693_BATTERY_RESERVED:
default:
- state = POWER_SUPPLY_HEALTH_UNKNOWN;
+ *val = POWER_SUPPLY_HEALTH_UNKNOWN;
break;
}
- return state;
+ return 0;
}
-static int max77693_get_present(struct regmap *regmap)
+static int max77693_get_present(struct regmap *regmap, int *val)
{
unsigned int data;
+ int ret;
/*
* Read CHG_INT_OK register. High DETBAT bit here should be
* equal to value 0x0 in CHG_DETAILS_01/BAT field.
*/
- regmap_read(regmap, MAX77693_CHG_REG_CHG_INT_OK, &data);
- if (data & CHG_INT_OK_DETBAT_MASK)
- return 0;
- return 1;
+ ret = regmap_read(regmap, MAX77693_CHG_REG_CHG_INT_OK, &data);
+ if (ret < 0)
+ return ret;
+
+ *val = (data & CHG_INT_OK_DETBAT_MASK) ? 0 : 1;
+
+ return 0;
}
-static int max77693_get_online(struct regmap *regmap)
+static int max77693_get_online(struct regmap *regmap, int *val)
{
unsigned int data;
+ int ret;
+
+ ret = regmap_read(regmap, MAX77693_CHG_REG_CHG_INT_OK, &data);
+ if (ret < 0)
+ return ret;
+
+ *val = (data & CHG_INT_OK_CHGIN_MASK) ? 1 : 0;
- regmap_read(regmap, MAX77693_CHG_REG_CHG_INT_OK, &data);
- if (data & CHG_INT_OK_CHGIN_MASK)
- return 1;
return 0;
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max77693_charger *chg = container_of(psy,
- struct max77693_charger,
- charger);
+ struct max77693_charger *chg = power_supply_get_drvdata(psy);
struct regmap *regmap = chg->max77693->regmap;
int ret = 0;
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
- val->intval = max77693_get_charger_state(regmap);
+ ret = max77693_get_charger_state(regmap, &val->intval);
break;
case POWER_SUPPLY_PROP_CHARGE_TYPE:
- val->intval = max77693_get_charge_type(regmap);
+ ret = max77693_get_charge_type(regmap, &val->intval);
break;
case POWER_SUPPLY_PROP_HEALTH:
- val->intval = max77693_get_battery_health(regmap);
+ ret = max77693_get_battery_health(regmap, &val->intval);
break;
case POWER_SUPPLY_PROP_PRESENT:
- val->intval = max77693_get_present(regmap);
+ ret = max77693_get_present(regmap, &val->intval);
break;
case POWER_SUPPLY_PROP_ONLINE:
- val->intval = max77693_get_online(regmap);
+ ret = max77693_get_online(regmap, &val->intval);
break;
case POWER_SUPPLY_PROP_MODEL_NAME:
val->strval = max77693_charger_model;
return ret;
}
+static const struct power_supply_desc max77693_charger_desc = {
+ .name = MAX77693_CHARGER_NAME,
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = max77693_charger_props,
+ .num_properties = ARRAY_SIZE(max77693_charger_props),
+ .get_property = max77693_charger_get_property,
+};
+
static ssize_t device_attr_store(struct device *dev,
struct device_attribute *attr, const char *buf, size_t count,
int (*fn)(struct max77693_charger *, unsigned long))
static int max77693_charger_probe(struct platform_device *pdev)
{
struct max77693_charger *chg;
+ struct power_supply_config psy_cfg = {};
struct max77693_dev *max77693 = dev_get_drvdata(pdev->dev.parent);
int ret;
if (ret)
return ret;
- chg->charger.name = max77693_charger_name;
- chg->charger.type = POWER_SUPPLY_TYPE_BATTERY;
- chg->charger.properties = max77693_charger_props;
- chg->charger.num_properties = ARRAY_SIZE(max77693_charger_props);
- chg->charger.get_property = max77693_charger_get_property;
+ psy_cfg.drv_data = chg;
ret = device_create_file(&pdev->dev, &dev_attr_fast_charge_timer);
if (ret) {
goto err;
}
- ret = power_supply_register(&pdev->dev, &chg->charger);
- if (ret) {
+ chg->charger = power_supply_register(&pdev->dev,
+ &max77693_charger_desc,
+ &psy_cfg);
+ if (IS_ERR(chg->charger)) {
dev_err(&pdev->dev, "failed: power supply register\n");
+ ret = PTR_ERR(chg->charger);
goto err;
}
device_remove_file(&pdev->dev, &dev_attr_top_off_threshold_current);
device_remove_file(&pdev->dev, &dev_attr_fast_charge_timer);
- power_supply_unregister(&chg->charger);
+ power_supply_unregister(chg->charger);
return 0;
}
struct max8903_data {
struct max8903_pdata pdata;
struct device *dev;
- struct power_supply psy;
+ struct power_supply *psy;
+ struct power_supply_desc psy_desc;
bool fault;
bool usb_in;
bool ta_in;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max8903_data *data = container_of(psy,
- struct max8903_data, psy);
+ struct max8903_data *data = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
dev_dbg(data->dev, "TA(DC-IN) Charger %s.\n", ta_in ?
"Connected" : "Disconnected");
- old_type = data->psy.type;
+ old_type = data->psy_desc.type;
if (data->ta_in)
- data->psy.type = POWER_SUPPLY_TYPE_MAINS;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_MAINS;
else if (data->usb_in)
- data->psy.type = POWER_SUPPLY_TYPE_USB;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_USB;
else
- data->psy.type = POWER_SUPPLY_TYPE_BATTERY;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_BATTERY;
- if (old_type != data->psy.type)
- power_supply_changed(&data->psy);
+ if (old_type != data->psy_desc.type)
+ power_supply_changed(data->psy);
return IRQ_HANDLED;
}
dev_dbg(data->dev, "USB Charger %s.\n", usb_in ?
"Connected" : "Disconnected");
- old_type = data->psy.type;
+ old_type = data->psy_desc.type;
if (data->ta_in)
- data->psy.type = POWER_SUPPLY_TYPE_MAINS;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_MAINS;
else if (data->usb_in)
- data->psy.type = POWER_SUPPLY_TYPE_USB;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_USB;
else
- data->psy.type = POWER_SUPPLY_TYPE_BATTERY;
+ data->psy_desc.type = POWER_SUPPLY_TYPE_BATTERY;
- if (old_type != data->psy.type)
- power_supply_changed(&data->psy);
+ if (old_type != data->psy_desc.type)
+ power_supply_changed(data->psy);
return IRQ_HANDLED;
}
struct max8903_data *data;
struct device *dev = &pdev->dev;
struct max8903_pdata *pdata = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
int ret = 0;
int gpio;
int ta_in = 0;
data->ta_in = ta_in;
data->usb_in = usb_in;
- data->psy.name = "max8903_charger";
- data->psy.type = (ta_in) ? POWER_SUPPLY_TYPE_MAINS :
+ data->psy_desc.name = "max8903_charger";
+ data->psy_desc.type = (ta_in) ? POWER_SUPPLY_TYPE_MAINS :
((usb_in) ? POWER_SUPPLY_TYPE_USB :
POWER_SUPPLY_TYPE_BATTERY);
- data->psy.get_property = max8903_get_property;
- data->psy.properties = max8903_charger_props;
- data->psy.num_properties = ARRAY_SIZE(max8903_charger_props);
+ data->psy_desc.get_property = max8903_get_property;
+ data->psy_desc.properties = max8903_charger_props;
+ data->psy_desc.num_properties = ARRAY_SIZE(max8903_charger_props);
- ret = power_supply_register(dev, &data->psy);
- if (ret) {
+ psy_cfg.drv_data = data;
+
+ data->psy = power_supply_register(dev, &data->psy_desc, &psy_cfg);
+ if (IS_ERR(data->psy)) {
dev_err(dev, "failed: power supply register.\n");
+ ret = PTR_ERR(data->psy);
goto err;
}
if (pdata->dc_valid)
free_irq(gpio_to_irq(pdata->dok), data);
err_psy:
- power_supply_unregister(&data->psy);
+ power_supply_unregister(data->psy);
err:
return ret;
}
free_irq(gpio_to_irq(pdata->uok), data);
if (pdata->dc_valid)
free_irq(gpio_to_irq(pdata->dok), data);
- power_supply_unregister(&data->psy);
+ power_supply_unregister(data->psy);
}
return 0;
struct i2c_client *gpm;
struct i2c_client *adc;
- struct power_supply ac;
- struct power_supply usb;
- struct power_supply battery;
+ struct power_supply *ac;
+ struct power_supply *usb;
+ struct power_supply *battery;
int irq_base;
unsigned ac_online:1;
unsigned usb_online:1;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max8925_power_info *info = dev_get_drvdata(psy->dev->parent);
+ struct max8925_power_info *info = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max8925_power_info *info = dev_get_drvdata(psy->dev->parent);
+ struct max8925_power_info *info = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max8925_power_info *info = dev_get_drvdata(psy->dev->parent);
+ struct max8925_power_info *info = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
POWER_SUPPLY_PROP_STATUS,
};
+static const struct power_supply_desc ac_desc = {
+ .name = "max8925-ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = max8925_ac_props,
+ .num_properties = ARRAY_SIZE(max8925_ac_props),
+ .get_property = max8925_ac_get_prop,
+};
+
+static const struct power_supply_desc usb_desc = {
+ .name = "max8925-usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = max8925_usb_props,
+ .num_properties = ARRAY_SIZE(max8925_usb_props),
+ .get_property = max8925_usb_get_prop,
+};
+
+static const struct power_supply_desc battery_desc = {
+ .name = "max8925-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = max8925_battery_props,
+ .num_properties = ARRAY_SIZE(max8925_battery_props),
+ .get_property = max8925_bat_get_prop,
+};
+
#define REQUEST_IRQ(_irq, _name) \
do { \
ret = request_threaded_irq(chip->irq_base + _irq, NULL, \
static int max8925_power_probe(struct platform_device *pdev)
{
struct max8925_chip *chip = dev_get_drvdata(pdev->dev.parent);
+ struct power_supply_config psy_cfg = {}; /* Only for ac and usb */
struct max8925_power_pdata *pdata = NULL;
struct max8925_power_info *info;
int ret;
info->adc = chip->adc;
platform_set_drvdata(pdev, info);
- info->ac.name = "max8925-ac";
- info->ac.type = POWER_SUPPLY_TYPE_MAINS;
- info->ac.properties = max8925_ac_props;
- info->ac.num_properties = ARRAY_SIZE(max8925_ac_props);
- info->ac.get_property = max8925_ac_get_prop;
- info->ac.supplied_to = pdata->supplied_to;
- info->ac.num_supplicants = pdata->num_supplicants;
- ret = power_supply_register(&pdev->dev, &info->ac);
- if (ret)
+ psy_cfg.supplied_to = pdata->supplied_to;
+ psy_cfg.num_supplicants = pdata->num_supplicants;
+
+ info->ac = power_supply_register(&pdev->dev, &ac_desc, &psy_cfg);
+ if (IS_ERR(info->ac)) {
+ ret = PTR_ERR(info->ac);
goto out;
- info->ac.dev->parent = &pdev->dev;
-
- info->usb.name = "max8925-usb";
- info->usb.type = POWER_SUPPLY_TYPE_USB;
- info->usb.properties = max8925_usb_props;
- info->usb.num_properties = ARRAY_SIZE(max8925_usb_props);
- info->usb.get_property = max8925_usb_get_prop;
- info->usb.supplied_to = pdata->supplied_to;
- info->usb.num_supplicants = pdata->num_supplicants;
-
- ret = power_supply_register(&pdev->dev, &info->usb);
- if (ret)
+ }
+ info->ac->dev.parent = &pdev->dev;
+
+ info->usb = power_supply_register(&pdev->dev, &usb_desc, &psy_cfg);
+ if (IS_ERR(info->usb)) {
+ ret = PTR_ERR(info->usb);
goto out_usb;
- info->usb.dev->parent = &pdev->dev;
-
- info->battery.name = "max8925-battery";
- info->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- info->battery.properties = max8925_battery_props;
- info->battery.num_properties = ARRAY_SIZE(max8925_battery_props);
- info->battery.get_property = max8925_bat_get_prop;
- ret = power_supply_register(&pdev->dev, &info->battery);
- if (ret)
+ }
+ info->usb->dev.parent = &pdev->dev;
+
+ info->battery = power_supply_register(&pdev->dev, &battery_desc, NULL);
+ if (IS_ERR(info->battery)) {
+ ret = PTR_ERR(info->battery);
goto out_battery;
- info->battery.dev->parent = &pdev->dev;
+ }
+ info->battery->dev.parent = &pdev->dev;
info->batt_detect = pdata->batt_detect;
info->topoff_threshold = pdata->topoff_threshold;
max8925_init_charger(chip, info);
return 0;
out_battery:
- power_supply_unregister(&info->battery);
+ power_supply_unregister(info->battery);
out_usb:
- power_supply_unregister(&info->ac);
+ power_supply_unregister(info->ac);
out:
return ret;
}
struct max8925_power_info *info = platform_get_drvdata(pdev);
if (info) {
- power_supply_unregister(&info->ac);
- power_supply_unregister(&info->usb);
- power_supply_unregister(&info->battery);
+ power_supply_unregister(info->ac);
+ power_supply_unregister(info->usb);
+ power_supply_unregister(info->battery);
max8925_deinit_charger(info);
}
return 0;
struct charger_data {
struct device *dev;
struct max8997_dev *iodev;
- struct power_supply battery;
+ struct power_supply *battery;
};
static enum power_supply_property max8997_battery_props[] = {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct charger_data *charger = container_of(psy,
- struct charger_data, battery);
+ struct charger_data *charger = power_supply_get_drvdata(psy);
struct i2c_client *i2c = charger->iodev->i2c;
int ret;
u8 reg;
return 0;
}
+static const struct power_supply_desc max8997_battery_desc = {
+ .name = "max8997_pmic",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = max8997_battery_get_property,
+ .properties = max8997_battery_props,
+ .num_properties = ARRAY_SIZE(max8997_battery_props),
+};
+
static int max8997_battery_probe(struct platform_device *pdev)
{
int ret = 0;
struct charger_data *charger;
struct max8997_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct max8997_platform_data *pdata = dev_get_platdata(iodev->dev);
+ struct power_supply_config psy_cfg = {};
if (!pdata)
return -EINVAL;
platform_set_drvdata(pdev, charger);
- charger->battery.name = "max8997_pmic";
- charger->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- charger->battery.get_property = max8997_battery_get_property;
- charger->battery.properties = max8997_battery_props;
- charger->battery.num_properties = ARRAY_SIZE(max8997_battery_props);
charger->dev = &pdev->dev;
charger->iodev = iodev;
- ret = power_supply_register(&pdev->dev, &charger->battery);
- if (ret) {
+ psy_cfg.drv_data = charger;
+
+ charger->battery = power_supply_register(&pdev->dev,
+ &max8997_battery_desc,
+ &psy_cfg);
+ if (IS_ERR(charger->battery)) {
dev_err(&pdev->dev, "failed: power supply register\n");
- return ret;
+ return PTR_ERR(charger->battery);
}
return 0;
{
struct charger_data *charger = platform_get_drvdata(pdev);
- power_supply_unregister(&charger->battery);
+ power_supply_unregister(charger->battery);
return 0;
}
struct max8998_battery_data {
struct device *dev;
struct max8998_dev *iodev;
- struct power_supply battery;
+ struct power_supply *battery;
};
static enum power_supply_property max8998_battery_props[] = {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct max8998_battery_data *max8998 = container_of(psy,
- struct max8998_battery_data, battery);
+ struct max8998_battery_data *max8998 = power_supply_get_drvdata(psy);
struct i2c_client *i2c = max8998->iodev->i2c;
int ret;
u8 reg;
return 0;
}
+static const struct power_supply_desc max8998_battery_desc = {
+ .name = "max8998_pmic",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = max8998_battery_get_property,
+ .properties = max8998_battery_props,
+ .num_properties = ARRAY_SIZE(max8998_battery_props),
+};
+
static int max8998_battery_probe(struct platform_device *pdev)
{
struct max8998_dev *iodev = dev_get_drvdata(pdev->dev.parent);
struct max8998_platform_data *pdata = dev_get_platdata(iodev->dev);
+ struct power_supply_config psy_cfg = {};
struct max8998_battery_data *max8998;
struct i2c_client *i2c;
int ret = 0;
goto err;
}
- max8998->battery.name = "max8998_pmic";
- max8998->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- max8998->battery.get_property = max8998_battery_get_property;
- max8998->battery.properties = max8998_battery_props;
- max8998->battery.num_properties = ARRAY_SIZE(max8998_battery_props);
+ psy_cfg.drv_data = max8998;
- ret = power_supply_register(max8998->dev, &max8998->battery);
- if (ret) {
- dev_err(max8998->dev, "failed: power supply register\n");
+ max8998->battery = power_supply_register(max8998->dev,
+ &max8998_battery_desc,
+ &psy_cfg);
+ if (IS_ERR(max8998->battery)) {
+ ret = PTR_ERR(max8998->battery);
+ dev_err(max8998->dev, "failed: power supply register: %d\n",
+ ret);
goto err;
}
{
struct max8998_battery_data *max8998 = platform_get_drvdata(pdev);
- power_supply_unregister(&max8998->battery);
+ power_supply_unregister(max8998->battery);
return 0;
}
POWER_SUPPLY_PROP_ONLINE,
};
-static struct power_supply olpc_ac = {
+static const struct power_supply_desc olpc_ac_desc = {
.name = "olpc-ac",
.type = POWER_SUPPLY_TYPE_MAINS,
.properties = olpc_ac_props,
.get_property = olpc_ac_get_prop,
};
+static struct power_supply *olpc_ac;
+
static char bat_serial[17]; /* Ick */
static int olpc_bat_get_status(union power_supply_propval *val, uint8_t ec_byte)
* Initialisation
*********************************************************************/
-static struct power_supply olpc_bat = {
+static struct power_supply_desc olpc_bat_desc = {
.name = "olpc-battery",
.get_property = olpc_bat_get_property,
.use_for_apm = 1,
};
+static struct power_supply *olpc_bat;
+
static int olpc_battery_suspend(struct platform_device *pdev,
pm_message_t state)
{
- if (device_may_wakeup(olpc_ac.dev))
+ if (device_may_wakeup(&olpc_ac->dev))
olpc_ec_wakeup_set(EC_SCI_SRC_ACPWR);
else
olpc_ec_wakeup_clear(EC_SCI_SRC_ACPWR);
- if (device_may_wakeup(olpc_bat.dev))
+ if (device_may_wakeup(&olpc_bat->dev))
olpc_ec_wakeup_set(EC_SCI_SRC_BATTERY | EC_SCI_SRC_BATSOC
| EC_SCI_SRC_BATERR);
else
/* Ignore the status. It doesn't actually matter */
- ret = power_supply_register(&pdev->dev, &olpc_ac);
- if (ret)
- return ret;
+ olpc_ac = power_supply_register(&pdev->dev, &olpc_ac_desc, NULL);
+ if (IS_ERR(olpc_ac))
+ return PTR_ERR(olpc_ac);
if (olpc_board_at_least(olpc_board_pre(0xd0))) { /* XO-1.5 */
- olpc_bat.properties = olpc_xo15_bat_props;
- olpc_bat.num_properties = ARRAY_SIZE(olpc_xo15_bat_props);
+ olpc_bat_desc.properties = olpc_xo15_bat_props;
+ olpc_bat_desc.num_properties = ARRAY_SIZE(olpc_xo15_bat_props);
} else { /* XO-1 */
- olpc_bat.properties = olpc_xo1_bat_props;
- olpc_bat.num_properties = ARRAY_SIZE(olpc_xo1_bat_props);
+ olpc_bat_desc.properties = olpc_xo1_bat_props;
+ olpc_bat_desc.num_properties = ARRAY_SIZE(olpc_xo1_bat_props);
}
- ret = power_supply_register(&pdev->dev, &olpc_bat);
- if (ret)
+ olpc_bat = power_supply_register(&pdev->dev, &olpc_bat_desc, NULL);
+ if (IS_ERR(olpc_bat)) {
+ ret = PTR_ERR(olpc_bat);
goto battery_failed;
+ }
- ret = device_create_bin_file(olpc_bat.dev, &olpc_bat_eeprom);
+ ret = device_create_bin_file(&olpc_bat->dev, &olpc_bat_eeprom);
if (ret)
goto eeprom_failed;
- ret = device_create_file(olpc_bat.dev, &olpc_bat_error);
+ ret = device_create_file(&olpc_bat->dev, &olpc_bat_error);
if (ret)
goto error_failed;
if (olpc_ec_wakeup_available()) {
- device_set_wakeup_capable(olpc_ac.dev, true);
- device_set_wakeup_capable(olpc_bat.dev, true);
+ device_set_wakeup_capable(&olpc_ac->dev, true);
+ device_set_wakeup_capable(&olpc_bat->dev, true);
}
return 0;
error_failed:
- device_remove_bin_file(olpc_bat.dev, &olpc_bat_eeprom);
+ device_remove_bin_file(&olpc_bat->dev, &olpc_bat_eeprom);
eeprom_failed:
- power_supply_unregister(&olpc_bat);
+ power_supply_unregister(olpc_bat);
battery_failed:
- power_supply_unregister(&olpc_ac);
+ power_supply_unregister(olpc_ac);
return ret;
}
static int olpc_battery_remove(struct platform_device *pdev)
{
- device_remove_file(olpc_bat.dev, &olpc_bat_error);
- device_remove_bin_file(olpc_bat.dev, &olpc_bat_eeprom);
- power_supply_unregister(&olpc_bat);
- power_supply_unregister(&olpc_ac);
+ device_remove_file(&olpc_bat->dev, &olpc_bat_error);
+ device_remove_bin_file(&olpc_bat->dev, &olpc_bat_eeprom);
+ power_supply_unregister(olpc_bat);
+ power_supply_unregister(olpc_ac);
return 0;
}
int adapter_online;
int usb_online;
- struct power_supply usb;
- struct power_supply adapter;
- struct power_supply ac;
+ struct power_supply *usb;
+ struct power_supply *adapter;
+ struct power_supply *ac;
};
int pcf50633_mbc_usb_curlim_set(struct pcf50633 *pcf, int ma)
PCF50633_MBCC1_CHGENA, PCF50633_MBCC1_CHGENA);
}
- power_supply_changed(&mbc->usb);
+ power_supply_changed(mbc->usb);
return ret;
}
else if (irq == PCF50633_IRQ_ADPREM)
mbc->adapter_online = 0;
- power_supply_changed(&mbc->ac);
- power_supply_changed(&mbc->usb);
- power_supply_changed(&mbc->adapter);
+ power_supply_changed(mbc->ac);
+ power_supply_changed(mbc->usb);
+ power_supply_changed(mbc->adapter);
if (mbc->pcf->pdata->mbc_event_callback)
mbc->pcf->pdata->mbc_event_callback(mbc->pcf, irq);
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pcf50633_mbc *mbc = container_of(psy,
- struct pcf50633_mbc, adapter);
+ struct pcf50633_mbc *mbc = power_supply_get_drvdata(psy);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pcf50633_mbc *mbc = container_of(psy, struct pcf50633_mbc, usb);
+ struct pcf50633_mbc *mbc = power_supply_get_drvdata(psy);
int ret = 0;
u8 usblim = pcf50633_reg_read(mbc->pcf, PCF50633_REG_MBCC7) &
PCF50633_MBCC7_USB_MASK;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct pcf50633_mbc *mbc = container_of(psy, struct pcf50633_mbc, ac);
+ struct pcf50633_mbc *mbc = power_supply_get_drvdata(psy);
int ret = 0;
u8 usblim = pcf50633_reg_read(mbc->pcf, PCF50633_REG_MBCC7) &
PCF50633_MBCC7_USB_MASK;
PCF50633_IRQ_LOWBAT,
};
+static const struct power_supply_desc pcf50633_mbc_adapter_desc = {
+ .name = "adapter",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = power_props,
+ .num_properties = ARRAY_SIZE(power_props),
+ .get_property = &adapter_get_property,
+};
+
+static const struct power_supply_desc pcf50633_mbc_usb_desc = {
+ .name = "usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = power_props,
+ .num_properties = ARRAY_SIZE(power_props),
+ .get_property = usb_get_property,
+};
+
+static const struct power_supply_desc pcf50633_mbc_ac_desc = {
+ .name = "ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = power_props,
+ .num_properties = ARRAY_SIZE(power_props),
+ .get_property = ac_get_property,
+};
+
static int pcf50633_mbc_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
struct pcf50633_mbc *mbc;
int ret;
int i;
pcf50633_register_irq(mbc->pcf, mbc_irq_handlers[i],
pcf50633_mbc_irq_handler, mbc);
+ psy_cfg.supplied_to = mbc->pcf->pdata->batteries;
+ psy_cfg.num_supplicants = mbc->pcf->pdata->num_batteries;
+ psy_cfg.drv_data = mbc;
+
/* Create power supplies */
- mbc->adapter.name = "adapter";
- mbc->adapter.type = POWER_SUPPLY_TYPE_MAINS;
- mbc->adapter.properties = power_props;
- mbc->adapter.num_properties = ARRAY_SIZE(power_props);
- mbc->adapter.get_property = &adapter_get_property;
- mbc->adapter.supplied_to = mbc->pcf->pdata->batteries;
- mbc->adapter.num_supplicants = mbc->pcf->pdata->num_batteries;
-
- mbc->usb.name = "usb";
- mbc->usb.type = POWER_SUPPLY_TYPE_USB;
- mbc->usb.properties = power_props;
- mbc->usb.num_properties = ARRAY_SIZE(power_props);
- mbc->usb.get_property = usb_get_property;
- mbc->usb.supplied_to = mbc->pcf->pdata->batteries;
- mbc->usb.num_supplicants = mbc->pcf->pdata->num_batteries;
-
- mbc->ac.name = "ac";
- mbc->ac.type = POWER_SUPPLY_TYPE_MAINS;
- mbc->ac.properties = power_props;
- mbc->ac.num_properties = ARRAY_SIZE(power_props);
- mbc->ac.get_property = ac_get_property;
- mbc->ac.supplied_to = mbc->pcf->pdata->batteries;
- mbc->ac.num_supplicants = mbc->pcf->pdata->num_batteries;
-
- ret = power_supply_register(&pdev->dev, &mbc->adapter);
- if (ret) {
+ mbc->adapter = power_supply_register(&pdev->dev,
+ &pcf50633_mbc_adapter_desc,
+ &psy_cfg);
+ if (IS_ERR(mbc->adapter)) {
dev_err(mbc->pcf->dev, "failed to register adapter\n");
+ ret = PTR_ERR(mbc->adapter);
return ret;
}
- ret = power_supply_register(&pdev->dev, &mbc->usb);
- if (ret) {
+ mbc->usb = power_supply_register(&pdev->dev, &pcf50633_mbc_usb_desc,
+ &psy_cfg);
+ if (IS_ERR(mbc->usb)) {
dev_err(mbc->pcf->dev, "failed to register usb\n");
- power_supply_unregister(&mbc->adapter);
+ power_supply_unregister(mbc->adapter);
+ ret = PTR_ERR(mbc->usb);
return ret;
}
- ret = power_supply_register(&pdev->dev, &mbc->ac);
- if (ret) {
+ mbc->ac = power_supply_register(&pdev->dev, &pcf50633_mbc_ac_desc,
+ &psy_cfg);
+ if (IS_ERR(mbc->ac)) {
dev_err(mbc->pcf->dev, "failed to register ac\n");
- power_supply_unregister(&mbc->adapter);
- power_supply_unregister(&mbc->usb);
+ power_supply_unregister(mbc->adapter);
+ power_supply_unregister(mbc->usb);
+ ret = PTR_ERR(mbc->ac);
return ret;
}
pcf50633_free_irq(mbc->pcf, mbc_irq_handlers[i]);
sysfs_remove_group(&pdev->dev.kobj, &mbc_attr_group);
- power_supply_unregister(&mbc->usb);
- power_supply_unregister(&mbc->adapter);
- power_supply_unregister(&mbc->ac);
+ power_supply_unregister(mbc->usb);
+ power_supply_unregister(mbc->adapter);
+ power_supply_unregister(mbc->ac);
return 0;
}
static struct timer_list supply_timer;
static struct timer_list polling_timer;
static int polling;
+static struct power_supply *pda_psy_ac, *pda_psy_usb;
#if IS_ENABLED(CONFIG_USB_PHY)
static struct usb_phy *transceiver;
{
switch (psp) {
case POWER_SUPPLY_PROP_ONLINE:
- if (psy->type == POWER_SUPPLY_TYPE_MAINS)
+ if (psy->desc->type == POWER_SUPPLY_TYPE_MAINS)
val->intval = pdata->is_ac_online ?
pdata->is_ac_online() : 0;
else
"backup-battery",
};
-static struct power_supply pda_psy_ac = {
+static const struct power_supply_desc pda_psy_ac_desc = {
.name = "ac",
.type = POWER_SUPPLY_TYPE_MAINS,
- .supplied_to = pda_power_supplied_to,
- .num_supplicants = ARRAY_SIZE(pda_power_supplied_to),
.properties = pda_power_props,
.num_properties = ARRAY_SIZE(pda_power_props),
.get_property = pda_power_get_property,
};
-static struct power_supply pda_psy_usb = {
+static const struct power_supply_desc pda_psy_usb_desc = {
.name = "usb",
.type = POWER_SUPPLY_TYPE_USB,
- .supplied_to = pda_power_supplied_to,
- .num_supplicants = ARRAY_SIZE(pda_power_supplied_to),
.properties = pda_power_props,
.num_properties = ARRAY_SIZE(pda_power_props),
.get_property = pda_power_get_property,
{
if (ac_status == PDA_PSY_TO_CHANGE) {
ac_status = new_ac_status;
- power_supply_changed(&pda_psy_ac);
+ power_supply_changed(pda_psy_ac);
}
if (usb_status == PDA_PSY_TO_CHANGE) {
usb_status = new_usb_status;
- power_supply_changed(&pda_psy_usb);
+ power_supply_changed(pda_psy_usb);
}
}
static irqreturn_t power_changed_isr(int irq, void *power_supply)
{
- if (power_supply == &pda_psy_ac)
+ if (power_supply == pda_psy_ac)
ac_status = PDA_PSY_TO_CHANGE;
- else if (power_supply == &pda_psy_usb)
+ else if (power_supply == pda_psy_usb)
usb_status = PDA_PSY_TO_CHANGE;
else
return IRQ_NONE;
static int pda_power_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
int ret = 0;
dev = &pdev->dev;
usb_irq = platform_get_resource_byname(pdev, IORESOURCE_IRQ, "usb");
if (pdata->supplied_to) {
- pda_psy_ac.supplied_to = pdata->supplied_to;
- pda_psy_ac.num_supplicants = pdata->num_supplicants;
- pda_psy_usb.supplied_to = pdata->supplied_to;
- pda_psy_usb.num_supplicants = pdata->num_supplicants;
+ psy_cfg.supplied_to = pdata->supplied_to;
+ psy_cfg.num_supplicants = pdata->num_supplicants;
+ } else {
+ psy_cfg.supplied_to = pda_power_supplied_to;
+ psy_cfg.num_supplicants = ARRAY_SIZE(pda_power_supplied_to);
}
#if IS_ENABLED(CONFIG_USB_PHY)
#endif
if (pdata->is_ac_online) {
- ret = power_supply_register(&pdev->dev, &pda_psy_ac);
- if (ret) {
+ pda_psy_ac = power_supply_register(&pdev->dev,
+ &pda_psy_ac_desc, &psy_cfg);
+ if (IS_ERR(pda_psy_ac)) {
dev_err(dev, "failed to register %s power supply\n",
- pda_psy_ac.name);
+ pda_psy_ac_desc.name);
+ ret = PTR_ERR(pda_psy_ac);
goto ac_supply_failed;
}
if (ac_irq) {
ret = request_irq(ac_irq->start, power_changed_isr,
get_irq_flags(ac_irq), ac_irq->name,
- &pda_psy_ac);
+ pda_psy_ac);
if (ret) {
dev_err(dev, "request ac irq failed\n");
goto ac_irq_failed;
}
if (pdata->is_usb_online) {
- ret = power_supply_register(&pdev->dev, &pda_psy_usb);
- if (ret) {
+ pda_psy_usb = power_supply_register(&pdev->dev,
+ &pda_psy_usb_desc,
+ &psy_cfg);
+ if (IS_ERR(pda_psy_usb)) {
dev_err(dev, "failed to register %s power supply\n",
- pda_psy_usb.name);
+ pda_psy_usb_desc.name);
+ ret = PTR_ERR(pda_psy_usb);
goto usb_supply_failed;
}
if (usb_irq) {
ret = request_irq(usb_irq->start, power_changed_isr,
get_irq_flags(usb_irq),
- usb_irq->name, &pda_psy_usb);
+ usb_irq->name, pda_psy_usb);
if (ret) {
dev_err(dev, "request usb irq failed\n");
goto usb_irq_failed;
#if IS_ENABLED(CONFIG_USB_PHY)
otg_reg_notifier_failed:
if (pdata->is_usb_online && usb_irq)
- free_irq(usb_irq->start, &pda_psy_usb);
+ free_irq(usb_irq->start, pda_psy_usb);
#endif
usb_irq_failed:
if (pdata->is_usb_online)
- power_supply_unregister(&pda_psy_usb);
+ power_supply_unregister(pda_psy_usb);
usb_supply_failed:
if (pdata->is_ac_online && ac_irq)
- free_irq(ac_irq->start, &pda_psy_ac);
+ free_irq(ac_irq->start, pda_psy_ac);
#if IS_ENABLED(CONFIG_USB_PHY)
if (!IS_ERR_OR_NULL(transceiver))
usb_put_phy(transceiver);
#endif
ac_irq_failed:
if (pdata->is_ac_online)
- power_supply_unregister(&pda_psy_ac);
+ power_supply_unregister(pda_psy_ac);
ac_supply_failed:
if (ac_draw) {
regulator_put(ac_draw);
static int pda_power_remove(struct platform_device *pdev)
{
if (pdata->is_usb_online && usb_irq)
- free_irq(usb_irq->start, &pda_psy_usb);
+ free_irq(usb_irq->start, pda_psy_usb);
if (pdata->is_ac_online && ac_irq)
- free_irq(ac_irq->start, &pda_psy_ac);
+ free_irq(ac_irq->start, pda_psy_ac);
if (polling)
del_timer_sync(&polling_timer);
del_timer_sync(&supply_timer);
if (pdata->is_usb_online)
- power_supply_unregister(&pda_psy_usb);
+ power_supply_unregister(pda_psy_usb);
if (pdata->is_ac_online)
- power_supply_unregister(&pda_psy_ac);
+ power_supply_unregister(pda_psy_ac);
#if IS_ENABLED(CONFIG_USB_PHY)
if (!IS_ERR_OR_NULL(transceiver))
usb_put_phy(transceiver);
{
dev_err(pm2->dev, "Overvoltage detected\n");
pm2->flags.ovv = true;
- power_supply_changed(&pm2->ac_chg.psy);
+ power_supply_changed(pm2->ac_chg.psy);
/* Schedule a new HW failure check */
queue_delayed_work(pm2->charger_wq, &pm2->check_hw_failure_work, 0);
dev_dbg(pm2->dev , "20 minutes watchdog expired\n");
pm2->ac.wd_expired = true;
- power_supply_changed(&pm2->ac_chg.psy);
+ power_supply_changed(pm2->ac_chg.psy);
return 0;
}
struct pm2xxx_charger *pm2;
u8 val;
- if (charger->psy.type == POWER_SUPPLY_TYPE_MAINS)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_MAINS)
pm2 = to_pm2xxx_charger_ac_device_info(charger);
else
return -ENXIO;
dev_dbg(pm2->dev, "PM2301: " "Disabled AC charging\n");
}
- power_supply_changed(&pm2->ac_chg.psy);
+ power_supply_changed(pm2->ac_chg.psy);
error_occured:
return ret;
int ret;
struct pm2xxx_charger *pm2;
- if (charger->psy.type == POWER_SUPPLY_TYPE_MAINS)
+ if (charger->psy->desc->type == POWER_SUPPLY_TYPE_MAINS)
pm2 = to_pm2xxx_charger_ac_device_info(charger);
else
return -ENXIO;
struct pm2xxx_charger, ac_work);
- power_supply_changed(&pm2->ac_chg.psy);
- sysfs_notify(&pm2->ac_chg.psy.dev->kobj, NULL, "present");
+ power_supply_changed(pm2->ac_chg.psy);
+ sysfs_notify(&pm2->ac_chg.psy->dev.kobj, NULL, "present");
};
static void pm2xxx_charger_check_hw_failure_work(struct work_struct *work)
if (!(reg_value & (PM2XXX_INT4_S_ITVPWR1OVV |
PM2XXX_INT4_S_ITVPWR2OVV))) {
pm2->flags.ovv = false;
- power_supply_changed(&pm2->ac_chg.psy);
+ power_supply_changed(pm2->ac_chg.psy);
}
}
| PM2XXX_INT5_S_ITTHERMALSHUTDOWNFALL))
pm2->flags.main_thermal_prot = false;
- power_supply_changed(&pm2->ac_chg.psy);
+ power_supply_changed(pm2->ac_chg.psy);
}
static struct pm2xxx_interrupts pm2xxx_int = {
const struct i2c_device_id *id)
{
struct pm2xxx_platform_data *pl_data = i2c_client->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
struct pm2xxx_charger *pm2;
int ret = 0;
u8 val;
/* AC supply */
/* power_supply base class */
- pm2->ac_chg.psy.name = pm2->pdata->label;
- pm2->ac_chg.psy.type = POWER_SUPPLY_TYPE_MAINS;
- pm2->ac_chg.psy.properties = pm2xxx_charger_ac_props;
- pm2->ac_chg.psy.num_properties = ARRAY_SIZE(pm2xxx_charger_ac_props);
- pm2->ac_chg.psy.get_property = pm2xxx_charger_ac_get_property;
- pm2->ac_chg.psy.supplied_to = pm2->pdata->supplied_to;
- pm2->ac_chg.psy.num_supplicants = pm2->pdata->num_supplicants;
+ pm2->ac_chg_desc.name = pm2->pdata->label;
+ pm2->ac_chg_desc.type = POWER_SUPPLY_TYPE_MAINS;
+ pm2->ac_chg_desc.properties = pm2xxx_charger_ac_props;
+ pm2->ac_chg_desc.num_properties = ARRAY_SIZE(pm2xxx_charger_ac_props);
+ pm2->ac_chg_desc.get_property = pm2xxx_charger_ac_get_property;
+
+ psy_cfg.supplied_to = pm2->pdata->supplied_to;
+ psy_cfg.num_supplicants = pm2->pdata->num_supplicants;
/* pm2xxx_charger sub-class */
pm2->ac_chg.ops.enable = &pm2xxx_charger_ac_en;
pm2->ac_chg.ops.kick_wd = &pm2xxx_charger_watchdog_kick;
}
/* Register AC charger class */
- ret = power_supply_register(pm2->dev, &pm2->ac_chg.psy);
- if (ret) {
+ pm2->ac_chg.psy = power_supply_register(pm2->dev, &pm2->ac_chg_desc,
+ &psy_cfg);
+ if (IS_ERR(pm2->ac_chg.psy)) {
dev_err(pm2->dev, "failed to register AC charger\n");
+ ret = PTR_ERR(pm2->ac_chg.psy);
goto free_regulator;
}
ab8500_override_turn_on_stat(~AB8500_POW_KEY_1_ON,
AB8500_MAIN_CH_DET);
pm2->ac_conn = true;
- power_supply_changed(&pm2->ac_chg.psy);
- sysfs_notify(&pm2->ac_chg.psy.dev->kobj, NULL, "present");
+ power_supply_changed(pm2->ac_chg.psy);
+ sysfs_notify(&pm2->ac_chg.psy->dev.kobj, NULL, "present");
}
return 0;
free_irq(gpio_to_irq(pm2->pdata->gpio_irq_number), pm2);
unregister_pm2xxx_charger:
/* unregister power supply */
- power_supply_unregister(&pm2->ac_chg.psy);
+ power_supply_unregister(pm2->ac_chg.psy);
free_regulator:
/* disable the regulator */
regulator_put(pm2->regu);
/* disable the regulator */
regulator_put(pm2->regu);
- power_supply_unregister(&pm2->ac_chg.psy);
+ power_supply_unregister(pm2->ac_chg.psy);
if (gpio_is_valid(pm2->lpn_pin))
gpio_free(pm2->lpn_pin);
struct work_struct check_main_thermal_prot_work;
struct delayed_work check_hw_failure_work;
struct ux500_charger ac_chg;
+ struct power_supply_desc ac_chg_desc;
struct pm2xxx_charger_event_flags flags;
};
#include <linux/slab.h>
static struct pmu_battery_dev {
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct pmu_battery_info *pbi;
char name[16];
int propval;
} *pbats[PMU_MAX_BATTERIES];
-#define to_pmu_battery_dev(x) container_of(x, struct pmu_battery_dev, bat)
+#define to_pmu_battery_dev(x) power_supply_get_drvdata(x)
/*********************************************************************
* Power
POWER_SUPPLY_PROP_ONLINE,
};
-static struct power_supply pmu_ac = {
+static const struct power_supply_desc pmu_ac_desc = {
.name = "pmu-ac",
.type = POWER_SUPPLY_TYPE_MAINS,
.properties = pmu_ac_props,
.get_property = pmu_get_ac_prop,
};
+static struct power_supply *pmu_ac;
+
/*********************************************************************
* Battery properties
*********************************************************************/
static int __init pmu_bat_init(void)
{
- int ret;
+ int ret = 0;
int i;
bat_pdev = platform_device_register_simple("pmu-battery",
goto pdev_register_failed;
}
- ret = power_supply_register(&bat_pdev->dev, &pmu_ac);
- if (ret)
+ pmu_ac = power_supply_register(&bat_pdev->dev, &pmu_ac_desc, NULL);
+ if (IS_ERR(pmu_ac)) {
+ ret = PTR_ERR(pmu_ac);
goto ac_register_failed;
+ }
for (i = 0; i < pmu_battery_count; i++) {
+ struct power_supply_config psy_cfg = {};
struct pmu_battery_dev *pbat = kzalloc(sizeof(*pbat),
GFP_KERNEL);
if (!pbat)
break;
sprintf(pbat->name, "PMU_battery_%d", i);
- pbat->bat.name = pbat->name;
- pbat->bat.properties = pmu_bat_props;
- pbat->bat.num_properties = ARRAY_SIZE(pmu_bat_props);
- pbat->bat.get_property = pmu_bat_get_property;
+ pbat->bat_desc.name = pbat->name;
+ pbat->bat_desc.properties = pmu_bat_props;
+ pbat->bat_desc.num_properties = ARRAY_SIZE(pmu_bat_props);
+ pbat->bat_desc.get_property = pmu_bat_get_property;
pbat->pbi = &pmu_batteries[i];
+ psy_cfg.drv_data = pbat;
- ret = power_supply_register(&bat_pdev->dev, &pbat->bat);
- if (ret) {
+ pbat->bat = power_supply_register(&bat_pdev->dev,
+ &pbat->bat_desc,
+ &psy_cfg);
+ if (IS_ERR(pbat->bat)) {
+ ret = PTR_ERR(pbat->bat);
kfree(pbat);
goto battery_register_failed;
}
while (i--) {
if (!pbats[i])
continue;
- power_supply_unregister(&pbats[i]->bat);
+ power_supply_unregister(pbats[i]->bat);
kfree(pbats[i]);
}
- power_supply_unregister(&pmu_ac);
+ power_supply_unregister(pmu_ac);
ac_register_failed:
platform_device_unregister(bat_pdev);
pdev_register_failed:
for (i = 0; i < PMU_MAX_BATTERIES; i++) {
if (!pbats[i])
continue;
- power_supply_unregister(&pbats[i]->bat);
+ power_supply_unregister(pbats[i]->bat);
kfree(pbats[i]);
}
- power_supply_unregister(&pmu_ac);
+ power_supply_unregister(pmu_ac);
platform_device_unregister(bat_pdev);
}
/* Support both supplied_to and supplied_from modes */
if (supply->supplied_from) {
- if (!supplier->name)
+ if (!supplier->desc->name)
return false;
for (i = 0; i < supply->num_supplies; i++)
- if (!strcmp(supplier->name, supply->supplied_from[i]))
+ if (!strcmp(supplier->desc->name, supply->supplied_from[i]))
return true;
} else {
- if (!supply->name)
+ if (!supply->desc->name)
return false;
for (i = 0; i < supplier->num_supplicants; i++)
- if (!strcmp(supplier->supplied_to[i], supply->name))
+ if (!strcmp(supplier->supplied_to[i], supply->desc->name))
return true;
}
struct power_supply *pst = dev_get_drvdata(dev);
if (__power_supply_is_supplied_by(psy, pst)) {
- if (pst->external_power_changed)
- pst->external_power_changed(pst);
+ if (pst->desc->external_power_changed)
+ pst->desc->external_power_changed(pst);
}
return 0;
struct power_supply *psy = container_of(work, struct power_supply,
changed_work);
- dev_dbg(psy->dev, "%s\n", __func__);
+ dev_dbg(&psy->dev, "%s\n", __func__);
spin_lock_irqsave(&psy->changed_lock, flags);
/*
power_supply_update_leds(psy);
atomic_notifier_call_chain(&power_supply_notifier,
PSY_EVENT_PROP_CHANGED, psy);
- kobject_uevent(&psy->dev->kobj, KOBJ_CHANGE);
+ kobject_uevent(&psy->dev.kobj, KOBJ_CHANGE);
spin_lock_irqsave(&psy->changed_lock, flags);
}
* to true.
*/
if (likely(!psy->changed))
- pm_relax(psy->dev);
+ pm_relax(&psy->dev);
spin_unlock_irqrestore(&psy->changed_lock, flags);
}
{
unsigned long flags;
- dev_dbg(psy->dev, "%s\n", __func__);
+ dev_dbg(&psy->dev, "%s\n", __func__);
spin_lock_irqsave(&psy->changed_lock, flags);
psy->changed = true;
- pm_stay_awake(psy->dev);
+ pm_stay_awake(&psy->dev);
spin_unlock_irqrestore(&psy->changed_lock, flags);
schedule_work(&psy->changed_work);
}
break;
if (np == epsy->of_node) {
- dev_info(psy->dev, "%s: Found supply : %s\n",
- psy->name, epsy->name);
- psy->supplied_from[i-1] = (char *)epsy->name;
+ dev_info(&psy->dev, "%s: Found supply : %s\n",
+ psy->desc->name, epsy->desc->name);
+ psy->supplied_from[i-1] = (char *)epsy->desc->name;
psy->num_supplies++;
of_node_put(np);
break;
error = class_for_each_device(power_supply_class, NULL, psy,
__power_supply_populate_supplied_from);
- dev_dbg(psy->dev, "%s %d\n", __func__, error);
+ dev_dbg(&psy->dev, "%s %d\n", __func__, error);
return error;
}
of_node_put(np);
if (ret) {
- dev_dbg(psy->dev, "Failed to find supply!\n");
+ dev_dbg(&psy->dev, "Failed to find supply!\n");
return ret;
}
} while (np);
return 0;
/* All supplies found, allocate char ** array for filling */
- psy->supplied_from = devm_kzalloc(psy->dev, sizeof(psy->supplied_from),
+ psy->supplied_from = devm_kzalloc(&psy->dev, sizeof(psy->supplied_from),
GFP_KERNEL);
if (!psy->supplied_from) {
- dev_err(psy->dev, "Couldn't allocate memory for supply list\n");
+ dev_err(&psy->dev, "Couldn't allocate memory for supply list\n");
return -ENOMEM;
}
- *psy->supplied_from = devm_kzalloc(psy->dev, sizeof(char *) * (cnt - 1),
+ *psy->supplied_from = devm_kzalloc(&psy->dev,
+ sizeof(char *) * (cnt - 1),
GFP_KERNEL);
if (!*psy->supplied_from) {
- dev_err(psy->dev, "Couldn't allocate memory for supply list\n");
+ dev_err(&psy->dev, "Couldn't allocate memory for supply list\n");
return -ENOMEM;
}
struct power_supply *epsy = dev_get_drvdata(dev);
if (__power_supply_is_supplied_by(epsy, psy))
- if (!epsy->get_property(epsy, POWER_SUPPLY_PROP_ONLINE, &ret))
+ if (!epsy->desc->get_property(epsy, POWER_SUPPLY_PROP_ONLINE,
+ &ret))
return ret.intval;
return 0;
error = class_for_each_device(power_supply_class, NULL, psy,
__power_supply_am_i_supplied);
- dev_dbg(psy->dev, "%s %d\n", __func__, error);
+ dev_dbg(&psy->dev, "%s %d\n", __func__, error);
return error;
}
unsigned int *count = data;
(*count)++;
- if (psy->type != POWER_SUPPLY_TYPE_BATTERY)
- if (!psy->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &ret))
+ if (psy->desc->type != POWER_SUPPLY_TYPE_BATTERY)
+ if (!psy->desc->get_property(psy, POWER_SUPPLY_PROP_ONLINE,
+ &ret))
return ret.intval;
return 0;
int power_supply_set_battery_charged(struct power_supply *psy)
{
- if (psy->type == POWER_SUPPLY_TYPE_BATTERY && psy->set_charged) {
- psy->set_charged(psy);
+ if (atomic_read(&psy->use_cnt) >= 0 &&
+ psy->desc->type == POWER_SUPPLY_TYPE_BATTERY &&
+ psy->desc->set_charged) {
+ psy->desc->set_charged(psy);
return 0;
}
const char *name = data;
struct power_supply *psy = dev_get_drvdata(dev);
- return strcmp(psy->name, name) == 0;
+ return strcmp(psy->desc->name, name) == 0;
}
+/**
+ * power_supply_get_by_name() - Search for a power supply and returns its ref
+ * @name: Power supply name to fetch
+ *
+ * If power supply was found, it increases reference count for the
+ * internal power supply's device. The user should power_supply_put()
+ * after usage.
+ *
+ * Return: On success returns a reference to a power supply with
+ * matching name equals to @name, a NULL otherwise.
+ */
struct power_supply *power_supply_get_by_name(const char *name)
{
+ struct power_supply *psy = NULL;
struct device *dev = class_find_device(power_supply_class, NULL, name,
power_supply_match_device_by_name);
- return dev ? dev_get_drvdata(dev) : NULL;
+ if (dev) {
+ psy = dev_get_drvdata(dev);
+ atomic_inc(&psy->use_cnt);
+ }
+
+ return psy;
}
EXPORT_SYMBOL_GPL(power_supply_get_by_name);
+/**
+ * power_supply_put() - Drop reference obtained with power_supply_get_by_name
+ * @psy: Reference to put
+ *
+ * The reference to power supply should be put before unregistering
+ * the power supply.
+ */
+void power_supply_put(struct power_supply *psy)
+{
+ might_sleep();
+
+ atomic_dec(&psy->use_cnt);
+ put_device(&psy->dev);
+}
+EXPORT_SYMBOL_GPL(power_supply_put);
+
#ifdef CONFIG_OF
static int power_supply_match_device_node(struct device *dev, const void *data)
{
return dev->parent && dev->parent->of_node == data;
}
+/**
+ * power_supply_get_by_phandle() - Search for a power supply and returns its ref
+ * @np: Pointer to device node holding phandle property
+ * @phandle_name: Name of property holding a power supply name
+ *
+ * If power supply was found, it increases reference count for the
+ * internal power supply's device. The user should power_supply_put()
+ * after usage.
+ *
+ * Return: On success returns a reference to a power supply with
+ * matching name equals to value under @property, NULL or ERR_PTR otherwise.
+ */
struct power_supply *power_supply_get_by_phandle(struct device_node *np,
const char *property)
{
struct device_node *power_supply_np;
+ struct power_supply *psy = NULL;
struct device *dev;
power_supply_np = of_parse_phandle(np, property, 0);
of_node_put(power_supply_np);
- return dev ? dev_get_drvdata(dev) : NULL;
+ if (dev) {
+ psy = dev_get_drvdata(dev);
+ atomic_inc(&psy->use_cnt);
+ }
+
+ return psy;
}
EXPORT_SYMBOL_GPL(power_supply_get_by_phandle);
#endif /* CONFIG_OF */
+int power_supply_get_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ union power_supply_propval *val)
+{
+ if (atomic_read(&psy->use_cnt) <= 0)
+ return -ENODEV;
+
+ return psy->desc->get_property(psy, psp, val);
+}
+EXPORT_SYMBOL_GPL(power_supply_get_property);
+
+int power_supply_set_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ const union power_supply_propval *val)
+{
+ if (atomic_read(&psy->use_cnt) <= 0 || !psy->desc->set_property)
+ return -ENODEV;
+
+ return psy->desc->set_property(psy, psp, val);
+}
+EXPORT_SYMBOL_GPL(power_supply_set_property);
+
+int power_supply_property_is_writeable(struct power_supply *psy,
+ enum power_supply_property psp)
+{
+ if (atomic_read(&psy->use_cnt) <= 0 ||
+ !psy->desc->property_is_writeable)
+ return -ENODEV;
+
+ return psy->desc->property_is_writeable(psy, psp);
+}
+EXPORT_SYMBOL_GPL(power_supply_property_is_writeable);
+
+void power_supply_external_power_changed(struct power_supply *psy)
+{
+ if (atomic_read(&psy->use_cnt) <= 0 ||
+ !psy->desc->external_power_changed)
+ return;
+
+ psy->desc->external_power_changed(psy);
+}
+EXPORT_SYMBOL_GPL(power_supply_external_power_changed);
+
int power_supply_powers(struct power_supply *psy, struct device *dev)
{
- return sysfs_create_link(&psy->dev->kobj, &dev->kobj, "powers");
+ return sysfs_create_link(&psy->dev.kobj, &dev->kobj, "powers");
}
EXPORT_SYMBOL_GPL(power_supply_powers);
static void power_supply_dev_release(struct device *dev)
{
+ struct power_supply *psy = container_of(dev, struct power_supply, dev);
pr_debug("device: '%s': %s\n", dev_name(dev), __func__);
- kfree(dev);
+ kfree(psy);
}
int power_supply_reg_notifier(struct notifier_block *nb)
WARN_ON(tzd == NULL);
psy = tzd->devdata;
- ret = psy->get_property(psy, POWER_SUPPLY_PROP_TEMP, &val);
+ ret = psy->desc->get_property(psy, POWER_SUPPLY_PROP_TEMP, &val);
/* Convert tenths of degree Celsius to milli degree Celsius. */
if (!ret)
{
int i;
- if (psy->no_thermal)
+ if (psy->desc->no_thermal)
return 0;
/* Register battery zone device psy reports temperature */
- for (i = 0; i < psy->num_properties; i++) {
- if (psy->properties[i] == POWER_SUPPLY_PROP_TEMP) {
- psy->tzd = thermal_zone_device_register(psy->name, 0, 0,
- psy, &psy_tzd_ops, NULL, 0, 0);
+ for (i = 0; i < psy->desc->num_properties; i++) {
+ if (psy->desc->properties[i] == POWER_SUPPLY_PROP_TEMP) {
+ psy->tzd = thermal_zone_device_register(psy->desc->name,
+ 0, 0, psy, &psy_tzd_ops, NULL, 0, 0);
return PTR_ERR_OR_ZERO(psy->tzd);
}
}
int ret;
psy = tcd->devdata;
- ret = psy->get_property(psy,
+ ret = psy->desc->get_property(psy,
POWER_SUPPLY_PROP_CHARGE_CONTROL_LIMIT_MAX, &val);
if (!ret)
*state = val.intval;
int ret;
psy = tcd->devdata;
- ret = psy->get_property(psy,
+ ret = psy->desc->get_property(psy,
POWER_SUPPLY_PROP_CHARGE_CONTROL_LIMIT, &val);
if (!ret)
*state = val.intval;
psy = tcd->devdata;
val.intval = state;
- ret = psy->set_property(psy,
+ ret = psy->desc->set_property(psy,
POWER_SUPPLY_PROP_CHARGE_CONTROL_LIMIT, &val);
return ret;
int i;
/* Register for cooling device if psy can control charging */
- for (i = 0; i < psy->num_properties; i++) {
- if (psy->properties[i] ==
+ for (i = 0; i < psy->desc->num_properties; i++) {
+ if (psy->desc->properties[i] ==
POWER_SUPPLY_PROP_CHARGE_CONTROL_LIMIT) {
psy->tcd = thermal_cooling_device_register(
- (char *)psy->name,
+ (char *)psy->desc->name,
psy, &psy_tcd_ops);
return PTR_ERR_OR_ZERO(psy->tcd);
}
}
#endif
-static int __power_supply_register(struct device *parent,
- struct power_supply *psy, bool ws)
+static struct power_supply *__must_check
+__power_supply_register(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg,
+ bool ws)
{
struct device *dev;
+ struct power_supply *psy;
int rc;
- dev = kzalloc(sizeof(*dev), GFP_KERNEL);
- if (!dev)
- return -ENOMEM;
+ psy = kzalloc(sizeof(*psy), GFP_KERNEL);
+ if (!psy)
+ return ERR_PTR(-ENOMEM);
+
+ dev = &psy->dev;
device_initialize(dev);
dev->parent = parent;
dev->release = power_supply_dev_release;
dev_set_drvdata(dev, psy);
- psy->dev = dev;
+ psy->desc = desc;
+ atomic_inc(&psy->use_cnt);
+ if (cfg) {
+ psy->drv_data = cfg->drv_data;
+ psy->of_node = cfg->of_node;
+ psy->supplied_to = cfg->supplied_to;
+ psy->num_supplicants = cfg->num_supplicants;
+ }
- rc = dev_set_name(dev, "%s", psy->name);
+ rc = dev_set_name(dev, "%s", desc->name);
if (rc)
goto dev_set_name_failed;
power_supply_changed(psy);
- return 0;
+ return psy;
create_triggers_failed:
psy_unregister_cooler(psy);
check_supplies_failed:
dev_set_name_failed:
put_device(dev);
- return rc;
+ return ERR_PTR(rc);
}
-int power_supply_register(struct device *parent, struct power_supply *psy)
+/**
+ * power_supply_register() - Register new power supply
+ * @parent: Device to be a parent of power supply's device
+ * @desc: Description of power supply, must be valid through whole
+ * lifetime of this power supply
+ * @cfg: Run-time specific configuration accessed during registering,
+ * may be NULL
+ *
+ * Return: A pointer to newly allocated power_supply on success
+ * or ERR_PTR otherwise.
+ * Use power_supply_unregister() on returned power_supply pointer to release
+ * resources.
+ */
+struct power_supply *__must_check power_supply_register(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg)
{
- return __power_supply_register(parent, psy, true);
+ return __power_supply_register(parent, desc, cfg, true);
}
EXPORT_SYMBOL_GPL(power_supply_register);
-int power_supply_register_no_ws(struct device *parent, struct power_supply *psy)
+/**
+ * power_supply_register() - Register new non-waking-source power supply
+ * @parent: Device to be a parent of power supply's device
+ * @desc: Description of power supply, must be valid through whole
+ * lifetime of this power supply
+ * @cfg: Run-time specific configuration accessed during registering,
+ * may be NULL
+ *
+ * Return: A pointer to newly allocated power_supply on success
+ * or ERR_PTR otherwise.
+ * Use power_supply_unregister() on returned power_supply pointer to release
+ * resources.
+ */
+struct power_supply *__must_check
+power_supply_register_no_ws(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg)
{
- return __power_supply_register(parent, psy, false);
+ return __power_supply_register(parent, desc, cfg, false);
}
EXPORT_SYMBOL_GPL(power_supply_register_no_ws);
+static void devm_power_supply_release(struct device *dev, void *res)
+{
+ struct power_supply **psy = res;
+
+ power_supply_unregister(*psy);
+}
+
+/**
+ * power_supply_register() - Register managed power supply
+ * @parent: Device to be a parent of power supply's device
+ * @desc: Description of power supply, must be valid through whole
+ * lifetime of this power supply
+ * @cfg: Run-time specific configuration accessed during registering,
+ * may be NULL
+ *
+ * Return: A pointer to newly allocated power_supply on success
+ * or ERR_PTR otherwise.
+ * The returned power_supply pointer will be automatically unregistered
+ * on driver detach.
+ */
+struct power_supply *__must_check
+devm_power_supply_register(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg)
+{
+ struct power_supply **ptr, *psy;
+
+ ptr = devres_alloc(devm_power_supply_release, sizeof(*ptr), GFP_KERNEL);
+
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+ psy = __power_supply_register(parent, desc, cfg, true);
+ if (IS_ERR(psy)) {
+ devres_free(ptr);
+ } else {
+ *ptr = psy;
+ devres_add(parent, ptr);
+ }
+ return psy;
+}
+EXPORT_SYMBOL_GPL(devm_power_supply_register);
+
+/**
+ * power_supply_register() - Register managed non-waking-source power supply
+ * @parent: Device to be a parent of power supply's device
+ * @desc: Description of power supply, must be valid through whole
+ * lifetime of this power supply
+ * @cfg: Run-time specific configuration accessed during registering,
+ * may be NULL
+ *
+ * Return: A pointer to newly allocated power_supply on success
+ * or ERR_PTR otherwise.
+ * The returned power_supply pointer will be automatically unregistered
+ * on driver detach.
+ */
+struct power_supply *__must_check
+devm_power_supply_register_no_ws(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg)
+{
+ struct power_supply **ptr, *psy;
+
+ ptr = devres_alloc(devm_power_supply_release, sizeof(*ptr), GFP_KERNEL);
+
+ if (!ptr)
+ return ERR_PTR(-ENOMEM);
+ psy = __power_supply_register(parent, desc, cfg, false);
+ if (IS_ERR(psy)) {
+ devres_free(ptr);
+ } else {
+ *ptr = psy;
+ devres_add(parent, ptr);
+ }
+ return psy;
+}
+EXPORT_SYMBOL_GPL(devm_power_supply_register_no_ws);
+
+/**
+ * power_supply_unregister() - Remove this power supply from system
+ * @psy: Pointer to power supply to unregister
+ *
+ * Remove this power supply from the system. The resources of power supply
+ * will be freed here or on last power_supply_put() call.
+ */
void power_supply_unregister(struct power_supply *psy)
{
+ WARN_ON(atomic_dec_return(&psy->use_cnt));
cancel_work_sync(&psy->changed_work);
- sysfs_remove_link(&psy->dev->kobj, "powers");
+ sysfs_remove_link(&psy->dev.kobj, "powers");
power_supply_remove_triggers(psy);
psy_unregister_cooler(psy);
psy_unregister_thermal(psy);
- device_init_wakeup(psy->dev, false);
- device_unregister(psy->dev);
+ device_init_wakeup(&psy->dev, false);
+ device_unregister(&psy->dev);
}
EXPORT_SYMBOL_GPL(power_supply_unregister);
+void *power_supply_get_drvdata(struct power_supply *psy)
+{
+ return psy->drv_data;
+}
+EXPORT_SYMBOL_GPL(power_supply_get_drvdata);
+
static int __init power_supply_class_init(void)
{
power_supply_class = class_create(THIS_MODULE, "power_supply");
unsigned long delay_on = 0;
unsigned long delay_off = 0;
- if (psy->get_property(psy, POWER_SUPPLY_PROP_STATUS, &status))
+ if (psy->desc->get_property(psy, POWER_SUPPLY_PROP_STATUS, &status))
return;
- dev_dbg(psy->dev, "%s %d\n", __func__, status.intval);
+ dev_dbg(&psy->dev, "%s %d\n", __func__, status.intval);
switch (status.intval) {
case POWER_SUPPLY_STATUS_FULL:
static int power_supply_create_bat_triggers(struct power_supply *psy)
{
psy->charging_full_trig_name = kasprintf(GFP_KERNEL,
- "%s-charging-or-full", psy->name);
+ "%s-charging-or-full", psy->desc->name);
if (!psy->charging_full_trig_name)
goto charging_full_failed;
psy->charging_trig_name = kasprintf(GFP_KERNEL,
- "%s-charging", psy->name);
+ "%s-charging", psy->desc->name);
if (!psy->charging_trig_name)
goto charging_failed;
- psy->full_trig_name = kasprintf(GFP_KERNEL, "%s-full", psy->name);
+ psy->full_trig_name = kasprintf(GFP_KERNEL, "%s-full", psy->desc->name);
if (!psy->full_trig_name)
goto full_failed;
psy->charging_blink_full_solid_trig_name = kasprintf(GFP_KERNEL,
- "%s-charging-blink-full-solid", psy->name);
+ "%s-charging-blink-full-solid", psy->desc->name);
if (!psy->charging_blink_full_solid_trig_name)
goto charging_blink_full_solid_failed;
{
union power_supply_propval online;
- if (psy->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &online))
+ if (psy->desc->get_property(psy, POWER_SUPPLY_PROP_ONLINE, &online))
return;
- dev_dbg(psy->dev, "%s %d\n", __func__, online.intval);
+ dev_dbg(&psy->dev, "%s %d\n", __func__, online.intval);
if (online.intval)
led_trigger_event(psy->online_trig, LED_FULL);
static int power_supply_create_gen_triggers(struct power_supply *psy)
{
- psy->online_trig_name = kasprintf(GFP_KERNEL, "%s-online", psy->name);
+ psy->online_trig_name = kasprintf(GFP_KERNEL, "%s-online",
+ psy->desc->name);
if (!psy->online_trig_name)
return -ENOMEM;
void power_supply_update_leds(struct power_supply *psy)
{
- if (psy->type == POWER_SUPPLY_TYPE_BATTERY)
+ if (psy->desc->type == POWER_SUPPLY_TYPE_BATTERY)
power_supply_update_bat_leds(psy);
else
power_supply_update_gen_leds(psy);
int power_supply_create_triggers(struct power_supply *psy)
{
- if (psy->type == POWER_SUPPLY_TYPE_BATTERY)
+ if (psy->desc->type == POWER_SUPPLY_TYPE_BATTERY)
return power_supply_create_bat_triggers(psy);
return power_supply_create_gen_triggers(psy);
}
void power_supply_remove_triggers(struct power_supply *psy)
{
- if (psy->type == POWER_SUPPLY_TYPE_BATTERY)
+ if (psy->desc->type == POWER_SUPPLY_TYPE_BATTERY)
power_supply_remove_bat_triggers(psy);
else
power_supply_remove_gen_triggers(psy);
union power_supply_propval value;
if (off == POWER_SUPPLY_PROP_TYPE) {
- value.intval = psy->type;
+ value.intval = psy->desc->type;
} else {
- ret = psy->get_property(psy, off, &value);
+ ret = power_supply_get_property(psy, off, &value);
if (ret < 0) {
if (ret == -ENODATA)
value.intval = long_val;
- ret = psy->set_property(psy, off, &value);
+ ret = psy->desc->set_property(psy, off, &value);
if (ret < 0)
return ret;
if (attrno == POWER_SUPPLY_PROP_TYPE)
return mode;
- for (i = 0; i < psy->num_properties; i++) {
- int property = psy->properties[i];
+ for (i = 0; i < psy->desc->num_properties; i++) {
+ int property = psy->desc->properties[i];
if (property == attrno) {
- if (psy->property_is_writeable &&
- psy->property_is_writeable(psy, property) > 0)
+ if (psy->desc->property_is_writeable &&
+ power_supply_property_is_writeable(psy, property) > 0)
mode |= S_IWUSR;
return mode;
dev_dbg(dev, "uevent\n");
- if (!psy || !psy->dev) {
+ if (!psy || !psy->desc) {
dev_dbg(dev, "No power supply yet\n");
return ret;
}
- dev_dbg(dev, "POWER_SUPPLY_NAME=%s\n", psy->name);
+ dev_dbg(dev, "POWER_SUPPLY_NAME=%s\n", psy->desc->name);
- ret = add_uevent_var(env, "POWER_SUPPLY_NAME=%s", psy->name);
+ ret = add_uevent_var(env, "POWER_SUPPLY_NAME=%s", psy->desc->name);
if (ret)
return ret;
if (!prop_buf)
return -ENOMEM;
- for (j = 0; j < psy->num_properties; j++) {
+ for (j = 0; j < psy->desc->num_properties; j++) {
struct device_attribute *attr;
char *line;
- attr = &power_supply_attrs[psy->properties[j]];
+ attr = &power_supply_attrs[psy->desc->properties[j]];
ret = power_supply_show_property(dev, attr, prop_buf);
if (ret == -ENODEV || ret == -ENODATA) {
help
Reboot support for generic SYSCON mapped register reset.
+config POWER_RESET_SYSCON_POWEROFF
+ bool "Generic SYSCON regmap poweroff driver"
+ depends on OF
+ select MFD_SYSCON
+ help
+ Poweroff support for generic SYSCON mapped register poweroff.
+
config POWER_RESET_RMOBILE
tristate "Renesas R-Mobile reset driver"
depends on ARCH_RMOBILE || COMPILE_TEST
+ depends on HAS_IOMEM
help
Reboot support for Renesas R-Mobile and SH-Mobile SoCs.
obj-$(CONFIG_POWER_RESET_XGENE) += xgene-reboot.o
obj-$(CONFIG_POWER_RESET_KEYSTONE) += keystone-reset.o
obj-$(CONFIG_POWER_RESET_SYSCON) += syscon-reboot.o
+obj-$(CONFIG_POWER_RESET_SYSCON_POWEROFF) += syscon-poweroff.o
obj-$(CONFIG_POWER_RESET_RMOBILE) += rmobile-reset.o
return 0;
}
-static struct of_device_id at91_poweroff_of_match[] = {
+static const struct of_device_id at91_poweroff_of_match[] = {
{ .compatible = "atmel,at91sam9260-shdwc", },
{ .compatible = "atmel,at91sam9rl-shdwc", },
{ .compatible = "atmel,at91sam9x5-shdwc", },
: "r" (at91_ramc_base[0]),
"r" (at91_rstc_base),
"r" (1),
- "r" (AT91_SDRAMC_LPCB_POWER_DOWN),
- "r" (AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST));
+ "r" cpu_to_le32(AT91_SDRAMC_LPCB_POWER_DOWN),
+ "r" cpu_to_le32(AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST));
return NOTIFY_DONE;
}
"r" (at91_ramc_base[1]),
"r" (at91_rstc_base),
"r" (1),
- "r" (AT91_DDRSDRC_LPCB_POWER_DOWN),
- "r" (AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST)
+ "r" cpu_to_le32(AT91_DDRSDRC_LPCB_POWER_DOWN),
+ "r" cpu_to_le32(AT91_RSTC_KEY | AT91_RSTC_PERRST | AT91_RSTC_PROCRST)
: "r0");
return NOTIFY_DONE;
pr_info("AT91: Starting after %s\n", reason);
}
-static struct of_device_id at91_ramc_of_match[] = {
+static const struct of_device_id at91_ramc_of_match[] = {
{ .compatible = "atmel,at91sam9260-sdramc", },
{ .compatible = "atmel,at91sam9g45-ddramc", },
{ .compatible = "atmel,sama5d3-ddramc", },
{ /* sentinel */ }
};
-static struct of_device_id at91_reset_of_match[] = {
+static const struct of_device_id at91_reset_of_match[] = {
{ .compatible = "atmel,at91sam9260-rstc", .data = at91sam9260_restart },
{ .compatible = "atmel,at91sam9g45-rstc", .data = at91sam9g45_restart },
{ /* sentinel */ }
return err;
}
-static struct of_device_id hisi_reboot_of_match[] = {
+static const struct of_device_id hisi_reboot_of_match[] = {
{ .compatible = "hisilicon,sysctrl" },
{}
};
.priority = 128,
};
-static struct of_device_id rsctrl_of_match[] = {
+static const struct of_device_id rsctrl_of_match[] = {
{.compatible = "ti,keystone-reset", },
{},
};
.priority = 192,
};
-static struct of_device_id st_reset_of_match[] = {
+static const struct of_device_id st_reset_of_match[] = {
{
.compatible = "st,stih415-restart",
.data = (void *)&stih415_reset,
--- /dev/null
+/*
+ * Generic Syscon Poweroff Driver
+ *
+ * Copyright (c) 2015, National Instruments Corp.
+ * Author: Moritz Fischer <moritz.fischer@ettus.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of
+ * the License, or (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <linux/kallsyms.h>
+#include <linux/delay.h>
+#include <linux/io.h>
+#include <linux/notifier.h>
+#include <linux/mfd/syscon.h>
+#include <linux/of_address.h>
+#include <linux/of_device.h>
+#include <linux/platform_device.h>
+#include <linux/pm.h>
+#include <linux/regmap.h>
+
+static struct regmap *map;
+static u32 offset;
+static u32 mask;
+
+void syscon_poweroff(void)
+{
+ /* Issue the poweroff */
+ regmap_write(map, offset, mask);
+
+ mdelay(1000);
+
+ pr_emerg("Unable to poweroff system\n");
+}
+
+static int syscon_poweroff_probe(struct platform_device *pdev)
+{
+ char symname[KSYM_NAME_LEN];
+
+ map = syscon_regmap_lookup_by_phandle(pdev->dev.of_node, "regmap");
+ if (IS_ERR(map)) {
+ dev_err(&pdev->dev, "unable to get syscon");
+ return PTR_ERR(map);
+ }
+
+ if (of_property_read_u32(pdev->dev.of_node, "offset", &offset)) {
+ dev_err(&pdev->dev, "unable to read 'offset'");
+ return -EINVAL;
+ }
+
+ if (of_property_read_u32(pdev->dev.of_node, "mask", &mask)) {
+ dev_err(&pdev->dev, "unable to read 'mask'");
+ return -EINVAL;
+ }
+
+ if (pm_power_off) {
+ lookup_symbol_name((ulong)pm_power_off, symname);
+ dev_err(&pdev->dev,
+ "pm_power_off already claimed %p %s",
+ pm_power_off, symname);
+ return -EBUSY;
+ }
+
+ pm_power_off = syscon_poweroff;
+
+ return 0;
+}
+
+static int syscon_poweroff_remove(struct platform_device *pdev)
+{
+ if (pm_power_off == syscon_poweroff)
+ pm_power_off = NULL;
+
+ return 0;
+}
+
+static const struct of_device_id syscon_poweroff_of_match[] = {
+ { .compatible = "syscon-poweroff" },
+ {}
+};
+
+static struct platform_driver syscon_poweroff_driver = {
+ .probe = syscon_poweroff_probe,
+ .remove = syscon_poweroff_remove,
+ .driver = {
+ .name = "syscon-poweroff",
+ .of_match_table = syscon_poweroff_of_match,
+ },
+};
+
+static int __init syscon_poweroff_register(void)
+{
+ return platform_driver_register(&syscon_poweroff_driver);
+}
+device_initcall(syscon_poweroff_register);
return err;
}
-static struct of_device_id syscon_reboot_of_match[] = {
+static const struct of_device_id syscon_reboot_of_match[] = {
{ .compatible = "syscon-reboot" },
{}
};
enum vexpress_reset_func { FUNC_RESET, FUNC_SHUTDOWN, FUNC_REBOOT };
-static struct of_device_id vexpress_reset_of_match[] = {
+static const struct of_device_id vexpress_reset_of_match[] = {
{
.compatible = "arm,vexpress-reset",
.data = (void *)FUNC_RESET,
return err;
}
-static struct of_device_id xgene_reboot_of_match[] = {
+static const struct of_device_id xgene_reboot_of_match[] = {
{ .compatible = "apm,xgene-reboot" },
{}
};
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct rt5033_battery *battery = container_of(psy,
- struct rt5033_battery, psy);
+ struct rt5033_battery *battery = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_VOLTAGE_NOW:
POWER_SUPPLY_PROP_CAPACITY,
};
-static struct regmap_config rt5033_battery_regmap_config = {
+static const struct regmap_config rt5033_battery_regmap_config = {
.reg_bits = 8,
.val_bits = 8,
.max_register = RT5033_FUEL_REG_END,
};
+static const struct power_supply_desc rt5033_battery_desc = {
+ .name = "rt5033-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = rt5033_battery_get_property,
+ .properties = rt5033_battery_props,
+ .num_properties = ARRAY_SIZE(rt5033_battery_props),
+};
+
static int rt5033_battery_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct i2c_adapter *adapter = to_i2c_adapter(client->dev.parent);
+ struct power_supply_config psy_cfg = {};
struct rt5033_battery *battery;
u32 ret;
}
i2c_set_clientdata(client, battery);
+ psy_cfg.drv_data = battery;
- battery->psy.name = "rt5033-battery";
- battery->psy.type = POWER_SUPPLY_TYPE_BATTERY;
- battery->psy.get_property = rt5033_battery_get_property;
- battery->psy.properties = rt5033_battery_props;
- battery->psy.num_properties = ARRAY_SIZE(rt5033_battery_props);
-
- ret = power_supply_register(&client->dev, &battery->psy);
- if (ret) {
+ battery->psy = power_supply_register(&client->dev,
+ &rt5033_battery_desc, &psy_cfg);
+ if (IS_ERR(battery->psy)) {
dev_err(&client->dev, "Failed to register power supply\n");
+ ret = PTR_ERR(battery->psy);
return ret;
}
{
struct rt5033_battery *battery = i2c_get_clientdata(client);
- power_supply_unregister(&battery->psy);
+ power_supply_unregister(battery->psy);
return 0;
}
struct rx51_device_info {
struct device *dev;
- struct power_supply bat;
+ struct power_supply *bat;
+ struct power_supply_desc bat_desc;
struct iio_channel *channel_temp;
struct iio_channel *channel_bsi;
struct iio_channel *channel_vbat;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct rx51_device_info *di = container_of((psy),
- struct rx51_device_info, bat);
+ struct rx51_device_info *di = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_TECHNOLOGY:
static int rx51_battery_probe(struct platform_device *pdev)
{
+ struct power_supply_config psy_cfg = {};
struct rx51_device_info *di;
int ret;
platform_set_drvdata(pdev, di);
di->dev = &pdev->dev;
- di->bat.name = dev_name(&pdev->dev);
- di->bat.type = POWER_SUPPLY_TYPE_BATTERY;
- di->bat.properties = rx51_battery_props;
- di->bat.num_properties = ARRAY_SIZE(rx51_battery_props);
- di->bat.get_property = rx51_battery_get_property;
+ di->bat_desc.name = dev_name(&pdev->dev);
+ di->bat_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ di->bat_desc.properties = rx51_battery_props;
+ di->bat_desc.num_properties = ARRAY_SIZE(rx51_battery_props);
+ di->bat_desc.get_property = rx51_battery_get_property;
+
+ psy_cfg.drv_data = di;
di->channel_temp = iio_channel_get(di->dev, "temp");
if (IS_ERR(di->channel_temp)) {
goto error_channel_bsi;
}
- ret = power_supply_register(di->dev, &di->bat);
- if (ret)
+ di->bat = power_supply_register(di->dev, &di->bat_desc, &psy_cfg);
+ if (IS_ERR(di->bat)) {
+ ret = PTR_ERR(di->bat);
goto error_channel_vbat;
+ }
return 0;
{
struct rx51_device_info *di = platform_get_drvdata(pdev);
- power_supply_unregister(&di->bat);
+ power_supply_unregister(di->bat);
iio_channel_release(di->channel_vbat);
iio_channel_release(di->channel_bsi);
#define JITTER_DELAY 500 /* ms */
struct s3c_adc_bat {
- struct power_supply psy;
+ struct power_supply *psy;
struct s3c_adc_client *client;
struct s3c_adc_bat_pdata *pdata;
int volt_value;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct s3c_adc_bat *bat = container_of(psy, struct s3c_adc_bat, psy);
+ struct s3c_adc_bat *bat = power_supply_get_drvdata(psy);
if (!bat) {
- dev_err(psy->dev, "%s: no battery infos ?!\n", __func__);
+ dev_err(&psy->dev, "%s: no battery infos ?!\n", __func__);
return -EINVAL;
}
}
}
-static struct s3c_adc_bat backup_bat = {
- .psy = {
- .name = "backup-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = s3c_adc_backup_bat_props,
- .num_properties = ARRAY_SIZE(s3c_adc_backup_bat_props),
- .get_property = s3c_adc_backup_bat_get_property,
- .use_for_apm = 1,
- },
+static const struct power_supply_desc backup_bat_desc = {
+ .name = "backup-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = s3c_adc_backup_bat_props,
+ .num_properties = ARRAY_SIZE(s3c_adc_backup_bat_props),
+ .get_property = s3c_adc_backup_bat_get_property,
+ .use_for_apm = 1,
};
+static struct s3c_adc_bat backup_bat;
+
static enum power_supply_property s3c_adc_main_bat_props[] = {
POWER_SUPPLY_PROP_STATUS,
POWER_SUPPLY_PROP_CHARGE_FULL_DESIGN,
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct s3c_adc_bat *bat = container_of(psy, struct s3c_adc_bat, psy);
+ struct s3c_adc_bat *bat = power_supply_get_drvdata(psy);
int new_level;
int full_volt;
unsigned int lut_size;
if (!bat) {
- dev_err(psy->dev, "no battery infos ?!\n");
+ dev_err(&psy->dev, "no battery infos ?!\n");
return -EINVAL;
}
}
}
-static struct s3c_adc_bat main_bat = {
- .psy = {
- .name = "main-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = s3c_adc_main_bat_props,
- .num_properties = ARRAY_SIZE(s3c_adc_main_bat_props),
- .get_property = s3c_adc_bat_get_property,
- .external_power_changed = s3c_adc_bat_ext_power_changed,
- .use_for_apm = 1,
- },
+static const struct power_supply_desc main_bat_desc = {
+ .name = "main-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = s3c_adc_main_bat_props,
+ .num_properties = ARRAY_SIZE(s3c_adc_main_bat_props),
+ .get_property = s3c_adc_bat_get_property,
+ .external_power_changed = s3c_adc_bat_ext_power_changed,
+ .use_for_apm = 1,
};
+static struct s3c_adc_bat main_bat;
+
static void s3c_adc_bat_work(struct work_struct *work)
{
struct s3c_adc_bat *bat = &main_bat;
int is_plugged;
static int was_plugged;
- is_plugged = power_supply_am_i_supplied(&bat->psy);
+ is_plugged = power_supply_am_i_supplied(bat->psy);
bat->cable_plugged = is_plugged;
if (is_plugged != was_plugged) {
was_plugged = is_plugged;
}
}
- power_supply_changed(&bat->psy);
+ power_supply_changed(bat->psy);
}
static irqreturn_t s3c_adc_bat_charged(int irq, void *dev_id)
main_bat.cable_plugged = 0;
main_bat.status = POWER_SUPPLY_STATUS_DISCHARGING;
- ret = power_supply_register(&pdev->dev, &main_bat.psy);
- if (ret)
+ main_bat.psy = power_supply_register(&pdev->dev, &main_bat_desc, NULL);
+ if (IS_ERR(main_bat.psy)) {
+ ret = PTR_ERR(main_bat.psy);
goto err_reg_main;
+ }
if (pdata->backup_volt_mult) {
+ const struct power_supply_config psy_cfg
+ = { .drv_data = &backup_bat, };
+
backup_bat.client = client;
backup_bat.pdata = pdev->dev.platform_data;
backup_bat.volt_value = -1;
- ret = power_supply_register(&pdev->dev, &backup_bat.psy);
- if (ret)
+ backup_bat.psy = power_supply_register(&pdev->dev,
+ &backup_bat_desc,
+ &psy_cfg);
+ if (IS_ERR(backup_bat.psy)) {
+ ret = PTR_ERR(backup_bat.psy);
goto err_reg_backup;
+ }
}
INIT_DELAYED_WORK(&bat_work, s3c_adc_bat_work);
gpio_free(pdata->gpio_charge_finished);
err_gpio:
if (pdata->backup_volt_mult)
- power_supply_unregister(&backup_bat.psy);
+ power_supply_unregister(backup_bat.psy);
err_reg_backup:
- power_supply_unregister(&main_bat.psy);
+ power_supply_unregister(main_bat.psy);
err_reg_main:
return ret;
}
struct s3c_adc_client *client = platform_get_drvdata(pdev);
struct s3c_adc_bat_pdata *pdata = pdev->dev.platform_data;
- power_supply_unregister(&main_bat.psy);
+ power_supply_unregister(main_bat.psy);
if (pdata->backup_volt_mult)
- power_supply_unregister(&backup_bat.psy);
+ power_supply_unregister(backup_bat.psy);
s3c_adc_release(client);
struct sbs_info {
struct i2c_client *client;
- struct power_supply power_supply;
+ struct power_supply *power_supply;
struct sbs_platform_data *pdata;
bool is_present;
bool gpio_detect;
chip->last_state = val->intval;
else if (chip->last_state != val->intval) {
cancel_delayed_work_sync(&chip->work);
- power_supply_changed(&chip->power_supply);
+ power_supply_changed(chip->power_supply);
chip->poll_time = 0;
}
} else {
union power_supply_propval *val)
{
int ret = 0;
- struct sbs_info *chip = container_of(psy,
- struct sbs_info, power_supply);
+ struct sbs_info *chip = power_supply_get_drvdata(psy);
struct i2c_client *client = chip->client;
switch (psp) {
if (!chip->gpio_detect &&
chip->is_present != (ret >= 0)) {
chip->is_present = (ret >= 0);
- power_supply_changed(&chip->power_supply);
+ power_supply_changed(chip->power_supply);
}
done:
static void sbs_external_power_changed(struct power_supply *psy)
{
- struct sbs_info *chip;
-
- chip = container_of(psy, struct sbs_info, power_supply);
+ struct sbs_info *chip = power_supply_get_drvdata(psy);
if (chip->ignore_changes > 0) {
chip->ignore_changes--;
if (chip->last_state != ret) {
chip->poll_time = 0;
- power_supply_changed(&chip->power_supply);
+ power_supply_changed(chip->power_supply);
return;
}
if (chip->poll_time > 0) {
}
#endif
+static const struct power_supply_desc sbs_default_desc = {
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = sbs_properties,
+ .num_properties = ARRAY_SIZE(sbs_properties),
+ .get_property = sbs_get_property,
+ .external_power_changed = sbs_external_power_changed,
+};
+
static int sbs_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
struct sbs_info *chip;
+ struct power_supply_desc *sbs_desc;
struct sbs_platform_data *pdata = client->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
int rc;
int irq;
- char *name;
- name = kasprintf(GFP_KERNEL, "sbs-%s", dev_name(&client->dev));
- if (!name) {
- dev_err(&client->dev, "Failed to allocate device name\n");
+ sbs_desc = devm_kmemdup(&client->dev, &sbs_default_desc,
+ sizeof(*sbs_desc), GFP_KERNEL);
+ if (!sbs_desc)
+ return -ENOMEM;
+
+ sbs_desc->name = devm_kasprintf(&client->dev, GFP_KERNEL, "sbs-%s",
+ dev_name(&client->dev));
+ if (!sbs_desc->name)
return -ENOMEM;
- }
chip = kzalloc(sizeof(struct sbs_info), GFP_KERNEL);
- if (!chip) {
- rc = -ENOMEM;
- goto exit_free_name;
- }
+ if (!chip)
+ return -ENOMEM;
chip->client = client;
chip->enable_detection = false;
chip->gpio_detect = false;
- chip->power_supply.name = name;
- chip->power_supply.type = POWER_SUPPLY_TYPE_BATTERY;
- chip->power_supply.properties = sbs_properties;
- chip->power_supply.num_properties = ARRAY_SIZE(sbs_properties);
- chip->power_supply.get_property = sbs_get_property;
- chip->power_supply.of_node = client->dev.of_node;
+ psy_cfg.of_node = client->dev.of_node;
+ psy_cfg.drv_data = chip;
/* ignore first notification of external change, it is generated
* from the power_supply_register call back
*/
chip->ignore_changes = 1;
chip->last_state = POWER_SUPPLY_STATUS_UNKNOWN;
- chip->power_supply.external_power_changed = sbs_external_power_changed;
pdata = sbs_of_populate_pdata(client);
rc = request_irq(irq, sbs_irq,
IRQF_TRIGGER_RISING | IRQF_TRIGGER_FALLING,
- dev_name(&client->dev), &chip->power_supply);
+ dev_name(&client->dev), chip->power_supply);
if (rc) {
dev_warn(&client->dev, "Failed to request irq: %d\n", rc);
gpio_free(pdata->battery_detect);
goto exit_psupply;
}
- rc = power_supply_register(&client->dev, &chip->power_supply);
- if (rc) {
+ chip->power_supply = power_supply_register(&client->dev, sbs_desc,
+ &psy_cfg);
+ if (IS_ERR(chip->power_supply)) {
dev_err(&client->dev,
"%s: Failed to register power supply\n", __func__);
+ rc = PTR_ERR(chip->power_supply);
goto exit_psupply;
}
exit_psupply:
if (chip->irq)
- free_irq(chip->irq, &chip->power_supply);
+ free_irq(chip->irq, chip->power_supply);
if (chip->gpio_detect)
gpio_free(pdata->battery_detect);
kfree(chip);
-exit_free_name:
- kfree(name);
-
return rc;
}
struct sbs_info *chip = i2c_get_clientdata(client);
if (chip->irq)
- free_irq(chip->irq, &chip->power_supply);
+ free_irq(chip->irq, chip->power_supply);
if (chip->gpio_detect)
gpio_free(chip->pdata->battery_detect);
- power_supply_unregister(&chip->power_supply);
+ power_supply_unregister(chip->power_supply);
cancel_delayed_work_sync(&chip->work);
- kfree(chip->power_supply.name);
kfree(chip);
chip = NULL;
struct mutex lock;
struct device *dev;
struct regmap *regmap;
- struct power_supply mains;
- struct power_supply usb;
- struct power_supply battery;
+ struct power_supply *mains;
+ struct power_supply *usb;
+ struct power_supply *battery;
bool mains_online;
bool usb_online;
bool charging_enabled;
*/
if (stat_c & STAT_C_CHARGER_ERROR) {
dev_err(smb->dev, "charging stopped due to charger error\n");
- power_supply_changed(&smb->battery);
+ power_supply_changed(smb->battery);
handled = true;
}
*/
if (irqstat_c & (IRQSTAT_C_TERMINATION_IRQ | IRQSTAT_C_TAPER_IRQ)) {
if (irqstat_c & IRQSTAT_C_TERMINATION_STAT)
- power_supply_changed(&smb->battery);
+ power_supply_changed(smb->battery);
dev_dbg(smb->dev, "going to HW maintenance mode\n");
handled = true;
}
if (irqstat_d & IRQSTAT_D_CHARGE_TIMEOUT_STAT)
dev_warn(smb->dev, "charging stopped due to timeout\n");
- power_supply_changed(&smb->battery);
+ power_supply_changed(smb->battery);
handled = true;
}
if (smb347_update_ps_status(smb) > 0) {
smb347_start_stop_charging(smb);
if (smb->pdata->use_mains)
- power_supply_changed(&smb->mains);
+ power_supply_changed(smb->mains);
if (smb->pdata->use_usb)
- power_supply_changed(&smb->usb);
+ power_supply_changed(smb->usb);
}
handled = true;
}
goto fail;
ret = request_threaded_irq(irq, NULL, smb347_interrupt,
- IRQF_TRIGGER_FALLING, client->name, smb);
+ IRQF_TRIGGER_FALLING | IRQF_ONESHOT,
+ client->name, smb);
if (ret < 0)
goto fail_gpio;
enum power_supply_property prop,
union power_supply_propval *val)
{
- struct smb347_charger *smb =
- container_of(psy, struct smb347_charger, mains);
+ struct smb347_charger *smb = power_supply_get_drvdata(psy);
int ret;
switch (prop) {
enum power_supply_property prop,
union power_supply_propval *val)
{
- struct smb347_charger *smb =
- container_of(psy, struct smb347_charger, usb);
+ struct smb347_charger *smb = power_supply_get_drvdata(psy);
int ret;
switch (prop) {
enum power_supply_property prop,
union power_supply_propval *val)
{
- struct smb347_charger *smb =
- container_of(psy, struct smb347_charger, battery);
+ struct smb347_charger *smb = power_supply_get_drvdata(psy);
const struct smb347_charger_platform_data *pdata = smb->pdata;
int ret;
.readable_reg = smb347_readable_reg,
};
+static const struct power_supply_desc smb347_mains_desc = {
+ .name = "smb347-mains",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .get_property = smb347_mains_get_property,
+ .properties = smb347_mains_properties,
+ .num_properties = ARRAY_SIZE(smb347_mains_properties),
+};
+
+static const struct power_supply_desc smb347_usb_desc = {
+ .name = "smb347-usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .get_property = smb347_usb_get_property,
+ .properties = smb347_usb_properties,
+ .num_properties = ARRAY_SIZE(smb347_usb_properties),
+};
+
+static const struct power_supply_desc smb347_battery_desc = {
+ .name = "smb347-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .get_property = smb347_battery_get_property,
+ .properties = smb347_battery_properties,
+ .num_properties = ARRAY_SIZE(smb347_battery_properties),
+};
+
static int smb347_probe(struct i2c_client *client,
const struct i2c_device_id *id)
{
static char *battery[] = { "smb347-battery" };
const struct smb347_charger_platform_data *pdata;
+ struct power_supply_config mains_usb_cfg = {}, battery_cfg = {};
struct device *dev = &client->dev;
struct smb347_charger *smb;
int ret;
if (ret < 0)
return ret;
+ mains_usb_cfg.supplied_to = battery;
+ mains_usb_cfg.num_supplicants = ARRAY_SIZE(battery);
+ mains_usb_cfg.drv_data = smb;
if (smb->pdata->use_mains) {
- smb->mains.name = "smb347-mains";
- smb->mains.type = POWER_SUPPLY_TYPE_MAINS;
- smb->mains.get_property = smb347_mains_get_property;
- smb->mains.properties = smb347_mains_properties;
- smb->mains.num_properties = ARRAY_SIZE(smb347_mains_properties);
- smb->mains.supplied_to = battery;
- smb->mains.num_supplicants = ARRAY_SIZE(battery);
- ret = power_supply_register(dev, &smb->mains);
- if (ret < 0)
- return ret;
+ smb->mains = power_supply_register(dev, &smb347_mains_desc,
+ &mains_usb_cfg);
+ if (IS_ERR(smb->mains))
+ return PTR_ERR(smb->mains);
}
if (smb->pdata->use_usb) {
- smb->usb.name = "smb347-usb";
- smb->usb.type = POWER_SUPPLY_TYPE_USB;
- smb->usb.get_property = smb347_usb_get_property;
- smb->usb.properties = smb347_usb_properties;
- smb->usb.num_properties = ARRAY_SIZE(smb347_usb_properties);
- smb->usb.supplied_to = battery;
- smb->usb.num_supplicants = ARRAY_SIZE(battery);
- ret = power_supply_register(dev, &smb->usb);
- if (ret < 0) {
+ smb->usb = power_supply_register(dev, &smb347_usb_desc,
+ &mains_usb_cfg);
+ if (IS_ERR(smb->usb)) {
if (smb->pdata->use_mains)
- power_supply_unregister(&smb->mains);
- return ret;
+ power_supply_unregister(smb->mains);
+ return PTR_ERR(smb->usb);
}
}
- smb->battery.name = "smb347-battery";
- smb->battery.type = POWER_SUPPLY_TYPE_BATTERY;
- smb->battery.get_property = smb347_battery_get_property;
- smb->battery.properties = smb347_battery_properties;
- smb->battery.num_properties = ARRAY_SIZE(smb347_battery_properties);
-
-
- ret = power_supply_register(dev, &smb->battery);
- if (ret < 0) {
+ battery_cfg.drv_data = smb;
+ smb->battery = power_supply_register(dev, &smb347_battery_desc,
+ &battery_cfg);
+ if (IS_ERR(smb->battery)) {
if (smb->pdata->use_usb)
- power_supply_unregister(&smb->usb);
+ power_supply_unregister(smb->usb);
if (smb->pdata->use_mains)
- power_supply_unregister(&smb->mains);
- return ret;
+ power_supply_unregister(smb->mains);
+ return PTR_ERR(smb->battery);
}
/*
gpio_free(smb->pdata->irq_gpio);
}
- power_supply_unregister(&smb->battery);
+ power_supply_unregister(smb->battery);
if (smb->pdata->use_usb)
- power_supply_unregister(&smb->usb);
+ power_supply_unregister(smb->usb);
if (smb->pdata->use_mains)
- power_supply_unregister(&smb->mains);
+ power_supply_unregister(smb->mains);
return 0;
}
"test_battery",
};
-static struct power_supply test_power_supplies[] = {
+static struct power_supply *test_power_supplies[TEST_POWER_NUM];
+
+static const struct power_supply_desc test_power_desc[] = {
[TEST_AC] = {
.name = "test_ac",
.type = POWER_SUPPLY_TYPE_MAINS,
- .supplied_to = test_power_ac_supplied_to,
- .num_supplicants = ARRAY_SIZE(test_power_ac_supplied_to),
.properties = test_power_ac_props,
.num_properties = ARRAY_SIZE(test_power_ac_props),
.get_property = test_power_get_ac_property,
[TEST_USB] = {
.name = "test_usb",
.type = POWER_SUPPLY_TYPE_USB,
- .supplied_to = test_power_ac_supplied_to,
- .num_supplicants = ARRAY_SIZE(test_power_ac_supplied_to),
.properties = test_power_ac_props,
.num_properties = ARRAY_SIZE(test_power_ac_props),
.get_property = test_power_get_usb_property,
},
};
+static const struct power_supply_config test_power_configs[] = {
+ {
+ /* test_ac */
+ .supplied_to = test_power_ac_supplied_to,
+ .num_supplicants = ARRAY_SIZE(test_power_ac_supplied_to),
+ }, {
+ /* test_battery */
+ }, {
+ /* test_usb */
+ .supplied_to = test_power_ac_supplied_to,
+ .num_supplicants = ARRAY_SIZE(test_power_ac_supplied_to),
+ },
+};
static int __init test_power_init(void)
{
int ret;
BUILD_BUG_ON(TEST_POWER_NUM != ARRAY_SIZE(test_power_supplies));
+ BUILD_BUG_ON(TEST_POWER_NUM != ARRAY_SIZE(test_power_configs));
for (i = 0; i < ARRAY_SIZE(test_power_supplies); i++) {
- ret = power_supply_register(NULL, &test_power_supplies[i]);
- if (ret) {
+ test_power_supplies[i] = power_supply_register(NULL,
+ &test_power_desc[i],
+ &test_power_configs[i]);
+ if (IS_ERR(test_power_supplies[i])) {
pr_err("%s: failed to register %s\n", __func__,
- test_power_supplies[i].name);
+ test_power_desc[i].name);
+ ret = PTR_ERR(test_power_supplies[i]);
goto failed;
}
}
return 0;
failed:
while (--i >= 0)
- power_supply_unregister(&test_power_supplies[i]);
+ power_supply_unregister(test_power_supplies[i]);
return ret;
}
module_init(test_power_init);
usb_online = 0;
battery_status = POWER_SUPPLY_STATUS_DISCHARGING;
for (i = 0; i < ARRAY_SIZE(test_power_supplies); i++)
- power_supply_changed(&test_power_supplies[i]);
+ power_supply_changed(test_power_supplies[i]);
pr_info("%s: 'changed' event sent, sleeping for 10 seconds...\n",
__func__);
ssleep(10);
for (i = 0; i < ARRAY_SIZE(test_power_supplies); i++)
- power_supply_unregister(&test_power_supplies[i]);
+ power_supply_unregister(test_power_supplies[i]);
module_initialized = false;
}
static int param_set_ac_online(const char *key, const struct kernel_param *kp)
{
ac_online = map_get_value(map_ac_online, key, ac_online);
- signal_power_supply_changed(&test_power_supplies[TEST_AC]);
+ signal_power_supply_changed(test_power_supplies[TEST_AC]);
return 0;
}
static int param_set_usb_online(const char *key, const struct kernel_param *kp)
{
usb_online = map_get_value(map_ac_online, key, usb_online);
- signal_power_supply_changed(&test_power_supplies[TEST_USB]);
+ signal_power_supply_changed(test_power_supplies[TEST_USB]);
return 0;
}
const struct kernel_param *kp)
{
battery_status = map_get_value(map_status, key, battery_status);
- signal_power_supply_changed(&test_power_supplies[TEST_BATTERY]);
+ signal_power_supply_changed(test_power_supplies[TEST_BATTERY]);
return 0;
}
const struct kernel_param *kp)
{
battery_health = map_get_value(map_health, key, battery_health);
- signal_power_supply_changed(&test_power_supplies[TEST_BATTERY]);
+ signal_power_supply_changed(test_power_supplies[TEST_BATTERY]);
return 0;
}
const struct kernel_param *kp)
{
battery_present = map_get_value(map_present, key, battery_present);
- signal_power_supply_changed(&test_power_supplies[TEST_AC]);
+ signal_power_supply_changed(test_power_supplies[TEST_AC]);
return 0;
}
{
battery_technology = map_get_value(map_technology, key,
battery_technology);
- signal_power_supply_changed(&test_power_supplies[TEST_BATTERY]);
+ signal_power_supply_changed(test_power_supplies[TEST_BATTERY]);
return 0;
}
return -EINVAL;
battery_capacity = tmp;
- signal_power_supply_changed(&test_power_supplies[TEST_BATTERY]);
+ signal_power_supply_changed(test_power_supplies[TEST_BATTERY]);
return 0;
}
return -EINVAL;
battery_voltage = tmp;
- signal_power_supply_changed(&test_power_supplies[TEST_BATTERY]);
+ signal_power_supply_changed(test_power_supplies[TEST_BATTERY]);
return 0;
}
struct tosa_bat {
int status;
- struct power_supply psy;
+ struct power_supply *psy;
int full_chrg;
struct mutex work_lock; /* protects data */
mutex_lock(&bat_lock);
gpio_set_value(bat->gpio_bat, 1);
msleep(5);
- value = wm97xx_read_aux_adc(dev_get_drvdata(bat->psy.dev->parent),
+ value = wm97xx_read_aux_adc(dev_get_drvdata(bat->psy->dev.parent),
bat->adc_bat);
gpio_set_value(bat->gpio_bat, 0);
mutex_unlock(&bat_lock);
mutex_lock(&bat_lock);
gpio_set_value(bat->gpio_temp, 1);
msleep(5);
- value = wm97xx_read_aux_adc(dev_get_drvdata(bat->psy.dev->parent),
+ value = wm97xx_read_aux_adc(dev_get_drvdata(bat->psy->dev.parent),
bat->adc_temp);
gpio_set_value(bat->gpio_temp, 0);
mutex_unlock(&bat_lock);
union power_supply_propval *val)
{
int ret = 0;
- struct tosa_bat *bat = container_of(psy, struct tosa_bat, psy);
+ struct tosa_bat *bat = power_supply_get_drvdata(psy);
if (bat->is_present && !bat->is_present(bat)
&& psp != POWER_SUPPLY_PROP_PRESENT) {
static void tosa_bat_update(struct tosa_bat *bat)
{
int old;
- struct power_supply *psy = &bat->psy;
+ struct power_supply *psy = bat->psy;
mutex_lock(&bat->work_lock);
old = bat->status;
if (bat->is_present && !bat->is_present(bat)) {
- printk(KERN_NOTICE "%s not present\n", psy->name);
+ printk(KERN_NOTICE "%s not present\n", psy->desc->name);
bat->status = POWER_SUPPLY_STATUS_UNKNOWN;
bat->full_chrg = -1;
} else if (power_supply_am_i_supplied(psy)) {
POWER_SUPPLY_PROP_PRESENT,
};
+static const struct power_supply_desc tosa_bat_main_desc = {
+ .name = "main-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = tosa_bat_main_props,
+ .num_properties = ARRAY_SIZE(tosa_bat_main_props),
+ .get_property = tosa_bat_get_property,
+ .external_power_changed = tosa_bat_external_power_changed,
+ .use_for_apm = 1,
+};
+
+static const struct power_supply_desc tosa_bat_jacket_desc = {
+ .name = "jacket-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = tosa_bat_main_props,
+ .num_properties = ARRAY_SIZE(tosa_bat_main_props),
+ .get_property = tosa_bat_get_property,
+ .external_power_changed = tosa_bat_external_power_changed,
+};
+
+static const struct power_supply_desc tosa_bat_bu_desc = {
+ .name = "backup-battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = tosa_bat_bu_props,
+ .num_properties = ARRAY_SIZE(tosa_bat_bu_props),
+ .get_property = tosa_bat_get_property,
+ .external_power_changed = tosa_bat_external_power_changed,
+};
+
static struct tosa_bat tosa_bat_main = {
.status = POWER_SUPPLY_STATUS_DISCHARGING,
.full_chrg = -1,
- .psy = {
- .name = "main-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = tosa_bat_main_props,
- .num_properties = ARRAY_SIZE(tosa_bat_main_props),
- .get_property = tosa_bat_get_property,
- .external_power_changed = tosa_bat_external_power_changed,
- .use_for_apm = 1,
- },
+ .psy = NULL,
.gpio_full = TOSA_GPIO_BAT0_CRG,
.gpio_charge_off = TOSA_GPIO_CHARGE_OFF,
static struct tosa_bat tosa_bat_jacket = {
.status = POWER_SUPPLY_STATUS_DISCHARGING,
.full_chrg = -1,
- .psy = {
- .name = "jacket-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = tosa_bat_main_props,
- .num_properties = ARRAY_SIZE(tosa_bat_main_props),
- .get_property = tosa_bat_get_property,
- .external_power_changed = tosa_bat_external_power_changed,
- },
+ .psy = NULL,
.is_present = tosa_jacket_bat_is_present,
.gpio_full = TOSA_GPIO_BAT1_CRG,
static struct tosa_bat tosa_bat_bu = {
.status = POWER_SUPPLY_STATUS_UNKNOWN,
.full_chrg = -1,
-
- .psy = {
- .name = "backup-battery",
- .type = POWER_SUPPLY_TYPE_BATTERY,
- .properties = tosa_bat_bu_props,
- .num_properties = ARRAY_SIZE(tosa_bat_bu_props),
- .get_property = tosa_bat_get_property,
- .external_power_changed = tosa_bat_external_power_changed,
- },
+ .psy = NULL,
.gpio_full = -1,
.gpio_charge_off = -1,
static int tosa_bat_probe(struct platform_device *dev)
{
int ret;
+ struct power_supply_config main_psy_cfg = {},
+ jacket_psy_cfg = {},
+ bu_psy_cfg = {};
if (!machine_is_tosa())
return -ENODEV;
INIT_WORK(&bat_work, tosa_bat_work);
- ret = power_supply_register(&dev->dev, &tosa_bat_main.psy);
- if (ret)
+ main_psy_cfg.drv_data = &tosa_bat_main;
+ tosa_bat_main.psy = power_supply_register(&dev->dev,
+ &tosa_bat_main_desc,
+ &main_psy_cfg);
+ if (IS_ERR(tosa_bat_main.psy)) {
+ ret = PTR_ERR(tosa_bat_main.psy);
goto err_psy_reg_main;
- ret = power_supply_register(&dev->dev, &tosa_bat_jacket.psy);
- if (ret)
+ }
+
+ jacket_psy_cfg.drv_data = &tosa_bat_jacket;
+ tosa_bat_jacket.psy = power_supply_register(&dev->dev,
+ &tosa_bat_jacket_desc,
+ &jacket_psy_cfg);
+ if (IS_ERR(tosa_bat_jacket.psy)) {
+ ret = PTR_ERR(tosa_bat_jacket.psy);
goto err_psy_reg_jacket;
- ret = power_supply_register(&dev->dev, &tosa_bat_bu.psy);
- if (ret)
+ }
+
+ bu_psy_cfg.drv_data = &tosa_bat_bu;
+ tosa_bat_bu.psy = power_supply_register(&dev->dev, &tosa_bat_bu_desc,
+ &bu_psy_cfg);
+ if (IS_ERR(tosa_bat_bu.psy)) {
+ ret = PTR_ERR(tosa_bat_bu.psy);
goto err_psy_reg_bu;
+ }
ret = request_irq(gpio_to_irq(TOSA_GPIO_BAT0_CRG),
tosa_bat_gpio_isr,
err_req_jacket:
free_irq(gpio_to_irq(TOSA_GPIO_BAT0_CRG), &tosa_bat_main);
err_req_main:
- power_supply_unregister(&tosa_bat_bu.psy);
+ power_supply_unregister(tosa_bat_bu.psy);
err_psy_reg_bu:
- power_supply_unregister(&tosa_bat_jacket.psy);
+ power_supply_unregister(tosa_bat_jacket.psy);
err_psy_reg_jacket:
- power_supply_unregister(&tosa_bat_main.psy);
+ power_supply_unregister(tosa_bat_main.psy);
err_psy_reg_main:
/* see comment in tosa_bat_remove */
free_irq(gpio_to_irq(TOSA_GPIO_BAT1_CRG), &tosa_bat_jacket);
free_irq(gpio_to_irq(TOSA_GPIO_BAT0_CRG), &tosa_bat_main);
- power_supply_unregister(&tosa_bat_bu.psy);
- power_supply_unregister(&tosa_bat_jacket.psy);
- power_supply_unregister(&tosa_bat_main.psy);
+ power_supply_unregister(tosa_bat_bu.psy);
+ power_supply_unregister(tosa_bat_jacket.psy);
+ power_supply_unregister(tosa_bat_main.psy);
/*
* Now cancel the bat_work. We won't get any more schedules,
int irq;
struct task_struct *poll_task;
bool passive_mode;
- struct power_supply ac;
+ struct power_supply *ac;
struct tps65090_platform_data *pdata;
};
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct tps65090_charger *charger = container_of(psy,
- struct tps65090_charger, ac);
+ struct tps65090_charger *charger = power_supply_get_drvdata(psy);
if (psp == POWER_SUPPLY_PROP_ONLINE) {
val->intval = charger->ac_online;
}
if (charger->prev_ac_online != charger->ac_online)
- power_supply_changed(&charger->ac);
+ power_supply_changed(charger->ac);
return IRQ_HANDLED;
}
return 0;
}
+static const struct power_supply_desc tps65090_charger_desc = {
+ .name = "tps65090-ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .get_property = tps65090_ac_get_property,
+ .properties = tps65090_ac_props,
+ .num_properties = ARRAY_SIZE(tps65090_ac_props),
+};
+
static int tps65090_charger_probe(struct platform_device *pdev)
{
struct tps65090_charger *cdata;
struct tps65090_platform_data *pdata;
+ struct power_supply_config psy_cfg = {};
uint8_t status1 = 0;
int ret;
int irq;
cdata->dev = &pdev->dev;
cdata->pdata = pdata;
- cdata->ac.name = "tps65090-ac";
- cdata->ac.type = POWER_SUPPLY_TYPE_MAINS;
- cdata->ac.get_property = tps65090_ac_get_property;
- cdata->ac.properties = tps65090_ac_props;
- cdata->ac.num_properties = ARRAY_SIZE(tps65090_ac_props);
- cdata->ac.supplied_to = pdata->supplied_to;
- cdata->ac.num_supplicants = pdata->num_supplicants;
- cdata->ac.of_node = pdev->dev.of_node;
-
- ret = power_supply_register(&pdev->dev, &cdata->ac);
- if (ret) {
+ psy_cfg.supplied_to = pdata->supplied_to;
+ psy_cfg.num_supplicants = pdata->num_supplicants;
+ psy_cfg.of_node = pdev->dev.of_node;
+ psy_cfg.drv_data = cdata;
+
+ cdata->ac = power_supply_register(&pdev->dev, &tps65090_charger_desc,
+ &psy_cfg);
+ if (IS_ERR(cdata->ac)) {
dev_err(&pdev->dev, "failed: power supply register\n");
- return ret;
+ return PTR_ERR(cdata->ac);
}
irq = platform_get_irq(pdev, 0);
goto fail_unregister_supply;
}
cdata->ac_online = 1;
- power_supply_changed(&cdata->ac);
+ power_supply_changed(cdata->ac);
}
if (irq != -ENXIO) {
return 0;
fail_unregister_supply:
- power_supply_unregister(&cdata->ac);
+ power_supply_unregister(cdata->ac);
return ret;
}
if (cdata->irq == -ENXIO)
kthread_stop(cdata->poll_task);
- power_supply_unregister(&cdata->ac);
+ power_supply_unregister(cdata->ac);
return 0;
}
-static struct of_device_id of_tps65090_charger_match[] = {
+static const struct of_device_id of_tps65090_charger_match[] = {
{ .compatible = "ti,tps65090-charger", },
{ /* end */ }
};
struct twl4030_bci {
struct device *dev;
- struct power_supply ac;
- struct power_supply usb;
+ struct power_supply *ac;
+ struct power_supply *usb;
struct usb_phy *transceiver;
struct notifier_block usb_nb;
struct work_struct work;
struct twl4030_bci *bci = arg;
dev_dbg(bci->dev, "CHG_PRES irq\n");
- power_supply_changed(&bci->ac);
- power_supply_changed(&bci->usb);
+ power_supply_changed(bci->ac);
+ power_supply_changed(bci->usb);
return IRQ_HANDLED;
}
if (irqs1 & (TWL4030_ICHGLOW | TWL4030_ICHGEOC)) {
/* charger state change, inform the core */
- power_supply_changed(&bci->ac);
- power_supply_changed(&bci->usb);
+ power_supply_changed(bci->ac);
+ power_supply_changed(bci->usb);
}
/* various monitoring events, for now we just log them here */
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct twl4030_bci *bci = dev_get_drvdata(psy->dev->parent);
+ struct twl4030_bci *bci = dev_get_drvdata(psy->dev.parent);
int is_charging;
int state;
int ret;
if (state < 0)
return state;
- if (psy->type == POWER_SUPPLY_TYPE_USB)
+ if (psy->desc->type == POWER_SUPPLY_TYPE_USB)
is_charging = state & TWL4030_MSTATEC_USB;
else
is_charging = state & TWL4030_MSTATEC_AC;
/* charging must be active for meaningful result */
if (!is_charging)
return -ENODATA;
- if (psy->type == POWER_SUPPLY_TYPE_USB) {
+ if (psy->desc->type == POWER_SUPPLY_TYPE_USB) {
ret = twl4030bci_read_adc_val(TWL4030_BCIVBUS);
if (ret < 0)
return ret;
}
#endif
+static const struct power_supply_desc twl4030_bci_ac_desc = {
+ .name = "twl4030_ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = twl4030_charger_props,
+ .num_properties = ARRAY_SIZE(twl4030_charger_props),
+ .get_property = twl4030_bci_get_property,
+};
+
+static const struct power_supply_desc twl4030_bci_usb_desc = {
+ .name = "twl4030_usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = twl4030_charger_props,
+ .num_properties = ARRAY_SIZE(twl4030_charger_props),
+ .get_property = twl4030_bci_get_property,
+};
+
static int __init twl4030_bci_probe(struct platform_device *pdev)
{
struct twl4030_bci *bci;
}
platform_set_drvdata(pdev, bci);
- bci->ac.name = "twl4030_ac";
- bci->ac.type = POWER_SUPPLY_TYPE_MAINS;
- bci->ac.properties = twl4030_charger_props;
- bci->ac.num_properties = ARRAY_SIZE(twl4030_charger_props);
- bci->ac.get_property = twl4030_bci_get_property;
- ret = power_supply_register(&pdev->dev, &bci->ac);
- if (ret) {
+ bci->ac = power_supply_register(&pdev->dev, &twl4030_bci_ac_desc,
+ NULL);
+ if (IS_ERR(bci->ac)) {
+ ret = PTR_ERR(bci->ac);
dev_err(&pdev->dev, "failed to register ac: %d\n", ret);
goto fail_register_ac;
}
- bci->usb.name = "twl4030_usb";
- bci->usb.type = POWER_SUPPLY_TYPE_USB;
- bci->usb.properties = twl4030_charger_props;
- bci->usb.num_properties = ARRAY_SIZE(twl4030_charger_props);
- bci->usb.get_property = twl4030_bci_get_property;
-
bci->usb_reg = regulator_get(bci->dev, "bci3v1");
- ret = power_supply_register(&pdev->dev, &bci->usb);
- if (ret) {
+ bci->usb = power_supply_register(&pdev->dev, &twl4030_bci_usb_desc,
+ NULL);
+ if (IS_ERR(bci->usb)) {
+ ret = PTR_ERR(bci->usb);
dev_err(&pdev->dev, "failed to register usb: %d\n", ret);
goto fail_register_usb;
}
fail_bci_irq:
free_irq(bci->irq_chg, bci);
fail_chg_irq:
- power_supply_unregister(&bci->usb);
+ power_supply_unregister(bci->usb);
fail_register_usb:
- power_supply_unregister(&bci->ac);
+ power_supply_unregister(bci->ac);
fail_register_ac:
fail_no_battery:
kfree(bci);
}
free_irq(bci->irq_bci, bci);
free_irq(bci->irq_chg, bci);
- power_supply_unregister(&bci->usb);
- power_supply_unregister(&bci->ac);
+ power_supply_unregister(bci->usb);
+ power_supply_unregister(bci->ac);
kfree(bci);
return 0;
#include <linux/sort.h>
#include <linux/i2c/twl4030-madc.h>
#include <linux/power/twl4030_madc_battery.h>
+#include <linux/iio/consumer.h>
struct twl4030_madc_battery {
- struct power_supply psy;
+ struct power_supply *psy;
struct twl4030_madc_bat_platform_data *pdata;
+ struct iio_channel *channel_temp;
+ struct iio_channel *channel_ichg;
+ struct iio_channel *channel_vbat;
};
static enum power_supply_property twl4030_madc_bat_props[] = {
POWER_SUPPLY_PROP_TIME_TO_EMPTY_NOW,
};
-static int madc_read(int index)
+static int madc_read(struct iio_channel *channel)
{
- struct twl4030_madc_request req;
- int val;
+ int val, err;
+ err = iio_read_channel_processed(channel, &val);
+ if (err < 0)
+ return err;
- req.channels = index;
- req.method = TWL4030_MADC_SW2;
- req.type = TWL4030_MADC_WAIT;
- req.do_avg = 0;
- req.raw = false;
- req.func_cb = NULL;
-
- val = twl4030_madc_conversion(&req);
- if (val < 0)
- return val;
-
- return req.rbuf[ffs(index) - 1];
+ return val;
}
-static int twl4030_madc_bat_get_charging_status(void)
+static int twl4030_madc_bat_get_charging_status(struct twl4030_madc_battery *bt)
{
- return (madc_read(TWL4030_MADC_ICHG) > 0) ? 1 : 0;
+ return (madc_read(bt->channel_ichg) > 0) ? 1 : 0;
}
-static int twl4030_madc_bat_get_voltage(void)
+static int twl4030_madc_bat_get_voltage(struct twl4030_madc_battery *bt)
{
- return madc_read(TWL4030_MADC_VBAT);
+ return madc_read(bt->channel_vbat);
}
-static int twl4030_madc_bat_get_current(void)
+static int twl4030_madc_bat_get_current(struct twl4030_madc_battery *bt)
{
- return madc_read(TWL4030_MADC_ICHG) * 1000;
+ return madc_read(bt->channel_ichg) * 1000;
}
-static int twl4030_madc_bat_get_temp(void)
+static int twl4030_madc_bat_get_temp(struct twl4030_madc_battery *bt)
{
- return madc_read(TWL4030_MADC_BTEMP) * 10;
+ return madc_read(bt->channel_temp) * 10;
}
static int twl4030_madc_bat_voltscale(struct twl4030_madc_battery *bat,
int i, res = 0;
/* choose charging curve */
- if (twl4030_madc_bat_get_charging_status())
+ if (twl4030_madc_bat_get_charging_status(bat))
calibration = bat->pdata->charging;
else
calibration = bat->pdata->discharging;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct twl4030_madc_battery *bat = container_of(psy,
- struct twl4030_madc_battery, psy);
+ struct twl4030_madc_battery *bat = power_supply_get_drvdata(psy);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
if (twl4030_madc_bat_voltscale(bat,
- twl4030_madc_bat_get_voltage()) > 95)
+ twl4030_madc_bat_get_voltage(bat)) > 95)
val->intval = POWER_SUPPLY_STATUS_FULL;
else {
- if (twl4030_madc_bat_get_charging_status())
+ if (twl4030_madc_bat_get_charging_status(bat))
val->intval = POWER_SUPPLY_STATUS_CHARGING;
else
val->intval = POWER_SUPPLY_STATUS_DISCHARGING;
}
break;
case POWER_SUPPLY_PROP_VOLTAGE_NOW:
- val->intval = twl4030_madc_bat_get_voltage() * 1000;
+ val->intval = twl4030_madc_bat_get_voltage(bat) * 1000;
break;
case POWER_SUPPLY_PROP_TECHNOLOGY:
val->intval = POWER_SUPPLY_TECHNOLOGY_LION;
break;
case POWER_SUPPLY_PROP_CURRENT_NOW:
- val->intval = twl4030_madc_bat_get_current();
+ val->intval = twl4030_madc_bat_get_current(bat);
break;
case POWER_SUPPLY_PROP_PRESENT:
/* assume battery is always present */
break;
case POWER_SUPPLY_PROP_CHARGE_NOW: {
int percent = twl4030_madc_bat_voltscale(bat,
- twl4030_madc_bat_get_voltage());
+ twl4030_madc_bat_get_voltage(bat));
val->intval = (percent * bat->pdata->capacity) / 100;
break;
}
case POWER_SUPPLY_PROP_CAPACITY:
val->intval = twl4030_madc_bat_voltscale(bat,
- twl4030_madc_bat_get_voltage());
+ twl4030_madc_bat_get_voltage(bat));
break;
case POWER_SUPPLY_PROP_CHARGE_FULL:
val->intval = bat->pdata->capacity;
break;
case POWER_SUPPLY_PROP_TEMP:
- val->intval = twl4030_madc_bat_get_temp();
+ val->intval = twl4030_madc_bat_get_temp(bat);
break;
case POWER_SUPPLY_PROP_TIME_TO_EMPTY_NOW: {
int percent = twl4030_madc_bat_voltscale(bat,
- twl4030_madc_bat_get_voltage());
+ twl4030_madc_bat_get_voltage(bat));
/* in mAh */
int chg = (percent * (bat->pdata->capacity/1000))/100;
static void twl4030_madc_bat_ext_changed(struct power_supply *psy)
{
- struct twl4030_madc_battery *bat = container_of(psy,
- struct twl4030_madc_battery, psy);
-
- power_supply_changed(&bat->psy);
+ power_supply_changed(psy);
}
+static const struct power_supply_desc twl4030_madc_bat_desc = {
+ .name = "twl4030_battery",
+ .type = POWER_SUPPLY_TYPE_BATTERY,
+ .properties = twl4030_madc_bat_props,
+ .num_properties = ARRAY_SIZE(twl4030_madc_bat_props),
+ .get_property = twl4030_madc_bat_get_property,
+ .external_power_changed = twl4030_madc_bat_ext_changed,
+
+};
+
static int twl4030_cmp(const void *a, const void *b)
{
return ((struct twl4030_madc_bat_calibration *)b)->voltage -
{
struct twl4030_madc_battery *twl4030_madc_bat;
struct twl4030_madc_bat_platform_data *pdata = pdev->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
+ int ret = 0;
- twl4030_madc_bat = kzalloc(sizeof(*twl4030_madc_bat), GFP_KERNEL);
+ twl4030_madc_bat = devm_kzalloc(&pdev->dev, sizeof(*twl4030_madc_bat),
+ GFP_KERNEL);
if (!twl4030_madc_bat)
return -ENOMEM;
- twl4030_madc_bat->psy.name = "twl4030_battery";
- twl4030_madc_bat->psy.type = POWER_SUPPLY_TYPE_BATTERY;
- twl4030_madc_bat->psy.properties = twl4030_madc_bat_props;
- twl4030_madc_bat->psy.num_properties =
- ARRAY_SIZE(twl4030_madc_bat_props);
- twl4030_madc_bat->psy.get_property = twl4030_madc_bat_get_property;
- twl4030_madc_bat->psy.external_power_changed =
- twl4030_madc_bat_ext_changed;
+ twl4030_madc_bat->channel_temp = iio_channel_get(&pdev->dev, "temp");
+ if (IS_ERR(twl4030_madc_bat->channel_temp)) {
+ ret = PTR_ERR(twl4030_madc_bat->channel_temp);
+ goto err;
+ }
+
+ twl4030_madc_bat->channel_ichg = iio_channel_get(&pdev->dev, "ichg");
+ if (IS_ERR(twl4030_madc_bat->channel_ichg)) {
+ ret = PTR_ERR(twl4030_madc_bat->channel_ichg);
+ goto err_temp;
+ }
+
+ twl4030_madc_bat->channel_vbat = iio_channel_get(&pdev->dev, "vbat");
+ if (IS_ERR(twl4030_madc_bat->channel_vbat)) {
+ ret = PTR_ERR(twl4030_madc_bat->channel_vbat);
+ goto err_ichg;
+ }
/* sort charging and discharging calibration data */
sort(pdata->charging, pdata->charging_size,
twl4030_madc_bat->pdata = pdata;
platform_set_drvdata(pdev, twl4030_madc_bat);
- power_supply_register(&pdev->dev, &twl4030_madc_bat->psy);
+ psy_cfg.drv_data = twl4030_madc_bat;
+ twl4030_madc_bat->psy = power_supply_register(&pdev->dev,
+ &twl4030_madc_bat_desc,
+ &psy_cfg);
+ if (IS_ERR(twl4030_madc_bat->psy)) {
+ ret = PTR_ERR(twl4030_madc_bat->psy);
+ goto err_vbat;
+ }
return 0;
+
+err_vbat:
+ iio_channel_release(twl4030_madc_bat->channel_vbat);
+err_ichg:
+ iio_channel_release(twl4030_madc_bat->channel_ichg);
+err_temp:
+ iio_channel_release(twl4030_madc_bat->channel_temp);
+err:
+ return ret;
}
static int twl4030_madc_battery_remove(struct platform_device *pdev)
{
struct twl4030_madc_battery *bat = platform_get_drvdata(pdev);
- power_supply_unregister(&bat->psy);
- kfree(bat);
+ power_supply_unregister(bat->psy);
+
+ iio_channel_release(bat->channel_vbat);
+ iio_channel_release(bat->channel_ichg);
+ iio_channel_release(bat->channel_temp);
return 0;
}
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Lukas Märdian <lukas@goldelico.com>");
MODULE_DESCRIPTION("twl4030_madc battery driver");
+MODULE_ALIAS("platform:twl4030_madc_battery");
struct wm831x_backup {
struct wm831x *wm831x;
- struct power_supply backup;
+ struct power_supply *backup;
+ struct power_supply_desc backup_desc;
char name[20];
};
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm831x_backup *devdata = dev_get_drvdata(psy->dev->parent);
+ struct wm831x_backup *devdata = dev_get_drvdata(psy->dev.parent);
struct wm831x *wm831x = devdata->wm831x;
int ret = 0;
struct wm831x *wm831x = dev_get_drvdata(pdev->dev.parent);
struct wm831x_pdata *wm831x_pdata = wm831x->dev->platform_data;
struct wm831x_backup *devdata;
- struct power_supply *backup;
- int ret;
devdata = devm_kzalloc(&pdev->dev, sizeof(struct wm831x_backup),
GFP_KERNEL);
devdata->wm831x = wm831x;
platform_set_drvdata(pdev, devdata);
- backup = &devdata->backup;
-
/* We ignore configuration failures since we can still read
* back the status without enabling the charger (which may
* already be enabled anyway).
snprintf(devdata->name, sizeof(devdata->name),
"wm831x-backup");
- backup->name = devdata->name;
- backup->type = POWER_SUPPLY_TYPE_BATTERY;
- backup->properties = wm831x_backup_props;
- backup->num_properties = ARRAY_SIZE(wm831x_backup_props);
- backup->get_property = wm831x_backup_get_prop;
- ret = power_supply_register(&pdev->dev, backup);
+ devdata->backup_desc.name = devdata->name;
+ devdata->backup_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ devdata->backup_desc.properties = wm831x_backup_props;
+ devdata->backup_desc.num_properties = ARRAY_SIZE(wm831x_backup_props);
+ devdata->backup_desc.get_property = wm831x_backup_get_prop;
+ devdata->backup = power_supply_register(&pdev->dev,
+ &devdata->backup_desc, NULL);
- return ret;
+ return PTR_ERR_OR_ZERO(devdata->backup);
}
static int wm831x_backup_remove(struct platform_device *pdev)
{
struct wm831x_backup *devdata = platform_get_drvdata(pdev);
- power_supply_unregister(&devdata->backup);
+ power_supply_unregister(devdata->backup);
return 0;
}
struct wm831x_power {
struct wm831x *wm831x;
- struct power_supply wall;
- struct power_supply usb;
- struct power_supply battery;
+ struct power_supply *wall;
+ struct power_supply *usb;
+ struct power_supply *battery;
+ struct power_supply_desc wall_desc;
+ struct power_supply_desc usb_desc;
+ struct power_supply_desc battery_desc;
char wall_name[20];
char usb_name[20];
char battery_name[20];
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev->parent);
+ struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev.parent);
struct wm831x *wm831x = wm831x_power->wm831x;
int ret = 0;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev->parent);
+ struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev.parent);
struct wm831x *wm831x = wm831x_power->wm831x;
int ret = 0;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev->parent);
+ struct wm831x_power *wm831x_power = dev_get_drvdata(psy->dev.parent);
struct wm831x *wm831x = wm831x_power->wm831x;
int ret = 0;
/* The battery charger is autonomous so we don't need to do
* anything except kick user space */
if (wm831x_power->have_battery)
- power_supply_changed(&wm831x_power->battery);
+ power_supply_changed(wm831x_power->battery);
return IRQ_HANDLED;
}
/* Just notify for everything - little harm in overnotifying. */
if (wm831x_power->have_battery)
- power_supply_changed(&wm831x_power->battery);
- power_supply_changed(&wm831x_power->usb);
- power_supply_changed(&wm831x_power->wall);
+ power_supply_changed(wm831x_power->battery);
+ power_supply_changed(wm831x_power->usb);
+ power_supply_changed(wm831x_power->wall);
return IRQ_HANDLED;
}
struct wm831x *wm831x = dev_get_drvdata(pdev->dev.parent);
struct wm831x_pdata *wm831x_pdata = wm831x->dev->platform_data;
struct wm831x_power *power;
- struct power_supply *usb;
- struct power_supply *battery;
- struct power_supply *wall;
int ret, irq, i;
power = kzalloc(sizeof(struct wm831x_power), GFP_KERNEL);
power->wm831x = wm831x;
platform_set_drvdata(pdev, power);
- usb = &power->usb;
- battery = &power->battery;
- wall = &power->wall;
-
if (wm831x_pdata && wm831x_pdata->wm831x_num) {
snprintf(power->wall_name, sizeof(power->wall_name),
"wm831x-wall.%d", wm831x_pdata->wm831x_num);
*/
wm831x_config_battery(wm831x);
- wall->name = power->wall_name;
- wall->type = POWER_SUPPLY_TYPE_MAINS;
- wall->properties = wm831x_wall_props;
- wall->num_properties = ARRAY_SIZE(wm831x_wall_props);
- wall->get_property = wm831x_wall_get_prop;
- ret = power_supply_register(&pdev->dev, wall);
- if (ret)
+ power->wall_desc.name = power->wall_name;
+ power->wall_desc.type = POWER_SUPPLY_TYPE_MAINS;
+ power->wall_desc.properties = wm831x_wall_props;
+ power->wall_desc.num_properties = ARRAY_SIZE(wm831x_wall_props);
+ power->wall_desc.get_property = wm831x_wall_get_prop;
+ power->wall = power_supply_register(&pdev->dev, &power->wall_desc,
+ NULL);
+ if (IS_ERR(power->wall)) {
+ ret = PTR_ERR(power->wall);
goto err_kmalloc;
+ }
- usb->name = power->usb_name,
- usb->type = POWER_SUPPLY_TYPE_USB;
- usb->properties = wm831x_usb_props;
- usb->num_properties = ARRAY_SIZE(wm831x_usb_props);
- usb->get_property = wm831x_usb_get_prop;
- ret = power_supply_register(&pdev->dev, usb);
- if (ret)
+ power->usb_desc.name = power->usb_name,
+ power->usb_desc.type = POWER_SUPPLY_TYPE_USB;
+ power->usb_desc.properties = wm831x_usb_props;
+ power->usb_desc.num_properties = ARRAY_SIZE(wm831x_usb_props);
+ power->usb_desc.get_property = wm831x_usb_get_prop;
+ power->usb = power_supply_register(&pdev->dev, &power->usb_desc, NULL);
+ if (IS_ERR(power->usb)) {
+ ret = PTR_ERR(power->usb);
goto err_wall;
+ }
ret = wm831x_reg_read(wm831x, WM831X_CHARGER_CONTROL_1);
if (ret < 0)
power->have_battery = ret & WM831X_CHG_ENA;
if (power->have_battery) {
- battery->name = power->battery_name;
- battery->properties = wm831x_bat_props;
- battery->num_properties = ARRAY_SIZE(wm831x_bat_props);
- battery->get_property = wm831x_bat_get_prop;
- battery->use_for_apm = 1;
- ret = power_supply_register(&pdev->dev, battery);
- if (ret)
- goto err_usb;
+ power->battery_desc.name = power->battery_name;
+ power->battery_desc.properties = wm831x_bat_props;
+ power->battery_desc.num_properties = ARRAY_SIZE(wm831x_bat_props);
+ power->battery_desc.get_property = wm831x_bat_get_prop;
+ power->battery_desc.use_for_apm = 1;
+ power->battery = power_supply_register(&pdev->dev,
+ &power->battery_desc,
+ NULL);
+ if (IS_ERR(power->battery)) {
+ ret = PTR_ERR(power->battery);
+ goto err_usb;
+ }
}
irq = wm831x_irq(wm831x, platform_get_irq_byname(pdev, "SYSLO"));
free_irq(irq, power);
err_battery:
if (power->have_battery)
- power_supply_unregister(battery);
+ power_supply_unregister(power->battery);
err_usb:
- power_supply_unregister(usb);
+ power_supply_unregister(power->usb);
err_wall:
- power_supply_unregister(wall);
+ power_supply_unregister(power->wall);
err_kmalloc:
kfree(power);
return ret;
free_irq(irq, wm831x_power);
if (wm831x_power->have_battery)
- power_supply_unregister(&wm831x_power->battery);
- power_supply_unregister(&wm831x_power->wall);
- power_supply_unregister(&wm831x_power->usb);
+ power_supply_unregister(wm831x_power->battery);
+ power_supply_unregister(wm831x_power->wall);
+ power_supply_unregister(wm831x_power->usb);
kfree(wm831x_power);
return 0;
}
break;
case WM8350_IRQ_CHG_TO:
dev_err(wm8350->dev, "charger timeout\n");
- power_supply_changed(&power->battery);
+ power_supply_changed(power->battery);
break;
case WM8350_IRQ_CHG_BAT_HOT:
case WM8350_IRQ_CHG_BAT_COLD:
case WM8350_IRQ_CHG_START:
case WM8350_IRQ_CHG_END:
- power_supply_changed(&power->battery);
+ power_supply_changed(power->battery);
break;
case WM8350_IRQ_CHG_FAST_RDY:
case WM8350_IRQ_EXT_WALL_FB:
wm8350_charger_config(wm8350, policy);
case WM8350_IRQ_EXT_BAT_FB: /* Fall through */
- power_supply_changed(&power->battery);
- power_supply_changed(&power->usb);
- power_supply_changed(&power->ac);
+ power_supply_changed(power->battery);
+ power_supply_changed(power->usb);
+ power_supply_changed(power->ac);
break;
default:
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm8350 *wm8350 = dev_get_drvdata(psy->dev->parent);
+ struct wm8350 *wm8350 = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm8350 *wm8350 = dev_get_drvdata(psy->dev->parent);
+ struct wm8350 *wm8350 = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm8350 *wm8350 = dev_get_drvdata(psy->dev->parent);
+ struct wm8350 *wm8350 = dev_get_drvdata(psy->dev.parent);
int ret = 0;
switch (psp) {
POWER_SUPPLY_PROP_CHARGE_TYPE,
};
+static const struct power_supply_desc wm8350_ac_desc = {
+ .name = "wm8350-ac",
+ .type = POWER_SUPPLY_TYPE_MAINS,
+ .properties = wm8350_ac_props,
+ .num_properties = ARRAY_SIZE(wm8350_ac_props),
+ .get_property = wm8350_ac_get_prop,
+};
+
+static const struct power_supply_desc wm8350_battery_desc = {
+ .name = "wm8350-battery",
+ .properties = wm8350_bat_props,
+ .num_properties = ARRAY_SIZE(wm8350_bat_props),
+ .get_property = wm8350_bat_get_property,
+ .use_for_apm = 1,
+};
+
+static const struct power_supply_desc wm8350_usb_desc = {
+ .name = "wm8350-usb",
+ .type = POWER_SUPPLY_TYPE_USB,
+ .properties = wm8350_usb_props,
+ .num_properties = ARRAY_SIZE(wm8350_usb_props),
+ .get_property = wm8350_usb_get_prop,
+};
+
/*********************************************************************
* Initialisation
*********************************************************************/
struct wm8350 *wm8350 = platform_get_drvdata(pdev);
struct wm8350_power *power = &wm8350->power;
struct wm8350_charger_policy *policy = power->policy;
- struct power_supply *usb = &power->usb;
- struct power_supply *battery = &power->battery;
- struct power_supply *ac = &power->ac;
int ret;
- ac->name = "wm8350-ac";
- ac->type = POWER_SUPPLY_TYPE_MAINS;
- ac->properties = wm8350_ac_props;
- ac->num_properties = ARRAY_SIZE(wm8350_ac_props);
- ac->get_property = wm8350_ac_get_prop;
- ret = power_supply_register(&pdev->dev, ac);
- if (ret)
- return ret;
-
- battery->name = "wm8350-battery";
- battery->properties = wm8350_bat_props;
- battery->num_properties = ARRAY_SIZE(wm8350_bat_props);
- battery->get_property = wm8350_bat_get_property;
- battery->use_for_apm = 1;
- ret = power_supply_register(&pdev->dev, battery);
- if (ret)
+ power->ac = power_supply_register(&pdev->dev, &wm8350_ac_desc, NULL);
+ if (IS_ERR(power->ac))
+ return PTR_ERR(power->ac);
+
+ power->battery = power_supply_register(&pdev->dev, &wm8350_battery_desc,
+ NULL);
+ if (IS_ERR(power->battery)) {
+ ret = PTR_ERR(power->battery);
goto battery_failed;
+ }
- usb->name = "wm8350-usb",
- usb->type = POWER_SUPPLY_TYPE_USB;
- usb->properties = wm8350_usb_props;
- usb->num_properties = ARRAY_SIZE(wm8350_usb_props);
- usb->get_property = wm8350_usb_get_prop;
- ret = power_supply_register(&pdev->dev, usb);
- if (ret)
+ power->usb = power_supply_register(&pdev->dev, &wm8350_usb_desc, NULL);
+ if (IS_ERR(power->usb)) {
+ ret = PTR_ERR(power->usb);
goto usb_failed;
+ }
ret = device_create_file(&pdev->dev, &dev_attr_charger_state);
if (ret < 0)
return ret;
usb_failed:
- power_supply_unregister(battery);
+ power_supply_unregister(power->battery);
battery_failed:
- power_supply_unregister(ac);
+ power_supply_unregister(power->ac);
return ret;
}
free_charger_irq(wm8350);
device_remove_file(&pdev->dev, &dev_attr_charger_state);
- power_supply_unregister(&power->battery);
- power_supply_unregister(&power->ac);
- power_supply_unregister(&power->usb);
+ power_supply_unregister(power->battery);
+ power_supply_unregister(power->ac);
+ power_supply_unregister(power->usb);
return 0;
}
static unsigned long wm97xx_read_bat(struct power_supply *bat_ps)
{
- struct wm97xx_pdata *wmdata = bat_ps->dev->parent->platform_data;
+ struct wm97xx_pdata *wmdata = bat_ps->dev.parent->platform_data;
struct wm97xx_batt_pdata *pdata = wmdata->batt_pdata;
- return wm97xx_read_aux_adc(dev_get_drvdata(bat_ps->dev->parent),
+ return wm97xx_read_aux_adc(dev_get_drvdata(bat_ps->dev.parent),
pdata->batt_aux) * pdata->batt_mult /
pdata->batt_div;
}
static unsigned long wm97xx_read_temp(struct power_supply *bat_ps)
{
- struct wm97xx_pdata *wmdata = bat_ps->dev->parent->platform_data;
+ struct wm97xx_pdata *wmdata = bat_ps->dev.parent->platform_data;
struct wm97xx_batt_pdata *pdata = wmdata->batt_pdata;
- return wm97xx_read_aux_adc(dev_get_drvdata(bat_ps->dev->parent),
+ return wm97xx_read_aux_adc(dev_get_drvdata(bat_ps->dev.parent),
pdata->temp_aux) * pdata->temp_mult /
pdata->temp_div;
}
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct wm97xx_pdata *wmdata = bat_ps->dev->parent->platform_data;
+ struct wm97xx_pdata *wmdata = bat_ps->dev.parent->platform_data;
struct wm97xx_batt_pdata *pdata = wmdata->batt_pdata;
switch (psp) {
static void wm97xx_bat_update(struct power_supply *bat_ps)
{
int old_status = bat_status;
- struct wm97xx_pdata *wmdata = bat_ps->dev->parent->platform_data;
+ struct wm97xx_pdata *wmdata = bat_ps->dev.parent->platform_data;
struct wm97xx_batt_pdata *pdata = wmdata->batt_pdata;
mutex_lock(&work_lock);
POWER_SUPPLY_STATUS_UNKNOWN;
if (old_status != bat_status) {
- pr_debug("%s: %i -> %i\n", bat_ps->name, old_status,
+ pr_debug("%s: %i -> %i\n", bat_ps->desc->name, old_status,
bat_status);
power_supply_changed(bat_ps);
}
mutex_unlock(&work_lock);
}
-static struct power_supply bat_ps = {
+static struct power_supply *bat_psy;
+static struct power_supply_desc bat_psy_desc = {
.type = POWER_SUPPLY_TYPE_BATTERY,
.get_property = wm97xx_bat_get_property,
.external_power_changed = wm97xx_bat_external_power_changed,
static void wm97xx_bat_work(struct work_struct *work)
{
- wm97xx_bat_update(&bat_ps);
+ wm97xx_bat_update(bat_psy);
}
static irqreturn_t wm97xx_chrg_irq(int irq, void *data)
dev_info(&dev->dev, "Please consider setting proper battery "
"name in platform definition file, falling "
"back to name \"wm97xx-batt\"\n");
- bat_ps.name = "wm97xx-batt";
+ bat_psy_desc.name = "wm97xx-batt";
} else
- bat_ps.name = pdata->batt_name;
+ bat_psy_desc.name = pdata->batt_name;
- bat_ps.properties = prop;
- bat_ps.num_properties = props;
+ bat_psy_desc.properties = prop;
+ bat_psy_desc.num_properties = props;
- ret = power_supply_register(&dev->dev, &bat_ps);
- if (!ret)
+ bat_psy = power_supply_register(&dev->dev, &bat_psy_desc, NULL);
+ if (!IS_ERR(bat_psy)) {
schedule_work(&bat_work);
- else
+ } else {
+ ret = PTR_ERR(bat_psy);
goto err4;
+ }
return 0;
err4:
gpio_free(pdata->charge_gpio);
}
cancel_work_sync(&bat_work);
- power_supply_unregister(&bat_ps);
+ power_supply_unregister(bat_psy);
kfree(prop);
return 0;
}
#define Z2_DEFAULT_NAME "Z2"
struct z2_charger {
- struct z2_battery_info *info;
- int bat_status;
- struct i2c_client *client;
- struct power_supply batt_ps;
- struct mutex work_lock;
- struct work_struct bat_work;
+ struct z2_battery_info *info;
+ int bat_status;
+ struct i2c_client *client;
+ struct power_supply *batt_ps;
+ struct power_supply_desc batt_ps_desc;
+ struct mutex work_lock;
+ struct work_struct bat_work;
};
static unsigned long z2_read_bat(struct z2_charger *charger)
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct z2_charger *charger = container_of(batt_ps, struct z2_charger,
- batt_ps);
+ struct z2_charger *charger = power_supply_get_drvdata(batt_ps);
struct z2_battery_info *info = charger->info;
switch (psp) {
static void z2_batt_ext_power_changed(struct power_supply *batt_ps)
{
- struct z2_charger *charger = container_of(batt_ps, struct z2_charger,
- batt_ps);
+ struct z2_charger *charger = power_supply_get_drvdata(batt_ps);
+
schedule_work(&charger->bat_work);
}
POWER_SUPPLY_STATUS_UNKNOWN;
if (old_status != charger->bat_status) {
- pr_debug("%s: %i -> %i\n", charger->batt_ps.name, old_status,
- charger->bat_status);
- power_supply_changed(&charger->batt_ps);
+ pr_debug("%s: %i -> %i\n", charger->batt_ps->desc->name,
+ old_status,
+ charger->bat_status);
+ power_supply_changed(charger->batt_ps);
}
mutex_unlock(&charger->work_lock);
"Please consider setting proper battery "
"name in platform definition file, falling "
"back to name \" Z2_DEFAULT_NAME \"\n");
- charger->batt_ps.name = Z2_DEFAULT_NAME;
+ charger->batt_ps_desc.name = Z2_DEFAULT_NAME;
} else
- charger->batt_ps.name = info->batt_name;
+ charger->batt_ps_desc.name = info->batt_name;
- charger->batt_ps.properties = prop;
- charger->batt_ps.num_properties = props;
- charger->batt_ps.type = POWER_SUPPLY_TYPE_BATTERY;
- charger->batt_ps.get_property = z2_batt_get_property;
- charger->batt_ps.external_power_changed = z2_batt_ext_power_changed;
- charger->batt_ps.use_for_apm = 1;
+ charger->batt_ps_desc.properties = prop;
+ charger->batt_ps_desc.num_properties = props;
+ charger->batt_ps_desc.type = POWER_SUPPLY_TYPE_BATTERY;
+ charger->batt_ps_desc.get_property = z2_batt_get_property;
+ charger->batt_ps_desc.external_power_changed =
+ z2_batt_ext_power_changed;
+ charger->batt_ps_desc.use_for_apm = 1;
return 0;
}
int props = 1; /* POWER_SUPPLY_PROP_PRESENT */
struct z2_charger *charger;
struct z2_battery_info *info = client->dev.platform_data;
+ struct power_supply_config psy_cfg = {};
if (info == NULL) {
dev_err(&client->dev,
charger->info = info;
charger->client = client;
i2c_set_clientdata(client, charger);
+ psy_cfg.drv_data = charger;
mutex_init(&charger->work_lock);
INIT_WORK(&charger->bat_work, z2_batt_work);
- ret = power_supply_register(&client->dev, &charger->batt_ps);
- if (ret)
+ charger->batt_ps = power_supply_register(&client->dev,
+ &charger->batt_ps_desc,
+ &psy_cfg);
+ if (IS_ERR(charger->batt_ps)) {
+ ret = PTR_ERR(charger->batt_ps);
goto err4;
+ }
schedule_work(&charger->bat_work);
return 0;
err4:
- kfree(charger->batt_ps.properties);
+ kfree(charger->batt_ps_desc.properties);
err3:
if (info->charge_gpio >= 0 && gpio_is_valid(info->charge_gpio))
free_irq(gpio_to_irq(info->charge_gpio), charger);
struct z2_battery_info *info = charger->info;
cancel_work_sync(&charger->bat_work);
- power_supply_unregister(&charger->batt_ps);
+ power_supply_unregister(charger->batt_ps);
- kfree(charger->batt_ps.properties);
+ kfree(charger->batt_ps_desc.properties);
if (info->charge_gpio >= 0 && gpio_is_valid(info->charge_gpio)) {
free_irq(gpio_to_irq(info->charge_gpio), charger);
gpio_free(info->charge_gpio);
#define TIME_WINDOW_MAX_MSEC 40000
#define TIME_WINDOW_MIN_MSEC 250
-
+#define ENERGY_UNIT_SCALE 1000 /* scale from driver unit to powercap unit */
enum unit_type {
ARBITRARY_UNIT, /* no translation */
POWER_UNIT,
struct rapl_power_limit rpl[NR_POWER_LIMITS];
u64 attr_map; /* track capabilities */
unsigned int state;
+ unsigned int domain_energy_unit;
int package_id;
};
#define power_zone_to_rapl_domain(_zone) \
void (*set_floor_freq)(struct rapl_domain *rd, bool mode);
u64 (*compute_time_window)(struct rapl_package *rp, u64 val,
bool to_raw);
+ unsigned int dram_domain_energy_unit;
};
static struct rapl_defaults *rapl_defaults;
static int rapl_write_data_raw(struct rapl_domain *rd,
enum rapl_primitives prim,
unsigned long long value);
-static u64 rapl_unit_xlate(int package, enum unit_type type, u64 value,
+static u64 rapl_unit_xlate(struct rapl_domain *rd, int package,
+ enum unit_type type, u64 value,
int to_raw);
static void package_power_limit_irq_save(int package_id);
static int get_max_energy_counter(struct powercap_zone *pcd_dev, u64 *energy)
{
- *energy = rapl_unit_xlate(0, ENERGY_UNIT, ENERGY_STATUS_MASK, 0);
+ struct rapl_domain *rd = power_zone_to_rapl_domain(pcd_dev);
+
+ *energy = rapl_unit_xlate(rd, 0, ENERGY_UNIT, ENERGY_STATUS_MASK, 0);
return 0;
}
rd->msrs[4] = MSR_DRAM_POWER_INFO;
rd->rpl[0].prim_id = PL1_ENABLE;
rd->rpl[0].name = pl1_name;
+ rd->domain_energy_unit =
+ rapl_defaults->dram_domain_energy_unit;
+ if (rd->domain_energy_unit)
+ pr_info("DRAM domain energy unit %dpj\n",
+ rd->domain_energy_unit);
break;
}
if (mask) {
}
}
-static u64 rapl_unit_xlate(int package, enum unit_type type, u64 value,
+static u64 rapl_unit_xlate(struct rapl_domain *rd, int package,
+ enum unit_type type, u64 value,
int to_raw)
{
u64 units = 1;
struct rapl_package *rp;
+ u64 scale = 1;
rp = find_package_by_id(package);
if (!rp)
units = rp->power_unit;
break;
case ENERGY_UNIT:
- units = rp->energy_unit;
+ scale = ENERGY_UNIT_SCALE;
+ /* per domain unit takes precedence */
+ if (rd && rd->domain_energy_unit)
+ units = rd->domain_energy_unit;
+ else
+ units = rp->energy_unit;
break;
case TIME_UNIT:
return rapl_defaults->compute_time_window(rp, value, to_raw);
};
if (to_raw)
- return div64_u64(value, units);
+ return div64_u64(value, units) * scale;
value *= units;
- return value;
+ return div64_u64(value, scale);
}
/* in the order of enum rapl_primitives */
final = value & rp->mask;
final = final >> rp->shift;
if (xlate)
- *data = rapl_unit_xlate(rd->package_id, rp->unit, final, 0);
+ *data = rapl_unit_xlate(rd, rd->package_id, rp->unit, final, 0);
else
*data = final;
"failed to read msr 0x%x on cpu %d\n", msr, cpu);
return -EIO;
}
- value = rapl_unit_xlate(rd->package_id, rp->unit, value, 1);
+ value = rapl_unit_xlate(rd, rd->package_id, rp->unit, value, 1);
msr_val &= ~rp->mask;
msr_val |= value << rp->shift;
if (wrmsrl_safe_on_cpu(cpu, msr, msr_val)) {
* calculate units differ on different CPUs.
* We convert the units to below format based on CPUs.
* i.e.
- * energy unit: microJoules : Represented in microJoules by default
+ * energy unit: picoJoules : Represented in picoJoules by default
* power unit : microWatts : Represented in milliWatts by default
* time unit : microseconds: Represented in seconds by default
*/
}
value = (msr_val & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
- rp->energy_unit = 1000000 / (1 << value);
+ rp->energy_unit = ENERGY_UNIT_SCALE * 1000000 / (1 << value);
value = (msr_val & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
rp->power_unit = 1000000 / (1 << value);
value = (msr_val & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
rp->time_unit = 1000000 / (1 << value);
- pr_debug("Core CPU package %d energy=%duJ, time=%dus, power=%duW\n",
+ pr_debug("Core CPU package %d energy=%dpJ, time=%dus, power=%duW\n",
rp->id, rp->energy_unit, rp->time_unit, rp->power_unit);
return 0;
return -ENODEV;
}
value = (msr_val & ENERGY_UNIT_MASK) >> ENERGY_UNIT_OFFSET;
- rp->energy_unit = 1 << value;
+ rp->energy_unit = ENERGY_UNIT_SCALE * 1 << value;
value = (msr_val & POWER_UNIT_MASK) >> POWER_UNIT_OFFSET;
rp->power_unit = (1 << value) * 1000;
value = (msr_val & TIME_UNIT_MASK) >> TIME_UNIT_OFFSET;
rp->time_unit = 1000000 / (1 << value);
- pr_debug("Atom package %d energy=%duJ, time=%dus, power=%duW\n",
+ pr_debug("Atom package %d energy=%dpJ, time=%dus, power=%duW\n",
rp->id, rp->energy_unit, rp->time_unit, rp->power_unit);
return 0;
.compute_time_window = rapl_compute_time_window_core,
};
+static const struct rapl_defaults rapl_defaults_hsw_server = {
+ .check_unit = rapl_check_unit_core,
+ .set_floor_freq = set_floor_freq_default,
+ .compute_time_window = rapl_compute_time_window_core,
+ .dram_domain_energy_unit = 15300,
+};
+
static const struct rapl_defaults rapl_defaults_atom = {
.check_unit = rapl_check_unit_atom,
.set_floor_freq = set_floor_freq_atom,
RAPL_CPU(0x3a, rapl_defaults_core),/* Ivy Bridge */
RAPL_CPU(0x3c, rapl_defaults_core),/* Haswell */
RAPL_CPU(0x3d, rapl_defaults_core),/* Broadwell */
- RAPL_CPU(0x3f, rapl_defaults_core),/* Haswell */
+ RAPL_CPU(0x3f, rapl_defaults_hsw_server),/* Haswell servers */
RAPL_CPU(0x45, rapl_defaults_core),/* Haswell ULT */
RAPL_CPU(0x4C, rapl_defaults_atom),/* Braswell */
RAPL_CPU(0x4A, rapl_defaults_atom),/* Tangier */
#include <linux/regulator/of_regulator.h>
#include <linux/regmap.h>
+/*
+ * ACT8600 Global Register Map.
+ */
+#define ACT8600_SYS_MODE 0x00
+#define ACT8600_SYS_CTRL 0x01
+#define ACT8600_DCDC1_VSET 0x10
+#define ACT8600_DCDC1_CTRL 0x12
+#define ACT8600_DCDC2_VSET 0x20
+#define ACT8600_DCDC2_CTRL 0x22
+#define ACT8600_DCDC3_VSET 0x30
+#define ACT8600_DCDC3_CTRL 0x32
+#define ACT8600_SUDCDC4_VSET 0x40
+#define ACT8600_SUDCDC4_CTRL 0x41
+#define ACT8600_LDO5_VSET 0x50
+#define ACT8600_LDO5_CTRL 0x51
+#define ACT8600_LDO6_VSET 0x60
+#define ACT8600_LDO6_CTRL 0x61
+#define ACT8600_LDO7_VSET 0x70
+#define ACT8600_LDO7_CTRL 0x71
+#define ACT8600_LDO8_VSET 0x80
+#define ACT8600_LDO8_CTRL 0x81
+#define ACT8600_LDO910_CTRL 0x91
+#define ACT8600_APCH0 0xA1
+#define ACT8600_APCH1 0xA8
+#define ACT8600_APCH2 0xA9
+#define ACT8600_APCH_STAT 0xAA
+#define ACT8600_OTG0 0xB0
+#define ACT8600_OTG1 0xB2
+
/*
* ACT8846 Global Register Map.
*/
#define ACT8865_ENA 0x80 /* ON - [7] */
#define ACT8865_VSEL_MASK 0x3F /* VSET - [5:0] */
+
+#define ACT8600_LDO10_ENA 0x40 /* ON - [6] */
+#define ACT8600_SUDCDC_VSEL_MASK 0xFF /* SUDCDC VSET - [7:0] */
+
/*
* ACT8865 voltage number
*/
#define ACT8865_VOLTAGE_NUM 64
+#define ACT8600_SUDCDC_VOLTAGE_NUM 255
struct act8865 {
struct regmap *regmap;
REGULATOR_LINEAR_RANGE(2400000, 48, 63, 100000),
};
+static const struct regulator_linear_range act8600_sudcdc_voltage_ranges[] = {
+ REGULATOR_LINEAR_RANGE(3000000, 0, 63, 0),
+ REGULATOR_LINEAR_RANGE(3000000, 64, 159, 100000),
+ REGULATOR_LINEAR_RANGE(12600000, 160, 191, 200000),
+ REGULATOR_LINEAR_RANGE(19000000, 191, 255, 400000),
+};
+
static struct regulator_ops act8865_ops = {
.list_voltage = regulator_list_voltage_linear_range,
.map_voltage = regulator_map_voltage_linear_range,
.is_enabled = regulator_is_enabled_regmap,
};
-#define ACT88xx_REG(_name, _family, _id, _vsel_reg) \
+static struct regulator_ops act8865_ldo_ops = {
+ .enable = regulator_enable_regmap,
+ .disable = regulator_disable_regmap,
+ .is_enabled = regulator_is_enabled_regmap,
+};
+
+#define ACT88xx_REG(_name, _family, _id, _vsel_reg, _supply) \
[_family##_ID_##_id] = { \
.name = _name, \
+ .supply_name = _supply, \
.id = _family##_ID_##_id, \
.type = REGULATOR_VOLTAGE, \
.ops = &act8865_ops, \
.owner = THIS_MODULE, \
}
+static const struct regulator_desc act8600_regulators[] = {
+ ACT88xx_REG("DCDC1", ACT8600, DCDC1, VSET, "vp1"),
+ ACT88xx_REG("DCDC2", ACT8600, DCDC2, VSET, "vp2"),
+ ACT88xx_REG("DCDC3", ACT8600, DCDC3, VSET, "vp3"),
+ {
+ .name = "SUDCDC_REG4",
+ .id = ACT8600_ID_SUDCDC4,
+ .ops = &act8865_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = ACT8600_SUDCDC_VOLTAGE_NUM,
+ .linear_ranges = act8600_sudcdc_voltage_ranges,
+ .n_linear_ranges = ARRAY_SIZE(act8600_sudcdc_voltage_ranges),
+ .vsel_reg = ACT8600_SUDCDC4_VSET,
+ .vsel_mask = ACT8600_SUDCDC_VSEL_MASK,
+ .enable_reg = ACT8600_SUDCDC4_CTRL,
+ .enable_mask = ACT8865_ENA,
+ .owner = THIS_MODULE,
+ },
+ ACT88xx_REG("LDO5", ACT8600, LDO5, VSET, "inl"),
+ ACT88xx_REG("LDO6", ACT8600, LDO6, VSET, "inl"),
+ ACT88xx_REG("LDO7", ACT8600, LDO7, VSET, "inl"),
+ ACT88xx_REG("LDO8", ACT8600, LDO8, VSET, "inl"),
+ {
+ .name = "LDO_REG9",
+ .id = ACT8600_ID_LDO9,
+ .ops = &act8865_ldo_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = 1,
+ .fixed_uV = 1800000,
+ .enable_reg = ACT8600_LDO910_CTRL,
+ .enable_mask = ACT8865_ENA,
+ .owner = THIS_MODULE,
+ },
+ {
+ .name = "LDO_REG10",
+ .id = ACT8600_ID_LDO10,
+ .ops = &act8865_ldo_ops,
+ .type = REGULATOR_VOLTAGE,
+ .n_voltages = 1,
+ .fixed_uV = 1200000,
+ .enable_reg = ACT8600_LDO910_CTRL,
+ .enable_mask = ACT8600_LDO10_ENA,
+ .owner = THIS_MODULE,
+ },
+};
+
static const struct regulator_desc act8846_regulators[] = {
- ACT88xx_REG("REG1", ACT8846, REG1, VSET),
- ACT88xx_REG("REG2", ACT8846, REG2, VSET0),
- ACT88xx_REG("REG3", ACT8846, REG3, VSET0),
- ACT88xx_REG("REG4", ACT8846, REG4, VSET0),
- ACT88xx_REG("REG5", ACT8846, REG5, VSET),
- ACT88xx_REG("REG6", ACT8846, REG6, VSET),
- ACT88xx_REG("REG7", ACT8846, REG7, VSET),
- ACT88xx_REG("REG8", ACT8846, REG8, VSET),
- ACT88xx_REG("REG9", ACT8846, REG9, VSET),
- ACT88xx_REG("REG10", ACT8846, REG10, VSET),
- ACT88xx_REG("REG11", ACT8846, REG11, VSET),
- ACT88xx_REG("REG12", ACT8846, REG12, VSET),
+ ACT88xx_REG("REG1", ACT8846, REG1, VSET, "vp1"),
+ ACT88xx_REG("REG2", ACT8846, REG2, VSET0, "vp2"),
+ ACT88xx_REG("REG3", ACT8846, REG3, VSET0, "vp3"),
+ ACT88xx_REG("REG4", ACT8846, REG4, VSET0, "vp4"),
+ ACT88xx_REG("REG5", ACT8846, REG5, VSET, "inl1"),
+ ACT88xx_REG("REG6", ACT8846, REG6, VSET, "inl1"),
+ ACT88xx_REG("REG7", ACT8846, REG7, VSET, "inl1"),
+ ACT88xx_REG("REG8", ACT8846, REG8, VSET, "inl2"),
+ ACT88xx_REG("REG9", ACT8846, REG9, VSET, "inl2"),
+ ACT88xx_REG("REG10", ACT8846, REG10, VSET, "inl3"),
+ ACT88xx_REG("REG11", ACT8846, REG11, VSET, "inl3"),
+ ACT88xx_REG("REG12", ACT8846, REG12, VSET, "inl3"),
};
static const struct regulator_desc act8865_regulators[] = {
- ACT88xx_REG("DCDC_REG1", ACT8865, DCDC1, VSET1),
- ACT88xx_REG("DCDC_REG2", ACT8865, DCDC2, VSET1),
- ACT88xx_REG("DCDC_REG3", ACT8865, DCDC3, VSET1),
- ACT88xx_REG("LDO_REG1", ACT8865, LDO1, VSET),
- ACT88xx_REG("LDO_REG2", ACT8865, LDO2, VSET),
- ACT88xx_REG("LDO_REG3", ACT8865, LDO3, VSET),
- ACT88xx_REG("LDO_REG4", ACT8865, LDO4, VSET),
+ ACT88xx_REG("DCDC_REG1", ACT8865, DCDC1, VSET1, "vp1"),
+ ACT88xx_REG("DCDC_REG2", ACT8865, DCDC2, VSET1, "vp2"),
+ ACT88xx_REG("DCDC_REG3", ACT8865, DCDC3, VSET1, "vp3"),
+ ACT88xx_REG("LDO_REG1", ACT8865, LDO1, VSET, "inl45"),
+ ACT88xx_REG("LDO_REG2", ACT8865, LDO2, VSET, "inl45"),
+ ACT88xx_REG("LDO_REG3", ACT8865, LDO3, VSET, "inl67"),
+ ACT88xx_REG("LDO_REG4", ACT8865, LDO4, VSET, "inl67"),
};
#ifdef CONFIG_OF
static const struct of_device_id act8865_dt_ids[] = {
+ { .compatible = "active-semi,act8600", .data = (void *)ACT8600 },
{ .compatible = "active-semi,act8846", .data = (void *)ACT8846 },
{ .compatible = "active-semi,act8865", .data = (void *)ACT8865 },
{ }
[ACT8865_ID_LDO4] = { .name = "LDO_REG4"},
};
+static struct of_regulator_match act8600_matches[] = {
+ [ACT8600_ID_DCDC1] = { .name = "DCDC_REG1"},
+ [ACT8600_ID_DCDC2] = { .name = "DCDC_REG2"},
+ [ACT8600_ID_DCDC3] = { .name = "DCDC_REG3"},
+ [ACT8600_ID_SUDCDC4] = { .name = "SUDCDC_REG4"},
+ [ACT8600_ID_LDO5] = { .name = "LDO_REG5"},
+ [ACT8600_ID_LDO6] = { .name = "LDO_REG6"},
+ [ACT8600_ID_LDO7] = { .name = "LDO_REG7"},
+ [ACT8600_ID_LDO8] = { .name = "LDO_REG8"},
+ [ACT8600_ID_LDO9] = { .name = "LDO_REG9"},
+ [ACT8600_ID_LDO10] = { .name = "LDO_REG10"},
+};
+
static int act8865_pdata_from_dt(struct device *dev,
struct device_node **of_node,
struct act8865_platform_data *pdata,
}
switch (type) {
+ case ACT8600:
+ matches = act8600_matches;
+ num_matches = ARRAY_SIZE(act8600_matches);
+ break;
case ACT8846:
matches = act8846_matches;
num_matches = ARRAY_SIZE(act8846_matches);
}
switch (type) {
+ case ACT8600:
+ regulators = act8600_regulators;
+ num_regulators = ARRAY_SIZE(act8600_regulators);
+ off_reg = -1;
+ off_mask = -1;
+ break;
case ACT8846:
regulators = act8846_regulators;
num_regulators = ARRAY_SIZE(act8846_regulators);
}
if (of_device_is_system_power_controller(dev->of_node)) {
- if (!pm_power_off) {
+ if (!pm_power_off && (off_reg > 0)) {
act8865_i2c_client = client;
act8865->off_reg = off_reg;
act8865->off_mask = off_mask;
}
static const struct i2c_device_id act8865_ids[] = {
+ { .name = "act8600", .driver_data = ACT8600 },
{ .name = "act8846", .driver_data = ACT8846 },
{ .name = "act8865", .driver_data = ACT8865 },
{ },
arizona->external_dcvdd = true;
ldo1->regulator = devm_regulator_register(&pdev->dev, desc, &config);
+
+ of_node_put(config.of_node);
+
if (IS_ERR(ldo1->regulator)) {
ret = PTR_ERR(ldo1->regulator);
dev_err(arizona->dev, "Failed to register LDO1 supply: %d\n",
return ret;
}
- of_node_put(config.of_node);
-
platform_set_drvdata(pdev, ldo1);
return 0;
micsupp->regulator = devm_regulator_register(&pdev->dev,
desc,
&config);
+
+ of_node_put(config.of_node);
+
if (IS_ERR(micsupp->regulator)) {
ret = PTR_ERR(micsupp->regulator);
dev_err(arizona->dev, "Failed to register mic supply: %d\n",
return ret;
}
- of_node_put(config.of_node);
-
platform_set_drvdata(pdev, micsupp);
return 0;
if (err < 0)
return 0;
- if (!rdev->desc->ops->get_optimum_mode)
+ if (!rdev->desc->ops->get_optimum_mode &&
+ !rdev->desc->ops->set_load)
return 0;
- if (!rdev->desc->ops->set_mode)
+ if (!rdev->desc->ops->set_mode &&
+ !rdev->desc->ops->set_load)
return -EINVAL;
/* get output voltage */
list_for_each_entry(sibling, &rdev->consumer_list, list)
current_uA += sibling->uA_load;
- /* now get the optimum mode for our new total regulator load */
- mode = rdev->desc->ops->get_optimum_mode(rdev, input_uV,
- output_uV, current_uA);
+ if (rdev->desc->ops->set_load) {
+ /* set the optimum mode for our new total regulator load */
+ err = rdev->desc->ops->set_load(rdev, current_uA);
+ if (err < 0)
+ rdev_err(rdev, "failed to set load %d\n", current_uA);
+ } else {
+ /* now get the optimum mode for our new total regulator load */
+ mode = rdev->desc->ops->get_optimum_mode(rdev, input_uV,
+ output_uV, current_uA);
+
+ /* check the new mode is allowed */
+ err = regulator_mode_constrain(rdev, &mode);
+ if (err < 0) {
+ rdev_err(rdev, "failed to get optimum mode @ %d uA %d -> %d uV\n",
+ current_uA, input_uV, output_uV);
+ return err;
+ }
- /* check the new mode is allowed */
- err = regulator_mode_constrain(rdev, &mode);
- if (err < 0) {
- rdev_err(rdev, "failed to get optimum mode @ %d uA %d -> %d uV\n",
- current_uA, input_uV, output_uV);
- return err;
+ err = rdev->desc->ops->set_mode(rdev, mode);
+ if (err < 0)
+ rdev_err(rdev, "failed to set optimum mode %x\n", mode);
}
- err = rdev->desc->ops->set_mode(rdev, mode);
- if (err < 0)
- rdev_err(rdev, "failed to set optimum mode %x\n", mode);
-
return err;
}
return NULL;
}
+static int regulator_resolve_supply(struct regulator_dev *rdev)
+{
+ struct regulator_dev *r;
+ struct device *dev = rdev->dev.parent;
+ int ret;
+
+ /* No supply to resovle? */
+ if (!rdev->supply_name)
+ return 0;
+
+ /* Supply already resolved? */
+ if (rdev->supply)
+ return 0;
+
+ r = regulator_dev_lookup(dev, rdev->supply_name, &ret);
+ if (ret == -ENODEV) {
+ /*
+ * No supply was specified for this regulator and
+ * there will never be one.
+ */
+ return 0;
+ }
+
+ if (!r) {
+ dev_err(dev, "Failed to resolve %s-supply for %s\n",
+ rdev->supply_name, rdev->desc->name);
+ return -EPROBE_DEFER;
+ }
+
+ /* Recursively resolve the supply of the supply */
+ ret = regulator_resolve_supply(r);
+ if (ret < 0)
+ return ret;
+
+ ret = set_supply(rdev, r);
+ if (ret < 0)
+ return ret;
+
+ /* Cascade always-on state to supply */
+ if (_regulator_is_enabled(rdev)) {
+ ret = regulator_enable(rdev->supply);
+ if (ret < 0)
+ return ret;
+ }
+
+ return 0;
+}
+
/* Internal regulator request function */
static struct regulator *_regulator_get(struct device *dev, const char *id,
bool exclusive, bool allow_dummy)
goto out;
}
+ ret = regulator_resolve_supply(rdev);
+ if (ret < 0) {
+ regulator = ERR_PTR(ret);
+ goto out;
+ }
+
if (!try_module_get(rdev->owner))
goto out;
}
if (rdev->ena_pin) {
- ret = regulator_ena_gpio_ctrl(rdev, true);
- if (ret < 0)
- return ret;
- rdev->ena_gpio_state = 1;
+ if (!rdev->ena_gpio_state) {
+ ret = regulator_ena_gpio_ctrl(rdev, true);
+ if (ret < 0)
+ return ret;
+ rdev->ena_gpio_state = 1;
+ }
} else if (rdev->desc->ops->enable) {
ret = rdev->desc->ops->enable(rdev);
if (ret < 0)
trace_regulator_disable(rdev_get_name(rdev));
if (rdev->ena_pin) {
- ret = regulator_ena_gpio_ctrl(rdev, false);
- if (ret < 0)
- return ret;
- rdev->ena_gpio_state = 0;
+ if (rdev->ena_gpio_state) {
+ ret = regulator_ena_gpio_ctrl(rdev, false);
+ if (ret < 0)
+ return ret;
+ rdev->ena_gpio_state = 0;
+ }
} else if (rdev->desc->ops->disable) {
ret = rdev->desc->ops->disable(rdev);
EXPORT_SYMBOL_GPL(regulator_get_mode);
/**
- * regulator_set_optimum_mode - set regulator optimum operating mode
+ * regulator_set_load - set regulator load
* @regulator: regulator source
* @uA_load: load current
*
* DRMS will sum the total requested load on the regulator and change
* to the most efficient operating mode if platform constraints allow.
*
- * Returns the new regulator mode or error.
+ * On error a negative errno is returned.
*/
-int regulator_set_optimum_mode(struct regulator *regulator, int uA_load)
+int regulator_set_load(struct regulator *regulator, int uA_load)
{
struct regulator_dev *rdev = regulator->rdev;
int ret;
return ret;
}
-EXPORT_SYMBOL_GPL(regulator_set_optimum_mode);
+EXPORT_SYMBOL_GPL(regulator_set_load);
/**
* regulator_allow_bypass - allow the regulator to go into bypass mode
if (attr == &dev_attr_requested_microamps.attr)
return rdev->desc->type == REGULATOR_CURRENT ? mode : 0;
- /* all the other attributes exist to support constraints;
- * don't show them if there are no constraints, or if the
- * relevant supporting methods are missing.
- */
- if (!rdev->constraints)
- return 0;
-
/* constraints need specific supporting methods */
if (attr == &dev_attr_min_microvolts.attr ||
attr == &dev_attr_max_microvolts.attr)
static void rdev_init_debugfs(struct regulator_dev *rdev)
{
- rdev->debugfs = debugfs_create_dir(rdev_get_name(rdev), debugfs_root);
+ struct device *parent = rdev->dev.parent;
+ const char *rname = rdev_get_name(rdev);
+ char name[NAME_MAX];
+
+ /* Avoid duplicate debugfs directory names */
+ if (parent && rname == rdev->desc->name) {
+ snprintf(name, sizeof(name), "%s-%s", dev_name(parent),
+ rname);
+ rname = name;
+ }
+
+ rdev->debugfs = debugfs_create_dir(rname, debugfs_root);
if (!rdev->debugfs) {
rdev_warn(rdev, "Failed to create debugfs directory\n");
return;
struct regulator_dev *rdev;
struct device *dev;
int ret, i;
- const char *supply = NULL;
if (regulator_desc == NULL || cfg == NULL)
return ERR_PTR(-EINVAL);
config->ena_gpio, ret);
goto wash;
}
-
- if (config->ena_gpio_flags & GPIOF_OUT_INIT_HIGH)
- rdev->ena_gpio_state = 1;
-
- if (config->ena_gpio_invert)
- rdev->ena_gpio_state = !rdev->ena_gpio_state;
}
/* set regulator constraints */
goto scrub;
if (init_data && init_data->supply_regulator)
- supply = init_data->supply_regulator;
+ rdev->supply_name = init_data->supply_regulator;
else if (regulator_desc->supply_name)
- supply = regulator_desc->supply_name;
-
- if (supply) {
- struct regulator_dev *r;
-
- r = regulator_dev_lookup(dev, supply, &ret);
-
- if (ret == -ENODEV) {
- /*
- * No supply was specified for this regulator and
- * there will never be one.
- */
- ret = 0;
- goto add_dev;
- } else if (!r) {
- dev_err(dev, "Failed to find supply %s\n", supply);
- ret = -EPROBE_DEFER;
- goto scrub;
- }
-
- ret = set_supply(rdev, r);
- if (ret < 0)
- goto scrub;
-
- /* Enable supply if rail is enabled */
- if (_regulator_is_enabled(rdev)) {
- ret = regulator_enable(rdev->supply);
- if (ret < 0)
- goto scrub;
- }
- }
+ rdev->supply_name = regulator_desc->supply_name;
-add_dev:
/* add consumers devices */
if (init_data) {
for (i = 0; i < init_data->num_consumer_supplies; i++) {
unset_regulator_supplies(rdev);
scrub:
- if (rdev->supply)
- _regulator_put(rdev->supply);
regulator_ena_gpio_free(rdev);
kfree(rdev->constraints);
wash:
list_for_each_entry(rdev, ®ulator_list, list) {
mutex_lock(&rdev->mutex);
if (rdev->use_count > 0 || rdev->constraints->always_on) {
- error = _regulator_do_enable(rdev);
- if (error)
- ret = error;
+ if (!_regulator_is_enabled(rdev)) {
+ error = _regulator_do_enable(rdev);
+ if (error)
+ ret = error;
+ }
} else {
if (!have_full_constraints())
goto unlock;
#endif
};
+#ifdef CONFIG_DEBUG_FS
+static void regulator_summary_show_subtree(struct seq_file *s,
+ struct regulator_dev *rdev,
+ int level)
+{
+ struct list_head *list = s->private;
+ struct regulator_dev *child;
+ struct regulation_constraints *c;
+ struct regulator *consumer;
+
+ if (!rdev)
+ return;
+
+ seq_printf(s, "%*s%-*s %3d %4d %6d ",
+ level * 3 + 1, "",
+ 30 - level * 3, rdev_get_name(rdev),
+ rdev->use_count, rdev->open_count, rdev->bypass_count);
+
+ seq_printf(s, "%5dmV ", _regulator_get_voltage(rdev) / 1000);
+ seq_printf(s, "%5dmA ", _regulator_get_current_limit(rdev) / 1000);
+
+ c = rdev->constraints;
+ if (c) {
+ switch (rdev->desc->type) {
+ case REGULATOR_VOLTAGE:
+ seq_printf(s, "%5dmV %5dmV ",
+ c->min_uV / 1000, c->max_uV / 1000);
+ break;
+ case REGULATOR_CURRENT:
+ seq_printf(s, "%5dmA %5dmA ",
+ c->min_uA / 1000, c->max_uA / 1000);
+ break;
+ }
+ }
+
+ seq_puts(s, "\n");
+
+ list_for_each_entry(consumer, &rdev->consumer_list, list) {
+ if (consumer->dev->class == ®ulator_class)
+ continue;
+
+ seq_printf(s, "%*s%-*s ",
+ (level + 1) * 3 + 1, "",
+ 30 - (level + 1) * 3, dev_name(consumer->dev));
+
+ switch (rdev->desc->type) {
+ case REGULATOR_VOLTAGE:
+ seq_printf(s, "%37dmV %5dmV",
+ consumer->min_uV / 1000,
+ consumer->max_uV / 1000);
+ break;
+ case REGULATOR_CURRENT:
+ break;
+ }
+
+ seq_puts(s, "\n");
+ }
+
+ list_for_each_entry(child, list, list) {
+ /* handle only non-root regulators supplied by current rdev */
+ if (!child->supply || child->supply->rdev != rdev)
+ continue;
+
+ regulator_summary_show_subtree(s, child, level + 1);
+ }
+}
+
+static int regulator_summary_show(struct seq_file *s, void *data)
+{
+ struct list_head *list = s->private;
+ struct regulator_dev *rdev;
+
+ seq_puts(s, " regulator use open bypass voltage current min max\n");
+ seq_puts(s, "-------------------------------------------------------------------------------\n");
+
+ mutex_lock(®ulator_list_mutex);
+
+ list_for_each_entry(rdev, list, list) {
+ if (rdev->supply)
+ continue;
+
+ regulator_summary_show_subtree(s, rdev, 0);
+ }
+
+ mutex_unlock(®ulator_list_mutex);
+
+ return 0;
+}
+
+static int regulator_summary_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, regulator_summary_show, inode->i_private);
+}
+#endif
+
+static const struct file_operations regulator_summary_fops = {
+#ifdef CONFIG_DEBUG_FS
+ .open = regulator_summary_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release = single_release,
+#endif
+};
+
static int __init regulator_init(void)
{
int ret;
debugfs_create_file("supply_map", 0444, debugfs_root, NULL,
&supply_map_fops);
+ debugfs_create_file("regulator_summary", 0444, debugfs_root,
+ ®ulator_list, ®ulator_summary_fops);
+
regulator_dummy_init();
return ret;
config.regmap = chip->regmap;
config.of_node = dev->of_node;
+ /* Mask all interrupt sources to deassert interrupt line */
+ error = regmap_write(chip->regmap, DA9210_REG_MASK_A, ~0);
+ if (!error)
+ error = regmap_write(chip->regmap, DA9210_REG_MASK_B, ~0);
+ if (error) {
+ dev_err(&i2c->dev, "Failed to write to mask reg: %d\n", error);
+ return error;
+ }
+
rdev = devm_regulator_register(&i2c->dev, &da9210_reg, &config);
if (IS_ERR(rdev)) {
dev_err(&i2c->dev, "Failed to register DA9210 regulator\n");
if (reg_val & DA9211_E_OV_CURR_A) {
regulator_notifier_call_chain(chip->rdev[0],
- REGULATOR_EVENT_OVER_CURRENT,
- rdev_get_drvdata(chip->rdev[0]));
+ REGULATOR_EVENT_OVER_CURRENT, NULL);
err = regmap_write(chip->regmap, DA9211_REG_EVENT_B,
DA9211_E_OV_CURR_A);
if (reg_val & DA9211_E_OV_CURR_B) {
regulator_notifier_call_chain(chip->rdev[1],
- REGULATOR_EVENT_OVER_CURRENT,
- rdev_get_drvdata(chip->rdev[1]));
+ REGULATOR_EVENT_OVER_CURRENT, NULL);
err = regmap_write(chip->regmap, DA9211_REG_EVENT_B,
DA9211_E_OV_CURR_B);
ret = regmap_read(chip->regmap, DA9211_REG_CONFIG_E, &data);
if (ret < 0) {
- dev_err(chip->dev, "Failed to read CONTROL_E reg: %d\n", ret);
+ dev_err(chip->dev, "Failed to read CONFIG_E reg: %d\n", ret);
return ret;
}
static int ux500_regulator_power_state_cnt_print(struct seq_file *s, void *p)
{
- struct device *dev = s->private;
- int err;
-
/* print power state count */
- err = seq_printf(s, "ux500-regulator power state count: %i\n",
- power_state_active_get());
- if (err < 0)
- dev_err(dev, "seq_printf overflow\n");
+ seq_printf(s, "ux500-regulator power state count: %i\n",
+ power_state_active_get());
return 0;
}
static int ux500_regulator_status_print(struct seq_file *s, void *p)
{
- struct device *dev = s->private;
- int err;
int i;
/* print dump header */
- err = seq_puts(s, "ux500-regulator status:\n");
- if (err < 0)
- dev_err(dev, "seq_puts overflow\n");
-
- err = seq_printf(s, "%31s : %8s : %8s\n", "current",
- "before", "after");
- if (err < 0)
- dev_err(dev, "seq_printf overflow\n");
+ seq_puts(s, "ux500-regulator status:\n");
+ seq_printf(s, "%31s : %8s : %8s\n", "current", "before", "after");
for (i = 0; i < rdebug.num_regulators; i++) {
struct dbx500_regulator_info *info;
info = &rdebug.regulator_array[i];
/* print status */
- err = seq_printf(s, "%20s : %8s : %8s : %8s\n", info->desc.name,
- info->is_enabled ? "enabled" : "disabled",
- rdebug.state_before_suspend[i] ? "enabled" : "disabled",
- rdebug.state_after_suspend[i] ? "enabled" : "disabled");
- if (err < 0)
- dev_err(dev, "seq_printf overflow\n");
+ seq_printf(s, "%20s : %8s : %8s : %8s\n",
+ info->desc.name,
+ info->is_enabled ? "enabled" : "disabled",
+ rdebug.state_before_suspend[i] ? "enabled" : "disabled",
+ rdebug.state_after_suspend[i] ? "enabled" : "disabled");
}
return 0;
devm_regulator_unregister_supply_alias(dev, id[i]);
}
EXPORT_SYMBOL_GPL(devm_regulator_bulk_unregister_supply_alias);
+
+struct regulator_notifier_match {
+ struct regulator *regulator;
+ struct notifier_block *nb;
+};
+
+static int devm_regulator_match_notifier(struct device *dev, void *res,
+ void *data)
+{
+ struct regulator_notifier_match *match = res;
+ struct regulator_notifier_match *target = data;
+
+ return match->regulator == target->regulator && match->nb == target->nb;
+}
+
+static void devm_regulator_destroy_notifier(struct device *dev, void *res)
+{
+ struct regulator_notifier_match *match = res;
+
+ regulator_unregister_notifier(match->regulator, match->nb);
+}
+
+/**
+ * devm_regulator_register_notifier - Resource managed
+ * regulator_register_notifier
+ *
+ * @regulator: regulator source
+ * @nb: notifier block
+ *
+ * The notifier will be registers under the consumer device and be
+ * automatically be unregistered when the source device is unbound.
+ */
+int devm_regulator_register_notifier(struct regulator *regulator,
+ struct notifier_block *nb)
+{
+ struct regulator_notifier_match *match;
+ int ret;
+
+ match = devres_alloc(devm_regulator_destroy_notifier,
+ sizeof(struct regulator_notifier_match),
+ GFP_KERNEL);
+ if (!match)
+ return -ENOMEM;
+
+ match->regulator = regulator;
+ match->nb = nb;
+
+ ret = regulator_register_notifier(regulator, nb);
+ if (ret < 0) {
+ devres_free(match);
+ return ret;
+ }
+
+ devres_add(regulator->dev, match);
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(devm_regulator_register_notifier);
+
+/**
+ * devm_regulator_unregister_notifier - Resource managed
+ * regulator_unregister_notifier()
+ *
+ * @regulator: regulator source
+ * @nb: notifier block
+ *
+ * Unregister a notifier registered with devm_regulator_register_notifier().
+ * Normally this function will not need to be called and the resource
+ * management code will ensure that the resource is freed.
+ */
+void devm_regulator_unregister_notifier(struct regulator *regulator,
+ struct notifier_block *nb)
+{
+ struct regulator_notifier_match match;
+ int rc;
+
+ match.regulator = regulator;
+ match.nb = nb;
+
+ rc = devres_release(regulator->dev, devm_regulator_destroy_notifier,
+ devm_regulator_match_notifier, &match);
+ if (rc != 0)
+ WARN_ON(rc);
+}
+EXPORT_SYMBOL_GPL(devm_regulator_unregister_notifier);
#define regulator_desc_esafeout(_num) { \
.name = "ESAFEOUT"#_num, \
.id = MAX77693_ESAFEOUT##_num, \
+ .of_match = of_match_ptr("ESAFEOUT"#_num), \
+ .regulators_node = of_match_ptr("regulators"), \
.n_voltages = 4, \
.ops = &max77693_safeout_ops, \
.type = REGULATOR_VOLTAGE, \
{
.name = "CHARGER",
.id = MAX77693_CHARGER,
+ .of_match = of_match_ptr("CHARGER"),
+ .regulators_node = of_match_ptr("regulators"),
.ops = &max77693_charger_ops,
.type = REGULATOR_CURRENT,
.owner = THIS_MODULE,
},
};
-#ifdef CONFIG_OF
-static int max77693_pmic_dt_parse_rdata(struct device *dev,
- struct max77693_regulator_data **rdata)
-{
- struct device_node *np;
- struct of_regulator_match *rmatch;
- struct max77693_regulator_data *tmp;
- int i, matched = 0;
-
- np = of_get_child_by_name(dev->parent->of_node, "regulators");
- if (!np)
- return -EINVAL;
-
- rmatch = devm_kzalloc(dev,
- sizeof(*rmatch) * ARRAY_SIZE(regulators), GFP_KERNEL);
- if (!rmatch) {
- of_node_put(np);
- return -ENOMEM;
- }
-
- for (i = 0; i < ARRAY_SIZE(regulators); i++)
- rmatch[i].name = regulators[i].name;
-
- matched = of_regulator_match(dev, np, rmatch, ARRAY_SIZE(regulators));
- of_node_put(np);
- if (matched <= 0)
- return matched;
- *rdata = devm_kzalloc(dev, sizeof(**rdata) * matched, GFP_KERNEL);
- if (!(*rdata))
- return -ENOMEM;
-
- tmp = *rdata;
-
- for (i = 0; i < matched; i++) {
- tmp->initdata = rmatch[i].init_data;
- tmp->of_node = rmatch[i].of_node;
- tmp->id = regulators[i].id;
- tmp++;
- }
-
- return matched;
-}
-#else
-static int max77693_pmic_dt_parse_rdata(struct device *dev,
- struct max77693_regulator_data **rdata)
-{
- return 0;
-}
-#endif /* CONFIG_OF */
-
-static int max77693_pmic_init_rdata(struct device *dev,
- struct max77693_regulator_data **rdata)
-{
- struct max77693_platform_data *pdata;
- int num_regulators = 0;
-
- pdata = dev_get_platdata(dev->parent);
- if (pdata) {
- *rdata = pdata->regulators;
- num_regulators = pdata->num_regulators;
- }
-
- if (!(*rdata) && dev->parent->of_node)
- num_regulators = max77693_pmic_dt_parse_rdata(dev, rdata);
-
- return num_regulators;
-}
-
static int max77693_pmic_probe(struct platform_device *pdev)
{
struct max77693_dev *iodev = dev_get_drvdata(pdev->dev.parent);
- struct max77693_regulator_data *rdata = NULL;
- int num_rdata, i;
+ int i;
struct regulator_config config = { };
- num_rdata = max77693_pmic_init_rdata(&pdev->dev, &rdata);
- if (!rdata || num_rdata <= 0) {
- dev_err(&pdev->dev, "No init data supplied.\n");
- return -ENODEV;
- }
-
- config.dev = &pdev->dev;
+ config.dev = iodev->dev;
config.regmap = iodev->regmap;
- for (i = 0; i < num_rdata; i++) {
- int id = rdata[i].id;
+ for (i = 0; i < ARRAY_SIZE(regulators); i++) {
struct regulator_dev *rdev;
- config.init_data = rdata[i].initdata;
- config.of_node = rdata[i].of_node;
-
rdev = devm_regulator_register(&pdev->dev,
- ®ulators[id], &config);
+ ®ulators[i], &config);
if (IS_ERR(rdev)) {
dev_err(&pdev->dev,
- "Failed to initialize regulator-%d\n", id);
+ "Failed to initialize regulator-%d\n", i);
return PTR_ERR(rdev);
}
}
const struct i2c_device_id *i2c_id)
{
struct device *dev = &client->dev;
- struct max8660_platform_data *pdata = dev_get_platdata(dev);
+ struct max8660_platform_data pdata_of, *pdata = dev_get_platdata(dev);
struct regulator_config config = { };
struct max8660 *max8660;
int boot_on, i, id, ret = -EINVAL;
if (dev->of_node && !pdata) {
const struct of_device_id *id;
- struct max8660_platform_data pdata_of;
id = of_match_device(of_match_ptr(max8660_dt_ids), dev);
if (!id)
for (i = 0; i < pdata->num_subdevs; i++) {
if (!pdata->subdevs[i].platform_data)
- return ret;
-
- boot_on = pdata->subdevs[i].platform_data->constraints.boot_on;
+ boot_on = false;
+ else
+ boot_on = pdata->subdevs[i].platform_data->constraints.boot_on;
switch (pdata->subdevs[i].id) {
case MAX8660_V3:
(id == PALMAS_REG_LDO6))
desc->enable_time = 2000;
} else {
+ if (!ddata->has_regen3 && id == PALMAS_REG_REGEN3)
+ continue;
+
desc->n_voltages = 1;
if (reg_init && reg_init->roof_floor)
desc->ops = &palmas_ops_ext_control_extreg;
.ldo_begin = PALMAS_REG_LDO1,
.ldo_end = PALMAS_REG_LDOUSB,
.max_reg = PALMAS_NUM_REGS,
+ .has_regen3 = true,
.palmas_regs_info = palmas_generic_regs_info,
.palmas_matches = palmas_matches,
.sleep_req_info = palma_sleep_req_info,
.ldo_begin = TPS65917_REG_LDO1,
.ldo_end = TPS65917_REG_LDO5,
.max_reg = TPS65917_NUM_REGS,
+ .has_regen3 = true,
.palmas_regs_info = tps65917_regs_info,
.palmas_matches = tps65917_matches,
.sleep_req_info = tps65917_sleep_req_info,
pdata->ldo6_vibrator = of_property_read_bool(node, "ti,ldo6-vibrator");
}
-static struct of_device_id of_palmas_match_tbl[] = {
+static const struct of_device_id of_palmas_match_tbl[] = {
{
.compatible = "ti,palmas-pmic",
.data = &palmas_ddata,
if (!pmic)
return -ENOMEM;
+ if (of_device_is_compatible(node, "ti,tps659038-pmic")) {
+ palmas_generic_regs_info[PALMAS_REG_REGEN2].ctrl_addr =
+ TPS659038_REGEN2_CTRL;
+ palmas_ddata.has_regen3 = false;
+ }
+
pmic->dev = &pdev->dev;
pmic->palmas = palmas;
palmas->pmic = pmic;
return vreg->is_enabled;
}
+static int rpm_reg_set_load(struct regulator_dev *rdev, int load_uA)
+{
+ struct qcom_rpm_reg *vreg = rdev_get_drvdata(rdev);
+ const struct rpm_reg_parts *parts = vreg->parts;
+ const struct request_member *req = &parts->ia;
+ int load_mA = load_uA / 1000;
+ int max_mA = req->mask >> req->shift;
+ int ret;
+
+ if (req->mask == 0)
+ return -EINVAL;
+
+ if (load_mA > max_mA)
+ load_mA = max_mA;
+
+ mutex_lock(&vreg->lock);
+ ret = rpm_reg_write(vreg, req, load_mA);
+ mutex_unlock(&vreg->lock);
+
+ return ret;
+}
+
static struct regulator_ops uV_ops = {
.list_voltage = regulator_list_voltage_linear_range,
.enable = rpm_reg_uV_enable,
.disable = rpm_reg_uV_disable,
.is_enabled = rpm_reg_is_enabled,
+
+ .set_load = rpm_reg_set_load,
};
static struct regulator_ops mV_ops = {
.enable = rpm_reg_mV_enable,
.disable = rpm_reg_mV_disable,
.is_enabled = rpm_reg_is_enabled,
+
+ .set_load = rpm_reg_set_load,
};
static struct regulator_ops switch_ops = {
.supports_force_mode_bypass = false,
};
-static const struct of_device_id rpm_of_match[] = {
- { .compatible = "qcom,rpm-pm8058-pldo", .data = &pm8058_pldo },
- { .compatible = "qcom,rpm-pm8058-nldo", .data = &pm8058_nldo },
- { .compatible = "qcom,rpm-pm8058-smps", .data = &pm8058_smps },
- { .compatible = "qcom,rpm-pm8058-ncp", .data = &pm8058_ncp },
- { .compatible = "qcom,rpm-pm8058-switch", .data = &pm8058_switch },
-
- { .compatible = "qcom,rpm-pm8901-pldo", .data = &pm8901_pldo },
- { .compatible = "qcom,rpm-pm8901-nldo", .data = &pm8901_nldo },
- { .compatible = "qcom,rpm-pm8901-ftsmps", .data = &pm8901_ftsmps },
- { .compatible = "qcom,rpm-pm8901-switch", .data = &pm8901_switch },
-
- { .compatible = "qcom,rpm-pm8921-pldo", .data = &pm8921_pldo },
- { .compatible = "qcom,rpm-pm8921-nldo", .data = &pm8921_nldo },
- { .compatible = "qcom,rpm-pm8921-nldo1200", .data = &pm8921_nldo1200 },
- { .compatible = "qcom,rpm-pm8921-smps", .data = &pm8921_smps },
- { .compatible = "qcom,rpm-pm8921-ftsmps", .data = &pm8921_ftsmps },
- { .compatible = "qcom,rpm-pm8921-ncp", .data = &pm8921_ncp },
- { .compatible = "qcom,rpm-pm8921-switch", .data = &pm8921_switch },
-
- { .compatible = "qcom,rpm-smb208", .data = &smb208_smps },
- { }
-};
-MODULE_DEVICE_TABLE(of, rpm_of_match);
-
static int rpm_reg_set(struct qcom_rpm_reg *vreg,
const struct request_member *req,
const int value)
return 0;
}
-static int rpm_reg_of_parse_freq(struct device *dev, struct qcom_rpm_reg *vreg)
+static int rpm_reg_of_parse_freq(struct device *dev,
+ struct device_node *node,
+ struct qcom_rpm_reg *vreg)
{
static const int freq_table[] = {
19200000, 9600000, 6400000, 4800000, 3840000, 3200000, 2740000,
int i;
key = "qcom,switch-mode-frequency";
- ret = of_property_read_u32(dev->of_node, key, &freq);
+ ret = of_property_read_u32(node, key, &freq);
if (ret) {
dev_err(dev, "regulator requires %s property\n", key);
return -EINVAL;
return -EINVAL;
}
-static int rpm_reg_probe(struct platform_device *pdev)
+static int rpm_reg_of_parse(struct device_node *node,
+ const struct regulator_desc *desc,
+ struct regulator_config *config)
{
- struct regulator_init_data *initdata;
- const struct qcom_rpm_reg *template;
- const struct of_device_id *match;
- struct regulator_config config = { };
- struct regulator_dev *rdev;
- struct qcom_rpm_reg *vreg;
+ struct qcom_rpm_reg *vreg = config->driver_data;
+ struct device *dev = config->dev;
const char *key;
u32 force_mode;
bool pwm;
u32 val;
int ret;
- match = of_match_device(rpm_of_match, &pdev->dev);
- template = match->data;
-
- vreg = devm_kmalloc(&pdev->dev, sizeof(*vreg), GFP_KERNEL);
- if (!vreg) {
- dev_err(&pdev->dev, "failed to allocate vreg\n");
- return -ENOMEM;
- }
- memcpy(vreg, template, sizeof(*vreg));
- mutex_init(&vreg->lock);
- vreg->dev = &pdev->dev;
- vreg->desc.id = -1;
- vreg->desc.owner = THIS_MODULE;
- vreg->desc.type = REGULATOR_VOLTAGE;
- vreg->desc.name = pdev->dev.of_node->name;
- vreg->desc.supply_name = "vin";
-
- vreg->rpm = dev_get_drvdata(pdev->dev.parent);
- if (!vreg->rpm) {
- dev_err(&pdev->dev, "unable to retrieve handle to rpm\n");
- return -ENODEV;
- }
-
- initdata = of_get_regulator_init_data(&pdev->dev, pdev->dev.of_node,
- &vreg->desc);
- if (!initdata)
- return -EINVAL;
-
- key = "reg";
- ret = of_property_read_u32(pdev->dev.of_node, key, &val);
- if (ret) {
- dev_err(&pdev->dev, "failed to read %s\n", key);
- return ret;
- }
- vreg->resource = val;
-
- if ((vreg->parts->uV.mask || vreg->parts->mV.mask) &&
- (!initdata->constraints.min_uV || !initdata->constraints.max_uV)) {
- dev_err(&pdev->dev, "no voltage specified for regulator\n");
- return -EINVAL;
- }
-
key = "bias-pull-down";
- if (of_property_read_bool(pdev->dev.of_node, key)) {
+ if (of_property_read_bool(node, key)) {
ret = rpm_reg_set(vreg, &vreg->parts->pd, 1);
if (ret) {
- dev_err(&pdev->dev, "%s is invalid", key);
+ dev_err(dev, "%s is invalid", key);
return ret;
}
}
if (vreg->parts->freq.mask) {
- ret = rpm_reg_of_parse_freq(&pdev->dev, vreg);
+ ret = rpm_reg_of_parse_freq(dev, node, vreg);
if (ret < 0)
return ret;
}
if (vreg->parts->pm.mask) {
key = "qcom,power-mode-hysteretic";
- pwm = !of_property_read_bool(pdev->dev.of_node, key);
+ pwm = !of_property_read_bool(node, key);
ret = rpm_reg_set(vreg, &vreg->parts->pm, pwm);
if (ret) {
- dev_err(&pdev->dev, "failed to set power mode\n");
+ dev_err(dev, "failed to set power mode\n");
return ret;
}
}
force_mode = -1;
key = "qcom,force-mode";
- ret = of_property_read_u32(pdev->dev.of_node, key, &val);
+ ret = of_property_read_u32(node, key, &val);
if (ret == -EINVAL) {
val = QCOM_RPM_FORCE_MODE_NONE;
} else if (ret < 0) {
- dev_err(&pdev->dev, "failed to read %s\n", key);
+ dev_err(dev, "failed to read %s\n", key);
return ret;
}
}
if (force_mode == -1) {
- dev_err(&pdev->dev, "invalid force mode\n");
+ dev_err(dev, "invalid force mode\n");
return -EINVAL;
}
ret = rpm_reg_set(vreg, &vreg->parts->fm, force_mode);
if (ret) {
- dev_err(&pdev->dev, "failed to set force mode\n");
+ dev_err(dev, "failed to set force mode\n");
return ret;
}
}
- config.dev = &pdev->dev;
- config.init_data = initdata;
- config.driver_data = vreg;
- config.of_node = pdev->dev.of_node;
- rdev = devm_regulator_register(&pdev->dev, &vreg->desc, &config);
- if (IS_ERR(rdev)) {
- dev_err(&pdev->dev, "can't register regulator\n");
- return PTR_ERR(rdev);
+ return 0;
+}
+
+struct rpm_regulator_data {
+ const char *name;
+ int resource;
+ const struct qcom_rpm_reg *template;
+ const char *supply;
+};
+
+static const struct rpm_regulator_data rpm_pm8058_regulators[] = {
+ { "l0", QCOM_RPM_PM8058_LDO0, &pm8058_nldo, "vdd_l0_l1_lvs" },
+ { "l1", QCOM_RPM_PM8058_LDO1, &pm8058_nldo, "vdd_l0_l1_lvs" },
+ { "l2", QCOM_RPM_PM8058_LDO2, &pm8058_pldo, "vdd_l2_l11_l12" },
+ { "l3", QCOM_RPM_PM8058_LDO3, &pm8058_pldo, "vdd_l3_l4_l5" },
+ { "l4", QCOM_RPM_PM8058_LDO4, &pm8058_pldo, "vdd_l3_l4_l5" },
+ { "l5", QCOM_RPM_PM8058_LDO5, &pm8058_pldo, "vdd_l3_l4_l5" },
+ { "l6", QCOM_RPM_PM8058_LDO6, &pm8058_pldo, "vdd_l6_l7" },
+ { "l7", QCOM_RPM_PM8058_LDO7, &pm8058_pldo, "vdd_l6_l7" },
+ { "l8", QCOM_RPM_PM8058_LDO8, &pm8058_pldo, "vdd_l8" },
+ { "l9", QCOM_RPM_PM8058_LDO9, &pm8058_pldo, "vdd_l9" },
+ { "l10", QCOM_RPM_PM8058_LDO10, &pm8058_pldo, "vdd_l10" },
+ { "l11", QCOM_RPM_PM8058_LDO11, &pm8058_pldo, "vdd_l2_l11_l12" },
+ { "l12", QCOM_RPM_PM8058_LDO12, &pm8058_pldo, "vdd_l2_l11_l12" },
+ { "l13", QCOM_RPM_PM8058_LDO13, &pm8058_pldo, "vdd_l13_l16" },
+ { "l14", QCOM_RPM_PM8058_LDO14, &pm8058_pldo, "vdd_l14_l15" },
+ { "l15", QCOM_RPM_PM8058_LDO15, &pm8058_pldo, "vdd_l14_l15" },
+ { "l16", QCOM_RPM_PM8058_LDO16, &pm8058_pldo, "vdd_l13_l16" },
+ { "l17", QCOM_RPM_PM8058_LDO17, &pm8058_pldo, "vdd_l17_l18" },
+ { "l18", QCOM_RPM_PM8058_LDO18, &pm8058_pldo, "vdd_l17_l18" },
+ { "l19", QCOM_RPM_PM8058_LDO19, &pm8058_pldo, "vdd_l19_l20" },
+ { "l20", QCOM_RPM_PM8058_LDO20, &pm8058_pldo, "vdd_l19_l20" },
+ { "l21", QCOM_RPM_PM8058_LDO21, &pm8058_nldo, "vdd_l21" },
+ { "l22", QCOM_RPM_PM8058_LDO22, &pm8058_nldo, "vdd_l22" },
+ { "l23", QCOM_RPM_PM8058_LDO23, &pm8058_nldo, "vdd_l23_l24_l25" },
+ { "l24", QCOM_RPM_PM8058_LDO24, &pm8058_nldo, "vdd_l23_l24_l25" },
+ { "l25", QCOM_RPM_PM8058_LDO25, &pm8058_nldo, "vdd_l23_l24_l25" },
+
+ { "s0", QCOM_RPM_PM8058_SMPS0, &pm8058_smps, "vdd_s0" },
+ { "s1", QCOM_RPM_PM8058_SMPS1, &pm8058_smps, "vdd_s1" },
+ { "s2", QCOM_RPM_PM8058_SMPS2, &pm8058_smps, "vdd_s2" },
+ { "s3", QCOM_RPM_PM8058_SMPS3, &pm8058_smps, "vdd_s3" },
+ { "s4", QCOM_RPM_PM8058_SMPS4, &pm8058_smps, "vdd_s4" },
+
+ { "lvs0", QCOM_RPM_PM8058_LVS0, &pm8058_switch, "vdd_l0_l1_lvs" },
+ { "lvs1", QCOM_RPM_PM8058_LVS1, &pm8058_switch, "vdd_l0_l1_lvs" },
+
+ { "ncp", QCOM_RPM_PM8058_NCP, &pm8058_ncp, "vdd_ncp" },
+ { }
+};
+
+static const struct rpm_regulator_data rpm_pm8901_regulators[] = {
+ { "l0", QCOM_RPM_PM8901_LDO0, &pm8901_nldo, "vdd_l0" },
+ { "l1", QCOM_RPM_PM8901_LDO1, &pm8901_pldo, "vdd_l1" },
+ { "l2", QCOM_RPM_PM8901_LDO2, &pm8901_pldo, "vdd_l2" },
+ { "l3", QCOM_RPM_PM8901_LDO3, &pm8901_pldo, "vdd_l3" },
+ { "l4", QCOM_RPM_PM8901_LDO4, &pm8901_pldo, "vdd_l4" },
+ { "l5", QCOM_RPM_PM8901_LDO5, &pm8901_pldo, "vdd_l5" },
+ { "l6", QCOM_RPM_PM8901_LDO6, &pm8901_pldo, "vdd_l6" },
+
+ { "s0", QCOM_RPM_PM8901_SMPS0, &pm8901_ftsmps, "vdd_s0" },
+ { "s1", QCOM_RPM_PM8901_SMPS1, &pm8901_ftsmps, "vdd_s1" },
+ { "s2", QCOM_RPM_PM8901_SMPS2, &pm8901_ftsmps, "vdd_s2" },
+ { "s3", QCOM_RPM_PM8901_SMPS3, &pm8901_ftsmps, "vdd_s3" },
+ { "s4", QCOM_RPM_PM8901_SMPS4, &pm8901_ftsmps, "vdd_s4" },
+
+ { "lvs0", QCOM_RPM_PM8901_LVS0, &pm8901_switch, "lvs0_in" },
+ { "lvs1", QCOM_RPM_PM8901_LVS1, &pm8901_switch, "lvs1_in" },
+ { "lvs2", QCOM_RPM_PM8901_LVS2, &pm8901_switch, "lvs2_in" },
+ { "lvs3", QCOM_RPM_PM8901_LVS3, &pm8901_switch, "lvs3_in" },
+
+ { "mvs", QCOM_RPM_PM8901_MVS, &pm8901_switch, "mvs_in" },
+ { }
+};
+
+static const struct rpm_regulator_data rpm_pm8921_regulators[] = {
+ { "s1", QCOM_RPM_PM8921_SMPS1, &pm8921_smps, "vdd_s1" },
+ { "s2", QCOM_RPM_PM8921_SMPS2, &pm8921_smps, "vdd_s2" },
+ { "s3", QCOM_RPM_PM8921_SMPS3, &pm8921_smps },
+ { "s4", QCOM_RPM_PM8921_SMPS4, &pm8921_smps, "vdd_s4" },
+ { "s7", QCOM_RPM_PM8921_SMPS7, &pm8921_smps, "vdd_s7" },
+ { "s8", QCOM_RPM_PM8921_SMPS8, &pm8921_smps, "vdd_s8" },
+
+ { "l1", QCOM_RPM_PM8921_LDO1, &pm8921_nldo, "vdd_l1_l2_l12_l18" },
+ { "l2", QCOM_RPM_PM8921_LDO2, &pm8921_nldo, "vdd_l1_l2_l12_l18" },
+ { "l3", QCOM_RPM_PM8921_LDO3, &pm8921_pldo, "vdd_l3_l15_l17" },
+ { "l4", QCOM_RPM_PM8921_LDO4, &pm8921_pldo, "vdd_l4_l14" },
+ { "l5", QCOM_RPM_PM8921_LDO5, &pm8921_pldo, "vdd_l5_l8_l16" },
+ { "l6", QCOM_RPM_PM8921_LDO6, &pm8921_pldo, "vdd_l6_l7" },
+ { "l7", QCOM_RPM_PM8921_LDO7, &pm8921_pldo, "vdd_l6_l7" },
+ { "l8", QCOM_RPM_PM8921_LDO8, &pm8921_pldo, "vdd_l5_l8_l16" },
+ { "l9", QCOM_RPM_PM8921_LDO9, &pm8921_pldo, "vdd_l9_l11" },
+ { "l10", QCOM_RPM_PM8921_LDO10, &pm8921_pldo, "vdd_l10_l22" },
+ { "l11", QCOM_RPM_PM8921_LDO11, &pm8921_pldo, "vdd_l9_l11" },
+ { "l12", QCOM_RPM_PM8921_LDO12, &pm8921_nldo, "vdd_l1_l2_l12_l18" },
+ { "l14", QCOM_RPM_PM8921_LDO14, &pm8921_pldo, "vdd_l4_l14" },
+ { "l15", QCOM_RPM_PM8921_LDO15, &pm8921_pldo, "vdd_l3_l15_l17" },
+ { "l16", QCOM_RPM_PM8921_LDO16, &pm8921_pldo, "vdd_l5_l8_l16" },
+ { "l17", QCOM_RPM_PM8921_LDO17, &pm8921_pldo, "vdd_l3_l15_l17" },
+ { "l18", QCOM_RPM_PM8921_LDO18, &pm8921_nldo, "vdd_l1_l2_l12_l18" },
+ { "l21", QCOM_RPM_PM8921_LDO21, &pm8921_pldo, "vdd_l21_l23_l29" },
+ { "l22", QCOM_RPM_PM8921_LDO22, &pm8921_pldo, "vdd_l10_l22" },
+ { "l23", QCOM_RPM_PM8921_LDO23, &pm8921_pldo, "vdd_l21_l23_l29" },
+ { "l24", QCOM_RPM_PM8921_LDO24, &pm8921_nldo1200, "vdd_l24" },
+ { "l25", QCOM_RPM_PM8921_LDO25, &pm8921_nldo1200, "vdd_l25" },
+ { "l26", QCOM_RPM_PM8921_LDO26, &pm8921_nldo1200, "vdd_l26" },
+ { "l27", QCOM_RPM_PM8921_LDO27, &pm8921_nldo1200, "vdd_l27" },
+ { "l28", QCOM_RPM_PM8921_LDO28, &pm8921_nldo1200, "vdd_l28" },
+ { "l29", QCOM_RPM_PM8921_LDO29, &pm8921_pldo, "vdd_l21_l23_l29" },
+
+ { "lvs1", QCOM_RPM_PM8921_LVS1, &pm8921_switch, "vin_lvs1_3_6" },
+ { "lvs2", QCOM_RPM_PM8921_LVS2, &pm8921_switch, "vin_lvs2" },
+ { "lvs3", QCOM_RPM_PM8921_LVS3, &pm8921_switch, "vin_lvs1_3_6" },
+ { "lvs4", QCOM_RPM_PM8921_LVS4, &pm8921_switch, "vin_lvs4_5_7" },
+ { "lvs5", QCOM_RPM_PM8921_LVS5, &pm8921_switch, "vin_lvs4_5_7" },
+ { "lvs6", QCOM_RPM_PM8921_LVS6, &pm8921_switch, "vin_lvs1_3_6" },
+ { "lvs7", QCOM_RPM_PM8921_LVS7, &pm8921_switch, "vin_lvs4_5_7" },
+
+ { "usb-switch", QCOM_RPM_USB_OTG_SWITCH, &pm8921_switch, "vin_5vs" },
+ { "hdmi-switch", QCOM_RPM_HDMI_SWITCH, &pm8921_switch, "vin_5vs" },
+ { "ncp", QCOM_RPM_PM8921_NCP, &pm8921_ncp, "vdd_ncp" },
+ { }
+};
+
+static const struct of_device_id rpm_of_match[] = {
+ { .compatible = "qcom,rpm-pm8058-regulators", .data = &rpm_pm8058_regulators },
+ { .compatible = "qcom,rpm-pm8901-regulators", .data = &rpm_pm8901_regulators },
+ { .compatible = "qcom,rpm-pm8921-regulators", .data = &rpm_pm8921_regulators },
+ { }
+};
+MODULE_DEVICE_TABLE(of, rpm_of_match);
+
+static int rpm_reg_probe(struct platform_device *pdev)
+{
+ const struct rpm_regulator_data *reg;
+ const struct of_device_id *match;
+ struct regulator_config config = { };
+ struct regulator_dev *rdev;
+ struct qcom_rpm_reg *vreg;
+ struct qcom_rpm *rpm;
+
+ rpm = dev_get_drvdata(pdev->dev.parent);
+ if (!rpm) {
+ dev_err(&pdev->dev, "unable to retrieve handle to rpm\n");
+ return -ENODEV;
+ }
+
+ match = of_match_device(rpm_of_match, &pdev->dev);
+ for (reg = match->data; reg->name; reg++) {
+ vreg = devm_kmalloc(&pdev->dev, sizeof(*vreg), GFP_KERNEL);
+ if (!vreg)
+ return -ENOMEM;
+
+ memcpy(vreg, reg->template, sizeof(*vreg));
+ mutex_init(&vreg->lock);
+
+ vreg->dev = &pdev->dev;
+ vreg->resource = reg->resource;
+ vreg->rpm = rpm;
+
+ vreg->desc.id = -1;
+ vreg->desc.owner = THIS_MODULE;
+ vreg->desc.type = REGULATOR_VOLTAGE;
+ vreg->desc.name = reg->name;
+ vreg->desc.supply_name = reg->supply;
+ vreg->desc.of_match = reg->name;
+ vreg->desc.of_parse_cb = rpm_reg_of_parse;
+
+ config.dev = &pdev->dev;
+ config.driver_data = vreg;
+ rdev = devm_regulator_register(&pdev->dev, &vreg->desc, &config);
+ if (IS_ERR(rdev)) {
+ dev_err(&pdev->dev, "failed to register %s\n", reg->name);
+ return PTR_ERR(rdev);
+ }
}
return 0;
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(0),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG2",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(1),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG3",
.vsel_mask = RK808_BUCK4_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(2),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG4",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(3),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG5",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(4),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG6",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(5),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG7",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(6),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "LDO_REG8",
.vsel_mask = RK808_LDO_VSEL_MASK,
.enable_reg = RK808_LDO_EN_REG,
.enable_mask = BIT(7),
+ .enable_time = 400,
.owner = THIS_MODULE,
}, {
.name = "SWITCH_REG1",
{
struct stw481x *stw481x = dev_get_platdata(&pdev->dev);
struct regulator_config config = { };
+ struct regulator_dev *rdev;
int ret;
/* First disable the external VMMC if it's active */
pdev->dev.of_node,
&vmmc_regulator);
- stw481x->vmmc_regulator = devm_regulator_register(&pdev->dev,
- &vmmc_regulator, &config);
- if (IS_ERR(stw481x->vmmc_regulator)) {
+ rdev = devm_regulator_register(&pdev->dev, &vmmc_regulator, &config);
+ if (IS_ERR(rdev)) {
dev_err(&pdev->dev,
"error initializing STw481x VMMC regulator\n");
- return PTR_ERR(stw481x->vmmc_regulator);
+ return PTR_ERR(rdev);
}
dev_info(&pdev->dev, "initialized STw481x VMMC regulator\n");
#include <linux/module.h>
#include <linux/init.h>
#include <linux/err.h>
+#include <linux/of.h>
#include <linux/platform_device.h>
#include <linux/regulator/driver.h>
#include <linux/regulator/machine.h>
static irqreturn_t pmic_uv_handler(int irq, void *data)
{
struct regulator_dev *rdev = (struct regulator_dev *)data;
- struct wm8350 *wm8350 = rdev_get_drvdata(rdev);
mutex_lock(&rdev->mutex);
if (irq == WM8350_IRQ_CS1 || irq == WM8350_IRQ_CS2)
regulator_notifier_call_chain(rdev,
REGULATOR_EVENT_REGULATION_OUT,
- wm8350);
+ NULL);
else
regulator_notifier_call_chain(rdev,
REGULATOR_EVENT_UNDER_VOLTAGE,
- wm8350);
+ NULL);
mutex_unlock(&rdev->mutex);
return IRQ_HANDLED;
void *bufs_va;
int err = 0, i;
size_t total_buf_space;
+ bool notify;
vrp = kzalloc(sizeof(*vrp), GFP_KERNEL);
if (!vrp)
}
}
+ /*
+ * Prepare to kick but don't notify yet - we can't do this before
+ * device is ready.
+ */
+ notify = virtqueue_kick_prepare(vrp->rvq);
+
+ /* From this point on, we can notify and get callbacks. */
+ virtio_device_ready(vdev);
+
/* tell the remote processor it can start sending messages */
- virtqueue_kick(vrp->rvq);
+ /*
+ * this might be concurrent with callbacks, but we are only
+ * doing notify, not a full kick here, so that's ok.
+ */
+ if (notify)
+ virtqueue_notify(vrp->rvq);
dev_info(&vdev->dev, "rpmsg host is online\n");
struct timespec64 delta, delta_delta;
int err;
- if (has_persistent_clock())
+ if (timekeeping_rtc_skipsuspend())
return 0;
if (strcmp(dev_name(&rtc->dev), CONFIG_RTC_HCTOSYS_DEVICE) != 0)
struct timespec64 sleep_time;
int err;
- if (has_persistent_clock())
+ if (timekeeping_rtc_skipresume())
return 0;
rtc_hctosys_ret = -ENODEV;
return 0;
}
- if (rtc_valid_tm(&tm) != 0) {
- pr_debug("%s: bogus resume time\n", dev_name(&rtc->dev));
- return 0;
- }
new_rtc.tv_sec = rtc_tm_to_time64(&tm);
new_rtc.tv_nsec = 0;
err = -ENODEV;
else if (rtc->ops->set_time)
err = rtc->ops->set_time(rtc->dev.parent, tm);
- else if (rtc->ops->set_mmss) {
+ else if (rtc->ops->set_mmss64) {
+ time64_t secs64 = rtc_tm_to_time64(tm);
+
+ err = rtc->ops->set_mmss64(rtc->dev.parent, secs64);
+ } else if (rtc->ops->set_mmss) {
time64_t secs64 = rtc_tm_to_time64(tm);
err = rtc->ops->set_mmss(rtc->dev.parent, secs64);
} else
if (!rtc->ops)
err = -ENODEV;
+ else if (rtc->ops->set_mmss64)
+ err = rtc->ops->set_mmss64(rtc->dev.parent, secs);
else if (rtc->ops->set_mmss)
err = rtc->ops->set_mmss(rtc->dev.parent, secs);
else if (rtc->ops->read_time && rtc->ops->set_time) {
/*
* RTC clock functions and device struct declaration
*/
-static int ab3100_rtc_set_mmss(struct device *dev, unsigned long secs)
+static int ab3100_rtc_set_mmss(struct device *dev, time64_t secs)
{
u8 regs[] = {AB3100_TI0, AB3100_TI1, AB3100_TI2,
AB3100_TI3, AB3100_TI4, AB3100_TI5};
unsigned char buf[6];
- u64 fat_time = (u64) secs * AB3100_RTC_CLOCK_RATE * 2;
+ u64 hw_counter = secs * AB3100_RTC_CLOCK_RATE * 2;
int err = 0;
int i;
- buf[0] = (fat_time) & 0xFF;
- buf[1] = (fat_time >> 8) & 0xFF;
- buf[2] = (fat_time >> 16) & 0xFF;
- buf[3] = (fat_time >> 24) & 0xFF;
- buf[4] = (fat_time >> 32) & 0xFF;
- buf[5] = (fat_time >> 40) & 0xFF;
+ buf[0] = (hw_counter) & 0xFF;
+ buf[1] = (hw_counter >> 8) & 0xFF;
+ buf[2] = (hw_counter >> 16) & 0xFF;
+ buf[3] = (hw_counter >> 24) & 0xFF;
+ buf[4] = (hw_counter >> 32) & 0xFF;
+ buf[5] = (hw_counter >> 40) & 0xFF;
for (i = 0; i < 6; i++) {
err = abx500_set_register_interruptible(dev, 0,
static int ab3100_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
- unsigned long time;
+ time64_t time;
u8 rtcval;
int err;
dev_info(dev, "clock not set (lost power)");
return -EINVAL;
} else {
- u64 fat_time;
+ u64 hw_counter;
u8 buf[6];
/* Read out time registers */
if (err != 0)
return err;
- fat_time = ((u64) buf[5] << 40) | ((u64) buf[4] << 32) |
+ hw_counter = ((u64) buf[5] << 40) | ((u64) buf[4] << 32) |
((u64) buf[3] << 24) | ((u64) buf[2] << 16) |
((u64) buf[1] << 8) | (u64) buf[0];
- time = (unsigned long) (fat_time /
- (u64) (AB3100_RTC_CLOCK_RATE * 2));
+ time = hw_counter / (u64) (AB3100_RTC_CLOCK_RATE * 2);
}
- rtc_time_to_tm(time, tm);
+ rtc_time64_to_tm(time, tm);
return rtc_valid_tm(tm);
}
static int ab3100_rtc_read_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
- unsigned long time;
- u64 fat_time;
+ time64_t time;
+ u64 hw_counter;
u8 buf[6];
u8 rtcval;
int err;
AB3100_AL0, buf, 4);
if (err)
return err;
- fat_time = ((u64) buf[3] << 40) | ((u64) buf[2] << 32) |
+ hw_counter = ((u64) buf[3] << 40) | ((u64) buf[2] << 32) |
((u64) buf[1] << 24) | ((u64) buf[0] << 16);
- time = (unsigned long) (fat_time / (u64) (AB3100_RTC_CLOCK_RATE * 2));
+ time = hw_counter / (u64) (AB3100_RTC_CLOCK_RATE * 2);
- rtc_time_to_tm(time, &alarm->time);
+ rtc_time64_to_tm(time, &alarm->time);
return rtc_valid_tm(&alarm->time);
}
{
u8 regs[] = {AB3100_AL0, AB3100_AL1, AB3100_AL2, AB3100_AL3};
unsigned char buf[4];
- unsigned long secs;
- u64 fat_time;
+ time64_t secs;
+ u64 hw_counter;
int err;
int i;
- rtc_tm_to_time(&alarm->time, &secs);
- fat_time = (u64) secs * AB3100_RTC_CLOCK_RATE * 2;
- buf[0] = (fat_time >> 16) & 0xFF;
- buf[1] = (fat_time >> 24) & 0xFF;
- buf[2] = (fat_time >> 32) & 0xFF;
- buf[3] = (fat_time >> 40) & 0xFF;
+ secs = rtc_tm_to_time64(&alarm->time);
+ hw_counter = secs * AB3100_RTC_CLOCK_RATE * 2;
+ buf[0] = (hw_counter >> 16) & 0xFF;
+ buf[1] = (hw_counter >> 24) & 0xFF;
+ buf[2] = (hw_counter >> 32) & 0xFF;
+ buf[3] = (hw_counter >> 40) & 0xFF;
/* Set the alarm */
for (i = 0; i < 4; i++) {
static const struct rtc_class_ops ab3100_rtc_ops = {
.read_time = ab3100_rtc_read_time,
- .set_mmss = ab3100_rtc_set_mmss,
+ .set_mmss64 = ab3100_rtc_set_mmss,
.read_alarm = ab3100_rtc_read_alarm,
.set_alarm = ab3100_rtc_set_alarm,
.alarm_irq_enable = ab3100_rtc_irq_enable,
#include <linux/io.h>
#include <linux/of.h>
#include <linux/of_device.h>
+#include <linux/suspend.h>
#include <linux/uaccess.h>
#include "rtc-at91rm9200.h"
static int irq;
static DEFINE_SPINLOCK(at91_rtc_lock);
static u32 at91_rtc_shadow_imr;
+static bool suspended;
+static DEFINE_SPINLOCK(suspended_lock);
+static unsigned long cached_events;
+static u32 at91_rtc_imr;
static void at91_rtc_write_ier(u32 mask)
{
struct rtc_device *rtc = platform_get_drvdata(pdev);
unsigned int rtsr;
unsigned long events = 0;
+ int ret = IRQ_NONE;
+ spin_lock(&suspended_lock);
rtsr = at91_rtc_read(AT91_RTC_SR) & at91_rtc_read_imr();
if (rtsr) { /* this interrupt is shared! Is it ours? */
if (rtsr & AT91_RTC_ALARM)
at91_rtc_write(AT91_RTC_SCCR, rtsr); /* clear status reg */
- rtc_update_irq(rtc, 1, events);
+ if (!suspended) {
+ rtc_update_irq(rtc, 1, events);
- dev_dbg(&pdev->dev, "%s(): num=%ld, events=0x%02lx\n", __func__,
- events >> 8, events & 0x000000FF);
+ dev_dbg(&pdev->dev, "%s(): num=%ld, events=0x%02lx\n",
+ __func__, events >> 8, events & 0x000000FF);
+ } else {
+ cached_events |= events;
+ at91_rtc_write_idr(at91_rtc_imr);
+ pm_system_wakeup();
+ }
- return IRQ_HANDLED;
+ ret = IRQ_HANDLED;
}
- return IRQ_NONE; /* not handled */
+ spin_unlock(&suspended_lock);
+
+ return ret;
}
static const struct at91_rtc_config at91rm9200_config = {
AT91_RTC_CALEV);
ret = devm_request_irq(&pdev->dev, irq, at91_rtc_interrupt,
- IRQF_SHARED,
- "at91_rtc", pdev);
+ IRQF_SHARED | IRQF_COND_SUSPEND,
+ "at91_rtc", pdev);
if (ret) {
dev_err(&pdev->dev, "IRQ %d already in use.\n", irq);
return ret;
/* AT91RM9200 RTC Power management control */
-static u32 at91_rtc_imr;
-
static int at91_rtc_suspend(struct device *dev)
{
/* this IRQ is shared with DBGU and other hardware which isn't
at91_rtc_imr = at91_rtc_read_imr()
& (AT91_RTC_ALARM|AT91_RTC_SECEV);
if (at91_rtc_imr) {
- if (device_may_wakeup(dev))
+ if (device_may_wakeup(dev)) {
+ unsigned long flags;
+
enable_irq_wake(irq);
- else
+
+ spin_lock_irqsave(&suspended_lock, flags);
+ suspended = true;
+ spin_unlock_irqrestore(&suspended_lock, flags);
+ } else {
at91_rtc_write_idr(at91_rtc_imr);
+ }
}
return 0;
}
static int at91_rtc_resume(struct device *dev)
{
+ struct rtc_device *rtc = dev_get_drvdata(dev);
+
if (at91_rtc_imr) {
- if (device_may_wakeup(dev))
+ if (device_may_wakeup(dev)) {
+ unsigned long flags;
+
+ spin_lock_irqsave(&suspended_lock, flags);
+
+ if (cached_events) {
+ rtc_update_irq(rtc, 1, cached_events);
+ cached_events = 0;
+ }
+
+ suspended = false;
+ spin_unlock_irqrestore(&suspended_lock, flags);
+
disable_irq_wake(irq);
- else
- at91_rtc_write_ier(at91_rtc_imr);
+ }
+ at91_rtc_write_ier(at91_rtc_imr);
}
return 0;
}
#include <linux/io.h>
#include <linux/mfd/syscon.h>
#include <linux/regmap.h>
+#include <linux/suspend.h>
#include <linux/clk.h>
/*
unsigned int gpbr_offset;
int irq;
struct clk *sclk;
+ bool suspended;
+ unsigned long events;
+ spinlock_t lock;
};
#define rtt_readl(rtc, field) \
return 0;
}
-/*
- * IRQ handler for the RTC
- */
-static irqreturn_t at91_rtc_interrupt(int irq, void *_rtc)
+static irqreturn_t at91_rtc_cache_events(struct sam9_rtc *rtc)
{
- struct sam9_rtc *rtc = _rtc;
u32 sr, mr;
- unsigned long events = 0;
/* Shared interrupt may be for another device. Note: reading
* SR clears it, so we must only read it in this irq handler!
/* alarm status */
if (sr & AT91_RTT_ALMS)
- events |= (RTC_AF | RTC_IRQF);
+ rtc->events |= (RTC_AF | RTC_IRQF);
/* timer update/increment */
if (sr & AT91_RTT_RTTINC)
- events |= (RTC_UF | RTC_IRQF);
+ rtc->events |= (RTC_UF | RTC_IRQF);
+
+ return IRQ_HANDLED;
+}
+
+static void at91_rtc_flush_events(struct sam9_rtc *rtc)
+{
+ if (!rtc->events)
+ return;
- rtc_update_irq(rtc->rtcdev, 1, events);
+ rtc_update_irq(rtc->rtcdev, 1, rtc->events);
+ rtc->events = 0;
pr_debug("%s: num=%ld, events=0x%02lx\n", __func__,
- events >> 8, events & 0x000000FF);
+ rtc->events >> 8, rtc->events & 0x000000FF);
+}
- return IRQ_HANDLED;
+/*
+ * IRQ handler for the RTC
+ */
+static irqreturn_t at91_rtc_interrupt(int irq, void *_rtc)
+{
+ struct sam9_rtc *rtc = _rtc;
+ int ret;
+
+ spin_lock(&rtc->lock);
+
+ ret = at91_rtc_cache_events(rtc);
+
+ /* We're called in suspended state */
+ if (rtc->suspended) {
+ /* Mask irqs coming from this peripheral */
+ rtt_writel(rtc, MR,
+ rtt_readl(rtc, MR) &
+ ~(AT91_RTT_ALMIEN | AT91_RTT_RTTINCIEN));
+ /* Trigger a system wakeup */
+ pm_system_wakeup();
+ } else {
+ at91_rtc_flush_events(rtc);
+ }
+
+ spin_unlock(&rtc->lock);
+
+ return ret;
}
static const struct rtc_class_ops at91_rtc_ops = {
/* register irq handler after we know what name we'll use */
ret = devm_request_irq(&pdev->dev, rtc->irq, at91_rtc_interrupt,
- IRQF_SHARED, dev_name(&rtc->rtcdev->dev), rtc);
+ IRQF_SHARED | IRQF_COND_SUSPEND,
+ dev_name(&rtc->rtcdev->dev), rtc);
if (ret) {
dev_dbg(&pdev->dev, "can't share IRQ %d?\n", rtc->irq);
return ret;
rtc->imr = mr & (AT91_RTT_ALMIEN | AT91_RTT_RTTINCIEN);
if (rtc->imr) {
if (device_may_wakeup(dev) && (mr & AT91_RTT_ALMIEN)) {
+ unsigned long flags;
+
enable_irq_wake(rtc->irq);
+ spin_lock_irqsave(&rtc->lock, flags);
+ rtc->suspended = true;
+ spin_unlock_irqrestore(&rtc->lock, flags);
/* don't let RTTINC cause wakeups */
if (mr & AT91_RTT_RTTINCIEN)
rtt_writel(rtc, MR, mr & ~AT91_RTT_RTTINCIEN);
u32 mr;
if (rtc->imr) {
+ unsigned long flags;
+
if (device_may_wakeup(dev))
disable_irq_wake(rtc->irq);
mr = rtt_readl(rtc, MR);
rtt_writel(rtc, MR, mr | rtc->imr);
+
+ spin_lock_irqsave(&rtc->lock, flags);
+ rtc->suspended = false;
+ at91_rtc_cache_events(rtc);
+ at91_rtc_flush_events(rtc);
+ spin_unlock_irqrestore(&rtc->lock, flags);
}
return 0;
* of this RTC chip. We check for it anyways in case support is
* added in the future.
*/
- if (unlikely((seconds >= 0xc0) && (seconds <= 0xff)))
+ if (unlikely(seconds >= 0xc0))
alrm->time.tm_sec = -1;
else
alrm->time.tm_sec = ds1685_rtc_bcd2bin(rtc, seconds,
RTC_SECS_BCD_MASK,
RTC_SECS_BIN_MASK);
- if (unlikely((minutes >= 0xc0) && (minutes <= 0xff)))
+ if (unlikely(minutes >= 0xc0))
alrm->time.tm_min = -1;
else
alrm->time.tm_min = ds1685_rtc_bcd2bin(rtc, minutes,
RTC_MINS_BCD_MASK,
RTC_MINS_BIN_MASK);
- if (unlikely((hours >= 0xc0) && (hours <= 0xff)))
+ if (unlikely(hours >= 0xc0))
alrm->time.tm_hour = -1;
else
alrm->time.tm_hour = ds1685_rtc_bcd2bin(rtc, hours,
* field, and we only support four fields. We put the support
* here anyways for the future.
*/
- if (unlikely((seconds >= 0xc0) && (seconds <= 0xff)))
+ if (unlikely(seconds >= 0xc0))
seconds = 0xff;
- if (unlikely((minutes >= 0xc0) && (minutes <= 0xff)))
+ if (unlikely(minutes >= 0xc0))
minutes = 0xff;
- if (unlikely((hours >= 0xc0) && (hours <= 0xff)))
+ if (unlikely(hours >= 0xc0))
hours = 0xff;
alrm->time.tm_mon = -1;
/* ----------------------------------------------------------------------- */
/* /dev/rtcX Interface functions */
-#ifdef CONFIG_RTC_INTF_DEV
/**
* ds1685_rtc_alarm_irq_enable - replaces ioctl() RTC_AIE on/off.
* @dev: pointer to device structure.
return 0;
}
-#endif
/* ----------------------------------------------------------------------- */
ds1685_rtc_sysfs_time_regs_lookup(attr->attr.name, false);
/* Make sure we actually matched something. */
- if (!bcd_reg_info && !bin_reg_info)
+ if (!bcd_reg_info || !bin_reg_info)
return -EINVAL;
/* bcd_reg_info->reg == bin_reg_info->reg. */
return -EINVAL;
/* Make sure we actually matched something. */
- if (!bcd_reg_info && !bin_reg_info)
+ if (!bcd_reg_info || !bin_reg_info)
return -EINVAL;
/* Check for a valid range. */
return ret;
} while (days1 != days2);
- rtc_time_to_tm(days1 * SEC_PER_DAY + seconds, tm);
+ rtc_time64_to_tm((time64_t)days1 * SEC_PER_DAY + seconds, tm);
return rtc_valid_tm(tm);
}
-static int mc13xxx_rtc_set_mmss(struct device *dev, unsigned long secs)
+static int mc13xxx_rtc_set_mmss(struct device *dev, time64_t secs)
{
struct mc13xxx_rtc *priv = dev_get_drvdata(dev);
unsigned int seconds, days;
unsigned int alarmseconds;
int ret;
- seconds = secs % SEC_PER_DAY;
- days = secs / SEC_PER_DAY;
+ days = div_s64_rem(secs, SEC_PER_DAY, &seconds);
mc13xxx_lock(priv->mc13xxx);
{
struct mc13xxx_rtc *priv = dev_get_drvdata(dev);
unsigned seconds, days;
- unsigned long s1970;
+ time64_t s1970;
int enabled, pending;
int ret;
alarm->enabled = enabled;
alarm->pending = pending;
- s1970 = days * SEC_PER_DAY + seconds;
+ s1970 = (time64_t)days * SEC_PER_DAY + seconds;
- rtc_time_to_tm(s1970, &alarm->time);
- dev_dbg(dev, "%s: %lu\n", __func__, s1970);
+ rtc_time64_to_tm(s1970, &alarm->time);
+ dev_dbg(dev, "%s: %lld\n", __func__, (long long)s1970);
return 0;
}
static int mc13xxx_rtc_set_alarm(struct device *dev, struct rtc_wkalrm *alarm)
{
struct mc13xxx_rtc *priv = dev_get_drvdata(dev);
- unsigned long s1970;
- unsigned seconds, days;
+ time64_t s1970;
+ u32 seconds, days;
int ret;
mc13xxx_lock(priv->mc13xxx);
if (unlikely(ret))
goto out;
- ret = rtc_tm_to_time(&alarm->time, &s1970);
- if (unlikely(ret))
- goto out;
+ s1970 = rtc_tm_to_time64(&alarm->time);
- dev_dbg(dev, "%s: o%2.s %lu\n", __func__, alarm->enabled ? "n" : "ff",
- s1970);
+ dev_dbg(dev, "%s: o%2.s %lld\n", __func__, alarm->enabled ? "n" : "ff",
+ (long long)s1970);
ret = mc13xxx_rtc_irq_enable_unlocked(dev, alarm->enabled,
MC13XXX_IRQ_TODA);
if (unlikely(ret))
goto out;
- seconds = s1970 % SEC_PER_DAY;
- days = s1970 / SEC_PER_DAY;
+ days = div_s64_rem(s1970, SEC_PER_DAY, &seconds);
ret = mc13xxx_reg_write(priv->mc13xxx, MC13XXX_RTCDAYA, days);
if (unlikely(ret))
static const struct rtc_class_ops mc13xxx_rtc_ops = {
.read_time = mc13xxx_rtc_read_time,
- .set_mmss = mc13xxx_rtc_set_mmss,
+ .set_mmss64 = mc13xxx_rtc_set_mmss,
.read_alarm = mc13xxx_rtc_read_alarm,
.set_alarm = mc13xxx_rtc_set_alarm,
.alarm_irq_enable = mc13xxx_rtc_alarm_irq_enable,
mrst->dev = NULL;
}
-#ifdef CONFIG_PM
-static int mrst_suspend(struct device *dev, pm_message_t mesg)
+#ifdef CONFIG_PM_SLEEP
+static int mrst_suspend(struct device *dev)
{
struct mrst_rtc *mrst = dev_get_drvdata(dev);
unsigned char tmp;
*/
static inline int mrst_poweroff(struct device *dev)
{
- return mrst_suspend(dev, PMSG_HIBERNATE);
+ return mrst_suspend(dev);
}
static int mrst_resume(struct device *dev)
return 0;
}
+static SIMPLE_DEV_PM_OPS(mrst_pm_ops, mrst_suspend, mrst_resume);
+#define MRST_PM_OPS (&mrst_pm_ops)
+
#else
-#define mrst_suspend NULL
-#define mrst_resume NULL
+#define MRST_PM_OPS NULL
static inline int mrst_poweroff(struct device *dev)
{
.remove = vrtc_mrst_platform_remove,
.shutdown = vrtc_mrst_platform_shutdown,
.driver = {
- .name = (char *) driver_name,
- .suspend = mrst_suspend,
- .resume = mrst_resume,
+ .name = driver_name,
+ .pm = MRST_PM_OPS,
}
};
* This function is used to obtain the RTC time or the alarm value in
* second.
*/
-static u32 get_alarm_or_time(struct device *dev, int time_alarm)
+static time64_t get_alarm_or_time(struct device *dev, int time_alarm)
{
struct platform_device *pdev = to_platform_device(dev);
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
hr = hr_min >> 8;
min = hr_min & 0xff;
- return (((day * 24 + hr) * 60) + min) * 60 + sec;
+ return ((((time64_t)day * 24 + hr) * 60) + min) * 60 + sec;
}
/*
* This function sets the RTC alarm value or the time value.
*/
-static void set_alarm_or_time(struct device *dev, int time_alarm, u32 time)
+static void set_alarm_or_time(struct device *dev, int time_alarm, time64_t time)
{
- u32 day, hr, min, sec, temp;
+ u32 tod, day, hr, min, sec, temp;
struct platform_device *pdev = to_platform_device(dev);
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
void __iomem *ioaddr = pdata->ioaddr;
- day = time / 86400;
- time -= day * 86400;
+ day = div_s64_rem(time, 86400, &tod);
/* time is within a day now */
- hr = time / 3600;
- time -= hr * 3600;
+ hr = tod / 3600;
+ tod -= hr * 3600;
/* time is within an hour now */
- min = time / 60;
- sec = time - min * 60;
+ min = tod / 60;
+ sec = tod - min * 60;
temp = (hr << 8) + min;
* This function updates the RTC alarm registers and then clears all the
* interrupt status bits.
*/
-static int rtc_update_alarm(struct device *dev, struct rtc_time *alrm)
+static void rtc_update_alarm(struct device *dev, struct rtc_time *alrm)
{
- struct rtc_time alarm_tm, now_tm;
- unsigned long now, time;
+ time64_t time;
struct platform_device *pdev = to_platform_device(dev);
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
void __iomem *ioaddr = pdata->ioaddr;
- now = get_alarm_or_time(dev, MXC_RTC_TIME);
- rtc_time_to_tm(now, &now_tm);
- alarm_tm.tm_year = now_tm.tm_year;
- alarm_tm.tm_mon = now_tm.tm_mon;
- alarm_tm.tm_mday = now_tm.tm_mday;
- alarm_tm.tm_hour = alrm->tm_hour;
- alarm_tm.tm_min = alrm->tm_min;
- alarm_tm.tm_sec = alrm->tm_sec;
- rtc_tm_to_time(&alarm_tm, &time);
+ time = rtc_tm_to_time64(alrm);
/* clear all the interrupt status bits */
writew(readw(ioaddr + RTC_RTCISR), ioaddr + RTC_RTCISR);
set_alarm_or_time(dev, MXC_RTC_ALARM, time);
-
- return 0;
}
static void mxc_rtc_irq_enable(struct device *dev, unsigned int bit,
*/
static int mxc_rtc_read_time(struct device *dev, struct rtc_time *tm)
{
- u32 val;
+ time64_t val;
/* Avoid roll-over from reading the different registers */
do {
val = get_alarm_or_time(dev, MXC_RTC_TIME);
} while (val != get_alarm_or_time(dev, MXC_RTC_TIME));
- rtc_time_to_tm(val, tm);
+ rtc_time64_to_tm(val, tm);
return 0;
}
/*
* This function sets the internal RTC time based on tm in Gregorian date.
*/
-static int mxc_rtc_set_mmss(struct device *dev, unsigned long time)
+static int mxc_rtc_set_mmss(struct device *dev, time64_t time)
{
struct platform_device *pdev = to_platform_device(dev);
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
if (is_imx1_rtc(pdata)) {
struct rtc_time tm;
- rtc_time_to_tm(time, &tm);
+ rtc_time64_to_tm(time, &tm);
tm.tm_year = 70;
- rtc_tm_to_time(&tm, &time);
+ time = rtc_tm_to_time64(&tm);
}
/* Avoid roll-over from reading the different registers */
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
void __iomem *ioaddr = pdata->ioaddr;
- rtc_time_to_tm(get_alarm_or_time(dev, MXC_RTC_ALARM), &alrm->time);
+ rtc_time64_to_tm(get_alarm_or_time(dev, MXC_RTC_ALARM), &alrm->time);
alrm->pending = ((readw(ioaddr + RTC_RTCISR) & RTC_ALM_BIT)) ? 1 : 0;
return 0;
{
struct platform_device *pdev = to_platform_device(dev);
struct rtc_plat_data *pdata = platform_get_drvdata(pdev);
- int ret;
- ret = rtc_update_alarm(dev, &alrm->time);
- if (ret)
- return ret;
+ rtc_update_alarm(dev, &alrm->time);
memcpy(&pdata->g_rtc_alarm, &alrm->time, sizeof(struct rtc_time));
mxc_rtc_irq_enable(dev, RTC_ALM_BIT, alrm->enabled);
static struct rtc_class_ops mxc_rtc_ops = {
.release = mxc_rtc_release,
.read_time = mxc_rtc_read_time,
- .set_mmss = mxc_rtc_set_mmss,
+ .set_mmss64 = mxc_rtc_set_mmss,
.read_alarm = mxc_rtc_read_alarm,
.set_alarm = mxc_rtc_set_alarm,
.alarm_irq_enable = mxc_rtc_alarm_irq_enable,
static struct s3c_rtc_data const s3c6410_rtc_data = {
.max_user_freq = 32768,
+ .needs_src_clk = true,
.irq_handler = s3c6410_rtc_irq,
.set_freq = s3c6410_rtc_setfreq,
.enable_tick = s3c6410_rtc_enable_tick,
#include <linux/rtc.h>
#include <linux/platform_device.h>
+static int test_mmss64;
+module_param(test_mmss64, int, 0644);
+MODULE_PARM_DESC(test_mmss64, "Test struct rtc_class_ops.set_mmss64().");
+
static struct platform_device *test0 = NULL, *test1 = NULL;
static int test_rtc_read_alarm(struct device *dev,
static int test_rtc_read_time(struct device *dev,
struct rtc_time *tm)
{
- rtc_time_to_tm(get_seconds(), tm);
+ rtc_time64_to_tm(ktime_get_real_seconds(), tm);
+ return 0;
+}
+
+static int test_rtc_set_mmss64(struct device *dev, time64_t secs)
+{
+ dev_info(dev, "%s, secs = %lld\n", __func__, (long long)secs);
return 0;
}
return 0;
}
-static const struct rtc_class_ops test_rtc_ops = {
+static struct rtc_class_ops test_rtc_ops = {
.proc = test_rtc_proc,
.read_time = test_rtc_read_time,
.read_alarm = test_rtc_read_alarm,
int err;
struct rtc_device *rtc;
+ if (test_mmss64) {
+ test_rtc_ops.set_mmss64 = test_rtc_set_mmss64;
+ test_rtc_ops.set_mmss = NULL;
+ }
+
rtc = devm_rtc_device_register(&plat_dev->dev, "test",
&test_rtc_ops, THIS_MODULE);
if (IS_ERR(rtc)) {
* rtc_set_ntp_time - Save NTP synchronized time to the RTC
* @now: Current time of day
*
- * Replacement for the NTP platform function update_persistent_clock
+ * Replacement for the NTP platform function update_persistent_clock64
* that stores time for later retrieval by rtc_hctosys.
*
* Returns 0 on successful RTC update, -ENODEV if a RTC update is not
if (rtc) {
/* rtc_hctosys exclusively uses UTC, so we call set_time here,
* not set_mmss. */
- if (rtc->ops && (rtc->ops->set_time || rtc->ops->set_mmss))
+ if (rtc->ops &&
+ (rtc->ops->set_time ||
+ rtc->ops->set_mmss64 ||
+ rtc->ops->set_mmss))
err = rtc_set_time(rtc, &tm);
rtc_class_close(rtc);
}
* parse input
*/
num_of_segments = 0;
- for (i = 0; ((buf[i] != '\0') && (buf[i] != '\n') && i < count); i++) {
+ for (i = 0; (i < count && (buf[i] != '\0') && (buf[i] != '\n')); i++) {
for (j = i; (buf[j] != ':') &&
(buf[j] != '\0') &&
(buf[j] != '\n') &&
add = 0;
continue;
}
- for (pos = 0; pos <= iter->aob->request.msb_count; pos++) {
+ for (pos = 0; pos < iter->aob->request.msb_count; pos++) {
if (clusters_intersect(req, iter->request[pos]) &&
(rq_data_dir(req) == WRITE ||
rq_data_dir(iter->request[pos]) == WRITE)) {
hba_free:
if (phba->msix_enabled)
pci_disable_msix(phba->pcidev);
- iscsi_host_remove(phba->shost);
pci_dev_put(phba->pcidev);
iscsi_host_free(phba->shost);
+ pci_set_drvdata(pcidev, NULL);
disable_pci:
pci_disable_device(pcidev);
return ret;
};
static struct ata_port_info sata_port_info = {
- .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA,
+ .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA |
+ ATA_FLAG_SAS_HOST,
.pio_mask = ATA_PIO4_ONLY,
.mwdma_mask = ATA_MWDMA2,
.udma_mask = ATA_UDMA6,
};
static struct ata_port_info sata_port_info = {
- .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA | ATA_FLAG_NCQ,
+ .flags = ATA_FLAG_SATA | ATA_FLAG_PIO_DMA | ATA_FLAG_NCQ |
+ ATA_FLAG_SAS_HOST,
.pio_mask = ATA_PIO4,
.mwdma_mask = ATA_MWDMA2,
.udma_mask = ATA_UDMA6,
struct sas_discovery_event *ev = to_sas_discovery_event(work);
struct asd_sas_port *port = ev->port;
struct sas_ha_struct *ha = port->ha;
+ struct domain_device *ddev = port->port_dev;
/* prevent revalidation from finding sata links in recovery */
mutex_lock(&ha->disco_mutex);
SAS_DPRINTK("REVALIDATING DOMAIN on port %d, pid:%d\n", port->id,
task_pid_nr(current));
- if (port->port_dev)
- res = sas_ex_revalidate_domain(port->port_dev);
+ if (ddev && (ddev->dev_type == SAS_FANOUT_EXPANDER_DEVICE ||
+ ddev->dev_type == SAS_EDGE_EXPANDER_DEVICE))
+ res = sas_ex_revalidate_domain(ddev);
SAS_DPRINTK("done REVALIDATING DOMAIN on port %d, pid:%d, res 0x%x\n",
port->id, task_pid_nr(current), res);
/*
* Finally register the new FC Nexus with TCM
*/
- __transport_register_session(se_nacl->se_tpg, se_nacl, se_sess, sess);
+ transport_register_session(se_nacl->se_tpg, se_nacl, se_sess, sess);
return 0;
}
#include <linux/blkdev.h>
#include <linux/delay.h>
#include <linux/jiffies.h>
+#include <asm/unaligned.h>
#include <scsi/scsi.h>
#include <scsi/scsi_cmnd.h>
}
}
EXPORT_SYMBOL(scsi_build_sense_buffer);
+
+/**
+ * scsi_set_sense_information - set the information field in a
+ * formatted sense data buffer
+ * @buf: Where to build sense data
+ * @info: 64-bit information value to be set
+ *
+ **/
+void scsi_set_sense_information(u8 *buf, u64 info)
+{
+ if ((buf[0] & 0x7f) == 0x72) {
+ u8 *ucp, len;
+
+ len = buf[7];
+ ucp = (char *)scsi_sense_desc_find(buf, len + 8, 0);
+ if (!ucp) {
+ buf[7] = len + 0xa;
+ ucp = buf + 8 + len;
+ }
+ ucp[0] = 0;
+ ucp[1] = 0xa;
+ ucp[2] = 0x80; /* Valid bit */
+ ucp[3] = 0;
+ put_unaligned_be64(info, &ucp[4]);
+ } else if ((buf[0] & 0x7f) == 0x70) {
+ buf[0] |= 0x80;
+ put_unaligned_be64(info, &buf[3]);
+ }
+}
+EXPORT_SYMBOL(scsi_set_sense_information);
"rejecting I/O to dead device\n");
ret = BLKPREP_KILL;
break;
- case SDEV_QUIESCE:
case SDEV_BLOCK:
case SDEV_CREATED_BLOCK:
+ ret = BLKPREP_DEFER;
+ break;
+ case SDEV_QUIESCE:
/*
* If the devices is blocked we defer normal commands.
*/
static int ufshcd_config_vreg_load(struct device *dev, struct ufs_vreg *vreg,
int ua)
{
- int ret = 0;
- struct regulator *reg = vreg->reg;
- const char *name = vreg->name;
+ int ret;
- BUG_ON(!vreg);
+ if (!vreg)
+ return 0;
- ret = regulator_set_optimum_mode(reg, ua);
- if (ret >= 0) {
- /*
- * regulator_set_optimum_mode() returns new regulator
- * mode upon success.
- */
- ret = 0;
- } else {
- dev_err(dev, "%s: %s set optimum mode(ua=%d) failed, err=%d\n",
- __func__, name, ua, ret);
+ ret = regulator_set_load(vreg->reg, ua);
+ if (ret < 0) {
+ dev_err(dev, "%s: %s set load (ua=%d) failed, err=%d\n",
+ __func__, vreg->name, ua, ret);
}
return ret;
static inline int ufshcd_config_vreg_lpm(struct ufs_hba *hba,
struct ufs_vreg *vreg)
{
- if (!vreg)
- return 0;
-
return ufshcd_config_vreg_load(hba->dev, vreg, UFS_VREG_LPM_LOAD_UA);
}
static inline int ufshcd_config_vreg_hpm(struct ufs_hba *hba,
struct ufs_vreg *vreg)
{
- if (!vreg)
- return 0;
-
return ufshcd_config_vreg_load(hba->dev, vreg, vreg->max_uA);
}
if (!of_machine_is_compatible("renesas,emev2") &&
!of_machine_is_compatible("renesas,r7s72100") &&
!of_machine_is_compatible("renesas,r8a73a4") &&
+#ifndef CONFIG_PM_GENERIC_DOMAINS_OF
!of_machine_is_compatible("renesas,r8a7740") &&
+#endif
!of_machine_is_compatible("renesas,r8a7778") &&
!of_machine_is_compatible("renesas,r8a7779") &&
!of_machine_is_compatible("renesas,r8a7790") &&
config SPI_CADENCE
tristate "Cadence SPI controller"
- depends on ARM
help
This selects the Cadence SPI controller master driver
- used by Xilinx Zynq.
+ used by Xilinx Zynq and ZynqMP.
config SPI_CLPS711X
tristate "CLPS711X host SPI controller"
config SPI_DW_MID_DMA
bool "DMA support for DW SPI controller on Intel MID platform"
- depends on SPI_DW_PCI && INTEL_MID_DMAC
+ depends on SPI_DW_PCI && DW_DMAC_PCI
config SPI_DW_MMIO
tristate "Memory-mapped io interface driver for DW SPI core"
| SPI_BF(name, value))
/* Register access macros */
+#ifdef CONFIG_AVR32
#define spi_readl(port, reg) \
__raw_readl((port)->regs + SPI_##reg)
#define spi_writel(port, reg, value) \
__raw_writel((value), (port)->regs + SPI_##reg)
-
+#else
+#define spi_readl(port, reg) \
+ readl_relaxed((port)->regs + SPI_##reg)
+#define spi_writel(port, reg, value) \
+ writel_relaxed((value), (port)->regs + SPI_##reg)
+#endif
/* use PIO for small transfers, avoiding DMA setup/teardown overhead and
* cache operations; better heuristics consider wordsize and bitrate.
*/
(unsigned long long)xfer->rx_dma);
}
- /* REVISIT: We're waiting for ENDRX before we start the next
+ /* REVISIT: We're waiting for RXBUFF before we start the next
* transfer because we need to handle some difficult timing
- * issues otherwise. If we wait for ENDTX in one transfer and
- * then starts waiting for ENDRX in the next, it's difficult
- * to tell the difference between the ENDRX interrupt we're
- * actually waiting for and the ENDRX interrupt of the
+ * issues otherwise. If we wait for TXBUFE in one transfer and
+ * then starts waiting for RXBUFF in the next, it's difficult
+ * to tell the difference between the RXBUFF interrupt we're
+ * actually waiting for and the RXBUFF interrupt of the
* previous transfer.
*
* It should be doable, though. Just not now...
*/
- spi_writel(as, IER, SPI_BIT(ENDRX) | SPI_BIT(OVRES));
+ spi_writel(as, IER, SPI_BIT(RXBUFF) | SPI_BIT(OVRES));
spi_writel(as, PTCR, SPI_BIT(TXTEN) | SPI_BIT(RXTEN));
}
*
* Copyright (C) 2012 Chris Boot
* Copyright (C) 2013 Stephen Warren
+ * Copyright (C) 2015 Martin Sperl
*
* This driver is inspired by:
* spi-ath79.c, Copyright (C) 2009-2011 Gabor Juhos <juhosg@openwrt.org>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_irq.h>
+#include <linux/of_gpio.h>
#include <linux/of_device.h>
#include <linux/spi/spi.h>
#define BCM2835_SPI_CS_CS_10 0x00000002
#define BCM2835_SPI_CS_CS_01 0x00000001
-#define BCM2835_SPI_TIMEOUT_MS 30000
-#define BCM2835_SPI_MODE_BITS (SPI_CPOL | SPI_CPHA | SPI_CS_HIGH | SPI_NO_CS)
+#define BCM2835_SPI_POLLING_LIMIT_US 30
+#define BCM2835_SPI_TIMEOUT_MS 30000
+#define BCM2835_SPI_MODE_BITS (SPI_CPOL | SPI_CPHA | SPI_CS_HIGH \
+ | SPI_NO_CS | SPI_3WIRE)
#define DRV_NAME "spi-bcm2835"
void __iomem *regs;
struct clk *clk;
int irq;
- struct completion done;
const u8 *tx_buf;
u8 *rx_buf;
- int len;
+ int tx_len;
+ int rx_len;
};
static inline u32 bcm2835_rd(struct bcm2835_spi *bs, unsigned reg)
writel(val, bs->regs + reg);
}
-static inline void bcm2835_rd_fifo(struct bcm2835_spi *bs, int len)
+static inline void bcm2835_rd_fifo(struct bcm2835_spi *bs)
{
u8 byte;
- while (len--) {
+ while ((bs->rx_len) &&
+ (bcm2835_rd(bs, BCM2835_SPI_CS) & BCM2835_SPI_CS_RXD)) {
byte = bcm2835_rd(bs, BCM2835_SPI_FIFO);
if (bs->rx_buf)
*bs->rx_buf++ = byte;
+ bs->rx_len--;
}
}
-static inline void bcm2835_wr_fifo(struct bcm2835_spi *bs, int len)
+static inline void bcm2835_wr_fifo(struct bcm2835_spi *bs)
{
u8 byte;
- if (len > bs->len)
- len = bs->len;
-
- while (len--) {
+ while ((bs->tx_len) &&
+ (bcm2835_rd(bs, BCM2835_SPI_CS) & BCM2835_SPI_CS_TXD)) {
byte = bs->tx_buf ? *bs->tx_buf++ : 0;
bcm2835_wr(bs, BCM2835_SPI_FIFO, byte);
- bs->len--;
+ bs->tx_len--;
}
}
+static void bcm2835_spi_reset_hw(struct spi_master *master)
+{
+ struct bcm2835_spi *bs = spi_master_get_devdata(master);
+ u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);
+
+ /* Disable SPI interrupts and transfer */
+ cs &= ~(BCM2835_SPI_CS_INTR |
+ BCM2835_SPI_CS_INTD |
+ BCM2835_SPI_CS_TA);
+ /* and reset RX/TX FIFOS */
+ cs |= BCM2835_SPI_CS_CLEAR_RX | BCM2835_SPI_CS_CLEAR_TX;
+
+ /* and reset the SPI_HW */
+ bcm2835_wr(bs, BCM2835_SPI_CS, cs);
+}
+
static irqreturn_t bcm2835_spi_interrupt(int irq, void *dev_id)
{
struct spi_master *master = dev_id;
struct bcm2835_spi *bs = spi_master_get_devdata(master);
- u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);
- /*
- * RXR - RX needs Reading. This means 12 (or more) bytes have been
- * transmitted and hence 12 (or more) bytes have been received.
- *
- * The FIFO is 16-bytes deep. We check for this interrupt to keep the
- * FIFO full; we have a 4-byte-time buffer for IRQ latency. We check
- * this before DONE (TX empty) just in case we delayed processing this
- * interrupt for some reason.
- *
- * We only check for this case if we have more bytes to TX; at the end
- * of the transfer, we ignore this pipelining optimization, and let
- * bcm2835_spi_finish_transfer() drain the RX FIFO.
+ /* Read as many bytes as possible from FIFO */
+ bcm2835_rd_fifo(bs);
+ /* Write as many bytes as possible to FIFO */
+ bcm2835_wr_fifo(bs);
+
+ /* based on flags decide if we can finish the transfer */
+ if (bcm2835_rd(bs, BCM2835_SPI_CS) & BCM2835_SPI_CS_DONE) {
+ /* Transfer complete - reset SPI HW */
+ bcm2835_spi_reset_hw(master);
+ /* wake up the framework */
+ complete(&master->xfer_completion);
+ }
+
+ return IRQ_HANDLED;
+}
+
+static int bcm2835_spi_transfer_one_poll(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *tfr,
+ u32 cs,
+ unsigned long xfer_time_us)
+{
+ struct bcm2835_spi *bs = spi_master_get_devdata(master);
+ unsigned long timeout = jiffies +
+ max(4 * xfer_time_us * HZ / 1000000, 2uL);
+
+ /* enable HW block without interrupts */
+ bcm2835_wr(bs, BCM2835_SPI_CS, cs | BCM2835_SPI_CS_TA);
+
+ /* set timeout to 4x the expected time, or 2 jiffies */
+ /* loop until finished the transfer */
+ while (bs->rx_len) {
+ /* read from fifo as much as possible */
+ bcm2835_rd_fifo(bs);
+ /* fill in tx fifo as much as possible */
+ bcm2835_wr_fifo(bs);
+ /* if we still expect some data after the read,
+ * check for a possible timeout
+ */
+ if (bs->rx_len && time_after(jiffies, timeout)) {
+ /* Transfer complete - reset SPI HW */
+ bcm2835_spi_reset_hw(master);
+ /* and return timeout */
+ return -ETIMEDOUT;
+ }
+ }
+
+ /* Transfer complete - reset SPI HW */
+ bcm2835_spi_reset_hw(master);
+ /* and return without waiting for completion */
+ return 0;
+}
+
+static int bcm2835_spi_transfer_one_irq(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *tfr,
+ u32 cs)
+{
+ struct bcm2835_spi *bs = spi_master_get_devdata(master);
+
+ /* fill in fifo if we have gpio-cs
+ * note that there have been rare events where the native-CS
+ * flapped for <1us which may change the behaviour
+ * with gpio-cs this does not happen, so it is implemented
+ * only for this case
*/
- if (bs->len && (cs & BCM2835_SPI_CS_RXR)) {
- /* Read 12 bytes of data */
- bcm2835_rd_fifo(bs, 12);
-
- /* Write up to 12 bytes */
- bcm2835_wr_fifo(bs, 12);
-
- /*
- * We must have written something to the TX FIFO due to the
- * bs->len check above, so cannot be DONE. Hence, return
- * early. Note that DONE could also be set if we serviced an
- * RXR interrupt really late.
+ if (gpio_is_valid(spi->cs_gpio)) {
+ /* enable HW block, but without interrupts enabled
+ * this would triggern an immediate interrupt
*/
- return IRQ_HANDLED;
+ bcm2835_wr(bs, BCM2835_SPI_CS,
+ cs | BCM2835_SPI_CS_TA);
+ /* fill in tx fifo as much as possible */
+ bcm2835_wr_fifo(bs);
}
/*
- * DONE - TX empty. This occurs when we first enable the transfer
- * since we do not pre-fill the TX FIFO. At any other time, given that
- * we refill the TX FIFO above based on RXR, and hence ignore DONE if
- * RXR is set, DONE really does mean end-of-transfer.
+ * Enable the HW block. This will immediately trigger a DONE (TX
+ * empty) interrupt, upon which we will fill the TX FIFO with the
+ * first TX bytes. Pre-filling the TX FIFO here to avoid the
+ * interrupt doesn't work:-(
*/
- if (cs & BCM2835_SPI_CS_DONE) {
- if (bs->len) { /* First interrupt in a transfer */
- bcm2835_wr_fifo(bs, 16);
- } else { /* Transfer complete */
- /* Disable SPI interrupts */
- cs &= ~(BCM2835_SPI_CS_INTR | BCM2835_SPI_CS_INTD);
- bcm2835_wr(bs, BCM2835_SPI_CS, cs);
-
- /*
- * Wake up bcm2835_spi_transfer_one(), which will call
- * bcm2835_spi_finish_transfer(), to drain the RX FIFO.
- */
- complete(&bs->done);
- }
-
- return IRQ_HANDLED;
- }
+ cs |= BCM2835_SPI_CS_INTR | BCM2835_SPI_CS_INTD | BCM2835_SPI_CS_TA;
+ bcm2835_wr(bs, BCM2835_SPI_CS, cs);
- return IRQ_NONE;
+ /* signal that we need to wait for completion */
+ return 1;
}
-static int bcm2835_spi_start_transfer(struct spi_device *spi,
- struct spi_transfer *tfr)
+static int bcm2835_spi_transfer_one(struct spi_master *master,
+ struct spi_device *spi,
+ struct spi_transfer *tfr)
{
- struct bcm2835_spi *bs = spi_master_get_devdata(spi->master);
+ struct bcm2835_spi *bs = spi_master_get_devdata(master);
unsigned long spi_hz, clk_hz, cdiv;
- u32 cs = BCM2835_SPI_CS_INTR | BCM2835_SPI_CS_INTD | BCM2835_SPI_CS_TA;
+ unsigned long spi_used_hz, xfer_time_us;
+ u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);
+ /* set clock */
spi_hz = tfr->speed_hz;
clk_hz = clk_get_rate(bs->clk);
if (spi_hz >= clk_hz / 2) {
cdiv = 2; /* clk_hz/2 is the fastest we can go */
} else if (spi_hz) {
- /* CDIV must be a power of two */
- cdiv = roundup_pow_of_two(DIV_ROUND_UP(clk_hz, spi_hz));
+ /* CDIV must be a multiple of two */
+ cdiv = DIV_ROUND_UP(clk_hz, spi_hz);
+ cdiv += (cdiv % 2);
if (cdiv >= 65536)
cdiv = 0; /* 0 is the slowest we can go */
- } else
+ } else {
cdiv = 0; /* 0 is the slowest we can go */
+ }
+ spi_used_hz = cdiv ? (clk_hz / cdiv) : (clk_hz / 65536);
+ bcm2835_wr(bs, BCM2835_SPI_CLK, cdiv);
+ /* handle all the modes */
+ if ((spi->mode & SPI_3WIRE) && (tfr->rx_buf))
+ cs |= BCM2835_SPI_CS_REN;
if (spi->mode & SPI_CPOL)
cs |= BCM2835_SPI_CS_CPOL;
if (spi->mode & SPI_CPHA)
cs |= BCM2835_SPI_CS_CPHA;
- if (!(spi->mode & SPI_NO_CS)) {
- if (spi->mode & SPI_CS_HIGH) {
- cs |= BCM2835_SPI_CS_CSPOL;
- cs |= BCM2835_SPI_CS_CSPOL0 << spi->chip_select;
- }
-
- cs |= spi->chip_select;
- }
+ /* for gpio_cs set dummy CS so that no HW-CS get changed
+ * we can not run this in bcm2835_spi_set_cs, as it does
+ * not get called for cs_gpio cases, so we need to do it here
+ */
+ if (gpio_is_valid(spi->cs_gpio) || (spi->mode & SPI_NO_CS))
+ cs |= BCM2835_SPI_CS_CS_10 | BCM2835_SPI_CS_CS_01;
- reinit_completion(&bs->done);
+ /* set transmit buffers and length */
bs->tx_buf = tfr->tx_buf;
bs->rx_buf = tfr->rx_buf;
- bs->len = tfr->len;
+ bs->tx_len = tfr->len;
+ bs->rx_len = tfr->len;
- bcm2835_wr(bs, BCM2835_SPI_CLK, cdiv);
- /*
- * Enable the HW block. This will immediately trigger a DONE (TX
- * empty) interrupt, upon which we will fill the TX FIFO with the
- * first TX bytes. Pre-filling the TX FIFO here to avoid the
- * interrupt doesn't work:-(
- */
- bcm2835_wr(bs, BCM2835_SPI_CS, cs);
+ /* calculate the estimated time in us the transfer runs */
+ xfer_time_us = tfr->len
+ * 9 /* clocks/byte - SPI-HW waits 1 clock after each byte */
+ * 1000000 / spi_used_hz;
- return 0;
+ /* for short requests run polling*/
+ if (xfer_time_us <= BCM2835_SPI_POLLING_LIMIT_US)
+ return bcm2835_spi_transfer_one_poll(master, spi, tfr,
+ cs, xfer_time_us);
+
+ return bcm2835_spi_transfer_one_irq(master, spi, tfr, cs);
}
-static int bcm2835_spi_finish_transfer(struct spi_device *spi,
- struct spi_transfer *tfr, bool cs_change)
+static void bcm2835_spi_handle_err(struct spi_master *master,
+ struct spi_message *msg)
{
- struct bcm2835_spi *bs = spi_master_get_devdata(spi->master);
- u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);
+ bcm2835_spi_reset_hw(master);
+}
+
+static void bcm2835_spi_set_cs(struct spi_device *spi, bool gpio_level)
+{
+ /*
+ * we can assume that we are "native" as per spi_set_cs
+ * calling us ONLY when cs_gpio is not set
+ * we can also assume that we are CS < 3 as per bcm2835_spi_setup
+ * we would not get called because of error handling there.
+ * the level passed is the electrical level not enabled/disabled
+ * so it has to get translated back to enable/disable
+ * see spi_set_cs in spi.c for the implementation
+ */
- /* Drain RX FIFO */
- while (cs & BCM2835_SPI_CS_RXD) {
- bcm2835_rd_fifo(bs, 1);
- cs = bcm2835_rd(bs, BCM2835_SPI_CS);
+ struct spi_master *master = spi->master;
+ struct bcm2835_spi *bs = spi_master_get_devdata(master);
+ u32 cs = bcm2835_rd(bs, BCM2835_SPI_CS);
+ bool enable;
+
+ /* calculate the enable flag from the passed gpio_level */
+ enable = (spi->mode & SPI_CS_HIGH) ? gpio_level : !gpio_level;
+
+ /* set flags for "reverse" polarity in the registers */
+ if (spi->mode & SPI_CS_HIGH) {
+ /* set the correct CS-bits */
+ cs |= BCM2835_SPI_CS_CSPOL;
+ cs |= BCM2835_SPI_CS_CSPOL0 << spi->chip_select;
+ } else {
+ /* clean the CS-bits */
+ cs &= ~BCM2835_SPI_CS_CSPOL;
+ cs &= ~(BCM2835_SPI_CS_CSPOL0 << spi->chip_select);
}
- if (tfr->delay_usecs)
- udelay(tfr->delay_usecs);
+ /* select the correct chip_select depending on disabled/enabled */
+ if (enable) {
+ /* set cs correctly */
+ if (spi->mode & SPI_NO_CS) {
+ /* use the "undefined" chip-select */
+ cs |= BCM2835_SPI_CS_CS_10 | BCM2835_SPI_CS_CS_01;
+ } else {
+ /* set the chip select */
+ cs &= ~(BCM2835_SPI_CS_CS_10 | BCM2835_SPI_CS_CS_01);
+ cs |= spi->chip_select;
+ }
+ } else {
+ /* disable CSPOL which puts HW-CS into deselected state */
+ cs &= ~BCM2835_SPI_CS_CSPOL;
+ /* use the "undefined" chip-select as precaution */
+ cs |= BCM2835_SPI_CS_CS_10 | BCM2835_SPI_CS_CS_01;
+ }
- if (cs_change)
- /* Clear TA flag */
- bcm2835_wr(bs, BCM2835_SPI_CS, cs & ~BCM2835_SPI_CS_TA);
+ /* finally set the calculated flags in SPI_CS */
+ bcm2835_wr(bs, BCM2835_SPI_CS, cs);
+}
- return 0;
+static int chip_match_name(struct gpio_chip *chip, void *data)
+{
+ return !strcmp(chip->label, data);
}
-static int bcm2835_spi_transfer_one(struct spi_master *master,
- struct spi_message *mesg)
+static int bcm2835_spi_setup(struct spi_device *spi)
{
- struct bcm2835_spi *bs = spi_master_get_devdata(master);
- struct spi_transfer *tfr;
- struct spi_device *spi = mesg->spi;
- int err = 0;
- unsigned int timeout;
- bool cs_change;
-
- list_for_each_entry(tfr, &mesg->transfers, transfer_list) {
- err = bcm2835_spi_start_transfer(spi, tfr);
- if (err)
- goto out;
-
- timeout = wait_for_completion_timeout(&bs->done,
- msecs_to_jiffies(BCM2835_SPI_TIMEOUT_MS));
- if (!timeout) {
- err = -ETIMEDOUT;
- goto out;
- }
+ int err;
+ struct gpio_chip *chip;
+ /*
+ * sanity checking the native-chipselects
+ */
+ if (spi->mode & SPI_NO_CS)
+ return 0;
+ if (gpio_is_valid(spi->cs_gpio))
+ return 0;
+ if (spi->chip_select > 1) {
+ /* error in the case of native CS requested with CS > 1
+ * officially there is a CS2, but it is not documented
+ * which GPIO is connected with that...
+ */
+ dev_err(&spi->dev,
+ "setup: only two native chip-selects are supported\n");
+ return -EINVAL;
+ }
+ /* now translate native cs to GPIO */
- cs_change = tfr->cs_change ||
- list_is_last(&tfr->transfer_list, &mesg->transfers);
+ /* get the gpio chip for the base */
+ chip = gpiochip_find("pinctrl-bcm2835", chip_match_name);
+ if (!chip)
+ return 0;
- err = bcm2835_spi_finish_transfer(spi, tfr, cs_change);
- if (err)
- goto out;
+ /* and calculate the real CS */
+ spi->cs_gpio = chip->base + 8 - spi->chip_select;
- mesg->actual_length += (tfr->len - bs->len);
- }
+ /* and set up the "mode" and level */
+ dev_info(&spi->dev, "setting up native-CS%i as GPIO %i\n",
+ spi->chip_select, spi->cs_gpio);
-out:
- /* Clear FIFOs, and disable the HW block */
- bcm2835_wr(bs, BCM2835_SPI_CS,
- BCM2835_SPI_CS_CLEAR_RX | BCM2835_SPI_CS_CLEAR_TX);
- mesg->status = err;
- spi_finalize_current_message(master);
+ /* set up GPIO as output and pull to the correct level */
+ err = gpio_direction_output(spi->cs_gpio,
+ (spi->mode & SPI_CS_HIGH) ? 0 : 1);
+ if (err) {
+ dev_err(&spi->dev,
+ "could not set CS%i gpio %i as output: %i",
+ spi->chip_select, spi->cs_gpio, err);
+ return err;
+ }
+ /* the implementation of pinctrl-bcm2835 currently does not
+ * set the GPIO value when using gpio_direction_output
+ * so we are setting it here explicitly
+ */
+ gpio_set_value(spi->cs_gpio, (spi->mode & SPI_CS_HIGH) ? 0 : 1);
return 0;
}
master->mode_bits = BCM2835_SPI_MODE_BITS;
master->bits_per_word_mask = SPI_BPW_MASK(8);
master->num_chipselect = 3;
- master->transfer_one_message = bcm2835_spi_transfer_one;
+ master->setup = bcm2835_spi_setup;
+ master->set_cs = bcm2835_spi_set_cs;
+ master->transfer_one = bcm2835_spi_transfer_one;
+ master->handle_err = bcm2835_spi_handle_err;
master->dev.of_node = pdev->dev.of_node;
bs = spi_master_get_devdata(master);
- init_completion(&bs->done);
-
res = platform_get_resource(pdev, IORESOURCE_MEM, 0);
bs->regs = devm_ioremap_resource(&pdev->dev, res);
if (IS_ERR(bs->regs)) {
clk_prepare_enable(bs->clk);
err = devm_request_irq(&pdev->dev, bs->irq, bcm2835_spi_interrupt, 0,
- dev_name(&pdev->dev), master);
+ dev_name(&pdev->dev), master);
if (err) {
dev_err(&pdev->dev, "could not request IRQ: %d\n", err);
goto out_clk_disable;
}
- /* initialise the hardware */
+ /* initialise the hardware with the default polarities */
bcm2835_wr(bs, BCM2835_SPI_CS,
BCM2835_SPI_CS_CLEAR_RX | BCM2835_SPI_CS_CLEAR_TX);
u32 tmp;
/* SPE bit has to be 0 before we read MSPI STATUS */
- deadline = jiffies + BCM53XXSPI_SPE_TIMEOUT_MS * HZ / 1000;
+ deadline = jiffies + msecs_to_jiffies(BCM53XXSPI_SPE_TIMEOUT_MS);
do {
tmp = bcm53xxspi_read(b53spi, B53SPI_MSPI_SPCR2);
if (!(tmp & B53SPI_MSPI_SPCR2_SPE))
goto spi_timeout;
/* Check status */
- deadline = jiffies + timeout_ms * HZ / 1000;
+ deadline = jiffies + msecs_to_jiffies(timeout_ms);
do {
tmp = bcm53xxspi_read(b53spi, B53SPI_MSPI_MSPI_STATUS);
if (tmp & B53SPI_MSPI_MSPI_STATUS_SPIF) {
struct spi_transfer *previous = NULL;
struct bfin_spi_slave_data *chip = NULL;
unsigned int bits_per_word;
- u16 cr, cr_width, dma_width, dma_config;
+ u16 cr, cr_width = 0, dma_width, dma_config;
u32 tranf_success = 1;
u8 full_duplex = 0;
} else if (bits_per_word == 8) {
drv_data->n_bytes = bits_per_word/8;
drv_data->len = transfer->len;
- cr_width = 0;
drv_data->ops = &bfin_bfin_spi_transfer_ops_u8;
}
cr = bfin_read(&drv_data->regs->ctl) & ~(BIT_CTL_TIMOD | BIT_CTL_WORDSIZE);
{
/* if (cpol == 0) this is SPI_MODE_0; else this is SPI_MODE_2 */
+ bool oldbit = !(word & 1);
/* clock starts at inactive polarity */
for (word <<= (32 - bits); likely(bits); bits--) {
/* setup MSB (to slave) on trailing edge */
- if ((flags & SPI_MASTER_NO_TX) == 0)
- setmosi(spi, word & (1 << 31));
+ if ((flags & SPI_MASTER_NO_TX) == 0) {
+ if ((word & (1 << 31)) != oldbit) {
+ setmosi(spi, word & (1 << 31));
+ oldbit = word & (1 << 31);
+ }
+ }
spidelay(nsecs); /* T(setup) */
setsck(spi, !cpol);
{
/* if (cpol == 0) this is SPI_MODE_1; else this is SPI_MODE_3 */
+ bool oldbit = !(word & (1 << 31));
/* clock starts at inactive polarity */
for (word <<= (32 - bits); likely(bits); bits--) {
/* setup MSB (to slave) on leading edge */
setsck(spi, !cpol);
- if ((flags & SPI_MASTER_NO_TX) == 0)
- setmosi(spi, word & (1 << 31));
+ if ((flags & SPI_MASTER_NO_TX) == 0) {
+ if ((word & (1 << 31)) != oldbit) {
+ setmosi(spi, word & (1 << 31));
+ oldbit = word & (1 << 31);
+ }
+ }
spidelay(nsecs); /* T(setup) */
setsck(spi, cpol);
#include "spi-dw.h"
#ifdef CONFIG_SPI_DW_MID_DMA
-#include <linux/intel_mid_dma.h>
#include <linux/pci.h>
+#include <linux/platform_data/dma-dw.h>
#define RX_BUSY 0
#define TX_BUSY 1
-struct mid_dma {
- struct intel_mid_dma_slave dmas_tx;
- struct intel_mid_dma_slave dmas_rx;
-};
+static struct dw_dma_slave mid_dma_tx = { .dst_id = 1 };
+static struct dw_dma_slave mid_dma_rx = { .src_id = 0 };
static bool mid_spi_dma_chan_filter(struct dma_chan *chan, void *param)
{
- struct dw_spi *dws = param;
+ struct dw_dma_slave *s = param;
+
+ if (s->dma_dev != chan->device->dev)
+ return false;
- return dws->dma_dev == chan->device->dev;
+ chan->private = s;
+ return true;
}
static int mid_spi_dma_init(struct dw_spi *dws)
{
- struct mid_dma *dw_dma = dws->dma_priv;
struct pci_dev *dma_dev;
- struct intel_mid_dma_slave *rxs, *txs;
+ struct dw_dma_slave *tx = dws->dma_tx;
+ struct dw_dma_slave *rx = dws->dma_rx;
dma_cap_mask_t mask;
/*
if (!dma_dev)
return -ENODEV;
- dws->dma_dev = &dma_dev->dev;
-
dma_cap_zero(mask);
dma_cap_set(DMA_SLAVE, mask);
/* 1. Init rx channel */
- dws->rxchan = dma_request_channel(mask, mid_spi_dma_chan_filter, dws);
+ rx->dma_dev = &dma_dev->dev;
+ dws->rxchan = dma_request_channel(mask, mid_spi_dma_chan_filter, rx);
if (!dws->rxchan)
goto err_exit;
- rxs = &dw_dma->dmas_rx;
- rxs->hs_mode = LNW_DMA_HW_HS;
- rxs->cfg_mode = LNW_DMA_PER_TO_MEM;
- dws->rxchan->private = rxs;
+ dws->master->dma_rx = dws->rxchan;
/* 2. Init tx channel */
- dws->txchan = dma_request_channel(mask, mid_spi_dma_chan_filter, dws);
+ tx->dma_dev = &dma_dev->dev;
+ dws->txchan = dma_request_channel(mask, mid_spi_dma_chan_filter, tx);
if (!dws->txchan)
goto free_rxchan;
- txs = &dw_dma->dmas_tx;
- txs->hs_mode = LNW_DMA_HW_HS;
- txs->cfg_mode = LNW_DMA_MEM_TO_PER;
- dws->txchan->private = txs;
+ dws->master->dma_tx = dws->txchan;
dws->dma_inited = 1;
return 0;
dma_release_channel(dws->rxchan);
}
+static irqreturn_t dma_transfer(struct dw_spi *dws)
+{
+ u16 irq_status = dw_readl(dws, DW_SPI_ISR);
+
+ if (!irq_status)
+ return IRQ_NONE;
+
+ dw_readl(dws, DW_SPI_ICR);
+ spi_reset_chip(dws);
+
+ dev_err(&dws->master->dev, "%s: FIFO overrun/underrun\n", __func__);
+ dws->master->cur_msg->status = -EIO;
+ spi_finalize_current_transfer(dws->master);
+ return IRQ_HANDLED;
+}
+
+static bool mid_spi_can_dma(struct spi_master *master, struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct dw_spi *dws = spi_master_get_devdata(master);
+
+ if (!dws->dma_inited)
+ return false;
+
+ return xfer->len > dws->fifo_len;
+}
+
+static enum dma_slave_buswidth convert_dma_width(u32 dma_width) {
+ if (dma_width == 1)
+ return DMA_SLAVE_BUSWIDTH_1_BYTE;
+ else if (dma_width == 2)
+ return DMA_SLAVE_BUSWIDTH_2_BYTES;
+
+ return DMA_SLAVE_BUSWIDTH_UNDEFINED;
+}
+
/*
* dws->dma_chan_busy is set before the dma transfer starts, callback for tx
* channel will clear a corresponding bit.
{
struct dw_spi *dws = arg;
- if (test_and_clear_bit(TX_BUSY, &dws->dma_chan_busy) & BIT(RX_BUSY))
+ clear_bit(TX_BUSY, &dws->dma_chan_busy);
+ if (test_bit(RX_BUSY, &dws->dma_chan_busy))
return;
- dw_spi_xfer_done(dws);
+ spi_finalize_current_transfer(dws->master);
}
-static struct dma_async_tx_descriptor *dw_spi_dma_prepare_tx(struct dw_spi *dws)
+static struct dma_async_tx_descriptor *dw_spi_dma_prepare_tx(struct dw_spi *dws,
+ struct spi_transfer *xfer)
{
struct dma_slave_config txconf;
struct dma_async_tx_descriptor *txdesc;
- if (!dws->tx_dma)
+ if (!xfer->tx_buf)
return NULL;
txconf.direction = DMA_MEM_TO_DEV;
txconf.dst_addr = dws->dma_addr;
- txconf.dst_maxburst = LNW_DMA_MSIZE_16;
+ txconf.dst_maxburst = 16;
txconf.src_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
- txconf.dst_addr_width = dws->dma_width;
+ txconf.dst_addr_width = convert_dma_width(dws->dma_width);
txconf.device_fc = false;
dmaengine_slave_config(dws->txchan, &txconf);
- memset(&dws->tx_sgl, 0, sizeof(dws->tx_sgl));
- dws->tx_sgl.dma_address = dws->tx_dma;
- dws->tx_sgl.length = dws->len;
-
txdesc = dmaengine_prep_slave_sg(dws->txchan,
- &dws->tx_sgl,
- 1,
+ xfer->tx_sg.sgl,
+ xfer->tx_sg.nents,
DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!txdesc)
+ return NULL;
+
txdesc->callback = dw_spi_dma_tx_done;
txdesc->callback_param = dws;
{
struct dw_spi *dws = arg;
- if (test_and_clear_bit(RX_BUSY, &dws->dma_chan_busy) & BIT(TX_BUSY))
+ clear_bit(RX_BUSY, &dws->dma_chan_busy);
+ if (test_bit(TX_BUSY, &dws->dma_chan_busy))
return;
- dw_spi_xfer_done(dws);
+ spi_finalize_current_transfer(dws->master);
}
-static struct dma_async_tx_descriptor *dw_spi_dma_prepare_rx(struct dw_spi *dws)
+static struct dma_async_tx_descriptor *dw_spi_dma_prepare_rx(struct dw_spi *dws,
+ struct spi_transfer *xfer)
{
struct dma_slave_config rxconf;
struct dma_async_tx_descriptor *rxdesc;
- if (!dws->rx_dma)
+ if (!xfer->rx_buf)
return NULL;
rxconf.direction = DMA_DEV_TO_MEM;
rxconf.src_addr = dws->dma_addr;
- rxconf.src_maxburst = LNW_DMA_MSIZE_16;
+ rxconf.src_maxburst = 16;
rxconf.dst_addr_width = DMA_SLAVE_BUSWIDTH_4_BYTES;
- rxconf.src_addr_width = dws->dma_width;
+ rxconf.src_addr_width = convert_dma_width(dws->dma_width);
rxconf.device_fc = false;
dmaengine_slave_config(dws->rxchan, &rxconf);
- memset(&dws->rx_sgl, 0, sizeof(dws->rx_sgl));
- dws->rx_sgl.dma_address = dws->rx_dma;
- dws->rx_sgl.length = dws->len;
-
rxdesc = dmaengine_prep_slave_sg(dws->rxchan,
- &dws->rx_sgl,
- 1,
+ xfer->rx_sg.sgl,
+ xfer->rx_sg.nents,
DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
+ if (!rxdesc)
+ return NULL;
+
rxdesc->callback = dw_spi_dma_rx_done;
rxdesc->callback_param = dws;
return rxdesc;
}
-static void dw_spi_dma_setup(struct dw_spi *dws)
+static int mid_spi_dma_setup(struct dw_spi *dws, struct spi_transfer *xfer)
{
u16 dma_ctrl = 0;
- spi_enable_chip(dws, 0);
-
- dw_writew(dws, DW_SPI_DMARDLR, 0xf);
- dw_writew(dws, DW_SPI_DMATDLR, 0x10);
+ dw_writel(dws, DW_SPI_DMARDLR, 0xf);
+ dw_writel(dws, DW_SPI_DMATDLR, 0x10);
- if (dws->tx_dma)
+ if (xfer->tx_buf)
dma_ctrl |= SPI_DMA_TDMAE;
- if (dws->rx_dma)
+ if (xfer->rx_buf)
dma_ctrl |= SPI_DMA_RDMAE;
- dw_writew(dws, DW_SPI_DMACR, dma_ctrl);
+ dw_writel(dws, DW_SPI_DMACR, dma_ctrl);
+
+ /* Set the interrupt mask */
+ spi_umask_intr(dws, SPI_INT_TXOI | SPI_INT_RXUI | SPI_INT_RXOI);
+
+ dws->transfer_handler = dma_transfer;
- spi_enable_chip(dws, 1);
+ return 0;
}
-static int mid_spi_dma_transfer(struct dw_spi *dws, int cs_change)
+static int mid_spi_dma_transfer(struct dw_spi *dws, struct spi_transfer *xfer)
{
struct dma_async_tx_descriptor *txdesc, *rxdesc;
- /* 1. setup DMA related registers */
- if (cs_change)
- dw_spi_dma_setup(dws);
-
- /* 2. Prepare the TX dma transfer */
- txdesc = dw_spi_dma_prepare_tx(dws);
+ /* Prepare the TX dma transfer */
+ txdesc = dw_spi_dma_prepare_tx(dws, xfer);
- /* 3. Prepare the RX dma transfer */
- rxdesc = dw_spi_dma_prepare_rx(dws);
+ /* Prepare the RX dma transfer */
+ rxdesc = dw_spi_dma_prepare_rx(dws, xfer);
/* rx must be started before tx due to spi instinct */
if (rxdesc) {
return 0;
}
+static void mid_spi_dma_stop(struct dw_spi *dws)
+{
+ if (test_bit(TX_BUSY, &dws->dma_chan_busy)) {
+ dmaengine_terminate_all(dws->txchan);
+ clear_bit(TX_BUSY, &dws->dma_chan_busy);
+ }
+ if (test_bit(RX_BUSY, &dws->dma_chan_busy)) {
+ dmaengine_terminate_all(dws->rxchan);
+ clear_bit(RX_BUSY, &dws->dma_chan_busy);
+ }
+}
+
static struct dw_spi_dma_ops mid_dma_ops = {
.dma_init = mid_spi_dma_init,
.dma_exit = mid_spi_dma_exit,
+ .dma_setup = mid_spi_dma_setup,
+ .can_dma = mid_spi_can_dma,
.dma_transfer = mid_spi_dma_transfer,
+ .dma_stop = mid_spi_dma_stop,
};
#endif
iounmap(clk_reg);
#ifdef CONFIG_SPI_DW_MID_DMA
- dws->dma_priv = kzalloc(sizeof(struct mid_dma), GFP_KERNEL);
- if (!dws->dma_priv)
- return -ENOMEM;
+ dws->dma_tx = &mid_dma_tx;
+ dws->dma_rx = &mid_dma_rx;
dws->dma_ops = &mid_dma_ops;
#endif
return 0;
static struct spi_pci_desc spi_pci_mid_desc_1 = {
.setup = dw_spi_mid_init,
- .num_cs = 32,
+ .num_cs = 5,
.bus_num = 0,
};
static struct spi_pci_desc spi_pci_mid_desc_2 = {
.setup = dw_spi_mid_init,
- .num_cs = 4,
+ .num_cs = 2,
.bus_num = 1,
};
#include <linux/debugfs.h>
#endif
-#define START_STATE ((void *)0)
-#define RUNNING_STATE ((void *)1)
-#define DONE_STATE ((void *)2)
-#define ERROR_STATE ((void *)-1)
-
/* Slave spi_dev related */
struct chip_data {
u16 cr0;
}
#endif /* CONFIG_DEBUG_FS */
+static void dw_spi_set_cs(struct spi_device *spi, bool enable)
+{
+ struct dw_spi *dws = spi_master_get_devdata(spi->master);
+ struct chip_data *chip = spi_get_ctldata(spi);
+
+ /* Chip select logic is inverted from spi_set_cs() */
+ if (chip && chip->cs_control)
+ chip->cs_control(!enable);
+
+ if (!enable)
+ dw_writel(dws, DW_SPI_SER, BIT(spi->chip_select));
+}
+
/* Return the max entries we can fill into tx fifo */
static inline u32 tx_max(struct dw_spi *dws)
{
u32 tx_left, tx_room, rxtx_gap;
tx_left = (dws->tx_end - dws->tx) / dws->n_bytes;
- tx_room = dws->fifo_len - dw_readw(dws, DW_SPI_TXFLR);
+ tx_room = dws->fifo_len - dw_readl(dws, DW_SPI_TXFLR);
/*
* Another concern is about the tx/rx mismatch, we
{
u32 rx_left = (dws->rx_end - dws->rx) / dws->n_bytes;
- return min_t(u32, rx_left, dw_readw(dws, DW_SPI_RXFLR));
+ return min_t(u32, rx_left, dw_readl(dws, DW_SPI_RXFLR));
}
static void dw_writer(struct dw_spi *dws)
else
txw = *(u16 *)(dws->tx);
}
- dw_writew(dws, DW_SPI_DR, txw);
+ dw_writel(dws, DW_SPI_DR, txw);
dws->tx += dws->n_bytes;
}
}
u16 rxw;
while (max--) {
- rxw = dw_readw(dws, DW_SPI_DR);
+ rxw = dw_readl(dws, DW_SPI_DR);
/* Care rx only if the transfer's original "rx" is not null */
if (dws->rx_end - dws->len) {
if (dws->n_bytes == 1)
}
}
-static void *next_transfer(struct dw_spi *dws)
-{
- struct spi_message *msg = dws->cur_msg;
- struct spi_transfer *trans = dws->cur_transfer;
-
- /* Move to next transfer */
- if (trans->transfer_list.next != &msg->transfers) {
- dws->cur_transfer =
- list_entry(trans->transfer_list.next,
- struct spi_transfer,
- transfer_list);
- return RUNNING_STATE;
- }
-
- return DONE_STATE;
-}
-
-/*
- * Note: first step is the protocol driver prepares
- * a dma-capable memory, and this func just need translate
- * the virt addr to physical
- */
-static int map_dma_buffers(struct dw_spi *dws)
-{
- if (!dws->cur_msg->is_dma_mapped
- || !dws->dma_inited
- || !dws->cur_chip->enable_dma
- || !dws->dma_ops)
- return 0;
-
- if (dws->cur_transfer->tx_dma)
- dws->tx_dma = dws->cur_transfer->tx_dma;
-
- if (dws->cur_transfer->rx_dma)
- dws->rx_dma = dws->cur_transfer->rx_dma;
-
- return 1;
-}
-
-/* Caller already set message->status; dma and pio irqs are blocked */
-static void giveback(struct dw_spi *dws)
-{
- struct spi_transfer *last_transfer;
- struct spi_message *msg;
-
- msg = dws->cur_msg;
- dws->cur_msg = NULL;
- dws->cur_transfer = NULL;
- dws->prev_chip = dws->cur_chip;
- dws->cur_chip = NULL;
- dws->dma_mapped = 0;
-
- last_transfer = list_last_entry(&msg->transfers, struct spi_transfer,
- transfer_list);
-
- if (!last_transfer->cs_change)
- spi_chip_sel(dws, msg->spi, 0);
-
- spi_finalize_current_message(dws->master);
-}
-
static void int_error_stop(struct dw_spi *dws, const char *msg)
{
- /* Stop the hw */
- spi_enable_chip(dws, 0);
+ spi_reset_chip(dws);
dev_err(&dws->master->dev, "%s\n", msg);
- dws->cur_msg->state = ERROR_STATE;
- tasklet_schedule(&dws->pump_transfers);
+ dws->master->cur_msg->status = -EIO;
+ spi_finalize_current_transfer(dws->master);
}
-void dw_spi_xfer_done(struct dw_spi *dws)
-{
- /* Update total byte transferred return count actual bytes read */
- dws->cur_msg->actual_length += dws->len;
-
- /* Move to next transfer */
- dws->cur_msg->state = next_transfer(dws);
-
- /* Handle end of message */
- if (dws->cur_msg->state == DONE_STATE) {
- dws->cur_msg->status = 0;
- giveback(dws);
- } else
- tasklet_schedule(&dws->pump_transfers);
-}
-EXPORT_SYMBOL_GPL(dw_spi_xfer_done);
-
static irqreturn_t interrupt_transfer(struct dw_spi *dws)
{
- u16 irq_status = dw_readw(dws, DW_SPI_ISR);
+ u16 irq_status = dw_readl(dws, DW_SPI_ISR);
/* Error handling */
if (irq_status & (SPI_INT_TXOI | SPI_INT_RXOI | SPI_INT_RXUI)) {
- dw_readw(dws, DW_SPI_TXOICR);
- dw_readw(dws, DW_SPI_RXOICR);
- dw_readw(dws, DW_SPI_RXUICR);
+ dw_readl(dws, DW_SPI_ICR);
int_error_stop(dws, "interrupt_transfer: fifo overrun/underrun");
return IRQ_HANDLED;
}
dw_reader(dws);
if (dws->rx_end == dws->rx) {
spi_mask_intr(dws, SPI_INT_TXEI);
- dw_spi_xfer_done(dws);
+ spi_finalize_current_transfer(dws->master);
return IRQ_HANDLED;
}
if (irq_status & SPI_INT_TXEI) {
static irqreturn_t dw_spi_irq(int irq, void *dev_id)
{
- struct dw_spi *dws = dev_id;
- u16 irq_status = dw_readw(dws, DW_SPI_ISR) & 0x3f;
+ struct spi_master *master = dev_id;
+ struct dw_spi *dws = spi_master_get_devdata(master);
+ u16 irq_status = dw_readl(dws, DW_SPI_ISR) & 0x3f;
if (!irq_status)
return IRQ_NONE;
- if (!dws->cur_msg) {
+ if (!master->cur_msg) {
spi_mask_intr(dws, SPI_INT_TXEI);
return IRQ_HANDLED;
}
}
/* Must be called inside pump_transfers() */
-static void poll_transfer(struct dw_spi *dws)
+static int poll_transfer(struct dw_spi *dws)
{
do {
dw_writer(dws);
cpu_relax();
} while (dws->rx_end > dws->rx);
- dw_spi_xfer_done(dws);
+ return 0;
}
-static void pump_transfers(unsigned long data)
+static int dw_spi_transfer_one(struct spi_master *master,
+ struct spi_device *spi, struct spi_transfer *transfer)
{
- struct dw_spi *dws = (struct dw_spi *)data;
- struct spi_message *message = NULL;
- struct spi_transfer *transfer = NULL;
- struct spi_transfer *previous = NULL;
- struct spi_device *spi = NULL;
- struct chip_data *chip = NULL;
- u8 bits = 0;
+ struct dw_spi *dws = spi_master_get_devdata(master);
+ struct chip_data *chip = spi_get_ctldata(spi);
u8 imask = 0;
- u8 cs_change = 0;
- u16 txint_level = 0;
+ u16 txlevel = 0;
u16 clk_div = 0;
u32 speed = 0;
u32 cr0 = 0;
+ int ret;
- /* Get current state information */
- message = dws->cur_msg;
- transfer = dws->cur_transfer;
- chip = dws->cur_chip;
- spi = message->spi;
-
- if (message->state == ERROR_STATE) {
- message->status = -EIO;
- goto early_exit;
- }
-
- /* Handle end of message */
- if (message->state == DONE_STATE) {
- message->status = 0;
- goto early_exit;
- }
-
- /* Delay if requested at end of transfer */
- if (message->state == RUNNING_STATE) {
- previous = list_entry(transfer->transfer_list.prev,
- struct spi_transfer,
- transfer_list);
- if (previous->delay_usecs)
- udelay(previous->delay_usecs);
- }
-
+ dws->dma_mapped = 0;
dws->n_bytes = chip->n_bytes;
dws->dma_width = chip->dma_width;
- dws->cs_control = chip->cs_control;
- dws->rx_dma = transfer->rx_dma;
- dws->tx_dma = transfer->tx_dma;
dws->tx = (void *)transfer->tx_buf;
dws->tx_end = dws->tx + transfer->len;
dws->rx = transfer->rx_buf;
dws->rx_end = dws->rx + transfer->len;
- dws->len = dws->cur_transfer->len;
- if (chip != dws->prev_chip)
- cs_change = 1;
+ dws->len = transfer->len;
+
+ spi_enable_chip(dws, 0);
cr0 = chip->cr0;
if (transfer->speed_hz) {
speed = chip->speed_hz;
- if ((transfer->speed_hz != speed) || (!chip->clk_div)) {
+ if ((transfer->speed_hz != speed) || !chip->clk_div) {
speed = transfer->speed_hz;
/* clk_div doesn't support odd number */
- clk_div = dws->max_freq / speed;
- clk_div = (clk_div + 1) & 0xfffe;
+ clk_div = (dws->max_freq / speed + 1) & 0xfffe;
chip->speed_hz = speed;
chip->clk_div = clk_div;
+
+ spi_set_clk(dws, chip->clk_div);
}
}
if (transfer->bits_per_word) {
- bits = transfer->bits_per_word;
- dws->n_bytes = dws->dma_width = bits >> 3;
- cr0 = (bits - 1)
+ if (transfer->bits_per_word == 8) {
+ dws->n_bytes = 1;
+ dws->dma_width = 1;
+ } else if (transfer->bits_per_word == 16) {
+ dws->n_bytes = 2;
+ dws->dma_width = 2;
+ }
+ cr0 = (transfer->bits_per_word - 1)
| (chip->type << SPI_FRF_OFFSET)
| (spi->mode << SPI_MODE_OFFSET)
| (chip->tmode << SPI_TMOD_OFFSET);
}
- message->state = RUNNING_STATE;
/*
* Adjust transfer mode if necessary. Requires platform dependent
* chipselect mechanism.
*/
- if (dws->cs_control) {
+ if (chip->cs_control) {
if (dws->rx && dws->tx)
chip->tmode = SPI_TMOD_TR;
else if (dws->rx)
cr0 |= (chip->tmode << SPI_TMOD_OFFSET);
}
+ dw_writel(dws, DW_SPI_CTRL0, cr0);
+
/* Check if current transfer is a DMA transaction */
- dws->dma_mapped = map_dma_buffers(dws);
+ if (master->can_dma && master->can_dma(master, spi, transfer))
+ dws->dma_mapped = master->cur_msg_mapped;
+
+ /* For poll mode just disable all interrupts */
+ spi_mask_intr(dws, 0xff);
/*
* Interrupt mode
* we only need set the TXEI IRQ, as TX/RX always happen syncronizely
*/
- if (!dws->dma_mapped && !chip->poll_mode) {
- int templen = dws->len / dws->n_bytes;
-
- txint_level = dws->fifo_len / 2;
- txint_level = (templen > txint_level) ? txint_level : templen;
+ if (dws->dma_mapped) {
+ ret = dws->dma_ops->dma_setup(dws, transfer);
+ if (ret < 0) {
+ spi_enable_chip(dws, 1);
+ return ret;
+ }
+ } else if (!chip->poll_mode) {
+ txlevel = min_t(u16, dws->fifo_len / 2, dws->len / dws->n_bytes);
+ dw_writel(dws, DW_SPI_TXFLTR, txlevel);
+ /* Set the interrupt mask */
imask |= SPI_INT_TXEI | SPI_INT_TXOI |
SPI_INT_RXUI | SPI_INT_RXOI;
+ spi_umask_intr(dws, imask);
+
dws->transfer_handler = interrupt_transfer;
}
- /*
- * Reprogram registers only if
- * 1. chip select changes
- * 2. clk_div is changed
- * 3. control value changes
- */
- if (dw_readw(dws, DW_SPI_CTRL0) != cr0 || cs_change || clk_div || imask) {
- spi_enable_chip(dws, 0);
-
- if (dw_readw(dws, DW_SPI_CTRL0) != cr0)
- dw_writew(dws, DW_SPI_CTRL0, cr0);
-
- spi_set_clk(dws, clk_div ? clk_div : chip->clk_div);
- spi_chip_sel(dws, spi, 1);
-
- /* Set the interrupt mask, for poll mode just disable all int */
- spi_mask_intr(dws, 0xff);
- if (imask)
- spi_umask_intr(dws, imask);
- if (txint_level)
- dw_writew(dws, DW_SPI_TXFLTR, txint_level);
+ spi_enable_chip(dws, 1);
- spi_enable_chip(dws, 1);
- if (cs_change)
- dws->prev_chip = chip;
+ if (dws->dma_mapped) {
+ ret = dws->dma_ops->dma_transfer(dws, transfer);
+ if (ret < 0)
+ return ret;
}
- if (dws->dma_mapped)
- dws->dma_ops->dma_transfer(dws, cs_change);
-
if (chip->poll_mode)
- poll_transfer(dws);
-
- return;
+ return poll_transfer(dws);
-early_exit:
- giveback(dws);
+ return 1;
}
-static int dw_spi_transfer_one_message(struct spi_master *master,
+static void dw_spi_handle_err(struct spi_master *master,
struct spi_message *msg)
{
struct dw_spi *dws = spi_master_get_devdata(master);
- dws->cur_msg = msg;
- /* Initial message state */
- dws->cur_msg->state = START_STATE;
- dws->cur_transfer = list_entry(dws->cur_msg->transfers.next,
- struct spi_transfer,
- transfer_list);
- dws->cur_chip = spi_get_ctldata(dws->cur_msg->spi);
-
- /* Launch transfers */
- tasklet_schedule(&dws->pump_transfers);
+ if (dws->dma_mapped)
+ dws->dma_ops->dma_stop(dws);
- return 0;
+ spi_reset_chip(dws);
}
/* This may be called twice for each spi dev */
chip->rx_threshold = 0;
chip->tx_threshold = 0;
-
- chip->enable_dma = chip_info->enable_dma;
}
if (spi->bits_per_word == 8) {
/* Restart the controller, disable all interrupts, clean rx fifo */
static void spi_hw_init(struct device *dev, struct dw_spi *dws)
{
- spi_enable_chip(dws, 0);
- spi_mask_intr(dws, 0xff);
- spi_enable_chip(dws, 1);
+ spi_reset_chip(dws);
/*
* Try to detect the FIFO depth if not set by interface driver,
if (!dws->fifo_len) {
u32 fifo;
- for (fifo = 2; fifo <= 256; fifo++) {
- dw_writew(dws, DW_SPI_TXFLTR, fifo);
- if (fifo != dw_readw(dws, DW_SPI_TXFLTR))
+ for (fifo = 1; fifo < 256; fifo++) {
+ dw_writel(dws, DW_SPI_TXFLTR, fifo);
+ if (fifo != dw_readl(dws, DW_SPI_TXFLTR))
break;
}
- dw_writew(dws, DW_SPI_TXFLTR, 0);
+ dw_writel(dws, DW_SPI_TXFLTR, 0);
- dws->fifo_len = (fifo == 2) ? 0 : fifo - 1;
+ dws->fifo_len = (fifo == 1) ? 0 : fifo;
dev_dbg(dev, "Detected FIFO size: %u bytes\n", dws->fifo_len);
}
}
dws->master = master;
dws->type = SSI_MOTO_SPI;
- dws->prev_chip = NULL;
dws->dma_inited = 0;
dws->dma_addr = (dma_addr_t)(dws->paddr + 0x60);
snprintf(dws->name, sizeof(dws->name), "dw_spi%d", dws->bus_num);
ret = devm_request_irq(dev, dws->irq, dw_spi_irq, IRQF_SHARED,
- dws->name, dws);
+ dws->name, master);
if (ret < 0) {
dev_err(&master->dev, "can not get IRQ\n");
goto err_free_master;
master->num_chipselect = dws->num_cs;
master->setup = dw_spi_setup;
master->cleanup = dw_spi_cleanup;
- master->transfer_one_message = dw_spi_transfer_one_message;
+ master->set_cs = dw_spi_set_cs;
+ master->transfer_one = dw_spi_transfer_one;
+ master->handle_err = dw_spi_handle_err;
master->max_speed_hz = dws->max_freq;
master->dev.of_node = dev->of_node;
if (ret) {
dev_warn(dev, "DMA init failed\n");
dws->dma_inited = 0;
+ } else {
+ master->can_dma = dws->dma_ops->can_dma;
}
}
- tasklet_init(&dws->pump_transfers, pump_transfers, (unsigned long)dws);
-
spi_master_set_devdata(master, dws);
ret = devm_spi_register_master(dev, master);
if (ret) {
struct dw_spi_dma_ops {
int (*dma_init)(struct dw_spi *dws);
void (*dma_exit)(struct dw_spi *dws);
- int (*dma_transfer)(struct dw_spi *dws, int cs_change);
+ int (*dma_setup)(struct dw_spi *dws, struct spi_transfer *xfer);
+ bool (*can_dma)(struct spi_master *master, struct spi_device *spi,
+ struct spi_transfer *xfer);
+ int (*dma_transfer)(struct dw_spi *dws, struct spi_transfer *xfer);
+ void (*dma_stop)(struct dw_spi *dws);
};
struct dw_spi {
struct spi_master *master;
- struct spi_device *cur_dev;
enum dw_ssi_type type;
char name[16];
u16 bus_num;
u16 num_cs; /* supported slave numbers */
- /* Message Transfer pump */
- struct tasklet_struct pump_transfers;
-
/* Current message transfer state info */
- struct spi_message *cur_msg;
- struct spi_transfer *cur_transfer;
- struct chip_data *cur_chip;
- struct chip_data *prev_chip;
size_t len;
void *tx;
void *tx_end;
void *rx;
void *rx_end;
int dma_mapped;
- dma_addr_t rx_dma;
- dma_addr_t tx_dma;
- size_t rx_map_len;
- size_t tx_map_len;
u8 n_bytes; /* current is a 1/2 bytes op */
- u8 max_bits_per_word; /* maxim is 16b */
u32 dma_width;
irqreturn_t (*transfer_handler)(struct dw_spi *dws);
- void (*cs_control)(u32 command);
- /* Dma info */
+ /* DMA info */
int dma_inited;
struct dma_chan *txchan;
- struct scatterlist tx_sgl;
struct dma_chan *rxchan;
- struct scatterlist rx_sgl;
unsigned long dma_chan_busy;
- struct device *dma_dev;
dma_addr_t dma_addr; /* phy address of the Data register */
struct dw_spi_dma_ops *dma_ops;
- void *dma_priv; /* platform relate info */
+ void *dma_tx;
+ void *dma_rx;
/* Bus interface info */
void *priv;
__raw_writel(val, dws->regs + offset);
}
-static inline u16 dw_readw(struct dw_spi *dws, u32 offset)
-{
- return __raw_readw(dws->regs + offset);
-}
-
-static inline void dw_writew(struct dw_spi *dws, u32 offset, u16 val)
-{
- __raw_writew(val, dws->regs + offset);
-}
-
static inline void spi_enable_chip(struct dw_spi *dws, int enable)
{
dw_writel(dws, DW_SPI_SSIENR, (enable ? 1 : 0));
dw_writel(dws, DW_SPI_BAUDR, div);
}
-static inline void spi_chip_sel(struct dw_spi *dws, struct spi_device *spi,
- int active)
-{
- u16 cs = spi->chip_select;
- int gpio_val = active ? (spi->mode & SPI_CS_HIGH) :
- !(spi->mode & SPI_CS_HIGH);
-
- if (dws->cs_control)
- dws->cs_control(active);
- if (gpio_is_valid(spi->cs_gpio))
- gpio_set_value(spi->cs_gpio, gpio_val);
-
- if (active)
- dw_writel(dws, DW_SPI_SER, 1 << cs);
-}
-
/* Disable IRQ bits */
static inline void spi_mask_intr(struct dw_spi *dws, u32 mask)
{
dw_writel(dws, DW_SPI_IMR, new_mask);
}
+/*
+ * This does disable the SPI controller, interrupts, and re-enable the
+ * controller back. Transmit and receive FIFO buffers are cleared when the
+ * device is disabled.
+ */
+static inline void spi_reset_chip(struct dw_spi *dws)
+{
+ spi_enable_chip(dws, 0);
+ spi_mask_intr(dws, 0xff);
+ spi_enable_chip(dws, 1);
+}
+
/*
* Each SPI slave device to work with dw_api controller should
- * has such a structure claiming its working mode (PIO/DMA etc),
+ * has such a structure claiming its working mode (poll or PIO/DMA),
* which can be save in the "controller_data" member of the
* struct spi_device.
*/
struct dw_spi_chip {
u8 poll_mode; /* 1 for controller polling mode */
u8 type; /* SPI/SSP/MicroWire */
- u8 enable_dma;
void (*cs_control)(u32 command);
};
extern void dw_spi_remove_host(struct dw_spi *dws);
extern int dw_spi_suspend_host(struct dw_spi *dws);
extern int dw_spi_resume_host(struct dw_spi *dws);
-extern void dw_spi_xfer_done(struct dw_spi *dws);
/* platform related setup */
extern int dw_spi_mid_init(struct dw_spi *dws); /* Intel MID platforms */
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/kernel.h>
+#include <linux/math64.h>
#include <linux/module.h>
#include <linux/of.h>
#include <linux/of_device.h>
#include <linux/sched.h>
#include <linux/spi/spi.h>
#include <linux/spi/spi_bitbang.h>
+#include <linux/time.h>
#define DRIVER_NAME "fsl-dspi"
#define SPI_CTAR_CPOL(x) ((x) << 26)
#define SPI_CTAR_CPHA(x) ((x) << 25)
#define SPI_CTAR_LSBFE(x) ((x) << 24)
-#define SPI_CTAR_PCSSCR(x) (((x) & 0x00000003) << 22)
+#define SPI_CTAR_PCSSCK(x) (((x) & 0x00000003) << 22)
#define SPI_CTAR_PASC(x) (((x) & 0x00000003) << 20)
#define SPI_CTAR_PDT(x) (((x) & 0x00000003) << 18)
#define SPI_CTAR_PBR(x) (((x) & 0x00000003) << 16)
#define SPI_CTAR_ASC(x) (((x) & 0x0000000f) << 8)
#define SPI_CTAR_DT(x) (((x) & 0x0000000f) << 4)
#define SPI_CTAR_BR(x) ((x) & 0x0000000f)
+#define SPI_CTAR_SCALE_BITS 0xf
#define SPI_CTAR0_SLAVE 0x0c
16, 32, 64, 128,
256, 512, 1024, 2048,
4096, 8192, 16384, 32768 };
- int temp, i = 0, j = 0;
+ int scale_needed, scale, minscale = INT_MAX;
+ int i, j;
+
+ scale_needed = clkrate / speed_hz;
+ if (clkrate % speed_hz)
+ scale_needed++;
+
+ for (i = 0; i < ARRAY_SIZE(brs); i++)
+ for (j = 0; j < ARRAY_SIZE(pbr_tbl); j++) {
+ scale = brs[i] * pbr_tbl[j];
+ if (scale >= scale_needed) {
+ if (scale < minscale) {
+ minscale = scale;
+ *br = i;
+ *pbr = j;
+ }
+ break;
+ }
+ }
- temp = clkrate / 2 / speed_hz;
+ if (minscale == INT_MAX) {
+ pr_warn("Can not find valid baud rate,speed_hz is %d,clkrate is %ld, we use the max prescaler value.\n",
+ speed_hz, clkrate);
+ *pbr = ARRAY_SIZE(pbr_tbl) - 1;
+ *br = ARRAY_SIZE(brs) - 1;
+ }
+}
- for (i = 0; i < ARRAY_SIZE(pbr_tbl); i++)
- for (j = 0; j < ARRAY_SIZE(brs); j++) {
- if (pbr_tbl[i] * brs[j] >= temp) {
- *pbr = i;
- *br = j;
- return;
+static void ns_delay_scale(char *psc, char *sc, int delay_ns,
+ unsigned long clkrate)
+{
+ int pscale_tbl[4] = {1, 3, 5, 7};
+ int scale_needed, scale, minscale = INT_MAX;
+ int i, j;
+ u32 remainder;
+
+ scale_needed = div_u64_rem((u64)delay_ns * clkrate, NSEC_PER_SEC,
+ &remainder);
+ if (remainder)
+ scale_needed++;
+
+ for (i = 0; i < ARRAY_SIZE(pscale_tbl); i++)
+ for (j = 0; j <= SPI_CTAR_SCALE_BITS; j++) {
+ scale = pscale_tbl[i] * (2 << j);
+ if (scale >= scale_needed) {
+ if (scale < minscale) {
+ minscale = scale;
+ *psc = i;
+ *sc = j;
+ }
+ break;
}
}
- pr_warn("Can not find valid baud rate,speed_hz is %d,clkrate is %ld\
- ,we use the max prescaler value.\n", speed_hz, clkrate);
- *pbr = ARRAY_SIZE(pbr_tbl) - 1;
- *br = ARRAY_SIZE(brs) - 1;
+ if (minscale == INT_MAX) {
+ pr_warn("Cannot find correct scale values for %dns delay at clkrate %ld, using max prescaler value",
+ delay_ns, clkrate);
+ *psc = ARRAY_SIZE(pscale_tbl) - 1;
+ *sc = SPI_CTAR_SCALE_BITS;
+ }
}
static int dspi_transfer_write(struct fsl_dspi *dspi)
{
struct chip_data *chip;
struct fsl_dspi *dspi = spi_master_get_devdata(spi->master);
- unsigned char br = 0, pbr = 0, fmsz = 0;
+ u32 cs_sck_delay = 0, sck_cs_delay = 0;
+ unsigned char br = 0, pbr = 0, pcssck = 0, cssck = 0;
+ unsigned char pasc = 0, asc = 0, fmsz = 0;
+ unsigned long clkrate;
if ((spi->bits_per_word >= 4) && (spi->bits_per_word <= 16)) {
fmsz = spi->bits_per_word - 1;
return -ENOMEM;
}
+ of_property_read_u32(spi->dev.of_node, "fsl,spi-cs-sck-delay",
+ &cs_sck_delay);
+
+ of_property_read_u32(spi->dev.of_node, "fsl,spi-sck-cs-delay",
+ &sck_cs_delay);
+
chip->mcr_val = SPI_MCR_MASTER | SPI_MCR_PCSIS |
SPI_MCR_CLR_TXF | SPI_MCR_CLR_RXF;
chip->void_write_data = 0;
- hz_to_spi_baud(&pbr, &br,
- spi->max_speed_hz, clk_get_rate(dspi->clk));
+ clkrate = clk_get_rate(dspi->clk);
+ hz_to_spi_baud(&pbr, &br, spi->max_speed_hz, clkrate);
+
+ /* Set PCS to SCK delay scale values */
+ ns_delay_scale(&pcssck, &cssck, cs_sck_delay, clkrate);
+
+ /* Set After SCK delay scale values */
+ ns_delay_scale(&pasc, &asc, sck_cs_delay, clkrate);
chip->ctar_val = SPI_CTAR_FMSZ(fmsz)
| SPI_CTAR_CPOL(spi->mode & SPI_CPOL ? 1 : 0)
| SPI_CTAR_CPHA(spi->mode & SPI_CPHA ? 1 : 0)
| SPI_CTAR_LSBFE(spi->mode & SPI_LSB_FIRST ? 1 : 0)
+ | SPI_CTAR_PCSSCK(pcssck)
+ | SPI_CTAR_CSSCK(cssck)
+ | SPI_CTAR_PASC(pasc)
+ | SPI_CTAR_ASC(asc)
| SPI_CTAR_PBR(pbr)
| SPI_CTAR_BR(br);
#include <linux/clk.h>
#include <linux/delay.h>
#include <linux/dmaengine.h>
+#include <linux/gpio.h>
#include <linux/interrupt.h>
#include <linux/io.h>
#include <linux/irq.h>
spfi_writel(spfi, val, SPFI_CONTROL);
}
-static inline void spfi_stop(struct img_spfi *spfi)
-{
- u32 val;
-
- val = spfi_readl(spfi, SPFI_CONTROL);
- val &= ~SPFI_CONTROL_SPFI_EN;
- spfi_writel(spfi, val, SPFI_CONTROL);
-}
-
static inline void spfi_reset(struct img_spfi *spfi)
{
spfi_writel(spfi, SPFI_CONTROL_SOFT_RESET, SPFI_CONTROL);
- udelay(1);
spfi_writel(spfi, 0, SPFI_CONTROL);
}
-static void spfi_flush_tx_fifo(struct img_spfi *spfi)
+static int spfi_wait_all_done(struct img_spfi *spfi)
{
- unsigned long timeout = jiffies + msecs_to_jiffies(10);
+ unsigned long timeout = jiffies + msecs_to_jiffies(50);
- spfi_writel(spfi, SPFI_INTERRUPT_SDE, SPFI_INTERRUPT_CLEAR);
while (time_before(jiffies, timeout)) {
- if (spfi_readl(spfi, SPFI_INTERRUPT_STATUS) &
- SPFI_INTERRUPT_SDE)
- return;
+ u32 status = spfi_readl(spfi, SPFI_INTERRUPT_STATUS);
+
+ if (status & SPFI_INTERRUPT_ALLDONETRIG) {
+ spfi_writel(spfi, SPFI_INTERRUPT_ALLDONETRIG,
+ SPFI_INTERRUPT_CLEAR);
+ return 0;
+ }
cpu_relax();
}
- dev_err(spfi->dev, "Timed out waiting for FIFO to drain\n");
+ dev_err(spfi->dev, "Timed out waiting for transaction to complete\n");
spfi_reset(spfi);
+
+ return -ETIMEDOUT;
}
static unsigned int spfi_pio_write32(struct img_spfi *spfi, const u32 *buf,
const void *tx_buf = xfer->tx_buf;
void *rx_buf = xfer->rx_buf;
unsigned long timeout;
+ int ret;
if (tx_buf)
tx_bytes = xfer->len;
cpu_relax();
}
+ ret = spfi_wait_all_done(spfi);
+ if (ret < 0)
+ return ret;
+
if (rx_bytes > 0 || tx_bytes > 0) {
dev_err(spfi->dev, "PIO transfer timed out\n");
- spfi_reset(spfi);
return -ETIMEDOUT;
}
- if (tx_buf)
- spfi_flush_tx_fifo(spfi);
- spfi_stop(spfi);
-
return 0;
}
struct img_spfi *spfi = data;
unsigned long flags;
- spin_lock_irqsave(&spfi->lock, flags);
+ spfi_wait_all_done(spfi);
+ spin_lock_irqsave(&spfi->lock, flags);
spfi->rx_dma_busy = false;
- if (!spfi->tx_dma_busy) {
- spfi_stop(spfi);
+ if (!spfi->tx_dma_busy)
spi_finalize_current_transfer(spfi->master);
- }
-
spin_unlock_irqrestore(&spfi->lock, flags);
}
struct img_spfi *spfi = data;
unsigned long flags;
- spfi_flush_tx_fifo(spfi);
+ spfi_wait_all_done(spfi);
spin_lock_irqsave(&spfi->lock, flags);
-
spfi->tx_dma_busy = false;
- if (!spfi->rx_dma_busy) {
- spfi_stop(spfi);
+ if (!spfi->rx_dma_busy)
spi_finalize_current_transfer(spfi->master);
- }
-
spin_unlock_irqrestore(&spfi->lock, flags);
}
return -EIO;
}
+static void img_spfi_handle_err(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct img_spfi *spfi = spi_master_get_devdata(master);
+ unsigned long flags;
+
+ /*
+ * Stop all DMA and reset the controller if the previous transaction
+ * timed-out and never completed it's DMA.
+ */
+ spin_lock_irqsave(&spfi->lock, flags);
+ if (spfi->tx_dma_busy || spfi->rx_dma_busy) {
+ spfi->tx_dma_busy = false;
+ spfi->rx_dma_busy = false;
+
+ dmaengine_terminate_all(spfi->tx_ch);
+ dmaengine_terminate_all(spfi->rx_ch);
+ }
+ spin_unlock_irqrestore(&spfi->lock, flags);
+}
+
+static int img_spfi_prepare(struct spi_master *master, struct spi_message *msg)
+{
+ struct img_spfi *spfi = spi_master_get_devdata(master);
+ u32 val;
+
+ val = spfi_readl(spfi, SPFI_PORT_STATE);
+ if (msg->spi->mode & SPI_CPHA)
+ val |= SPFI_PORT_STATE_CK_PHASE(msg->spi->chip_select);
+ else
+ val &= ~SPFI_PORT_STATE_CK_PHASE(msg->spi->chip_select);
+ if (msg->spi->mode & SPI_CPOL)
+ val |= SPFI_PORT_STATE_CK_POL(msg->spi->chip_select);
+ else
+ val &= ~SPFI_PORT_STATE_CK_POL(msg->spi->chip_select);
+ spfi_writel(spfi, val, SPFI_PORT_STATE);
+
+ return 0;
+}
+
+static int img_spfi_unprepare(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct img_spfi *spfi = spi_master_get_devdata(master);
+
+ spfi_reset(spfi);
+
+ return 0;
+}
+
+static int img_spfi_setup(struct spi_device *spi)
+{
+ int ret;
+
+ ret = gpio_request_one(spi->cs_gpio, (spi->mode & SPI_CS_HIGH) ?
+ GPIOF_OUT_INIT_LOW : GPIOF_OUT_INIT_HIGH,
+ dev_name(&spi->dev));
+ if (ret)
+ dev_err(&spi->dev, "can't request chipselect gpio %d\n",
+ spi->cs_gpio);
+
+ return ret;
+}
+
+static void img_spfi_cleanup(struct spi_device *spi)
+{
+ gpio_free(spi->cs_gpio);
+}
+
static void img_spfi_config(struct spi_master *master, struct spi_device *spi,
struct spi_transfer *xfer)
{
/*
* output = spfi_clk * (BITCLK / 512), where BITCLK must be a
- * power of 2 up to 256 (where 255 == 256 since BITCLK is 8 bits)
+ * power of 2 up to 128
*/
- div = DIV_ROUND_UP(master->max_speed_hz, xfer->speed_hz);
- div = clamp(512 / (1 << get_count_order(div)), 1, 255);
+ div = DIV_ROUND_UP(clk_get_rate(spfi->spfi_clk), xfer->speed_hz);
+ div = clamp(512 / (1 << get_count_order(div)), 1, 128);
val = spfi_readl(spfi, SPFI_DEVICE_PARAMETER(spi->chip_select));
val &= ~(SPFI_DEVICE_PARAMETER_BITCLK_MASK <<
val |= div << SPFI_DEVICE_PARAMETER_BITCLK_SHIFT;
spfi_writel(spfi, val, SPFI_DEVICE_PARAMETER(spi->chip_select));
+ spfi_writel(spfi, xfer->len << SPFI_TRANSACTION_TSIZE_SHIFT,
+ SPFI_TRANSACTION);
+
val = spfi_readl(spfi, SPFI_CONTROL);
val &= ~(SPFI_CONTROL_SEND_DMA | SPFI_CONTROL_GET_DMA);
if (xfer->tx_buf)
else if (xfer->tx_nbits == SPI_NBITS_QUAD &&
xfer->rx_nbits == SPI_NBITS_QUAD)
val |= SPFI_CONTROL_TMODE_QUAD << SPFI_CONTROL_TMODE_SHIFT;
- val &= ~SPFI_CONTROL_CONTINUE;
- if (!xfer->cs_change && !list_is_last(&xfer->transfer_list,
- &master->cur_msg->transfers))
- val |= SPFI_CONTROL_CONTINUE;
spfi_writel(spfi, val, SPFI_CONTROL);
-
- val = spfi_readl(spfi, SPFI_PORT_STATE);
- if (spi->mode & SPI_CPHA)
- val |= SPFI_PORT_STATE_CK_PHASE(spi->chip_select);
- else
- val &= ~SPFI_PORT_STATE_CK_PHASE(spi->chip_select);
- if (spi->mode & SPI_CPOL)
- val |= SPFI_PORT_STATE_CK_POL(spi->chip_select);
- else
- val &= ~SPFI_PORT_STATE_CK_POL(spi->chip_select);
- spfi_writel(spfi, val, SPFI_PORT_STATE);
-
- spfi_writel(spfi, xfer->len << SPFI_TRANSACTION_TSIZE_SHIFT,
- SPFI_TRANSACTION);
}
static int img_spfi_transfer_one(struct spi_master *master,
struct spi_transfer *xfer)
{
struct img_spfi *spfi = spi_master_get_devdata(spi->master);
- bool dma_reset = false;
- unsigned long flags;
int ret;
- /*
- * Stop all DMA and reset the controller if the previous transaction
- * timed-out and never completed it's DMA.
- */
- spin_lock_irqsave(&spfi->lock, flags);
- if (spfi->tx_dma_busy || spfi->rx_dma_busy) {
- dev_err(spfi->dev, "SPI DMA still busy\n");
- dma_reset = true;
- }
- spin_unlock_irqrestore(&spfi->lock, flags);
-
- if (dma_reset) {
- dmaengine_terminate_all(spfi->tx_ch);
- dmaengine_terminate_all(spfi->rx_ch);
- spfi_reset(spfi);
+ if (xfer->len > SPFI_TRANSACTION_TSIZE_MASK) {
+ dev_err(spfi->dev,
+ "Transfer length (%d) is greater than the max supported (%d)",
+ xfer->len, SPFI_TRANSACTION_TSIZE_MASK);
+ return -EINVAL;
}
img_spfi_config(master, spi, xfer);
return ret;
}
-static void img_spfi_set_cs(struct spi_device *spi, bool enable)
-{
- struct img_spfi *spfi = spi_master_get_devdata(spi->master);
- u32 val;
-
- val = spfi_readl(spfi, SPFI_PORT_STATE);
- val &= ~(SPFI_PORT_STATE_DEV_SEL_MASK << SPFI_PORT_STATE_DEV_SEL_SHIFT);
- val |= spi->chip_select << SPFI_PORT_STATE_DEV_SEL_SHIFT;
- spfi_writel(spfi, val, SPFI_PORT_STATE);
-}
-
static bool img_spfi_can_dma(struct spi_master *master, struct spi_device *spi,
struct spi_transfer *xfer)
{
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_TX_DUAL | SPI_RX_DUAL;
if (of_property_read_bool(spfi->dev->of_node, "img,supports-quad-mode"))
master->mode_bits |= SPI_TX_QUAD | SPI_RX_QUAD;
- master->num_chipselect = 5;
master->dev.of_node = pdev->dev.of_node;
master->bits_per_word_mask = SPI_BPW_MASK(32) | SPI_BPW_MASK(8);
- master->max_speed_hz = clk_get_rate(spfi->spfi_clk);
- master->min_speed_hz = master->max_speed_hz / 512;
+ master->max_speed_hz = clk_get_rate(spfi->spfi_clk) / 4;
+ master->min_speed_hz = clk_get_rate(spfi->spfi_clk) / 512;
- master->set_cs = img_spfi_set_cs;
+ master->setup = img_spfi_setup;
+ master->cleanup = img_spfi_cleanup;
master->transfer_one = img_spfi_transfer_one;
+ master->prepare_message = img_spfi_prepare;
+ master->unprepare_message = img_spfi_unprepare;
+ master->handle_err = img_spfi_handle_err;
spfi->tx_ch = dma_request_slave_channel(spfi->dev, "tx");
spfi->rx_ch = dma_request_slave_channel(spfi->dev, "rx");
if (spi_imx->dma_is_inited) {
dma = readl(spi_imx->base + MX51_ECSPI_DMA);
- spi_imx->tx_wml = spi_imx_get_fifosize(spi_imx) / 2;
- spi_imx->rx_wml = spi_imx_get_fifosize(spi_imx) / 2;
spi_imx->rxt_wml = spi_imx_get_fifosize(spi_imx) / 2;
rx_wml_cfg = spi_imx->rx_wml << MX51_ECSPI_DMA_RX_WML_OFFSET;
tx_wml_cfg = spi_imx->tx_wml << MX51_ECSPI_DMA_TX_WML_OFFSET;
master->max_dma_len = MAX_SDMA_BD_BYTES;
spi_imx->bitbang.master->flags = SPI_MASTER_MUST_RX |
SPI_MASTER_MUST_TX;
+ spi_imx->tx_wml = spi_imx_get_fifosize(spi_imx) / 2;
+ spi_imx->rx_wml = spi_imx_get_fifosize(spi_imx) / 2;
spi_imx->dma_is_inited = 1;
return 0;
if (tx) {
desc_tx = dmaengine_prep_slave_sg(master->dma_tx,
- tx->sgl, tx->nents, DMA_TO_DEVICE,
+ tx->sgl, tx->nents, DMA_MEM_TO_DEV,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc_tx)
goto no_dma;
if (rx) {
desc_rx = dmaengine_prep_slave_sg(master->dma_rx,
- rx->sgl, rx->nents, DMA_FROM_DEVICE,
+ rx->sgl, rx->nents, DMA_DEV_TO_MEM,
DMA_PREP_INTERRUPT | DMA_CTRL_ACK);
if (!desc_rx)
goto no_dma;
return mpc512x_psc_spi_do_remove(&op->dev);
}
-static struct of_device_id mpc512x_psc_spi_of_match[] = {
+static const struct of_device_id mpc512x_psc_spi_of_match[] = {
{ .compatible = "fsl,mpc5121-psc-spi", },
{},
};
return 0;
}
-static struct of_device_id octeon_spi_match[] = {
+static const struct of_device_id octeon_spi_match[] = {
{ .compatible = "cavium,octeon-3010-spi", },
{},
};
#include <linux/device.h>
#include <linux/delay.h>
#include <linux/platform_device.h>
+#include <linux/pm_runtime.h>
#include <linux/err.h>
#include <linux/clk.h>
#include <linux/io.h>
return ret;
}
-static int omap1_spi100k_prepare_hardware(struct spi_master *master)
-{
- struct omap1_spi100k *spi100k = spi_master_get_devdata(master);
-
- clk_prepare_enable(spi100k->ick);
- clk_prepare_enable(spi100k->fck);
-
- return 0;
-}
-
static int omap1_spi100k_transfer_one_message(struct spi_master *master,
struct spi_message *m)
{
return status;
}
-static int omap1_spi100k_unprepare_hardware(struct spi_master *master)
-{
- struct omap1_spi100k *spi100k = spi_master_get_devdata(master);
-
- clk_disable_unprepare(spi100k->ick);
- clk_disable_unprepare(spi100k->fck);
-
- return 0;
-}
-
static int omap1_spi100k_probe(struct platform_device *pdev)
{
struct spi_master *master;
master->setup = omap1_spi100k_setup;
master->transfer_one_message = omap1_spi100k_transfer_one_message;
- master->prepare_transfer_hardware = omap1_spi100k_prepare_hardware;
- master->unprepare_transfer_hardware = omap1_spi100k_unprepare_hardware;
- master->cleanup = NULL;
master->num_chipselect = 2;
master->mode_bits = MODEBITS;
master->bits_per_word_mask = SPI_BPW_RANGE_MASK(4, 32);
master->min_speed_hz = OMAP1_SPI100K_MAX_FREQ/(1<<16);
master->max_speed_hz = OMAP1_SPI100K_MAX_FREQ;
+ master->auto_runtime_pm = true;
spi100k = spi_master_get_devdata(master);
goto err;
}
+ status = clk_prepare_enable(spi100k->ick);
+ if (status != 0) {
+ dev_err(&pdev->dev, "failed to enable ick: %d\n", status);
+ goto err;
+ }
+
+ status = clk_prepare_enable(spi100k->fck);
+ if (status != 0) {
+ dev_err(&pdev->dev, "failed to enable fck: %d\n", status);
+ goto err_ick;
+ }
+
+ pm_runtime_enable(&pdev->dev);
+ pm_runtime_set_active(&pdev->dev);
+
status = devm_spi_register_master(&pdev->dev, master);
if (status < 0)
- goto err;
+ goto err_fck;
return status;
+err_fck:
+ clk_disable_unprepare(spi100k->fck);
+err_ick:
+ clk_disable_unprepare(spi100k->ick);
err:
spi_master_put(master);
return status;
}
+static int omap1_spi100k_remove(struct platform_device *pdev)
+{
+ struct spi_master *master = spi_master_get(platform_get_drvdata(pdev));
+ struct omap1_spi100k *spi100k = spi_master_get_devdata(master);
+
+ pm_runtime_disable(&pdev->dev);
+
+ clk_disable_unprepare(spi100k->fck);
+ clk_disable_unprepare(spi100k->ick);
+
+ return 0;
+}
+
+#ifdef CONFIG_PM
+static int omap1_spi100k_runtime_suspend(struct device *dev)
+{
+ struct spi_master *master = spi_master_get(dev_get_drvdata(dev));
+ struct omap1_spi100k *spi100k = spi_master_get_devdata(master);
+
+ clk_disable_unprepare(spi100k->ick);
+ clk_disable_unprepare(spi100k->fck);
+
+ return 0;
+}
+
+static int omap1_spi100k_runtime_resume(struct device *dev)
+{
+ struct spi_master *master = spi_master_get(dev_get_drvdata(dev));
+ struct omap1_spi100k *spi100k = spi_master_get_devdata(master);
+ int ret;
+
+ ret = clk_prepare_enable(spi100k->ick);
+ if (ret != 0) {
+ dev_err(dev, "Failed to enable ick: %d\n", ret);
+ return ret;
+ }
+
+ ret = clk_prepare_enable(spi100k->fck);
+ if (ret != 0) {
+ dev_err(dev, "Failed to enable fck: %d\n", ret);
+ clk_disable_unprepare(spi100k->ick);
+ return ret;
+ }
+
+ return 0;
+}
+#endif
+
+static const struct dev_pm_ops omap1_spi100k_pm = {
+ SET_RUNTIME_PM_OPS(omap1_spi100k_runtime_suspend,
+ omap1_spi100k_runtime_resume, NULL)
+};
+
static struct platform_driver omap1_spi100k_driver = {
.driver = {
.name = "omap1_spi100k",
+ .pm = &omap1_spi100k_pm,
},
.probe = omap1_spi100k_probe,
+ .remove = omap1_spi100k_remove,
};
module_platform_driver(omap1_spi100k_driver);
#include <linux/module.h>
#include <linux/io.h>
-#include <asm/irq.h>
#include <mach/hardware.h>
#include <asm/mach-types.h>
*/
#define DEFAULT_SSP_REG_IMSC 0x0UL
#define DISABLE_ALL_INTERRUPTS DEFAULT_SSP_REG_IMSC
-#define ENABLE_ALL_INTERRUPTS (~DEFAULT_SSP_REG_IMSC)
+#define ENABLE_ALL_INTERRUPTS ( \
+ SSP_IMSC_MASK_RORIM | \
+ SSP_IMSC_MASK_RTIM | \
+ SSP_IMSC_MASK_RXIM | \
+ SSP_IMSC_MASK_TXIM \
+)
#define CLEAR_ALL_INTERRUPTS 0x3
pl022->cur_msg = NULL;
pl022->cur_transfer = NULL;
pl022->cur_chip = NULL;
- spi_finalize_current_message(pl022->master);
/* disable the SPI/SSP operation */
writew((readw(SSP_CR1(pl022->virtbase)) &
(~SSP_CR1_MASK_SSE)), SSP_CR1(pl022->virtbase));
+ spi_finalize_current_message(pl022->master);
}
/**
struct pl022 *pl022 = dev_id;
struct spi_message *msg = pl022->cur_msg;
u16 irq_status = 0;
- u16 flag = 0;
if (unlikely(!msg)) {
dev_err(&pl022->adev->dev,
if (readw(SSP_SR(pl022->virtbase)) & SSP_SR_MASK_RFF)
dev_err(&pl022->adev->dev,
"RXFIFO is full\n");
- if (readw(SSP_SR(pl022->virtbase)) & SSP_SR_MASK_TNF)
- dev_err(&pl022->adev->dev,
- "TXFIFO is full\n");
/*
* Disable and clear interrupts, disable SSP,
readwriter(pl022);
- if ((pl022->tx == pl022->tx_end) && (flag == 0)) {
- flag = 1;
+ if (pl022->tx == pl022->tx_end) {
/* Disable Transmit interrupt, enable receive interrupt */
writew((readw(SSP_IMSC(pl022->virtbase)) &
~SSP_IMSC_MASK_TXIM) | SSP_IMSC_MASK_RXIM,
#include <linux/errno.h>
#include <linux/err.h>
#include <linux/interrupt.h>
+#include <linux/kernel.h>
#include <linux/platform_device.h>
#include <linux/spi/pxa2xx_spi.h>
#include <linux/spi/spi.h>
#include <linux/pm_runtime.h>
#include <linux/acpi.h>
-#include <asm/io.h>
-#include <asm/irq.h>
-#include <asm/delay.h>
-
#include "spi-pxa2xx.h"
MODULE_AUTHOR("Stephen Street");
#define LPSS_TX_LOTHRESH_DFLT 160
#define LPSS_TX_HITHRESH_DFLT 224
-struct quark_spi_rate {
- u32 bitrate;
- u32 dds_clk_rate;
- u32 clk_div;
-};
-
-/*
- * 'rate', 'dds', 'clk_div' lookup table, which is defined in
- * the Quark SPI datasheet.
- */
-static const struct quark_spi_rate quark_spi_rate_table[] = {
-/* bitrate, dds_clk_rate, clk_div */
- {50000000, 0x800000, 0},
- {40000000, 0x666666, 0},
- {25000000, 0x400000, 0},
- {20000000, 0x666666, 1},
- {16667000, 0x800000, 2},
- {13333000, 0x666666, 2},
- {12500000, 0x200000, 0},
- {10000000, 0x800000, 4},
- {8000000, 0x666666, 4},
- {6250000, 0x400000, 3},
- {5000000, 0x400000, 4},
- {4000000, 0x666666, 9},
- {3125000, 0x80000, 0},
- {2500000, 0x400000, 9},
- {2000000, 0x666666, 19},
- {1563000, 0x40000, 0},
- {1250000, 0x200000, 9},
- {1000000, 0x400000, 24},
- {800000, 0x666666, 49},
- {781250, 0x20000, 0},
- {625000, 0x200000, 19},
- {500000, 0x400000, 49},
- {400000, 0x666666, 99},
- {390625, 0x10000, 0},
- {250000, 0x400000, 99},
- {200000, 0x666666, 199},
- {195313, 0x8000, 0},
- {125000, 0x100000, 49},
- {100000, 0x200000, 124},
- {50000, 0x100000, 124},
- {25000, 0x80000, 124},
- {10016, 0x20000, 77},
- {5040, 0x20000, 154},
- {1002, 0x8000, 194},
-};
-
/* Offset from drv_data->lpss_base */
#define GENERAL_REG 0x08
#define GENERAL_REG_RXTO_HOLDOFF_DISABLE BIT(24)
}
/*
- * The Quark SPI data sheet gives a table, and for the given 'rate',
- * the 'dds' and 'clk_div' can be found in the table.
+ * The Quark SPI has an additional 24 bit register (DDS_CLK_RATE) to multiply
+ * input frequency by fractions of 2^24. It also has a divider by 5.
+ *
+ * There are formulas to get baud rate value for given input frequency and
+ * divider parameters, such as DDS_CLK_RATE and SCR:
+ *
+ * Fsys = 200MHz
+ *
+ * Fssp = Fsys * DDS_CLK_RATE / 2^24 (1)
+ * Baud rate = Fsclk = Fssp / (2 * (SCR + 1)) (2)
+ *
+ * DDS_CLK_RATE either 2^n or 2^n / 5.
+ * SCR is in range 0 .. 255
+ *
+ * Divisor = 5^i * 2^j * 2 * k
+ * i = [0, 1] i = 1 iff j = 0 or j > 3
+ * j = [0, 23] j = 0 iff i = 1
+ * k = [1, 256]
+ * Special case: j = 0, i = 1: Divisor = 2 / 5
+ *
+ * Accordingly to the specification the recommended values for DDS_CLK_RATE
+ * are:
+ * Case 1: 2^n, n = [0, 23]
+ * Case 2: 2^24 * 2 / 5 (0x666666)
+ * Case 3: less than or equal to 2^24 / 5 / 16 (0x33333)
+ *
+ * In all cases the lowest possible value is better.
+ *
+ * The function calculates parameters for all cases and chooses the one closest
+ * to the asked baud rate.
*/
-static u32 quark_x1000_set_clk_regvals(u32 rate, u32 *dds, u32 *clk_div)
+static unsigned int quark_x1000_get_clk_div(int rate, u32 *dds)
{
- unsigned int i;
-
- for (i = 0; i < ARRAY_SIZE(quark_spi_rate_table); i++) {
- if (rate >= quark_spi_rate_table[i].bitrate) {
- *dds = quark_spi_rate_table[i].dds_clk_rate;
- *clk_div = quark_spi_rate_table[i].clk_div;
- return quark_spi_rate_table[i].bitrate;
+ unsigned long xtal = 200000000;
+ unsigned long fref = xtal / 2; /* mandatory division by 2,
+ see (2) */
+ /* case 3 */
+ unsigned long fref1 = fref / 2; /* case 1 */
+ unsigned long fref2 = fref * 2 / 5; /* case 2 */
+ unsigned long scale;
+ unsigned long q, q1, q2;
+ long r, r1, r2;
+ u32 mul;
+
+ /* Case 1 */
+
+ /* Set initial value for DDS_CLK_RATE */
+ mul = (1 << 24) >> 1;
+
+ /* Calculate initial quot */
+ q1 = DIV_ROUND_CLOSEST(fref1, rate);
+
+ /* Scale q1 if it's too big */
+ if (q1 > 256) {
+ /* Scale q1 to range [1, 512] */
+ scale = fls_long(q1 - 1);
+ if (scale > 9) {
+ q1 >>= scale - 9;
+ mul >>= scale - 9;
}
+
+ /* Round the result if we have a remainder */
+ q1 += q1 & 1;
}
- *dds = quark_spi_rate_table[i-1].dds_clk_rate;
- *clk_div = quark_spi_rate_table[i-1].clk_div;
+ /* Decrease DDS_CLK_RATE as much as we can without loss in precision */
+ scale = __ffs(q1);
+ q1 >>= scale;
+ mul >>= scale;
+
+ /* Get the remainder */
+ r1 = abs(fref1 / (1 << (24 - fls_long(mul))) / q1 - rate);
+
+ /* Case 2 */
+
+ q2 = DIV_ROUND_CLOSEST(fref2, rate);
+ r2 = abs(fref2 / q2 - rate);
- return quark_spi_rate_table[i-1].bitrate;
+ /*
+ * Choose the best between two: less remainder we have the better. We
+ * can't go case 2 if q2 is greater than 256 since SCR register can
+ * hold only values 0 .. 255.
+ */
+ if (r2 >= r1 || q2 > 256) {
+ /* case 1 is better */
+ r = r1;
+ q = q1;
+ } else {
+ /* case 2 is better */
+ r = r2;
+ q = q2;
+ mul = (1 << 24) * 2 / 5;
+ }
+
+ /* Check case 3 only If the divisor is big enough */
+ if (fref / rate >= 80) {
+ u64 fssp;
+ u32 m;
+
+ /* Calculate initial quot */
+ q1 = DIV_ROUND_CLOSEST(fref, rate);
+ m = (1 << 24) / q1;
+
+ /* Get the remainder */
+ fssp = (u64)fref * m;
+ do_div(fssp, 1 << 24);
+ r1 = abs(fssp - rate);
+
+ /* Choose this one if it suits better */
+ if (r1 < r) {
+ /* case 3 is better */
+ q = 1;
+ mul = m;
+ }
+ }
+
+ *dds = mul;
+ return q - 1;
}
static unsigned int ssp_get_clk_div(struct driver_data *drv_data, int rate)
rate = min_t(int, ssp_clk, rate);
if (ssp->type == PXA25x_SSP || ssp->type == CE4100_SSP)
- return ((ssp_clk / (2 * rate) - 1) & 0xff) << 8;
+ return (ssp_clk / (2 * rate) - 1) & 0xff;
else
- return ((ssp_clk / rate - 1) & 0xfff) << 8;
+ return (ssp_clk / rate - 1) & 0xfff;
}
static unsigned int pxa2xx_ssp_get_clk_div(struct driver_data *drv_data,
struct chip_data *chip, int rate)
{
- u32 clk_div;
+ unsigned int clk_div;
switch (drv_data->ssp_type) {
case QUARK_X1000_SSP:
- quark_x1000_set_clk_regvals(rate, &chip->dds_rate, &clk_div);
- return clk_div << 8;
+ clk_div = quark_x1000_get_clk_div(rate, &chip->dds_rate);
+ break;
default:
- return ssp_get_clk_div(drv_data, rate);
+ clk_div = ssp_get_clk_div(drv_data, rate);
+ break;
}
+ return clk_div << 8;
}
static void pump_transfers(unsigned long data)
#include <linux/platform_device.h>
#include <linux/pm_runtime.h>
#include <linux/spi/spi.h>
+#include <linux/dmaengine.h>
+#include <linux/dma-mapping.h>
#define QUP_CONFIG 0x0000
#define QUP_STATE 0x0004
#define SPI_NUM_CHIPSELECTS 4
+#define SPI_MAX_DMA_XFER (SZ_64K - 64)
+
/* high speed mode is when bus rate is greater then 26MHz */
#define SPI_HS_MIN_RATE 26000000
#define SPI_MAX_RATE 50000000
struct completion done;
int error;
int w_size; /* bytes per SPI word */
+ int n_words;
int tx_bytes;
int rx_bytes;
int qup_v1;
+
+ int use_dma;
+ struct dma_slave_config rx_conf;
+ struct dma_slave_config tx_conf;
};
return 0;
}
-
static void spi_qup_fifo_read(struct spi_qup *controller,
struct spi_transfer *xfer)
{
}
}
+static void spi_qup_dma_done(void *data)
+{
+ struct spi_qup *qup = data;
+
+ complete(&qup->done);
+}
+
+static int spi_qup_prep_sg(struct spi_master *master, struct spi_transfer *xfer,
+ enum dma_transfer_direction dir,
+ dma_async_tx_callback callback)
+{
+ struct spi_qup *qup = spi_master_get_devdata(master);
+ unsigned long flags = DMA_PREP_INTERRUPT | DMA_PREP_FENCE;
+ struct dma_async_tx_descriptor *desc;
+ struct scatterlist *sgl;
+ struct dma_chan *chan;
+ dma_cookie_t cookie;
+ unsigned int nents;
+
+ if (dir == DMA_MEM_TO_DEV) {
+ chan = master->dma_tx;
+ nents = xfer->tx_sg.nents;
+ sgl = xfer->tx_sg.sgl;
+ } else {
+ chan = master->dma_rx;
+ nents = xfer->rx_sg.nents;
+ sgl = xfer->rx_sg.sgl;
+ }
+
+ desc = dmaengine_prep_slave_sg(chan, sgl, nents, dir, flags);
+ if (!desc)
+ return -EINVAL;
+
+ desc->callback = callback;
+ desc->callback_param = qup;
+
+ cookie = dmaengine_submit(desc);
+
+ return dma_submit_error(cookie);
+}
+
+static void spi_qup_dma_terminate(struct spi_master *master,
+ struct spi_transfer *xfer)
+{
+ if (xfer->tx_buf)
+ dmaengine_terminate_all(master->dma_tx);
+ if (xfer->rx_buf)
+ dmaengine_terminate_all(master->dma_rx);
+}
+
+static int spi_qup_do_dma(struct spi_master *master, struct spi_transfer *xfer)
+{
+ dma_async_tx_callback rx_done = NULL, tx_done = NULL;
+ int ret;
+
+ if (xfer->rx_buf)
+ rx_done = spi_qup_dma_done;
+ else if (xfer->tx_buf)
+ tx_done = spi_qup_dma_done;
+
+ if (xfer->rx_buf) {
+ ret = spi_qup_prep_sg(master, xfer, DMA_DEV_TO_MEM, rx_done);
+ if (ret)
+ return ret;
+
+ dma_async_issue_pending(master->dma_rx);
+ }
+
+ if (xfer->tx_buf) {
+ ret = spi_qup_prep_sg(master, xfer, DMA_MEM_TO_DEV, tx_done);
+ if (ret)
+ return ret;
+
+ dma_async_issue_pending(master->dma_tx);
+ }
+
+ return 0;
+}
+
+static int spi_qup_do_pio(struct spi_master *master, struct spi_transfer *xfer)
+{
+ struct spi_qup *qup = spi_master_get_devdata(master);
+ int ret;
+
+ ret = spi_qup_set_state(qup, QUP_STATE_RUN);
+ if (ret) {
+ dev_warn(qup->dev, "cannot set RUN state\n");
+ return ret;
+ }
+
+ ret = spi_qup_set_state(qup, QUP_STATE_PAUSE);
+ if (ret) {
+ dev_warn(qup->dev, "cannot set PAUSE state\n");
+ return ret;
+ }
+
+ spi_qup_fifo_write(qup, xfer);
+
+ return 0;
+}
+
static irqreturn_t spi_qup_qup_irq(int irq, void *dev_id)
{
struct spi_qup *controller = dev_id;
error = -EIO;
}
- if (opflags & QUP_OP_IN_SERVICE_FLAG)
- spi_qup_fifo_read(controller, xfer);
+ if (!controller->use_dma) {
+ if (opflags & QUP_OP_IN_SERVICE_FLAG)
+ spi_qup_fifo_read(controller, xfer);
- if (opflags & QUP_OP_OUT_SERVICE_FLAG)
- spi_qup_fifo_write(controller, xfer);
+ if (opflags & QUP_OP_OUT_SERVICE_FLAG)
+ spi_qup_fifo_write(controller, xfer);
+ }
spin_lock_irqsave(&controller->lock, flags);
controller->error = error;
return IRQ_HANDLED;
}
+static u32
+spi_qup_get_mode(struct spi_master *master, struct spi_transfer *xfer)
+{
+ struct spi_qup *qup = spi_master_get_devdata(master);
+ u32 mode;
+
+ qup->w_size = 4;
+
+ if (xfer->bits_per_word <= 8)
+ qup->w_size = 1;
+ else if (xfer->bits_per_word <= 16)
+ qup->w_size = 2;
+
+ qup->n_words = xfer->len / qup->w_size;
+
+ if (qup->n_words <= (qup->in_fifo_sz / sizeof(u32)))
+ mode = QUP_IO_M_MODE_FIFO;
+ else
+ mode = QUP_IO_M_MODE_BLOCK;
+
+ return mode;
+}
/* set clock freq ... bits per word */
static int spi_qup_io_config(struct spi_device *spi, struct spi_transfer *xfer)
{
struct spi_qup *controller = spi_master_get_devdata(spi->master);
u32 config, iomode, mode, control;
- int ret, n_words, w_size;
+ int ret, n_words;
if (spi->mode & SPI_LOOP && xfer->len > controller->in_fifo_sz) {
dev_err(controller->dev, "too big size for loopback %d > %d\n",
return -EIO;
}
- w_size = 4;
- if (xfer->bits_per_word <= 8)
- w_size = 1;
- else if (xfer->bits_per_word <= 16)
- w_size = 2;
+ mode = spi_qup_get_mode(spi->master, xfer);
+ n_words = controller->n_words;
- n_words = xfer->len / w_size;
- controller->w_size = w_size;
-
- if (n_words <= (controller->in_fifo_sz / sizeof(u32))) {
- mode = QUP_IO_M_MODE_FIFO;
+ if (mode == QUP_IO_M_MODE_FIFO) {
writel_relaxed(n_words, controller->base + QUP_MX_READ_CNT);
writel_relaxed(n_words, controller->base + QUP_MX_WRITE_CNT);
/* must be zero for FIFO */
writel_relaxed(0, controller->base + QUP_MX_INPUT_CNT);
writel_relaxed(0, controller->base + QUP_MX_OUTPUT_CNT);
- } else {
- mode = QUP_IO_M_MODE_BLOCK;
+ } else if (!controller->use_dma) {
writel_relaxed(n_words, controller->base + QUP_MX_INPUT_CNT);
writel_relaxed(n_words, controller->base + QUP_MX_OUTPUT_CNT);
/* must be zero for BLOCK and BAM */
writel_relaxed(0, controller->base + QUP_MX_READ_CNT);
writel_relaxed(0, controller->base + QUP_MX_WRITE_CNT);
+ } else {
+ mode = QUP_IO_M_MODE_BAM;
+ writel_relaxed(0, controller->base + QUP_MX_READ_CNT);
+ writel_relaxed(0, controller->base + QUP_MX_WRITE_CNT);
+
+ if (!controller->qup_v1) {
+ void __iomem *input_cnt;
+
+ input_cnt = controller->base + QUP_MX_INPUT_CNT;
+ /*
+ * for DMA transfers, both QUP_MX_INPUT_CNT and
+ * QUP_MX_OUTPUT_CNT must be zero to all cases but one.
+ * That case is a non-balanced transfer when there is
+ * only a rx_buf.
+ */
+ if (xfer->tx_buf)
+ writel_relaxed(0, input_cnt);
+ else
+ writel_relaxed(n_words, input_cnt);
+
+ writel_relaxed(0, controller->base + QUP_MX_OUTPUT_CNT);
+ }
}
iomode = readl_relaxed(controller->base + QUP_IO_M_MODES);
/* Set input and output transfer mode */
iomode &= ~(QUP_IO_M_INPUT_MODE_MASK | QUP_IO_M_OUTPUT_MODE_MASK);
- iomode &= ~(QUP_IO_M_PACK_EN | QUP_IO_M_UNPACK_EN);
+
+ if (!controller->use_dma)
+ iomode &= ~(QUP_IO_M_PACK_EN | QUP_IO_M_UNPACK_EN);
+ else
+ iomode |= QUP_IO_M_PACK_EN | QUP_IO_M_UNPACK_EN;
+
iomode |= (mode << QUP_IO_M_OUTPUT_MODE_MASK_SHIFT);
iomode |= (mode << QUP_IO_M_INPUT_MODE_MASK_SHIFT);
config &= ~(QUP_CONFIG_NO_INPUT | QUP_CONFIG_NO_OUTPUT | QUP_CONFIG_N);
config |= xfer->bits_per_word - 1;
config |= QUP_CONFIG_SPI_MODE;
+
+ if (controller->use_dma) {
+ if (!xfer->tx_buf)
+ config |= QUP_CONFIG_NO_OUTPUT;
+ if (!xfer->rx_buf)
+ config |= QUP_CONFIG_NO_INPUT;
+ }
+
writel_relaxed(config, controller->base + QUP_CONFIG);
/* only write to OPERATIONAL_MASK when register is present */
- if (!controller->qup_v1)
- writel_relaxed(0, controller->base + QUP_OPERATIONAL_MASK);
+ if (!controller->qup_v1) {
+ u32 mask = 0;
+
+ /*
+ * mask INPUT and OUTPUT service flags to prevent IRQs on FIFO
+ * status change in BAM mode
+ */
+
+ if (mode == QUP_IO_M_MODE_BAM)
+ mask = QUP_OP_IN_SERVICE_FLAG | QUP_OP_OUT_SERVICE_FLAG;
+
+ writel_relaxed(mask, controller->base + QUP_OPERATIONAL_MASK);
+ }
+
return 0;
}
controller->tx_bytes = 0;
spin_unlock_irqrestore(&controller->lock, flags);
- if (spi_qup_set_state(controller, QUP_STATE_RUN)) {
- dev_warn(controller->dev, "cannot set RUN state\n");
- goto exit;
- }
+ if (controller->use_dma)
+ ret = spi_qup_do_dma(master, xfer);
+ else
+ ret = spi_qup_do_pio(master, xfer);
- if (spi_qup_set_state(controller, QUP_STATE_PAUSE)) {
- dev_warn(controller->dev, "cannot set PAUSE state\n");
+ if (ret)
goto exit;
- }
-
- spi_qup_fifo_write(controller, xfer);
if (spi_qup_set_state(controller, QUP_STATE_RUN)) {
dev_warn(controller->dev, "cannot set EXECUTE state\n");
if (!wait_for_completion_timeout(&controller->done, timeout))
ret = -ETIMEDOUT;
+
exit:
spi_qup_set_state(controller, QUP_STATE_RESET);
spin_lock_irqsave(&controller->lock, flags);
if (!ret)
ret = controller->error;
spin_unlock_irqrestore(&controller->lock, flags);
+
+ if (ret && controller->use_dma)
+ spi_qup_dma_terminate(master, xfer);
+
+ return ret;
+}
+
+static bool spi_qup_can_dma(struct spi_master *master, struct spi_device *spi,
+ struct spi_transfer *xfer)
+{
+ struct spi_qup *qup = spi_master_get_devdata(master);
+ size_t dma_align = dma_get_cache_alignment();
+ u32 mode;
+
+ qup->use_dma = 0;
+
+ if (xfer->rx_buf && (xfer->len % qup->in_blk_sz ||
+ IS_ERR_OR_NULL(master->dma_rx) ||
+ !IS_ALIGNED((size_t)xfer->rx_buf, dma_align)))
+ return false;
+
+ if (xfer->tx_buf && (xfer->len % qup->out_blk_sz ||
+ IS_ERR_OR_NULL(master->dma_tx) ||
+ !IS_ALIGNED((size_t)xfer->tx_buf, dma_align)))
+ return false;
+
+ mode = spi_qup_get_mode(master, xfer);
+ if (mode == QUP_IO_M_MODE_FIFO)
+ return false;
+
+ qup->use_dma = 1;
+
+ return true;
+}
+
+static void spi_qup_release_dma(struct spi_master *master)
+{
+ if (!IS_ERR_OR_NULL(master->dma_rx))
+ dma_release_channel(master->dma_rx);
+ if (!IS_ERR_OR_NULL(master->dma_tx))
+ dma_release_channel(master->dma_tx);
+}
+
+static int spi_qup_init_dma(struct spi_master *master, resource_size_t base)
+{
+ struct spi_qup *spi = spi_master_get_devdata(master);
+ struct dma_slave_config *rx_conf = &spi->rx_conf,
+ *tx_conf = &spi->tx_conf;
+ struct device *dev = spi->dev;
+ int ret;
+
+ /* allocate dma resources, if available */
+ master->dma_rx = dma_request_slave_channel_reason(dev, "rx");
+ if (IS_ERR(master->dma_rx))
+ return PTR_ERR(master->dma_rx);
+
+ master->dma_tx = dma_request_slave_channel_reason(dev, "tx");
+ if (IS_ERR(master->dma_tx)) {
+ ret = PTR_ERR(master->dma_tx);
+ goto err_tx;
+ }
+
+ /* set DMA parameters */
+ rx_conf->direction = DMA_DEV_TO_MEM;
+ rx_conf->device_fc = 1;
+ rx_conf->src_addr = base + QUP_INPUT_FIFO;
+ rx_conf->src_maxburst = spi->in_blk_sz;
+
+ tx_conf->direction = DMA_MEM_TO_DEV;
+ tx_conf->device_fc = 1;
+ tx_conf->dst_addr = base + QUP_OUTPUT_FIFO;
+ tx_conf->dst_maxburst = spi->out_blk_sz;
+
+ ret = dmaengine_slave_config(master->dma_rx, rx_conf);
+ if (ret) {
+ dev_err(dev, "failed to configure RX channel\n");
+ goto err;
+ }
+
+ ret = dmaengine_slave_config(master->dma_tx, tx_conf);
+ if (ret) {
+ dev_err(dev, "failed to configure TX channel\n");
+ goto err;
+ }
+
+ return 0;
+
+err:
+ dma_release_channel(master->dma_tx);
+err_tx:
+ dma_release_channel(master->dma_rx);
return ret;
}
struct resource *res;
struct device *dev;
void __iomem *base;
- u32 max_freq, iomode;
+ u32 max_freq, iomode, num_cs;
int ret, irq, size;
dev = &pdev->dev;
}
/* use num-cs unless not present or out of range */
- if (of_property_read_u16(dev->of_node, "num-cs",
- &master->num_chipselect) ||
- (master->num_chipselect > SPI_NUM_CHIPSELECTS))
+ if (of_property_read_u32(dev->of_node, "num-cs", &num_cs) ||
+ num_cs > SPI_NUM_CHIPSELECTS)
master->num_chipselect = SPI_NUM_CHIPSELECTS;
+ else
+ master->num_chipselect = num_cs;
master->bus_num = pdev->id;
master->mode_bits = SPI_CPOL | SPI_CPHA | SPI_CS_HIGH | SPI_LOOP;
master->transfer_one = spi_qup_transfer_one;
master->dev.of_node = pdev->dev.of_node;
master->auto_runtime_pm = true;
+ master->dma_alignment = dma_get_cache_alignment();
+ master->max_dma_len = SPI_MAX_DMA_XFER;
platform_set_drvdata(pdev, master);
controller->cclk = cclk;
controller->irq = irq;
+ ret = spi_qup_init_dma(master, res->start);
+ if (ret == -EPROBE_DEFER)
+ goto error;
+ else if (!ret)
+ master->can_dma = spi_qup_can_dma;
+
/* set v1 flag if device is version 1 */
if (of_device_is_compatible(dev->of_node, "qcom,spi-qup-v1.1.1"))
controller->qup_v1 = 1;
ret = spi_qup_set_state(controller, QUP_STATE_RESET);
if (ret) {
dev_err(dev, "cannot set RESET state\n");
- goto error;
+ goto error_dma;
}
writel_relaxed(0, base + QUP_OPERATIONAL);
ret = devm_request_irq(dev, irq, spi_qup_qup_irq,
IRQF_TRIGGER_HIGH, pdev->name, controller);
if (ret)
- goto error;
+ goto error_dma;
pm_runtime_set_autosuspend_delay(dev, MSEC_PER_SEC);
pm_runtime_use_autosuspend(dev);
disable_pm:
pm_runtime_disable(&pdev->dev);
+error_dma:
+ spi_qup_release_dma(master);
error:
clk_disable_unprepare(cclk);
clk_disable_unprepare(iclk);
if (ret)
return ret;
+ spi_qup_release_dma(master);
+
clk_disable_unprepare(controller->cclk);
clk_disable_unprepare(controller->iclk);
u8 tmode;
u8 bpw;
u8 n_bytes;
+ u8 rsd_nsecs;
unsigned len;
u32 speed;
return 0;
}
-static int rockchip_spi_unprepare_message(struct spi_master *master,
- struct spi_message *msg)
+static void rockchip_spi_handle_err(struct spi_master *master,
+ struct spi_message *msg)
{
unsigned long flags;
struct rockchip_spi *rs = spi_master_get_devdata(master);
/*
* For DMA mode, we need terminate DMA channel and flush
* fifo for the next transfer if DMA thansfer timeout.
- * unprepare_message() was called by core if transfer complete
- * or timeout. Maybe it is reasonable for error handling here.
+ * handle_err() was called by core if transfer failed.
+ * Maybe it is reasonable for error handling here.
*/
if (rs->use_dma) {
if (rs->state & RXBUSY) {
}
spin_unlock_irqrestore(&rs->lock, flags);
+}
+
+static int rockchip_spi_unprepare_message(struct spi_master *master,
+ struct spi_message *msg)
+{
+ struct rockchip_spi *rs = spi_master_get_devdata(master);
spi_enable_chip(rs, 0);
{
u32 div = 0;
u32 dmacr = 0;
+ int rsd = 0;
u32 cr0 = (CR0_BHT_8BIT << CR0_BHT_OFFSET)
| (CR0_SSD_ONE << CR0_SSD_OFFSET);
}
/* div doesn't support odd number */
- div = max_t(u32, rs->max_freq / rs->speed, 1);
+ div = DIV_ROUND_UP(rs->max_freq, rs->speed);
div = (div + 1) & 0xfffe;
+ /* Rx sample delay is expressed in parent clock cycles (max 3) */
+ rsd = DIV_ROUND_CLOSEST(rs->rsd_nsecs * (rs->max_freq >> 8),
+ 1000000000 >> 8);
+ if (!rsd && rs->rsd_nsecs) {
+ pr_warn_once("rockchip-spi: %u Hz are too slow to express %u ns delay\n",
+ rs->max_freq, rs->rsd_nsecs);
+ } else if (rsd > 3) {
+ rsd = 3;
+ pr_warn_once("rockchip-spi: %u Hz are too fast to express %u ns delay, clamping at %u ns\n",
+ rs->max_freq, rs->rsd_nsecs,
+ rsd * 1000000000U / rs->max_freq);
+ }
+ cr0 |= rsd << CR0_RSD_OFFSET;
+
writel_relaxed(cr0, rs->regs + ROCKCHIP_SPI_CTRLR0);
writel_relaxed(rs->len - 1, rs->regs + ROCKCHIP_SPI_CTRLR1);
struct rockchip_spi *rs;
struct spi_master *master;
struct resource *mem;
+ u32 rsd_nsecs;
master = spi_alloc_master(&pdev->dev, sizeof(struct rockchip_spi));
if (!master)
rs->dev = &pdev->dev;
rs->max_freq = clk_get_rate(rs->spiclk);
+ if (!of_property_read_u32(pdev->dev.of_node, "rx-sample-delay-ns",
+ &rsd_nsecs))
+ rs->rsd_nsecs = rsd_nsecs;
+
rs->fifo_len = get_fifo_len(rs);
if (!rs->fifo_len) {
dev_err(&pdev->dev, "Failed to get fifo length\n");
master->prepare_message = rockchip_spi_prepare_message;
master->unprepare_message = rockchip_spi_unprepare_message;
master->transfer_one = rockchip_spi_transfer_one;
+ master->handle_err = rockchip_spi_handle_err;
rs->dma_tx.ch = dma_request_slave_channel(rs->dev, "tx");
if (!rs->dma_tx.ch)
#define SPBFCR_RXRST 0x40 /* Receive Buffer Data Reset */
#define SPBFCR_TXTRG_MASK 0x30 /* Transmit Buffer Data Triggering Number */
#define SPBFCR_RXTRG_MASK 0x07 /* Receive Buffer Data Triggering Number */
+/* QSPI on R-Car Gen2 */
+#define SPBFCR_TXTRG_1B 0x00 /* 31 bytes (1 byte available) */
+#define SPBFCR_TXTRG_32B 0x30 /* 0 byte (32 bytes available) */
+#define SPBFCR_RXTRG_1B 0x00 /* 1 byte (31 bytes available) */
+#define SPBFCR_RXTRG_32B 0x07 /* 32 bytes (0 byte available) */
+
+#define QSPI_BUFFER_SIZE 32u
struct rspi_data {
void __iomem *addr;
return 0;
}
+static void qspi_update(const struct rspi_data *rspi, u8 mask, u8 val, u8 reg)
+{
+ u8 data;
+
+ data = rspi_read8(rspi, reg);
+ data &= ~mask;
+ data |= (val & mask);
+ rspi_write8(rspi, data, reg);
+}
+
+static int qspi_set_send_trigger(struct rspi_data *rspi, unsigned int len)
+{
+ unsigned int n;
+
+ n = min(len, QSPI_BUFFER_SIZE);
+
+ if (len >= QSPI_BUFFER_SIZE) {
+ /* sets triggering number to 32 bytes */
+ qspi_update(rspi, SPBFCR_TXTRG_MASK,
+ SPBFCR_TXTRG_32B, QSPI_SPBFCR);
+ } else {
+ /* sets triggering number to 1 byte */
+ qspi_update(rspi, SPBFCR_TXTRG_MASK,
+ SPBFCR_TXTRG_1B, QSPI_SPBFCR);
+ }
+
+ return n;
+}
+
+static void qspi_set_receive_trigger(struct rspi_data *rspi, unsigned int len)
+{
+ unsigned int n;
+
+ n = min(len, QSPI_BUFFER_SIZE);
+
+ if (len >= QSPI_BUFFER_SIZE) {
+ /* sets triggering number to 32 bytes */
+ qspi_update(rspi, SPBFCR_RXTRG_MASK,
+ SPBFCR_RXTRG_32B, QSPI_SPBFCR);
+ } else {
+ /* sets triggering number to 1 byte */
+ qspi_update(rspi, SPBFCR_RXTRG_MASK,
+ SPBFCR_RXTRG_1B, QSPI_SPBFCR);
+ }
+}
+
#define set_config_register(spi, n) spi->ops->set_config_register(spi, n)
static void rspi_enable_irq(const struct rspi_data *rspi, u8 enable)
return __rspi_can_dma(rspi, xfer);
}
-static int rspi_common_transfer(struct rspi_data *rspi,
- struct spi_transfer *xfer)
+static int rspi_dma_check_then_transfer(struct rspi_data *rspi,
+ struct spi_transfer *xfer)
{
- int ret;
-
if (rspi->master->can_dma && __rspi_can_dma(rspi, xfer)) {
/* rx_buf can be NULL on RSPI on SH in TX-only Mode */
- ret = rspi_dma_transfer(rspi, &xfer->tx_sg,
+ int ret = rspi_dma_transfer(rspi, &xfer->tx_sg,
xfer->rx_buf ? &xfer->rx_sg : NULL);
if (ret != -EAGAIN)
- return ret;
+ return 0;
}
+ return -EAGAIN;
+}
+
+static int rspi_common_transfer(struct rspi_data *rspi,
+ struct spi_transfer *xfer)
+{
+ int ret;
+
+ ret = rspi_dma_check_then_transfer(rspi, xfer);
+ if (ret != -EAGAIN)
+ return ret;
+
ret = rspi_pio_transfer(rspi, xfer->tx_buf, xfer->rx_buf, xfer->len);
if (ret < 0)
return ret;
return rspi_common_transfer(rspi, xfer);
}
+static int qspi_trigger_transfer_out_int(struct rspi_data *rspi, const u8 *tx,
+ u8 *rx, unsigned int len)
+{
+ int i, n, ret;
+ int error;
+
+ while (len > 0) {
+ n = qspi_set_send_trigger(rspi, len);
+ qspi_set_receive_trigger(rspi, len);
+ if (n == QSPI_BUFFER_SIZE) {
+ error = rspi_wait_for_tx_empty(rspi);
+ if (error < 0) {
+ dev_err(&rspi->master->dev, "transmit timeout\n");
+ return error;
+ }
+ for (i = 0; i < n; i++)
+ rspi_write_data(rspi, *tx++);
+
+ error = rspi_wait_for_rx_full(rspi);
+ if (error < 0) {
+ dev_err(&rspi->master->dev, "receive timeout\n");
+ return error;
+ }
+ for (i = 0; i < n; i++)
+ *rx++ = rspi_read_data(rspi);
+ } else {
+ ret = rspi_pio_transfer(rspi, tx, rx, n);
+ if (ret < 0)
+ return ret;
+ }
+ len -= n;
+ }
+
+ return 0;
+}
+
static int qspi_transfer_out_in(struct rspi_data *rspi,
struct spi_transfer *xfer)
{
+ int ret;
+
qspi_receive_init(rspi);
- return rspi_common_transfer(rspi, xfer);
+ ret = rspi_dma_check_then_transfer(rspi, xfer);
+ if (ret != -EAGAIN)
+ return ret;
+
+ ret = qspi_trigger_transfer_out_int(rspi, xfer->tx_buf,
+ xfer->rx_buf, xfer->len);
+ if (ret < 0)
+ return ret;
+
+ return 0;
}
static int qspi_transfer_out(struct rspi_data *rspi, struct spi_transfer *xfer)
/* Acquire DMA channels */
sdd->rx_dma.ch = dma_request_slave_channel_compat(mask, filter,
- (void *)sdd->rx_dma.dmach, dev, "rx");
+ (void *)(long)sdd->rx_dma.dmach, dev, "rx");
if (!sdd->rx_dma.ch) {
dev_err(dev, "Failed to get RX DMA channel\n");
ret = -EBUSY;
spi->dma_rx = sdd->rx_dma.ch;
sdd->tx_dma.ch = dma_request_slave_channel_compat(mask, filter,
- (void *)sdd->tx_dma.dmach, dev, "tx");
+ (void *)(long)sdd->tx_dma.dmach, dev, "tx");
if (!sdd->tx_dma.ch) {
dev_err(dev, "Failed to get TX DMA channel\n");
ret = -EBUSY;
hw->freq = SC18IS602_CLOCK;
break;
}
- master->bus_num = client->adapter->nr;
+ master->bus_num = np ? -1 : client->adapter->nr;
master->mode_bits = SPI_CPHA | SPI_CPOL | SPI_LSB_FIRST;
master->bits_per_word_mask = SPI_BPW_MASK(8);
master->setup = sc18is602_setup;
SET_RUNTIME_PM_OPS(spi_st_runtime_suspend, spi_st_runtime_resume, NULL)
};
-static struct of_device_id stm_spi_match[] = {
+static const struct of_device_id stm_spi_match[] = {
{ .compatible = "st,comms-ssc4-spi", },
{},
};
#define QSPI_FLEN(n) ((n - 1) << 0)
/* STATUS REGISTER */
+#define BUSY 0x01
#define WC 0x02
/* INTERRUPT REGISTER */
ti_qspi_write(qspi, ctx_reg->clkctrl, QSPI_SPI_CLOCK_CNTRL_REG);
}
+static inline u32 qspi_is_busy(struct ti_qspi *qspi)
+{
+ u32 stat;
+ unsigned long timeout = jiffies + QSPI_COMPLETION_TIMEOUT;
+
+ stat = ti_qspi_read(qspi, QSPI_SPI_STATUS_REG);
+ while ((stat & BUSY) && time_after(timeout, jiffies)) {
+ cpu_relax();
+ stat = ti_qspi_read(qspi, QSPI_SPI_STATUS_REG);
+ }
+
+ WARN(stat & BUSY, "qspi busy\n");
+ return stat & BUSY;
+}
+
static int qspi_write_msg(struct ti_qspi *qspi, struct spi_transfer *t)
{
int wlen, count;
wlen = t->bits_per_word >> 3; /* in bytes */
while (count) {
+ if (qspi_is_busy(qspi))
+ return -EBUSY;
+
switch (wlen) {
case 1:
dev_dbg(qspi->dev, "tx cmd %08x dc %08x data %02x\n",
while (count) {
dev_dbg(qspi->dev, "rx cmd %08x dc %08x\n", cmd, qspi->dc);
+ if (qspi_is_busy(qspi))
+ return -EBUSY;
+
ti_qspi_write(qspi, cmd, QSPI_SPI_CMD_REG);
if (!wait_for_completion_timeout(&qspi->transfer_complete,
QSPI_COMPLETION_TIMEOUT)) {
*/
#include <linux/kernel.h>
-#include <linux/kmod.h>
#include <linux/device.h>
#include <linux/init.h>
#include <linux/cache.h>
return 0;
}
-#ifdef CONFIG_PM_SLEEP
-static int spi_legacy_suspend(struct device *dev, pm_message_t message)
-{
- int value = 0;
- struct spi_driver *drv = to_spi_driver(dev->driver);
-
- /* suspend will stop irqs and dma; no more i/o */
- if (drv) {
- if (drv->suspend)
- value = drv->suspend(to_spi_device(dev), message);
- else
- dev_dbg(dev, "... can't suspend\n");
- }
- return value;
-}
-
-static int spi_legacy_resume(struct device *dev)
-{
- int value = 0;
- struct spi_driver *drv = to_spi_driver(dev->driver);
-
- /* resume may restart the i/o queue */
- if (drv) {
- if (drv->resume)
- value = drv->resume(to_spi_device(dev));
- else
- dev_dbg(dev, "... can't resume\n");
- }
- return value;
-}
-
-static int spi_pm_suspend(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_suspend(dev);
- else
- return spi_legacy_suspend(dev, PMSG_SUSPEND);
-}
-
-static int spi_pm_resume(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_resume(dev);
- else
- return spi_legacy_resume(dev);
-}
-
-static int spi_pm_freeze(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_freeze(dev);
- else
- return spi_legacy_suspend(dev, PMSG_FREEZE);
-}
-
-static int spi_pm_thaw(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_thaw(dev);
- else
- return spi_legacy_resume(dev);
-}
-
-static int spi_pm_poweroff(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_poweroff(dev);
- else
- return spi_legacy_suspend(dev, PMSG_HIBERNATE);
-}
-
-static int spi_pm_restore(struct device *dev)
-{
- const struct dev_pm_ops *pm = dev->driver ? dev->driver->pm : NULL;
-
- if (pm)
- return pm_generic_restore(dev);
- else
- return spi_legacy_resume(dev);
-}
-#else
-#define spi_pm_suspend NULL
-#define spi_pm_resume NULL
-#define spi_pm_freeze NULL
-#define spi_pm_thaw NULL
-#define spi_pm_poweroff NULL
-#define spi_pm_restore NULL
-#endif
-
-static const struct dev_pm_ops spi_pm = {
- .suspend = spi_pm_suspend,
- .resume = spi_pm_resume,
- .freeze = spi_pm_freeze,
- .thaw = spi_pm_thaw,
- .poweroff = spi_pm_poweroff,
- .restore = spi_pm_restore,
- SET_RUNTIME_PM_OPS(
- pm_generic_runtime_suspend,
- pm_generic_runtime_resume,
- NULL
- )
-};
-
struct bus_type spi_bus_type = {
.name = "spi",
.dev_groups = spi_dev_groups,
.match = spi_match_device,
.uevent = spi_uevent,
- .pm = &spi_pm,
};
EXPORT_SYMBOL_GPL(spi_bus_type);
if (msg->status == -EINPROGRESS)
msg->status = ret;
+ if (msg->status && master->handle_err)
+ master->handle_err(master, msg);
+
spi_finalize_current_message(master);
return ret;
"failed to unprepare message: %d\n", ret);
}
}
+
+ trace_spi_message_done(mesg);
+
master->cur_msg_prepared = false;
mesg->state = NULL;
if (mesg->complete)
mesg->complete(mesg->context);
-
- trace_spi_message_done(mesg);
}
EXPORT_SYMBOL_GPL(spi_finalize_current_message);
spi->dev.of_node = nc;
/* Register the new device */
- request_module("%s%s", SPI_MODULE_PREFIX, spi->modalias);
rc = spi_add_device(spi);
if (rc) {
dev_err(&master->dev, "spi_device register error %s\n",
if (!spi->max_speed_hz)
spi->max_speed_hz = spi->master->max_speed_hz;
+ spi_set_cs(spi, false);
+
if (spi->master->setup)
status = spi->master->setup(spi);
struct spi_transfer *k_xfers;
struct spi_transfer *k_tmp;
struct spi_ioc_transfer *u_tmp;
- unsigned n, total;
+ unsigned n, total, tx_total, rx_total;
u8 *tx_buf, *rx_buf;
int status = -EFAULT;
tx_buf = spidev->tx_buffer;
rx_buf = spidev->rx_buffer;
total = 0;
+ tx_total = 0;
+ rx_total = 0;
for (n = n_xfers, k_tmp = k_xfers, u_tmp = u_xfers;
n;
n--, k_tmp++, u_tmp++) {
k_tmp->len = u_tmp->len;
total += k_tmp->len;
- if (total > bufsiz) {
+ /* Since the function returns the total length of transfers
+ * on success, restrict the total to positive int values to
+ * avoid the return value looking like an error. Also check
+ * each transfer length to avoid arithmetic overflow.
+ */
+ if (total > INT_MAX || k_tmp->len > INT_MAX) {
status = -EMSGSIZE;
goto done;
}
if (u_tmp->rx_buf) {
+ /* this transfer needs space in RX bounce buffer */
+ rx_total += k_tmp->len;
+ if (rx_total > bufsiz) {
+ status = -EMSGSIZE;
+ goto done;
+ }
k_tmp->rx_buf = rx_buf;
if (!access_ok(VERIFY_WRITE, (u8 __user *)
(uintptr_t) u_tmp->rx_buf,
u_tmp->len))
goto done;
+ rx_buf += k_tmp->len;
}
if (u_tmp->tx_buf) {
+ /* this transfer needs space in TX bounce buffer */
+ tx_total += k_tmp->len;
+ if (tx_total > bufsiz) {
+ status = -EMSGSIZE;
+ goto done;
+ }
k_tmp->tx_buf = tx_buf;
if (copy_from_user(tx_buf, (const u8 __user *)
(uintptr_t) u_tmp->tx_buf,
u_tmp->len))
goto done;
+ tx_buf += k_tmp->len;
}
- tx_buf += k_tmp->len;
- rx_buf += k_tmp->len;
k_tmp->cs_change = !!u_tmp->cs_change;
k_tmp->tx_nbits = u_tmp->tx_nbits;
status = -EFAULT;
goto done;
}
+ rx_buf += u_tmp->len;
}
- rx_buf += u_tmp->len;
}
status = total;
static struct class *spidev_class;
+#ifdef CONFIG_OF
+static const struct of_device_id spidev_dt_ids[] = {
+ { .compatible = "rohm,dh2228fv" },
+ {},
+};
+MODULE_DEVICE_TABLE(of, spidev_dt_ids);
+#endif
+
/*-------------------------------------------------------------------------*/
static int spidev_probe(struct spi_device *spi)
int status;
unsigned long minor;
+ /*
+ * spidev should never be referenced in DT without a specific
+ * compatbile string, it is a Linux implementation thing
+ * rather than a description of the hardware.
+ */
+ if (spi->dev.of_node && !of_match_device(spidev_dt_ids, &spi->dev)) {
+ dev_err(&spi->dev, "buggy DT: spidev listed directly in DT\n");
+ WARN_ON(spi->dev.of_node &&
+ !of_match_device(spidev_dt_ids, &spi->dev));
+ }
+
/* Allocate driver data */
spidev = kzalloc(sizeof(*spidev), GFP_KERNEL);
if (!spidev)
return 0;
}
-static const struct of_device_id spidev_dt_ids[] = {
- { .compatible = "rohm,dh2228fv" },
- {},
-};
-
-MODULE_DEVICE_TABLE(of, spidev_dt_ids);
-
static struct spi_driver spidev_spi_driver = {
.driver = {
.name = "spidev",
unsigned int *data)
{
struct pci1710_private *devpriv = dev->private;
- unsigned int chan = CR_CHAN(insn->chanspec);
int ret = 0;
int i;
if (ret)
break;
- ret = pci171x_ai_read_sample(dev, s, chan, &val);
+ ret = pci171x_ai_read_sample(dev, s, 0, &val);
if (ret)
break;
stalled++;
if (stalled > 10)
break;
+ } else {
+ residue = new_residue;
+ stalled = 0;
}
- residue = new_residue;
- stalled = 0;
}
return residue;
}
VMK8061_MODEL
};
-struct firmware_version {
- unsigned char ic3_vers[32]; /* USB-Controller */
- unsigned char ic6_vers[32]; /* CPU */
-};
-
static const struct comedi_lrange vmk8061_range = {
2, {
UNI_RANGE(5),
struct vmk80xx_private {
struct usb_endpoint_descriptor *ep_rx;
struct usb_endpoint_descriptor *ep_tx;
- struct firmware_version fw;
struct semaphore limit_sem;
unsigned char *usb_rx_buf;
unsigned char *usb_tx_buf;
enum vmk80xx_model model;
};
-static int vmk80xx_check_data_link(struct comedi_device *dev)
-{
- struct vmk80xx_private *devpriv = dev->private;
- struct usb_device *usb = comedi_to_usb_dev(dev);
- unsigned int tx_pipe;
- unsigned int rx_pipe;
- unsigned char tx[1];
- unsigned char rx[2];
-
- tx_pipe = usb_sndbulkpipe(usb, 0x01);
- rx_pipe = usb_rcvbulkpipe(usb, 0x81);
-
- tx[0] = VMK8061_CMD_RD_PWR_STAT;
-
- /*
- * Check that IC6 (PIC16F871) is powered and
- * running and the data link between IC3 and
- * IC6 is working properly
- */
- usb_bulk_msg(usb, tx_pipe, tx, 1, NULL, devpriv->ep_tx->bInterval);
- usb_bulk_msg(usb, rx_pipe, rx, 2, NULL, HZ * 10);
-
- return (int)rx[1];
-}
-
-static void vmk80xx_read_eeprom(struct comedi_device *dev, int flag)
-{
- struct vmk80xx_private *devpriv = dev->private;
- struct usb_device *usb = comedi_to_usb_dev(dev);
- unsigned int tx_pipe;
- unsigned int rx_pipe;
- unsigned char tx[1];
- unsigned char rx[64];
- int cnt;
-
- tx_pipe = usb_sndbulkpipe(usb, 0x01);
- rx_pipe = usb_rcvbulkpipe(usb, 0x81);
-
- tx[0] = VMK8061_CMD_RD_VERSION;
-
- /*
- * Read the firmware version info of IC3 and
- * IC6 from the internal EEPROM of the IC
- */
- usb_bulk_msg(usb, tx_pipe, tx, 1, NULL, devpriv->ep_tx->bInterval);
- usb_bulk_msg(usb, rx_pipe, rx, 64, &cnt, HZ * 10);
-
- rx[cnt] = '\0';
-
- if (flag & IC3_VERSION)
- strncpy(devpriv->fw.ic3_vers, rx + 1, 24);
- else /* IC6_VERSION */
- strncpy(devpriv->fw.ic6_vers, rx + 25, 24);
-}
-
static void vmk80xx_do_bulk_msg(struct comedi_device *dev)
{
struct vmk80xx_private *devpriv = dev->private;
usb_set_intfdata(intf, devpriv);
- if (devpriv->model == VMK8061_MODEL) {
- vmk80xx_read_eeprom(dev, IC3_VERSION);
- dev_info(&intf->dev, "%s\n", devpriv->fw.ic3_vers);
-
- if (vmk80xx_check_data_link(dev)) {
- vmk80xx_read_eeprom(dev, IC6_VERSION);
- dev_info(&intf->dev, "%s\n", devpriv->fw.ic6_vers);
- }
- }
-
if (devpriv->model == VMK8055_MODEL)
vmk80xx_reset_device(dev);
config IIO_SIMPLE_DUMMY_BUFFER
bool "Buffered capture support"
select IIO_BUFFER
+ select IIO_TRIGGER
select IIO_KFIFO_BUF
help
Add buffered data capture to the simple dummy driver.
unsigned long is_divided;
/*
- * Touchscreen LRADC channels receives a private slot in the CTRL4
- * register, the slot #7. Therefore only 7 slots instead of 8 in the
- * CTRL4 register can be mapped to LRADC channels when using the
- * touchscreen.
- *
+ * When the touchscreen is enabled, we give it two private virtual
+ * channels: #6 and #7. This means that only 6 virtual channels (instead
+ * of 8) will be available for buffered capture.
+ */
+#define TOUCHSCREEN_VCHANNEL1 7
+#define TOUCHSCREEN_VCHANNEL2 6
+#define BUFFER_VCHANS_LIMITED 0x3f
+#define BUFFER_VCHANS_ALL 0xff
+ u8 buffer_vchans;
+
+ /*
* Furthermore, certain LRADC channels are shared between touchscreen
* and/or touch-buttons and generic LRADC block. Therefore when using
* either of these, these channels are not available for the regular
#define LRADC_CTRL4 0x140
#define LRADC_CTRL4_LRADCSELECT_MASK(n) (0xf << ((n) * 4))
#define LRADC_CTRL4_LRADCSELECT_OFFSET(n) ((n) * 4)
+#define LRADC_CTRL4_LRADCSELECT(n, x) \
+ (((x) << LRADC_CTRL4_LRADCSELECT_OFFSET(n)) & \
+ LRADC_CTRL4_LRADCSELECT_MASK(n))
#define LRADC_RESOLUTION 12
#define LRADC_SINGLE_SAMPLE_MASK ((1 << LRADC_RESOLUTION) - 1)
LRADC_STATUS_TOUCH_DETECT_RAW);
}
+static void mxs_lradc_map_channel(struct mxs_lradc *lradc, unsigned vch,
+ unsigned ch)
+{
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL4_LRADCSELECT_MASK(vch),
+ LRADC_CTRL4);
+ mxs_lradc_reg_set(lradc, LRADC_CTRL4_LRADCSELECT(vch, ch), LRADC_CTRL4);
+}
+
static void mxs_lradc_setup_ts_channel(struct mxs_lradc *lradc, unsigned ch)
{
/*
LRADC_DELAY_DELAY(lradc->over_sample_delay - 1),
LRADC_DELAY(3));
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ(2) |
- LRADC_CTRL1_LRADC_IRQ(3) | LRADC_CTRL1_LRADC_IRQ(4) |
- LRADC_CTRL1_LRADC_IRQ(5), LRADC_CTRL1);
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ(ch), LRADC_CTRL1);
- /* wake us again, when the complete conversion is done */
- mxs_lradc_reg_set(lradc, LRADC_CTRL1_LRADC_IRQ_EN(ch), LRADC_CTRL1);
/*
* after changing the touchscreen plates setting
* the signals need some initial time to settle. Start the
LRADC_DELAY_DELAY(lradc->over_sample_delay - 1),
LRADC_DELAY(3));
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ(2) |
- LRADC_CTRL1_LRADC_IRQ(3) | LRADC_CTRL1_LRADC_IRQ(4) |
- LRADC_CTRL1_LRADC_IRQ(5), LRADC_CTRL1);
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ(ch2), LRADC_CTRL1);
- /* wake us again, when the conversions are done */
- mxs_lradc_reg_set(lradc, LRADC_CTRL1_LRADC_IRQ_EN(ch2), LRADC_CTRL1);
/*
* after changing the touchscreen plates setting
* the signals need some initial time to settle. Start the
#define TS_CH_XM 4
#define TS_CH_YM 5
-static int mxs_lradc_read_ts_channel(struct mxs_lradc *lradc)
-{
- u32 reg;
- int val;
-
- reg = readl(lradc->base + LRADC_CTRL1);
-
- /* only channels 3 to 5 are of interest here */
- if (reg & LRADC_CTRL1_LRADC_IRQ(TS_CH_YP)) {
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ_EN(TS_CH_YP) |
- LRADC_CTRL1_LRADC_IRQ(TS_CH_YP), LRADC_CTRL1);
- val = mxs_lradc_read_raw_channel(lradc, TS_CH_YP);
- } else if (reg & LRADC_CTRL1_LRADC_IRQ(TS_CH_XM)) {
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ_EN(TS_CH_XM) |
- LRADC_CTRL1_LRADC_IRQ(TS_CH_XM), LRADC_CTRL1);
- val = mxs_lradc_read_raw_channel(lradc, TS_CH_XM);
- } else if (reg & LRADC_CTRL1_LRADC_IRQ(TS_CH_YM)) {
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ_EN(TS_CH_YM) |
- LRADC_CTRL1_LRADC_IRQ(TS_CH_YM), LRADC_CTRL1);
- val = mxs_lradc_read_raw_channel(lradc, TS_CH_YM);
- } else {
- return -EIO;
- }
-
- mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(2));
- mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(3));
-
- return val;
-}
-
/*
* YP(open)--+-------------+
* | |--+
mxs_lradc_reg_set(lradc, mxs_lradc_drive_x_plate(lradc), LRADC_CTRL0);
lradc->cur_plate = LRADC_SAMPLE_X;
- mxs_lradc_setup_ts_channel(lradc, TS_CH_YP);
+ mxs_lradc_map_channel(lradc, TOUCHSCREEN_VCHANNEL1, TS_CH_YP);
+ mxs_lradc_setup_ts_channel(lradc, TOUCHSCREEN_VCHANNEL1);
}
/*
mxs_lradc_reg_set(lradc, mxs_lradc_drive_y_plate(lradc), LRADC_CTRL0);
lradc->cur_plate = LRADC_SAMPLE_Y;
- mxs_lradc_setup_ts_channel(lradc, TS_CH_XM);
+ mxs_lradc_map_channel(lradc, TOUCHSCREEN_VCHANNEL1, TS_CH_XM);
+ mxs_lradc_setup_ts_channel(lradc, TOUCHSCREEN_VCHANNEL1);
}
/*
mxs_lradc_reg_set(lradc, mxs_lradc_drive_pressure(lradc), LRADC_CTRL0);
lradc->cur_plate = LRADC_SAMPLE_PRESSURE;
- mxs_lradc_setup_ts_pressure(lradc, TS_CH_XP, TS_CH_YM);
+ mxs_lradc_map_channel(lradc, TOUCHSCREEN_VCHANNEL1, TS_CH_YM);
+ mxs_lradc_map_channel(lradc, TOUCHSCREEN_VCHANNEL2, TS_CH_XP);
+ mxs_lradc_setup_ts_pressure(lradc, TOUCHSCREEN_VCHANNEL2,
+ TOUCHSCREEN_VCHANNEL1);
}
static void mxs_lradc_enable_touch_detection(struct mxs_lradc *lradc)
mxs_lradc_reg_set(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ_EN, LRADC_CTRL1);
}
+static void mxs_lradc_start_touch_event(struct mxs_lradc *lradc)
+{
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ_EN,
+ LRADC_CTRL1);
+ mxs_lradc_reg_set(lradc,
+ LRADC_CTRL1_LRADC_IRQ_EN(TOUCHSCREEN_VCHANNEL1), LRADC_CTRL1);
+ /*
+ * start with the Y-pos, because it uses nearly the same plate
+ * settings like the touch detection
+ */
+ mxs_lradc_prepare_y_pos(lradc);
+}
+
static void mxs_lradc_report_ts_event(struct mxs_lradc *lradc)
{
input_report_abs(lradc->ts_input, ABS_X, lradc->ts_x_pos);
* start a dummy conversion to burn time to settle the signals
* note: we are not interested in the conversion's value
*/
- mxs_lradc_reg_wrt(lradc, 0, LRADC_CH(5));
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ(5), LRADC_CTRL1);
- mxs_lradc_reg_set(lradc, LRADC_CTRL1_LRADC_IRQ_EN(5), LRADC_CTRL1);
- mxs_lradc_reg_wrt(lradc, LRADC_DELAY_TRIGGER(1 << 5) |
+ mxs_lradc_reg_wrt(lradc, 0, LRADC_CH(TOUCHSCREEN_VCHANNEL1));
+ mxs_lradc_reg_clear(lradc,
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL1) |
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL2), LRADC_CTRL1);
+ mxs_lradc_reg_wrt(lradc,
+ LRADC_DELAY_TRIGGER(1 << TOUCHSCREEN_VCHANNEL1) |
LRADC_DELAY_KICK | LRADC_DELAY_DELAY(10), /* waste 5 ms */
LRADC_DELAY(2));
}
/* if it is released, wait for the next touch via IRQ */
lradc->cur_plate = LRADC_TOUCH;
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ, LRADC_CTRL1);
+ mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(2));
+ mxs_lradc_reg_wrt(lradc, 0, LRADC_DELAY(3));
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ |
+ LRADC_CTRL1_LRADC_IRQ_EN(TOUCHSCREEN_VCHANNEL1) |
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL1), LRADC_CTRL1);
mxs_lradc_reg_set(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ_EN, LRADC_CTRL1);
}
/* touchscreen's state machine */
static void mxs_lradc_handle_touch(struct mxs_lradc *lradc)
{
- int val;
-
switch (lradc->cur_plate) {
case LRADC_TOUCH:
- /*
- * start with the Y-pos, because it uses nearly the same plate
- * settings like the touch detection
- */
- if (mxs_lradc_check_touch_event(lradc)) {
- mxs_lradc_reg_clear(lradc,
- LRADC_CTRL1_TOUCH_DETECT_IRQ_EN,
- LRADC_CTRL1);
- mxs_lradc_prepare_y_pos(lradc);
- }
+ if (mxs_lradc_check_touch_event(lradc))
+ mxs_lradc_start_touch_event(lradc);
mxs_lradc_reg_clear(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ,
LRADC_CTRL1);
return;
case LRADC_SAMPLE_Y:
- val = mxs_lradc_read_ts_channel(lradc);
- if (val < 0) {
- mxs_lradc_enable_touch_detection(lradc); /* re-start */
- return;
- }
- lradc->ts_y_pos = val;
+ lradc->ts_y_pos = mxs_lradc_read_raw_channel(lradc,
+ TOUCHSCREEN_VCHANNEL1);
mxs_lradc_prepare_x_pos(lradc);
return;
case LRADC_SAMPLE_X:
- val = mxs_lradc_read_ts_channel(lradc);
- if (val < 0) {
- mxs_lradc_enable_touch_detection(lradc); /* re-start */
- return;
- }
- lradc->ts_x_pos = val;
+ lradc->ts_x_pos = mxs_lradc_read_raw_channel(lradc,
+ TOUCHSCREEN_VCHANNEL1);
mxs_lradc_prepare_pressure(lradc);
return;
case LRADC_SAMPLE_PRESSURE:
- lradc->ts_pressure =
- mxs_lradc_read_ts_pressure(lradc, TS_CH_XP, TS_CH_YM);
+ lradc->ts_pressure = mxs_lradc_read_ts_pressure(lradc,
+ TOUCHSCREEN_VCHANNEL2,
+ TOUCHSCREEN_VCHANNEL1);
mxs_lradc_complete_touch_event(lradc);
return;
case LRADC_SAMPLE_VALID:
- val = mxs_lradc_read_ts_channel(lradc); /* ignore the value */
mxs_lradc_finish_touch_event(lradc, 1);
break;
}
* used if doing raw sampling.
*/
if (lradc->soc == IMX28_LRADC)
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_MX28_LRADC_IRQ_EN_MASK,
+ mxs_lradc_reg_clear(lradc, LRADC_CTRL1_LRADC_IRQ_EN(0),
LRADC_CTRL1);
- mxs_lradc_reg_clear(lradc, 0xff, LRADC_CTRL0);
+ mxs_lradc_reg_clear(lradc, 0x1, LRADC_CTRL0);
/* Enable / disable the divider per requirement */
if (test_bit(chan, &lradc->is_divided))
{
/* stop all interrupts from firing */
mxs_lradc_reg_clear(lradc, LRADC_CTRL1_TOUCH_DETECT_IRQ_EN |
- LRADC_CTRL1_LRADC_IRQ_EN(2) | LRADC_CTRL1_LRADC_IRQ_EN(3) |
- LRADC_CTRL1_LRADC_IRQ_EN(4) | LRADC_CTRL1_LRADC_IRQ_EN(5),
- LRADC_CTRL1);
+ LRADC_CTRL1_LRADC_IRQ_EN(TOUCHSCREEN_VCHANNEL1) |
+ LRADC_CTRL1_LRADC_IRQ_EN(TOUCHSCREEN_VCHANNEL2), LRADC_CTRL1);
/* Power-down touchscreen touch-detect circuitry. */
mxs_lradc_reg_clear(lradc, mxs_lradc_plate_mask(lradc), LRADC_CTRL0);
struct iio_dev *iio = data;
struct mxs_lradc *lradc = iio_priv(iio);
unsigned long reg = readl(lradc->base + LRADC_CTRL1);
+ uint32_t clr_irq = mxs_lradc_irq_mask(lradc);
const uint32_t ts_irq_mask =
LRADC_CTRL1_TOUCH_DETECT_IRQ |
- LRADC_CTRL1_LRADC_IRQ(2) |
- LRADC_CTRL1_LRADC_IRQ(3) |
- LRADC_CTRL1_LRADC_IRQ(4) |
- LRADC_CTRL1_LRADC_IRQ(5);
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL1) |
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL2);
if (!(reg & mxs_lradc_irq_mask(lradc)))
return IRQ_NONE;
- if (lradc->use_touchscreen && (reg & ts_irq_mask))
+ if (lradc->use_touchscreen && (reg & ts_irq_mask)) {
mxs_lradc_handle_touch(lradc);
- if (iio_buffer_enabled(iio))
- iio_trigger_poll(iio->trig);
- else if (reg & LRADC_CTRL1_LRADC_IRQ(0))
+ /* Make sure we don't clear the next conversion's interrupt. */
+ clr_irq &= ~(LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL1) |
+ LRADC_CTRL1_LRADC_IRQ(TOUCHSCREEN_VCHANNEL2));
+ }
+
+ if (iio_buffer_enabled(iio)) {
+ if (reg & lradc->buffer_vchans)
+ iio_trigger_poll(iio->trig);
+ } else if (reg & LRADC_CTRL1_LRADC_IRQ(0)) {
complete(&lradc->completion);
+ }
- mxs_lradc_reg_clear(lradc, reg & mxs_lradc_irq_mask(lradc),
- LRADC_CTRL1);
+ mxs_lradc_reg_clear(lradc, reg & clr_irq, LRADC_CTRL1);
return IRQ_HANDLED;
}
}
if (lradc->soc == IMX28_LRADC)
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_MX28_LRADC_IRQ_EN_MASK,
- LRADC_CTRL1);
- mxs_lradc_reg_clear(lradc, 0xff, LRADC_CTRL0);
+ mxs_lradc_reg_clear(lradc,
+ lradc->buffer_vchans << LRADC_CTRL1_LRADC_IRQ_EN_OFFSET,
+ LRADC_CTRL1);
+ mxs_lradc_reg_clear(lradc, lradc->buffer_vchans, LRADC_CTRL0);
for_each_set_bit(chan, iio->active_scan_mask, LRADC_MAX_TOTAL_CHANS) {
ctrl4_set |= chan << LRADC_CTRL4_LRADCSELECT_OFFSET(ofs);
mxs_lradc_reg_clear(lradc, LRADC_DELAY_TRIGGER_LRADCS_MASK |
LRADC_DELAY_KICK, LRADC_DELAY(0));
- mxs_lradc_reg_clear(lradc, 0xff, LRADC_CTRL0);
+ mxs_lradc_reg_clear(lradc, lradc->buffer_vchans, LRADC_CTRL0);
if (lradc->soc == IMX28_LRADC)
- mxs_lradc_reg_clear(lradc, LRADC_CTRL1_MX28_LRADC_IRQ_EN_MASK,
- LRADC_CTRL1);
+ mxs_lradc_reg_clear(lradc,
+ lradc->buffer_vchans << LRADC_CTRL1_LRADC_IRQ_EN_OFFSET,
+ LRADC_CTRL1);
kfree(lradc->buffer);
mutex_unlock(&lradc->lock);
if (lradc->use_touchbutton)
rsvd_chans++;
if (lradc->use_touchscreen)
- rsvd_chans++;
+ rsvd_chans += 2;
/* Test for attempts to map channels with special mode of operation. */
if (bitmap_intersects(mask, &rsvd_mask, LRADC_MAX_TOTAL_CHANS))
.channel = 8,
.scan_type = {.sign = 'u', .realbits = 18, .storagebits = 32,},
},
+ /* Hidden channel to keep indexes */
+ {
+ .type = IIO_TEMP,
+ .indexed = 1,
+ .scan_index = -1,
+ .channel = 9,
+ },
MXS_ADC_CHAN(10, IIO_VOLTAGE), /* VDDIO */
MXS_ADC_CHAN(11, IIO_VOLTAGE), /* VTH */
MXS_ADC_CHAN(12, IIO_VOLTAGE), /* VDDA */
touch_ret = mxs_lradc_probe_touchscreen(lradc, node);
+ if (touch_ret == 0)
+ lradc->buffer_vchans = BUFFER_VCHANS_LIMITED;
+ else
+ lradc->buffer_vchans = BUFFER_VCHANS_ALL;
+
/* Grab all IRQ sources */
for (i = 0; i < of_cfg->irq_count; i++) {
lradc->irq[i] = platform_get_irq(pdev, i);
mutex_init(&data->lock);
indio_dev->dev.parent = dev;
+ indio_dev->name = dev->driver->name;
indio_dev->info = &hmc5843_info;
indio_dev->modes = INDIO_DIRECT_MODE;
indio_dev->channels = data->variant->channels;
#include <linux/delay.h>
#include <linux/gpio.h>
#include <linux/module.h>
+#include <linux/bitops.h>
#include <linux/iio/iio.h>
#include <linux/iio/sysfs.h>
break;
case IIO_ANGL_VEL:
vel = (((s16)(st->rx[0])) << 4) | ((st->rx[1] & 0xF0) >> 4);
- vel = (vel << 4) >> 4;
+ vel = sign_extend32(vel, 11);
*val = vel;
break;
default:
};
};
-static struct power_supply nvec_bat_psy;
-static struct power_supply nvec_psy;
+static struct power_supply *nvec_bat_psy;
+static struct power_supply *nvec_psy;
static int nvec_power_notifier(struct notifier_block *nb,
unsigned long event_type, void *data)
if (res->sub_type == 0) {
if (power->on != res->plu) {
power->on = res->plu;
- power_supply_changed(&nvec_psy);
+ power_supply_changed(nvec_psy);
}
return NOTIFY_STOP;
}
}
power->bat_cap = res->plc[1];
if (status_changed)
- power_supply_changed(&nvec_bat_psy);
+ power_supply_changed(nvec_bat_psy);
break;
case VOLTAGE:
power->bat_voltage_now = res->plu * 1000;
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct nvec_power *power = dev_get_drvdata(psy->dev->parent);
+ struct nvec_power *power = dev_get_drvdata(psy->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_ONLINE:
enum power_supply_property psp,
union power_supply_propval *val)
{
- struct nvec_power *power = dev_get_drvdata(psy->dev->parent);
+ struct nvec_power *power = dev_get_drvdata(psy->dev.parent);
switch (psp) {
case POWER_SUPPLY_PROP_STATUS:
"battery",
};
-static struct power_supply nvec_bat_psy = {
+static const struct power_supply_desc nvec_bat_psy_desc = {
.name = "battery",
.type = POWER_SUPPLY_TYPE_BATTERY,
.properties = nvec_battery_props,
.get_property = nvec_battery_get_property,
};
-static struct power_supply nvec_psy = {
+static const struct power_supply_desc nvec_psy_desc = {
.name = "ac",
.type = POWER_SUPPLY_TYPE_MAINS,
- .supplied_to = nvec_power_supplied_to,
- .num_supplicants = ARRAY_SIZE(nvec_power_supplied_to),
.properties = nvec_power_props,
.num_properties = ARRAY_SIZE(nvec_power_props),
.get_property = nvec_power_get_property,
static int nvec_power_probe(struct platform_device *pdev)
{
- struct power_supply *psy;
+ struct power_supply **psy;
+ const struct power_supply_desc *psy_desc;
struct nvec_power *power;
struct nvec_chip *nvec = dev_get_drvdata(pdev->dev.parent);
+ struct power_supply_config psy_cfg = {};
power = devm_kzalloc(&pdev->dev, sizeof(struct nvec_power), GFP_NOWAIT);
if (power == NULL)
switch (pdev->id) {
case AC:
psy = &nvec_psy;
+ psy_desc = &nvec_psy_desc;
+ psy_cfg.supplied_to = nvec_power_supplied_to;
+ psy_cfg.num_supplicants = ARRAY_SIZE(nvec_power_supplied_to);
power->notifier.notifier_call = nvec_power_notifier;
break;
case BAT:
psy = &nvec_bat_psy;
+ psy_desc = &nvec_bat_psy_desc;
power->notifier.notifier_call = nvec_power_bat_notifier;
break;
if (pdev->id == BAT)
get_bat_mfg_data(power);
- return power_supply_register(&pdev->dev, psy);
+ *psy = power_supply_register(&pdev->dev, psy_desc, &psy_cfg);
+
+ return PTR_ERR_OR_ZERO(*psy);
}
static int nvec_power_remove(struct platform_device *pdev)
nvec_unregister_notifier(power->nvec, &power->notifier);
switch (pdev->id) {
case AC:
- power_supply_unregister(&nvec_psy);
+ power_supply_unregister(nvec_psy);
break;
case BAT:
- power_supply_unregister(&nvec_bat_psy);
+ power_supply_unregister(nvec_bat_psy);
}
return 0;
/* zonetype initial */
pDevice->byOriginalZonetype = pDevice->abyEEPROM[EEP_OFS_ZONETYPE];
- /* Get RFType */
- pDevice->byRFType = SROMbyReadEmbedded(pDevice->PortOffset, EEP_OFS_RFTYPE);
-
- /* force change RevID for VT3253 emu */
- if ((pDevice->byRFType & RF_EMU) != 0)
- pDevice->byRevId = 0x80;
-
- pDevice->byRFType &= RF_MASK;
- pr_debug("pDevice->byRFType = %x\n", pDevice->byRFType);
-
if (!pDevice->bZoneRegExist)
pDevice->byZoneType = pDevice->abyEEPROM[EEP_OFS_ZONETYPE];
{
struct ieee80211_hdr *hdr = (struct ieee80211_hdr *)skb->data;
PSTxDesc head_td;
- u32 dma_idx = TYPE_AC0DMA;
+ u32 dma_idx;
unsigned long flags;
spin_lock_irqsave(&priv->lock, flags);
- if (!ieee80211_is_data(hdr->frame_control))
+ if (ieee80211_is_data(hdr->frame_control))
+ dma_idx = TYPE_AC0DMA;
+ else
dma_idx = TYPE_TXDMA0;
if (AVAIL_TD(priv, dma_idx) < 1) {
head_td->pTDInfo->skb = skb;
+ if (dma_idx == TYPE_AC0DMA)
+ head_td->pTDInfo->byFlags = TD_FLAGS_NETIF_SKB;
+
priv->iTDUsed[dma_idx]++;
/* Take ownership */
head_td->buff_addr = cpu_to_le32(head_td->pTDInfo->skb_dma);
- if (dma_idx == TYPE_AC0DMA) {
- head_td->pTDInfo->byFlags = TD_FLAGS_NETIF_SKB;
-
+ if (head_td->pTDInfo->byFlags & TD_FLAGS_NETIF_SKB)
MACvTransmitAC0(priv->PortOffset);
- } else {
+ else
MACvTransmit0(priv->PortOffset);
- }
spin_unlock_irqrestore(&priv->lock, flags);
MACvInitialize(priv->PortOffset);
MACvReadEtherAddress(priv->PortOffset, priv->abyCurrentNetAddr);
+ /* Get RFType */
+ priv->byRFType = SROMbyReadEmbedded(priv->PortOffset, EEP_OFS_RFTYPE);
+ priv->byRFType &= RF_MASK;
+
+ dev_dbg(&pcid->dev, "RF Type = %x\n", priv->byRFType);
+
device_get_options(priv);
device_set_options(priv);
/* Mask out the options cannot be set to the chip */
break;
case RATE_6M:
case RATE_9M:
+ case RATE_12M:
case RATE_18M:
byPwr = priv->abyOFDMPwrTbl[uCH];
if (priv->byRFType == RF_UW2452)
break;
case RATE_6M:
case RATE_9M:
+ case RATE_12M:
case RATE_18M:
case RATE_24M:
case RATE_36M:
* traditional iSCSI block I/O.
*/
if (iscsit_allocate_iovecs(cmd) < 0) {
- return iscsit_add_reject_cmd(cmd,
+ return iscsit_reject_cmd(cmd,
ISCSI_REASON_BOOKMARK_NO_RESOURCES, buf);
}
immed_data = cmd->immediate_data;
tpg_np_list) {
struct iscsi_np *np = tpg_np->tpg_np;
bool inaddr_any = iscsit_check_inaddr_any(np);
+ char *fmt_str;
if (np->np_network_transport != network_transport)
continue;
}
}
- len = sprintf(buf, "TargetAddress="
- "%s:%hu,%hu",
+ if (np->np_sockaddr.ss_family == AF_INET6)
+ fmt_str = "TargetAddress=[%s]:%hu,%hu";
+ else
+ fmt_str = "TargetAddress=%s:%hu,%hu";
+
+ len = sprintf(buf, fmt_str,
inaddr_any ? conn->local_ip : np->np_ip,
np->np_port,
tpg->tpgt);
pr_debug("Closing iSCSI connection CID %hu on SID:"
" %u\n", conn->cid, sess->sid);
/*
- * Always up conn_logout_comp just in case the RX Thread is sleeping
- * and the logout response never got sent because the connection
- * failed.
+ * Always up conn_logout_comp for the traditional TCP case just in case
+ * the RX Thread in iscsi_target_rx_opcode() is sleeping and the logout
+ * response never got sent because the connection failed.
+ *
+ * However for iser-target, isert_wait4logout() is using conn_logout_comp
+ * to signal logout response TX interrupt completion. Go ahead and skip
+ * this for iser since isert_rx_opcode() does not wait on logout failure,
+ * and to avoid iscsi_conn pointer dereference in iser-target code.
*/
- complete(&conn->conn_logout_comp);
+ if (conn->conn_transport->transport_type == ISCSI_TCP)
+ complete(&conn->conn_logout_comp);
iscsi_release_thread_set(conn);
#include <target/target_core_fabric.h>
#include <target/iscsi/iscsi_target_core.h>
-#include <target/iscsi/iscsi_transport.h>
#include "iscsi_target_seq_pdu_list.h"
#include "iscsi_target_tq.h"
#include "iscsi_target_erl0.h"
if (conn->conn_state == TARG_CONN_STATE_IN_LOGOUT) {
spin_unlock_bh(&conn->state_lock);
- if (conn->conn_transport->transport_type == ISCSI_TCP)
- iscsit_close_connection(conn);
+ iscsit_close_connection(conn);
return;
}
transport_free_session(tl_nexus->se_sess);
goto out;
}
- /*
- * Now, register the SAS I_T Nexus as active with the call to
- * transport_register_session()
- */
- __transport_register_session(se_tpg, tl_nexus->se_sess->se_node_acl,
+ /* Now, register the SAS I_T Nexus as active. */
+ transport_register_session(se_tpg, tl_nexus->se_sess->se_node_acl,
tl_nexus->se_sess, tl_nexus);
tl_tpg->tl_nexus = tl_nexus;
pr_debug("TCM_Loop_ConfigFS: Established I_T Nexus to emulated"
return aligned_max_sectors;
}
+bool se_dev_check_wce(struct se_device *dev)
+{
+ bool wce = false;
+
+ if (dev->transport->get_write_cache)
+ wce = dev->transport->get_write_cache(dev);
+ else if (dev->dev_attrib.emulate_write_cache > 0)
+ wce = true;
+
+ return wce;
+}
+
int se_dev_set_max_unmap_lba_count(
struct se_device *dev,
u32 max_unmap_lba_count)
pr_err("Illegal value %d\n", flag);
return -EINVAL;
}
+ if (flag &&
+ dev->transport->get_write_cache) {
+ pr_warn("emulate_fua_write not supported for this device, ignoring\n");
+ return 0;
+ }
+ if (dev->export_count) {
+ pr_err("emulate_fua_write cannot be changed with active"
+ " exports: %d\n", dev->export_count);
+ return -EINVAL;
+ }
dev->dev_attrib.emulate_fua_write = flag;
pr_debug("dev[%p]: SE Device Forced Unit Access WRITEs: %d\n",
dev, dev->dev_attrib.emulate_fua_write);
pr_err("emulate_write_cache not supported for this device\n");
return -EINVAL;
}
-
+ if (dev->export_count) {
+ pr_err("emulate_write_cache cannot be changed with active"
+ " exports: %d\n", dev->export_count);
+ return -EINVAL;
+ }
dev->dev_attrib.emulate_write_cache = flag;
pr_debug("dev[%p]: SE Device WRITE_CACHE_EMULATION flag: %d\n",
dev, dev->dev_attrib.emulate_write_cache);
ret = dev->transport->configure_device(dev);
if (ret)
goto out;
- dev->dev_flags |= DF_CONFIGURED;
-
/*
* XXX: there is not much point to have two different values here..
*/
list_add_tail(&dev->g_dev_node, &g_device_list);
mutex_unlock(&g_device_mutex);
+ dev->dev_flags |= DF_CONFIGURED;
+
return 0;
out_free_alua:
struct pscsi_dev_virt *pdv = PSCSI_DEV(dev);
struct scsi_device *sd = pdv->pdv_sd;
- return sd->type;
+ return (sd) ? sd->type : TYPE_NO_LUN;
}
static sector_t pscsi_get_blocks(struct se_device *dev)
}
}
if (cdb[1] & 0x8) {
- if (!dev->dev_attrib.emulate_fua_write ||
- !dev->dev_attrib.emulate_write_cache) {
+ if (!dev->dev_attrib.emulate_fua_write || !se_dev_check_wce(dev)) {
pr_err("Got CDB: 0x%02x with FUA bit set, but device"
" does not advertise support for FUA write\n",
cdb[0]);
}
EXPORT_SYMBOL(spc_emulate_evpd_83);
-static bool
-spc_check_dev_wce(struct se_device *dev)
-{
- bool wce = false;
-
- if (dev->transport->get_write_cache)
- wce = dev->transport->get_write_cache(dev);
- else if (dev->dev_attrib.emulate_write_cache > 0)
- wce = true;
-
- return wce;
-}
-
/* Extended INQUIRY Data VPD Page */
static sense_reason_t
spc_emulate_evpd_86(struct se_cmd *cmd, unsigned char *buf)
buf[5] = 0x07;
/* If WriteCache emulation is enabled, set V_SUP */
- if (spc_check_dev_wce(dev))
+ if (se_dev_check_wce(dev))
buf[6] = 0x01;
/* If an LBA map is present set R_SUP */
spin_lock(&cmd->se_dev->t10_alua.lba_map_lock);
if (pc == 1)
goto out;
- if (spc_check_dev_wce(dev))
+ if (se_dev_check_wce(dev))
p[2] = 0x04; /* Write Cache Enable */
p[12] = 0x20; /* Disabled Read Ahead */
(cmd->se_deve->lun_flags & TRANSPORT_LUNFLAGS_READ_ONLY)))
spc_modesense_write_protect(&buf[length], type);
- if ((spc_check_dev_wce(dev)) &&
+ if ((se_dev_check_wce(dev)) &&
(dev->dev_attrib.emulate_fua_write > 0))
spc_modesense_dpofua(&buf[length], type);
list_add_tail(&se_cmd->se_cmd_list, &se_sess->sess_cmd_list);
out:
spin_unlock_irqrestore(&se_sess->sess_cmd_lock, flags);
+
+ if (ret && ack_kref)
+ target_put_sess_cmd(se_sess, se_cmd);
+
return ret;
}
EXPORT_SYMBOL(target_get_sess_cmd);
ep = fc_seq_exch(seq);
if (ep) {
lport = ep->lp;
- if (lport && (ep->xid <= lport->lro_xid))
+ if (lport && (ep->xid <= lport->lro_xid)) {
/*
* "ddp_done" trigger invalidation of HW
* specific DDP context
* identified using ep->xid)
*/
cmd->was_ddp_setup = 0;
+ }
}
}
}
result = acpi_parse_art(priv->adev->handle, &priv->art_count,
&priv->arts, true);
if (result)
- goto free_priv;
-
+ dev_dbg(&pdev->dev, "_ART table parsing error\n");
result = acpi_parse_trt(priv->adev->handle, &priv->trt_count,
&priv->trts, true);
if (result)
- goto free_art;
+ dev_dbg(&pdev->dev, "_TRT table parsing error\n");
platform_set_drvdata(pdev, priv);
&int3400_thermal_params, 0, 0);
if (IS_ERR(priv->thermal)) {
result = PTR_ERR(priv->thermal);
- goto free_trt;
+ goto free_art_trt;
}
priv->rel_misc_dev_res = acpi_thermal_rel_misc_device_add(
free_zone:
thermal_zone_device_unregister(priv->thermal);
-free_trt:
+free_art_trt:
kfree(priv->trts);
-free_art:
kfree(priv->arts);
free_priv:
kfree(priv);
trip_cnt, GFP_KERNEL);
if (!int34x_thermal_zone->aux_trips) {
ret = -ENOMEM;
- goto free_mem;
+ goto err_trip_alloc;
}
trip_mask = BIT(trip_cnt) - 1;
int34x_thermal_zone->aux_trip_nr = trip_cnt;
0, 0);
if (IS_ERR(int34x_thermal_zone->zone)) {
ret = PTR_ERR(int34x_thermal_zone->zone);
- goto free_lpat;
+ goto err_thermal_zone;
}
return int34x_thermal_zone;
-free_lpat:
+err_thermal_zone:
acpi_lpat_free_conversion_table(int34x_thermal_zone->lpat_table);
-free_mem:
+ kfree(int34x_thermal_zone->aux_trips);
+err_trip_alloc:
kfree(int34x_thermal_zone);
return ERR_PTR(ret);
}
{
thermal_zone_device_unregister(int34x_thermal_zone->zone);
acpi_lpat_free_conversion_table(int34x_thermal_zone->lpat_table);
+ kfree(int34x_thermal_zone->aux_trips);
kfree(int34x_thermal_zone);
}
EXPORT_SYMBOL_GPL(int340x_thermal_zone_remove);
{ X86_VENDOR_INTEL, 6, 0x45},
{ X86_VENDOR_INTEL, 6, 0x46},
{ X86_VENDOR_INTEL, 6, 0x4c},
+ { X86_VENDOR_INTEL, 6, 0x4d},
{ X86_VENDOR_INTEL, 6, 0x56},
{}
};
irq = platform_get_resource(pdev, IORESOURCE_IRQ, 0);
if (irq) {
- int ret;
-
/*
* platform has IRQ support.
* Then, driver uses common registers
- */
-
- ret = devm_request_irq(dev, irq->start, rcar_thermal_irq, 0,
- dev_name(dev), common);
- if (ret) {
- dev_err(dev, "irq request failed\n ");
- return ret;
- }
-
- /*
* rcar_has_irq_support() will be enabled
*/
res = platform_get_resource(pdev, IORESOURCE_MEM, mres++);
}
/* enable temperature comparation */
- if (irq)
+ if (irq) {
+ ret = devm_request_irq(dev, irq->start, rcar_thermal_irq, 0,
+ dev_name(dev), common);
+ if (ret) {
+ dev_err(dev, "irq request failed\n ");
+ goto error_unregister;
+ }
+
rcar_thermal_common_write(common, ENR, enr_bits);
+ }
platform_set_drvdata(pdev, common);
error_unregister:
rcar_thermal_for_each_priv(priv, common) {
- thermal_zone_device_unregister(priv->zone);
if (rcar_has_irq_support(priv))
rcar_thermal_irq_disable(priv);
+ thermal_zone_device_unregister(priv->zone);
}
pm_runtime_put(dev);
struct rcar_thermal_priv *priv;
rcar_thermal_for_each_priv(priv, common) {
- thermal_zone_device_unregister(priv->zone);
if (rcar_has_irq_support(priv))
rcar_thermal_irq_disable(priv);
+ thermal_zone_device_unregister(priv->zone);
}
pm_runtime_put(dev);
if (on) {
con |= (1 << EXYNOS_TMU_CORE_EN_SHIFT);
+ con |= (1 << EXYNOS7_PD_DET_EN_SHIFT);
interrupt_en =
(of_thermal_is_trip_valid(tz, 7)
<< EXYNOS7_TMU_INTEN_RISE7_SHIFT) |
interrupt_en << EXYNOS_TMU_INTEN_FALL0_SHIFT;
} else {
con &= ~(1 << EXYNOS_TMU_CORE_EN_SHIFT);
+ con &= ~(1 << EXYNOS7_PD_DET_EN_SHIFT);
interrupt_en = 0; /* Disable all interrupts */
}
- con |= 1 << EXYNOS7_PD_DET_EN_SHIFT;
writel(interrupt_en, data->base + EXYNOS7_TMU_REG_INTEN);
writel(con, data->base + EXYNOS_TMU_REG_CONTROL);
}
static const struct of_device_id exynos_tmu_match[] = {
- {
- .compatible = "samsung,exynos3250-tmu",
- },
- {
- .compatible = "samsung,exynos4210-tmu",
- },
- {
- .compatible = "samsung,exynos4412-tmu",
- },
- {
- .compatible = "samsung,exynos5250-tmu",
- },
- {
- .compatible = "samsung,exynos5260-tmu",
- },
- {
- .compatible = "samsung,exynos5420-tmu",
- },
- {
- .compatible = "samsung,exynos5420-tmu-ext-triminfo",
- },
- {
- .compatible = "samsung,exynos5440-tmu",
- },
- {
- .compatible = "samsung,exynos7-tmu",
- },
- {},
+ { .compatible = "samsung,exynos3250-tmu", },
+ { .compatible = "samsung,exynos4210-tmu", },
+ { .compatible = "samsung,exynos4412-tmu", },
+ { .compatible = "samsung,exynos5250-tmu", },
+ { .compatible = "samsung,exynos5260-tmu", },
+ { .compatible = "samsung,exynos5420-tmu", },
+ { .compatible = "samsung,exynos5420-tmu-ext-triminfo", },
+ { .compatible = "samsung,exynos5440-tmu", },
+ { .compatible = "samsung,exynos7-tmu", },
+ { /* sentinel */ },
};
MODULE_DEVICE_TABLE(of, exynos_tmu_match);
* Function to allocate regfields which are common
* between syscfg and memory mapped based sensors
*/
-int st_thermal_alloc_regfields(struct st_thermal_sensor *sensor)
+static int st_thermal_alloc_regfields(struct st_thermal_sensor *sensor)
{
struct device *dev = sensor->dev;
struct regmap *regmap = sensor->regmap;
};
/* Compatible device data stih416 mpe thermal sensor */
-const struct st_thermal_compat_data st_416mpe_cdata = {
+static const struct st_thermal_compat_data st_416mpe_cdata = {
.reg_fields = st_mmap_thermal_regfields,
.ops = &st_mmap_sensor_ops,
.calibration_val = 14,
};
/* Compatible device data stih407 thermal sensor */
-const struct st_thermal_compat_data st_407_cdata = {
+static const struct st_thermal_compat_data st_407_cdata = {
.reg_fields = st_mmap_thermal_regfields,
.ops = &st_mmap_sensor_ops,
.calibration_val = 16,
.crit_temp = 120,
};
-static struct of_device_id st_mmap_thermal_of_match[] = {
+static const struct of_device_id st_mmap_thermal_of_match[] = {
{ .compatible = "st,stih416-mpe-thermal", .data = &st_416mpe_cdata },
{ .compatible = "st,stih407-thermal", .data = &st_407_cdata },
{ /* sentinel */ }
};
MODULE_DEVICE_TABLE(of, st_mmap_thermal_of_match);
-int st_mmap_probe(struct platform_device *pdev)
+static int st_mmap_probe(struct platform_device *pdev)
{
return st_thermal_register(pdev, st_mmap_thermal_of_match);
}
-int st_mmap_remove(struct platform_device *pdev)
+static int st_mmap_remove(struct platform_device *pdev)
{
return st_thermal_unregister(pdev);
}
};
/* Compatible device data for stih415 sas thermal sensor */
-const struct st_thermal_compat_data st_415sas_cdata = {
+static const struct st_thermal_compat_data st_415sas_cdata = {
.sys_compat = "st,stih415-front-syscfg",
.reg_fields = st_415sas_regfields,
.ops = &st_syscfg_sensor_ops,
};
/* Compatible device data for stih415 mpe thermal sensor */
-const struct st_thermal_compat_data st_415mpe_cdata = {
+static const struct st_thermal_compat_data st_415mpe_cdata = {
.sys_compat = "st,stih415-system-syscfg",
.reg_fields = st_415mpe_regfields,
.ops = &st_syscfg_sensor_ops,
};
/* Compatible device data for stih416 sas thermal sensor */
-const struct st_thermal_compat_data st_416sas_cdata = {
+static const struct st_thermal_compat_data st_416sas_cdata = {
.sys_compat = "st,stih416-front-syscfg",
.reg_fields = st_416sas_regfields,
.ops = &st_syscfg_sensor_ops,
};
/* Compatible device data for stid127 thermal sensor */
-const struct st_thermal_compat_data st_127_cdata = {
+static const struct st_thermal_compat_data st_127_cdata = {
.sys_compat = "st,stid127-cpu-syscfg",
.reg_fields = st_127_regfields,
.ops = &st_syscfg_sensor_ops,
.crit_temp = 120,
};
-static struct of_device_id st_syscfg_thermal_of_match[] = {
+static const struct of_device_id st_syscfg_thermal_of_match[] = {
{ .compatible = "st,stih415-sas-thermal", .data = &st_415sas_cdata },
{ .compatible = "st,stih415-mpe-thermal", .data = &st_415mpe_cdata },
{ .compatible = "st,stih416-sas-thermal", .data = &st_416sas_cdata },
};
MODULE_DEVICE_TABLE(of, st_syscfg_thermal_of_match);
-int st_syscfg_probe(struct platform_device *pdev)
+static int st_syscfg_probe(struct platform_device *pdev)
{
return st_thermal_register(pdev, st_syscfg_thermal_of_match);
}
-int st_syscfg_remove(struct platform_device *pdev)
+static int st_syscfg_remove(struct platform_device *pdev)
{
return st_thermal_unregister(pdev);
}
ret = thermal_zone_get_temp(tz, &temp);
if (ret) {
- dev_warn(&tz->device, "failed to read out thermal zone %d\n",
- tz->id);
+ if (ret != -EAGAIN)
+ dev_warn(&tz->device,
+ "failed to read out thermal zone (%d)\n",
+ ret);
return;
}
return sprintf(buf, "%d\n", instance->trip);
}
+static struct attribute *cooling_device_attrs[] = {
+ &dev_attr_cdev_type.attr,
+ &dev_attr_max_state.attr,
+ &dev_attr_cur_state.attr,
+ NULL,
+};
+
+static const struct attribute_group cooling_device_attr_group = {
+ .attrs = cooling_device_attrs,
+};
+
+static const struct attribute_group *cooling_device_attr_groups[] = {
+ &cooling_device_attr_group,
+ NULL,
+};
+
/* Device management */
/**
cdev->ops = ops;
cdev->updated = false;
cdev->device.class = &thermal_class;
+ cdev->device.groups = cooling_device_attr_groups;
cdev->devdata = devdata;
dev_set_name(&cdev->device, "cooling_device%d", cdev->id);
result = device_register(&cdev->device);
return ERR_PTR(result);
}
- /* sys I/F */
- if (type) {
- result = device_create_file(&cdev->device, &dev_attr_cdev_type);
- if (result)
- goto unregister;
- }
-
- result = device_create_file(&cdev->device, &dev_attr_max_state);
- if (result)
- goto unregister;
-
- result = device_create_file(&cdev->device, &dev_attr_cur_state);
- if (result)
- goto unregister;
-
/* Add 'this' new cdev to the global cdev list */
mutex_lock(&thermal_list_lock);
list_add(&cdev->node, &thermal_cdev_list);
bind_cdev(cdev);
return cdev;
-
-unregister:
- release_idr(&thermal_cdev_idr, &thermal_idr_lock, cdev->id);
- device_unregister(&cdev->device);
- return ERR_PTR(result);
}
/**
return 0;
}
-#ifdef CONFIG_PM
+#ifdef CONFIG_PM_SLEEP
static int ti_bandgap_save_ctxt(struct ti_bandgap *bgp)
{
int i;
data = ti_bandgap_get_sensor_data(bgp, id);
- if (data && data->cool_dev)
+ if (data)
cpufreq_cooling_unregister(data->cool_dev);
return 0;
return circ_cnt(&bfin_jc_write_buf);
}
-static void
-bfin_jc_wait_until_sent(struct tty_struct *tty, int timeout)
-{
- unsigned long expire = jiffies + timeout;
- while (!circ_empty(&bfin_jc_write_buf)) {
- if (signal_pending(current))
- break;
- if (time_after(jiffies, expire))
- break;
- }
-}
-
static const struct tty_operations bfin_jc_ops = {
.open = bfin_jc_open,
.close = bfin_jc_close,
.flush_chars = bfin_jc_flush_chars,
.write_room = bfin_jc_write_room,
.chars_in_buffer = bfin_jc_chars_in_buffer,
- .wait_until_sent = bfin_jc_wait_until_sent,
};
static int __init bfin_jc_init(void)
/*
* Clear the interrupt registers.
*/
- if (serial_port_in(port, UART_LSR) & UART_LSR_DR)
- serial_port_in(port, UART_RX);
+ serial_port_in(port, UART_LSR);
+ serial_port_in(port, UART_RX);
serial_port_in(port, UART_IIR);
serial_port_in(port, UART_MSR);
* saved flags to avoid getting false values from polling
* routines or the previous session.
*/
- if (serial_port_in(port, UART_LSR) & UART_LSR_DR)
- serial_port_in(port, UART_RX);
+ serial_port_in(port, UART_LSR);
+ serial_port_in(port, UART_RX);
serial_port_in(port, UART_IIR);
serial_port_in(port, UART_MSR);
up->lsr_saved_flags = 0;
* Read data port to reset things, and then unlink from
* the IRQ chain.
*/
- if (serial_port_in(port, UART_LSR) & UART_LSR_DR)
- serial_port_in(port, UART_RX);
+ serial_port_in(port, UART_RX);
serial8250_rpm_put(up);
del_timer_sync(&up->timer);
u8 usr_reg;
int last_mcr;
int line;
+ int msr_mask_on;
+ int msr_mask_off;
struct clk *clk;
struct clk *pclk;
struct reset_control *rst;
value &= ~UART_MSR_DCTS;
}
+ /* Override any modem control signals if needed */
+ if (offset == UART_MSR) {
+ value |= d->msr_mask_on;
+ value &= ~d->msr_mask_off;
+ }
+
return value;
}
dw8250_force_idle(p);
writeb(value, p->membase + (UART_LCR << p->regshift));
}
- dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ /*
+ * FIXME: this deadlocks if port->lock is already held
+ * dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ */
}
}
__raw_writeq(value & 0xff,
p->membase + (UART_LCR << p->regshift));
}
- dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ /*
+ * FIXME: this deadlocks if port->lock is already held
+ * dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ */
}
}
#endif /* CONFIG_64BIT */
dw8250_force_idle(p);
writel(value, p->membase + (UART_LCR << p->regshift));
}
- dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ /*
+ * FIXME: this deadlocks if port->lock is already held
+ * dev_err(p->dev, "Couldn't set LCR to %d\n", value);
+ */
}
}
if (id >= 0)
p->line = id;
+ if (of_property_read_bool(np, "dcd-override")) {
+ /* Always report DCD as active */
+ data->msr_mask_on |= UART_MSR_DCD;
+ data->msr_mask_off |= UART_MSR_DDCD;
+ }
+
+ if (of_property_read_bool(np, "dsr-override")) {
+ /* Always report DSR as active */
+ data->msr_mask_on |= UART_MSR_DSR;
+ data->msr_mask_off |= UART_MSR_DDSR;
+ }
+
+ if (of_property_read_bool(np, "cts-override")) {
+ /* Always report DSR as active */
+ data->msr_mask_on |= UART_MSR_DSR;
+ data->msr_mask_off |= UART_MSR_DDSR;
+ }
+
+ if (of_property_read_bool(np, "ri-override")) {
+ /* Always report Ring indicator as inactive */
+ data->msr_mask_off |= UART_MSR_RI;
+ data->msr_mask_off |= UART_MSR_TERI;
+ }
+
/* clock got configured through clk api, all done */
if (p->uartclk)
return 0;
"Please send the output of lspci -vv, this\n"
"message (0x%04x,0x%04x,0x%04x,0x%04x), the\n"
"manufacturer and name of serial board or\n"
- "modem board to rmk+serial@arm.linux.org.uk.\n",
+ "modem board to <linux-serial@vger.kernel.org>.\n",
pci_name(dev), str, dev->vendor, dev->device,
dev->subsystem_vendor, dev->subsystem_device);
}
.subdevice = PCI_ANY_ID,
.setup = byt_serial_setup,
},
- {
- .vendor = PCI_VENDOR_ID_INTEL,
- .device = PCI_DEVICE_ID_INTEL_QRK_UART,
- .subvendor = PCI_ANY_ID,
- .subdevice = PCI_ANY_ID,
- .setup = pci_default_setup,
- },
{
.vendor = PCI_VENDOR_ID_INTEL,
.device = PCI_DEVICE_ID_INTEL_BSW_UART1,
/*
* PLX
*/
- {
- .vendor = PCI_VENDOR_ID_PLX,
- .device = PCI_DEVICE_ID_PLX_9030,
- .subvendor = PCI_SUBVENDOR_ID_PERLE,
- .subdevice = PCI_ANY_ID,
- .setup = pci_default_setup,
- },
{
.vendor = PCI_VENDOR_ID_PLX,
.device = PCI_DEVICE_ID_PLX_9050,
PCI_ANY_ID, PCI_ANY_ID,
0, 0, pbn_b0_bt_2_115200 },
- { PCI_VENDOR_ID_WCH, PCI_DEVICE_ID_WCH_CH352_2S,
- PCI_ANY_ID, PCI_ANY_ID,
- 0, 0, pbn_b0_bt_2_115200 },
-
{ PCIE_VENDOR_ID_WCH, PCIE_DEVICE_ID_WCH_CH384_4S,
PCI_ANY_ID, PCI_ANY_ID,
0, 0, pbn_wch384_4 },
#include <linux/gpio/consumer.h>
#include <linux/err.h>
#include <linux/irq.h>
+#include <linux/suspend.h>
#include <asm/io.h>
#include <asm/ioctls.h>
bool ms_irq_enabled;
bool is_usart; /* usart or uart */
struct timer_list uart_timer; /* uart timer */
+
+ bool suspended;
+ unsigned int pending;
+ unsigned int pending_status;
+ spinlock_t lock_suspended;
+
int (*prepare_rx)(struct uart_port *port);
int (*prepare_tx)(struct uart_port *port);
void (*schedule_rx)(struct uart_port *port);
{
struct uart_port *port = dev_id;
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
- unsigned int status, pending, pass_counter = 0;
+ unsigned int status, pending, mask, pass_counter = 0;
bool gpio_handled = false;
+ spin_lock(&atmel_port->lock_suspended);
+
do {
status = atmel_get_lines_status(port);
- pending = status & UART_GET_IMR(port);
+ mask = UART_GET_IMR(port);
+ pending = status & mask;
if (!gpio_handled) {
/*
* Dealing with GPIO interrupt
if (!pending)
break;
+ if (atmel_port->suspended) {
+ atmel_port->pending |= pending;
+ atmel_port->pending_status = status;
+ UART_PUT_IDR(port, mask);
+ pm_system_wakeup();
+ break;
+ }
+
atmel_handle_receive(port, pending);
atmel_handle_status(port, pending, status);
atmel_handle_transmit(port, pending);
} while (pass_counter++ < ATMEL_ISR_PASS_LIMIT);
+ spin_unlock(&atmel_port->lock_suspended);
+
return pass_counter ? IRQ_HANDLED : IRQ_NONE;
}
/*
* Allocate the IRQ
*/
- retval = request_irq(port->irq, atmel_interrupt, IRQF_SHARED,
+ retval = request_irq(port->irq, atmel_interrupt,
+ IRQF_SHARED | IRQF_COND_SUSPEND,
tty ? tty->name : "atmel_serial", port);
if (retval) {
dev_err(port->dev, "atmel_startup - Can't get irq\n");
/* we can not wake up if we're running on slow clock */
atmel_port->may_wakeup = device_may_wakeup(&pdev->dev);
- if (atmel_serial_clk_will_stop())
+ if (atmel_serial_clk_will_stop()) {
+ unsigned long flags;
+
+ spin_lock_irqsave(&atmel_port->lock_suspended, flags);
+ atmel_port->suspended = true;
+ spin_unlock_irqrestore(&atmel_port->lock_suspended, flags);
device_set_wakeup_enable(&pdev->dev, 0);
+ }
uart_suspend_port(&atmel_uart, port);
{
struct uart_port *port = platform_get_drvdata(pdev);
struct atmel_uart_port *atmel_port = to_atmel_uart_port(port);
+ unsigned long flags;
+
+ spin_lock_irqsave(&atmel_port->lock_suspended, flags);
+ if (atmel_port->pending) {
+ atmel_handle_receive(port, atmel_port->pending);
+ atmel_handle_status(port, atmel_port->pending,
+ atmel_port->pending_status);
+ atmel_handle_transmit(port, atmel_port->pending);
+ atmel_port->pending = 0;
+ }
+ atmel_port->suspended = false;
+ spin_unlock_irqrestore(&atmel_port->lock_suspended, flags);
uart_resume_port(&atmel_uart, port);
device_set_wakeup_enable(&pdev->dev, atmel_port->may_wakeup);
port->backup_imr = 0;
port->uart.line = ret;
+ spin_lock_init(&port->lock_suspended);
+
ret = atmel_init_gpios(port, &pdev->dev);
if (ret < 0)
dev_err(&pdev->dev, "%s",
writeb(val | UARTPFIFO_TXFE | UARTPFIFO_RXFE,
sport->port.membase + UARTPFIFO);
+ /* explicitly clear RDRF */
+ readb(sport->port.membase + UARTSR1);
+
/* flush Tx and Rx FIFO */
writeb(UARTCFIFO_TXFLUSH | UARTCFIFO_RXFLUSH,
sport->port.membase + UARTCFIFO);
sport->txfifo_size = 0x1 << (((temp >> UARTPFIFO_TXSIZE_OFF) &
UARTPFIFO_FIFOSIZE_MASK) + 1);
+ sport->port.fifosize = sport->txfifo_size;
+
sport->rxfifo_size = 0x1 << (((temp >> UARTPFIFO_RXSIZE_OFF) &
UARTPFIFO_FIFOSIZE_MASK) + 1);
if (of_find_property(np, "no-loopback-test", NULL))
port->flags |= UPF_SKIP_TEST;
- ret = of_alias_get_id(np, "serial");
- if (ret >= 0)
- port->line = ret;
-
port->dev = &ofdev->dev;
switch (type) {
free_irq(ourport->tx_irq, ourport);
tx_enabled(port) = 0;
ourport->tx_claimed = 0;
+ ourport->tx_mode = 0;
}
if (ourport->rx_claimed) {
ims = serial_in(port, SPRD_IMSR);
- if (!ims)
+ if (!ims) {
+ spin_unlock(&port->lock);
return IRQ_NONE;
+ }
serial_out(port, SPRD_ICLR, ~0);
static void sysrq_handle_showstate(int key)
{
show_state();
+ show_workqueue_state();
}
static struct sysrq_key_op sysrq_showstate_op = {
.handler = sysrq_handle_showstate,
/* We limit tty time update visibility to every 8 seconds or so. */
static void tty_update_time(struct timespec *time)
{
- unsigned long sec = get_seconds() & ~7;
- if ((long)(sec - time->tv_sec) > 0)
+ unsigned long sec = get_seconds();
+ if (abs(sec - time->tv_sec) & ~7)
time->tv_sec = sec;
}
#endif
if (!timeout)
timeout = MAX_SCHEDULE_TIMEOUT;
- if (wait_event_interruptible_timeout(tty->write_wait,
- !tty_chars_in_buffer(tty), timeout) >= 0) {
- if (tty->ops->wait_until_sent)
- tty->ops->wait_until_sent(tty, timeout);
- }
+
+ timeout = wait_event_interruptible_timeout(tty->write_wait,
+ !tty_chars_in_buffer(tty), timeout);
+ if (timeout <= 0)
+ return;
+
+ if (timeout == MAX_SCHEDULE_TIMEOUT)
+ timeout = 0;
+
+ if (tty->ops->wait_until_sent)
+ tty->ops->wait_until_sent(tty, timeout);
}
EXPORT_SYMBOL(tty_wait_until_sent);
return retval;
}
+static int otg_a_alt_hnp_support(struct ci_hdrc *ci)
+{
+ dev_warn(&ci->gadget.dev,
+ "connect the device to an alternate port if you want HNP\n");
+ return isr_setup_status_phase(ci);
+}
+
/**
* isr_setup_packet_handler: setup packet handler
* @ci: UDC descriptor
ci);
}
break;
+ case USB_DEVICE_A_ALT_HNP_SUPPORT:
+ if (ci_otg_is_fsm_mode(ci))
+ err = otg_a_alt_hnp_support(ci);
+ break;
default:
goto delegate;
}
static const struct usb_device_id acm_ids[] = {
/* quirky and broken devices */
+ { USB_DEVICE(0x076d, 0x0006), /* Denso Cradle CU-321 */
+ .driver_info = NO_UNION_NORMAL, },/* has no union descriptor */
{ USB_DEVICE(0x17ef, 0x7000), /* Lenovo USB modem */
.driver_info = NO_UNION_NORMAL, },/* has no union descriptor */
{ USB_DEVICE(0x0870, 0x0001), /* Metricom GS Modem */
break;
case OTG_STATE_B_PERIPHERAL:
otg_chrg_vbus(fsm, 0);
- otg_loc_conn(fsm, 1);
otg_loc_sof(fsm, 0);
otg_set_protocol(fsm, PROTO_GADGET);
+ otg_loc_conn(fsm, 1);
break;
case OTG_STATE_B_WAIT_ACON:
otg_chrg_vbus(fsm, 0);
break;
case OTG_STATE_A_PERIPHERAL:
- otg_loc_conn(fsm, 1);
otg_loc_sof(fsm, 0);
otg_set_protocol(fsm, PROTO_GADGET);
otg_drv_vbus(fsm, 1);
+ otg_loc_conn(fsm, 1);
otg_add_timer(fsm, A_BIDL_ADIS);
break;
case OTG_STATE_A_WAIT_VFALL:
as->status = urb->status;
signr = as->signr;
if (signr) {
+ memset(&sinfo, 0, sizeof(sinfo));
sinfo.si_signo = as->signr;
sinfo.si_errno = as->status;
sinfo.si_code = SI_ASYNCIO;
wake_up_all(&ps->wait);
list_del_init(&ps->list);
if (ps->discsignr) {
+ memset(&sinfo, 0, sizeof(sinfo));
sinfo.si_signo = ps->discsignr;
sinfo.si_errno = EPIPE;
sinfo.si_code = SI_ASYNCIO;
dwc2_is_host_mode(hsotg) ? "Host" : "Device",
dwc2_op_state_str(hsotg));
+ if (hsotg->op_state == OTG_STATE_A_HOST)
+ dwc2_hcd_disconnect(hsotg);
+
/* Change to L3 (OFF) state */
hsotg->lx_state = DWC2_L3;
omap->irq0_offset, value);
}
+static void dwc3_omap_write_irqmisc_clr(struct dwc3_omap *omap, u32 value)
+{
+ dwc3_omap_writel(omap->base, USBOTGSS_IRQENABLE_CLR_MISC +
+ omap->irqmisc_offset, value);
+}
+
+static void dwc3_omap_write_irq0_clr(struct dwc3_omap *omap, u32 value)
+{
+ dwc3_omap_writel(omap->base, USBOTGSS_IRQENABLE_CLR_0 -
+ omap->irq0_offset, value);
+}
+
static void dwc3_omap_set_mailbox(struct dwc3_omap *omap,
enum omap_dwc3_vbus_id_status status)
{
static void dwc3_omap_disable_irqs(struct dwc3_omap *omap)
{
+ u32 reg;
+
/* disable all IRQs */
- dwc3_omap_write_irqmisc_set(omap, 0x00);
- dwc3_omap_write_irq0_set(omap, 0x00);
+ reg = USBOTGSS_IRQO_COREIRQ_ST;
+ dwc3_omap_write_irq0_clr(omap, reg);
+
+ reg = (USBOTGSS_IRQMISC_OEVT |
+ USBOTGSS_IRQMISC_DRVVBUS_RISE |
+ USBOTGSS_IRQMISC_CHRGVBUS_RISE |
+ USBOTGSS_IRQMISC_DISCHRGVBUS_RISE |
+ USBOTGSS_IRQMISC_IDPULLUP_RISE |
+ USBOTGSS_IRQMISC_DRVVBUS_FALL |
+ USBOTGSS_IRQMISC_CHRGVBUS_FALL |
+ USBOTGSS_IRQMISC_DISCHRGVBUS_FALL |
+ USBOTGSS_IRQMISC_IDPULLUP_FALL);
+
+ dwc3_omap_write_irqmisc_clr(omap, reg);
}
static u64 dwc3_omap_dma_mask = DMA_BIT_MASK(32);
if (desc->opts_mutex)
mutex_lock(desc->opts_mutex);
memcpy(desc->ext_compat_id, page, l);
- desc->ext_compat_id[l] = '\0';
if (desc->opts_mutex)
mutex_unlock(desc->opts_mutex);
if (desc->opts_mutex)
mutex_lock(desc->opts_mutex);
memcpy(desc->ext_compat_id + 8, page, l);
- desc->ext_compat_id[l + 8] = '\0';
if (desc->opts_mutex)
mutex_unlock(desc->opts_mutex);
bool read;
struct kiocb *kiocb;
- const struct iovec *iovec;
- unsigned long nr_segs;
- char __user *buf;
- size_t len;
+ struct iov_iter data;
+ const void *to_free;
+ char *buf;
struct mm_struct *mm;
struct work_struct work;
io_data->req->actual;
if (io_data->read && ret > 0) {
- int i;
- size_t pos = 0;
-
- /*
- * Since req->length may be bigger than io_data->len (after
- * being rounded up to maxpacketsize), we may end up with more
- * data then user space has space for.
- */
- ret = min_t(int, ret, io_data->len);
-
use_mm(io_data->mm);
- for (i = 0; i < io_data->nr_segs; i++) {
- size_t len = min_t(size_t, ret - pos,
- io_data->iovec[i].iov_len);
- if (!len)
- break;
- if (unlikely(copy_to_user(io_data->iovec[i].iov_base,
- &io_data->buf[pos], len))) {
- ret = -EFAULT;
- break;
- }
- pos += len;
- }
+ ret = copy_to_iter(io_data->buf, ret, &io_data->data);
+ if (iov_iter_count(&io_data->data))
+ ret = -EFAULT;
unuse_mm(io_data->mm);
}
io_data->kiocb->private = NULL;
if (io_data->read)
- kfree(io_data->iovec);
+ kfree(io_data->to_free);
kfree(io_data->buf);
kfree(io_data);
}
* before the waiting completes, so do not assign to 'gadget' earlier
*/
struct usb_gadget *gadget = epfile->ffs->gadget;
+ size_t copied;
spin_lock_irq(&epfile->ffs->eps_lock);
/* In the meantime, endpoint got disabled or changed. */
spin_unlock_irq(&epfile->ffs->eps_lock);
return -ESHUTDOWN;
}
+ data_len = iov_iter_count(&io_data->data);
/*
* Controller may require buffer size to be aligned to
* maxpacketsize of an out endpoint.
*/
- data_len = io_data->read ?
- usb_ep_align_maybe(gadget, ep->ep, io_data->len) :
- io_data->len;
+ if (io_data->read)
+ data_len = usb_ep_align_maybe(gadget, ep->ep, data_len);
spin_unlock_irq(&epfile->ffs->eps_lock);
data = kmalloc(data_len, GFP_KERNEL);
if (unlikely(!data))
return -ENOMEM;
- if (io_data->aio && !io_data->read) {
- int i;
- size_t pos = 0;
- for (i = 0; i < io_data->nr_segs; i++) {
- if (unlikely(copy_from_user(&data[pos],
- io_data->iovec[i].iov_base,
- io_data->iovec[i].iov_len))) {
- ret = -EFAULT;
- goto error;
- }
- pos += io_data->iovec[i].iov_len;
- }
- } else {
- if (!io_data->read &&
- unlikely(__copy_from_user(data, io_data->buf,
- io_data->len))) {
+ if (!io_data->read) {
+ copied = copy_from_iter(data, data_len, &io_data->data);
+ if (copied != data_len) {
ret = -EFAULT;
goto error;
}
*/
ret = ep->status;
if (io_data->read && ret > 0) {
- ret = min_t(size_t, ret, io_data->len);
-
- if (unlikely(copy_to_user(io_data->buf,
- data, ret)))
+ ret = copy_to_iter(data, ret, &io_data->data);
+ if (unlikely(iov_iter_count(&io_data->data)))
ret = -EFAULT;
}
}
return ret;
}
-static ssize_t
-ffs_epfile_write(struct file *file, const char __user *buf, size_t len,
- loff_t *ptr)
-{
- struct ffs_io_data io_data;
-
- ENTER();
-
- io_data.aio = false;
- io_data.read = false;
- io_data.buf = (char * __user)buf;
- io_data.len = len;
-
- return ffs_epfile_io(file, &io_data);
-}
-
-static ssize_t
-ffs_epfile_read(struct file *file, char __user *buf, size_t len, loff_t *ptr)
-{
- struct ffs_io_data io_data;
-
- ENTER();
-
- io_data.aio = false;
- io_data.read = true;
- io_data.buf = buf;
- io_data.len = len;
-
- return ffs_epfile_io(file, &io_data);
-}
-
static int
ffs_epfile_open(struct inode *inode, struct file *file)
{
return value;
}
-static ssize_t ffs_epfile_aio_write(struct kiocb *kiocb,
- const struct iovec *iovec,
- unsigned long nr_segs, loff_t loff)
+static ssize_t ffs_epfile_write_iter(struct kiocb *kiocb, struct iov_iter *from)
{
- struct ffs_io_data *io_data;
+ struct ffs_io_data io_data, *p = &io_data;
+ ssize_t res;
ENTER();
- io_data = kmalloc(sizeof(*io_data), GFP_KERNEL);
- if (unlikely(!io_data))
- return -ENOMEM;
+ if (!is_sync_kiocb(kiocb)) {
+ p = kmalloc(sizeof(io_data), GFP_KERNEL);
+ if (unlikely(!p))
+ return -ENOMEM;
+ p->aio = true;
+ } else {
+ p->aio = false;
+ }
- io_data->aio = true;
- io_data->read = false;
- io_data->kiocb = kiocb;
- io_data->iovec = iovec;
- io_data->nr_segs = nr_segs;
- io_data->len = kiocb->ki_nbytes;
- io_data->mm = current->mm;
+ p->read = false;
+ p->kiocb = kiocb;
+ p->data = *from;
+ p->mm = current->mm;
- kiocb->private = io_data;
+ kiocb->private = p;
kiocb_set_cancel_fn(kiocb, ffs_aio_cancel);
- return ffs_epfile_io(kiocb->ki_filp, io_data);
+ res = ffs_epfile_io(kiocb->ki_filp, p);
+ if (res == -EIOCBQUEUED)
+ return res;
+ if (p->aio)
+ kfree(p);
+ else
+ *from = p->data;
+ return res;
}
-static ssize_t ffs_epfile_aio_read(struct kiocb *kiocb,
- const struct iovec *iovec,
- unsigned long nr_segs, loff_t loff)
+static ssize_t ffs_epfile_read_iter(struct kiocb *kiocb, struct iov_iter *to)
{
- struct ffs_io_data *io_data;
- struct iovec *iovec_copy;
+ struct ffs_io_data io_data, *p = &io_data;
+ ssize_t res;
ENTER();
- iovec_copy = kmalloc_array(nr_segs, sizeof(*iovec_copy), GFP_KERNEL);
- if (unlikely(!iovec_copy))
- return -ENOMEM;
-
- memcpy(iovec_copy, iovec, sizeof(struct iovec)*nr_segs);
-
- io_data = kmalloc(sizeof(*io_data), GFP_KERNEL);
- if (unlikely(!io_data)) {
- kfree(iovec_copy);
- return -ENOMEM;
+ if (!is_sync_kiocb(kiocb)) {
+ p = kmalloc(sizeof(io_data), GFP_KERNEL);
+ if (unlikely(!p))
+ return -ENOMEM;
+ p->aio = true;
+ } else {
+ p->aio = false;
}
- io_data->aio = true;
- io_data->read = true;
- io_data->kiocb = kiocb;
- io_data->iovec = iovec_copy;
- io_data->nr_segs = nr_segs;
- io_data->len = kiocb->ki_nbytes;
- io_data->mm = current->mm;
+ p->read = true;
+ p->kiocb = kiocb;
+ if (p->aio) {
+ p->to_free = dup_iter(&p->data, to, GFP_KERNEL);
+ if (!p->to_free) {
+ kfree(p);
+ return -ENOMEM;
+ }
+ } else {
+ p->data = *to;
+ p->to_free = NULL;
+ }
+ p->mm = current->mm;
- kiocb->private = io_data;
+ kiocb->private = p;
kiocb_set_cancel_fn(kiocb, ffs_aio_cancel);
- return ffs_epfile_io(kiocb->ki_filp, io_data);
+ res = ffs_epfile_io(kiocb->ki_filp, p);
+ if (res == -EIOCBQUEUED)
+ return res;
+
+ if (p->aio) {
+ kfree(p->to_free);
+ kfree(p);
+ } else {
+ *to = p->data;
+ }
+ return res;
}
static int
.llseek = no_llseek,
.open = ffs_epfile_open,
- .write = ffs_epfile_write,
- .read = ffs_epfile_read,
- .aio_write = ffs_epfile_aio_write,
- .aio_read = ffs_epfile_aio_read,
+ .write = new_sync_write,
+ .read = new_sync_read,
+ .write_iter = ffs_epfile_write_iter,
+ .read_iter = ffs_epfile_read_iter,
.release = ffs_epfile_release,
.unlocked_ioctl = ffs_epfile_ioctl,
};
return status;
}
-const struct file_operations f_hidg_fops = {
+static const struct file_operations f_hidg_fops = {
.owner = THIS_MODULE,
.open = f_hidg_open,
.release = f_hidg_release,
struct usb_composite_dev *cdev;
cdev = loop->function.config->cdev;
- disable_endpoints(cdev, loop->in_ep, loop->out_ep, NULL, NULL, NULL,
- NULL);
+ disable_endpoints(cdev, loop->in_ep, loop->out_ep, NULL, NULL);
VDBG(cdev, "%s disabled\n", loop->function.name);
}
return -EINVAL;
spin_lock(&port->lock);
- __pn_reset(f);
+
+ if (fp->in_ep->driver_data)
+ __pn_reset(f);
+
if (alt == 1) {
int i;
#include "gadget_chips.h"
#include "u_f.h"
-#define USB_MS_TO_SS_INTERVAL(x) USB_MS_TO_HS_INTERVAL(x)
-
-enum eptype {
- EP_CONTROL = 0,
- EP_BULK,
- EP_ISOC,
- EP_INTERRUPT,
-};
-
/*
* SOURCE/SINK FUNCTION ... a primary testing vehicle for USB peripheral
* controller drivers.
struct usb_ep *out_ep;
struct usb_ep *iso_in_ep;
struct usb_ep *iso_out_ep;
- struct usb_ep *int_in_ep;
- struct usb_ep *int_out_ep;
int cur_alt;
};
static unsigned isoc_maxpacket;
static unsigned isoc_mult;
static unsigned isoc_maxburst;
-static unsigned int_interval; /* In ms */
-static unsigned int_maxpacket;
-static unsigned int_mult;
-static unsigned int_maxburst;
static unsigned buflen;
/*-------------------------------------------------------------------------*/
/* .iInterface = DYNAMIC */
};
-static struct usb_interface_descriptor source_sink_intf_alt2 = {
- .bLength = USB_DT_INTERFACE_SIZE,
- .bDescriptorType = USB_DT_INTERFACE,
-
- .bAlternateSetting = 2,
- .bNumEndpoints = 2,
- .bInterfaceClass = USB_CLASS_VENDOR_SPEC,
- /* .iInterface = DYNAMIC */
-};
-
/* full speed support: */
static struct usb_endpoint_descriptor fs_source_desc = {
.bInterval = 4,
};
-static struct usb_endpoint_descriptor fs_int_source_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bEndpointAddress = USB_DIR_IN,
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(64),
- .bInterval = GZERO_INT_INTERVAL,
-};
-
-static struct usb_endpoint_descriptor fs_int_sink_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bEndpointAddress = USB_DIR_OUT,
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(64),
- .bInterval = GZERO_INT_INTERVAL,
-};
-
static struct usb_descriptor_header *fs_source_sink_descs[] = {
(struct usb_descriptor_header *) &source_sink_intf_alt0,
(struct usb_descriptor_header *) &fs_sink_desc,
(struct usb_descriptor_header *) &fs_source_desc,
(struct usb_descriptor_header *) &fs_iso_sink_desc,
(struct usb_descriptor_header *) &fs_iso_source_desc,
- (struct usb_descriptor_header *) &source_sink_intf_alt2,
-#define FS_ALT_IFC_2_OFFSET 8
- (struct usb_descriptor_header *) &fs_int_sink_desc,
- (struct usb_descriptor_header *) &fs_int_source_desc,
NULL,
};
.bInterval = 4,
};
-static struct usb_endpoint_descriptor hs_int_source_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(1024),
- .bInterval = USB_MS_TO_HS_INTERVAL(GZERO_INT_INTERVAL),
-};
-
-static struct usb_endpoint_descriptor hs_int_sink_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(1024),
- .bInterval = USB_MS_TO_HS_INTERVAL(GZERO_INT_INTERVAL),
-};
-
static struct usb_descriptor_header *hs_source_sink_descs[] = {
(struct usb_descriptor_header *) &source_sink_intf_alt0,
(struct usb_descriptor_header *) &hs_source_desc,
(struct usb_descriptor_header *) &hs_sink_desc,
(struct usb_descriptor_header *) &hs_iso_source_desc,
(struct usb_descriptor_header *) &hs_iso_sink_desc,
- (struct usb_descriptor_header *) &source_sink_intf_alt2,
-#define HS_ALT_IFC_2_OFFSET 8
- (struct usb_descriptor_header *) &hs_int_source_desc,
- (struct usb_descriptor_header *) &hs_int_sink_desc,
NULL,
};
.wBytesPerInterval = cpu_to_le16(1024),
};
-static struct usb_endpoint_descriptor ss_int_source_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(1024),
- .bInterval = USB_MS_TO_SS_INTERVAL(GZERO_INT_INTERVAL),
-};
-
-struct usb_ss_ep_comp_descriptor ss_int_source_comp_desc = {
- .bLength = USB_DT_SS_EP_COMP_SIZE,
- .bDescriptorType = USB_DT_SS_ENDPOINT_COMP,
-
- .bMaxBurst = 0,
- .bmAttributes = 0,
- .wBytesPerInterval = cpu_to_le16(1024),
-};
-
-static struct usb_endpoint_descriptor ss_int_sink_desc = {
- .bLength = USB_DT_ENDPOINT_SIZE,
- .bDescriptorType = USB_DT_ENDPOINT,
-
- .bmAttributes = USB_ENDPOINT_XFER_INT,
- .wMaxPacketSize = cpu_to_le16(1024),
- .bInterval = USB_MS_TO_SS_INTERVAL(GZERO_INT_INTERVAL),
-};
-
-struct usb_ss_ep_comp_descriptor ss_int_sink_comp_desc = {
- .bLength = USB_DT_SS_EP_COMP_SIZE,
- .bDescriptorType = USB_DT_SS_ENDPOINT_COMP,
-
- .bMaxBurst = 0,
- .bmAttributes = 0,
- .wBytesPerInterval = cpu_to_le16(1024),
-};
-
static struct usb_descriptor_header *ss_source_sink_descs[] = {
(struct usb_descriptor_header *) &source_sink_intf_alt0,
(struct usb_descriptor_header *) &ss_source_desc,
(struct usb_descriptor_header *) &ss_iso_source_comp_desc,
(struct usb_descriptor_header *) &ss_iso_sink_desc,
(struct usb_descriptor_header *) &ss_iso_sink_comp_desc,
- (struct usb_descriptor_header *) &source_sink_intf_alt2,
-#define SS_ALT_IFC_2_OFFSET 14
- (struct usb_descriptor_header *) &ss_int_source_desc,
- (struct usb_descriptor_header *) &ss_int_source_comp_desc,
- (struct usb_descriptor_header *) &ss_int_sink_desc,
- (struct usb_descriptor_header *) &ss_int_sink_comp_desc,
NULL,
};
};
/*-------------------------------------------------------------------------*/
-static const char *get_ep_string(enum eptype ep_type)
-{
- switch (ep_type) {
- case EP_ISOC:
- return "ISOC-";
- case EP_INTERRUPT:
- return "INTERRUPT-";
- case EP_CONTROL:
- return "CTRL-";
- case EP_BULK:
- return "BULK-";
- default:
- return "UNKNOWN-";
- }
-}
static inline struct usb_request *ss_alloc_ep_req(struct usb_ep *ep, int len)
{
void disable_endpoints(struct usb_composite_dev *cdev,
struct usb_ep *in, struct usb_ep *out,
- struct usb_ep *iso_in, struct usb_ep *iso_out,
- struct usb_ep *int_in, struct usb_ep *int_out)
+ struct usb_ep *iso_in, struct usb_ep *iso_out)
{
disable_ep(cdev, in);
disable_ep(cdev, out);
disable_ep(cdev, iso_in);
if (iso_out)
disable_ep(cdev, iso_out);
- if (int_in)
- disable_ep(cdev, int_in);
- if (int_out)
- disable_ep(cdev, int_out);
}
static int
return id;
source_sink_intf_alt0.bInterfaceNumber = id;
source_sink_intf_alt1.bInterfaceNumber = id;
- source_sink_intf_alt2.bInterfaceNumber = id;
/* allocate bulk endpoints */
ss->in_ep = usb_ep_autoconfig(cdev->gadget, &fs_source_desc);
if (isoc_maxpacket > 1024)
isoc_maxpacket = 1024;
- /* sanity check the interrupt module parameters */
- if (int_interval < 1)
- int_interval = 1;
- if (int_interval > 4096)
- int_interval = 4096;
- if (int_mult > 2)
- int_mult = 2;
- if (int_maxburst > 15)
- int_maxburst = 15;
-
- /* fill in the FS interrupt descriptors from the module parameters */
- fs_int_source_desc.wMaxPacketSize = int_maxpacket > 64 ?
- 64 : int_maxpacket;
- fs_int_source_desc.bInterval = int_interval > 255 ?
- 255 : int_interval;
- fs_int_sink_desc.wMaxPacketSize = int_maxpacket > 64 ?
- 64 : int_maxpacket;
- fs_int_sink_desc.bInterval = int_interval > 255 ?
- 255 : int_interval;
-
- /* allocate int endpoints */
- ss->int_in_ep = usb_ep_autoconfig(cdev->gadget, &fs_int_source_desc);
- if (!ss->int_in_ep)
- goto no_int;
- ss->int_in_ep->driver_data = cdev; /* claim */
-
- ss->int_out_ep = usb_ep_autoconfig(cdev->gadget, &fs_int_sink_desc);
- if (ss->int_out_ep) {
- ss->int_out_ep->driver_data = cdev; /* claim */
- } else {
- ss->int_in_ep->driver_data = NULL;
- ss->int_in_ep = NULL;
-no_int:
- fs_source_sink_descs[FS_ALT_IFC_2_OFFSET] = NULL;
- hs_source_sink_descs[HS_ALT_IFC_2_OFFSET] = NULL;
- ss_source_sink_descs[SS_ALT_IFC_2_OFFSET] = NULL;
- }
-
- if (int_maxpacket > 1024)
- int_maxpacket = 1024;
-
/* support high speed hardware */
hs_source_desc.bEndpointAddress = fs_source_desc.bEndpointAddress;
hs_sink_desc.bEndpointAddress = fs_sink_desc.bEndpointAddress;
/*
- * Fill in the HS isoc and interrupt descriptors from the module
- * parameters. We assume that the user knows what they are doing and
- * won't give parameters that their UDC doesn't support.
+ * Fill in the HS isoc descriptors from the module parameters.
+ * We assume that the user knows what they are doing and won't
+ * give parameters that their UDC doesn't support.
*/
hs_iso_source_desc.wMaxPacketSize = isoc_maxpacket;
hs_iso_source_desc.wMaxPacketSize |= isoc_mult << 11;
hs_iso_sink_desc.bInterval = isoc_interval;
hs_iso_sink_desc.bEndpointAddress = fs_iso_sink_desc.bEndpointAddress;
- hs_int_source_desc.wMaxPacketSize = int_maxpacket;
- hs_int_source_desc.wMaxPacketSize |= int_mult << 11;
- hs_int_source_desc.bInterval = USB_MS_TO_HS_INTERVAL(int_interval);
- hs_int_source_desc.bEndpointAddress =
- fs_int_source_desc.bEndpointAddress;
-
- hs_int_sink_desc.wMaxPacketSize = int_maxpacket;
- hs_int_sink_desc.wMaxPacketSize |= int_mult << 11;
- hs_int_sink_desc.bInterval = USB_MS_TO_HS_INTERVAL(int_interval);
- hs_int_sink_desc.bEndpointAddress = fs_int_sink_desc.bEndpointAddress;
-
/* support super speed hardware */
ss_source_desc.bEndpointAddress =
fs_source_desc.bEndpointAddress;
fs_sink_desc.bEndpointAddress;
/*
- * Fill in the SS isoc and interrupt descriptors from the module
- * parameters. We assume that the user knows what they are doing and
- * won't give parameters that their UDC doesn't support.
+ * Fill in the SS isoc descriptors from the module parameters.
+ * We assume that the user knows what they are doing and won't
+ * give parameters that their UDC doesn't support.
*/
ss_iso_source_desc.wMaxPacketSize = isoc_maxpacket;
ss_iso_source_desc.bInterval = isoc_interval;
isoc_maxpacket * (isoc_mult + 1) * (isoc_maxburst + 1);
ss_iso_sink_desc.bEndpointAddress = fs_iso_sink_desc.bEndpointAddress;
- ss_int_source_desc.wMaxPacketSize = int_maxpacket;
- ss_int_source_desc.bInterval = USB_MS_TO_SS_INTERVAL(int_interval);
- ss_int_source_comp_desc.bmAttributes = int_mult;
- ss_int_source_comp_desc.bMaxBurst = int_maxburst;
- ss_int_source_comp_desc.wBytesPerInterval =
- int_maxpacket * (int_mult + 1) * (int_maxburst + 1);
- ss_int_source_desc.bEndpointAddress =
- fs_int_source_desc.bEndpointAddress;
-
- ss_int_sink_desc.wMaxPacketSize = int_maxpacket;
- ss_int_sink_desc.bInterval = USB_MS_TO_SS_INTERVAL(int_interval);
- ss_int_sink_comp_desc.bmAttributes = int_mult;
- ss_int_sink_comp_desc.bMaxBurst = int_maxburst;
- ss_int_sink_comp_desc.wBytesPerInterval =
- int_maxpacket * (int_mult + 1) * (int_maxburst + 1);
- ss_int_sink_desc.bEndpointAddress = fs_int_sink_desc.bEndpointAddress;
-
ret = usb_assign_descriptors(f, fs_source_sink_descs,
hs_source_sink_descs, ss_source_sink_descs);
if (ret)
return ret;
- DBG(cdev, "%s speed %s: IN/%s, OUT/%s, ISO-IN/%s, ISO-OUT/%s, "
- "INT-IN/%s, INT-OUT/%s\n",
+ DBG(cdev, "%s speed %s: IN/%s, OUT/%s, ISO-IN/%s, ISO-OUT/%s\n",
(gadget_is_superspeed(c->cdev->gadget) ? "super" :
(gadget_is_dualspeed(c->cdev->gadget) ? "dual" : "full")),
f->name, ss->in_ep->name, ss->out_ep->name,
ss->iso_in_ep ? ss->iso_in_ep->name : "<none>",
- ss->iso_out_ep ? ss->iso_out_ep->name : "<none>",
- ss->int_in_ep ? ss->int_in_ep->name : "<none>",
- ss->int_out_ep ? ss->int_out_ep->name : "<none>");
+ ss->iso_out_ep ? ss->iso_out_ep->name : "<none>");
return 0;
}
}
static int source_sink_start_ep(struct f_sourcesink *ss, bool is_in,
- enum eptype ep_type, int speed)
+ bool is_iso, int speed)
{
struct usb_ep *ep;
struct usb_request *req;
int i, size, status;
for (i = 0; i < 8; i++) {
- switch (ep_type) {
- case EP_ISOC:
+ if (is_iso) {
switch (speed) {
case USB_SPEED_SUPER:
size = isoc_maxpacket * (isoc_mult + 1) *
}
ep = is_in ? ss->iso_in_ep : ss->iso_out_ep;
req = ss_alloc_ep_req(ep, size);
- break;
- case EP_INTERRUPT:
- switch (speed) {
- case USB_SPEED_SUPER:
- size = int_maxpacket * (int_mult + 1) *
- (int_maxburst + 1);
- break;
- case USB_SPEED_HIGH:
- size = int_maxpacket * (int_mult + 1);
- break;
- default:
- size = int_maxpacket > 1023 ?
- 1023 : int_maxpacket;
- break;
- }
- ep = is_in ? ss->int_in_ep : ss->int_out_ep;
- req = ss_alloc_ep_req(ep, size);
- break;
- default:
+ } else {
ep = is_in ? ss->in_ep : ss->out_ep;
req = ss_alloc_ep_req(ep, 0);
- break;
}
if (!req)
cdev = ss->function.config->cdev;
ERROR(cdev, "start %s%s %s --> %d\n",
- get_ep_string(ep_type), is_in ? "IN" : "OUT",
- ep->name, status);
+ is_iso ? "ISO-" : "", is_in ? "IN" : "OUT",
+ ep->name, status);
free_ep_req(ep, req);
}
- if (!(ep_type == EP_ISOC))
+ if (!is_iso)
break;
}
cdev = ss->function.config->cdev;
disable_endpoints(cdev, ss->in_ep, ss->out_ep, ss->iso_in_ep,
- ss->iso_out_ep, ss->int_in_ep, ss->int_out_ep);
+ ss->iso_out_ep);
VDBG(cdev, "%s disabled\n", ss->function.name);
}
int speed = cdev->gadget->speed;
struct usb_ep *ep;
- if (alt == 2) {
- /* Configure for periodic interrupt endpoint */
- ep = ss->int_in_ep;
- if (ep) {
- result = config_ep_by_speed(cdev->gadget,
- &(ss->function), ep);
- if (result)
- return result;
-
- result = usb_ep_enable(ep);
- if (result < 0)
- return result;
-
- ep->driver_data = ss;
- result = source_sink_start_ep(ss, true, EP_INTERRUPT,
- speed);
- if (result < 0) {
-fail1:
- ep = ss->int_in_ep;
- if (ep) {
- usb_ep_disable(ep);
- ep->driver_data = NULL;
- }
- return result;
- }
- }
-
- /*
- * one interrupt endpoint reads (sinks) anything OUT (from the
- * host)
- */
- ep = ss->int_out_ep;
- if (ep) {
- result = config_ep_by_speed(cdev->gadget,
- &(ss->function), ep);
- if (result)
- goto fail1;
-
- result = usb_ep_enable(ep);
- if (result < 0)
- goto fail1;
-
- ep->driver_data = ss;
- result = source_sink_start_ep(ss, false, EP_INTERRUPT,
- speed);
- if (result < 0) {
- ep = ss->int_out_ep;
- usb_ep_disable(ep);
- ep->driver_data = NULL;
- goto fail1;
- }
- }
-
- goto out;
- }
-
/* one bulk endpoint writes (sources) zeroes IN (to the host) */
ep = ss->in_ep;
result = config_ep_by_speed(cdev->gadget, &(ss->function), ep);
return result;
ep->driver_data = ss;
- result = source_sink_start_ep(ss, true, EP_BULK, speed);
+ result = source_sink_start_ep(ss, true, false, speed);
if (result < 0) {
fail:
ep = ss->in_ep;
goto fail;
ep->driver_data = ss;
- result = source_sink_start_ep(ss, false, EP_BULK, speed);
+ result = source_sink_start_ep(ss, false, false, speed);
if (result < 0) {
fail2:
ep = ss->out_ep;
goto fail2;
ep->driver_data = ss;
- result = source_sink_start_ep(ss, true, EP_ISOC, speed);
+ result = source_sink_start_ep(ss, true, true, speed);
if (result < 0) {
fail3:
ep = ss->iso_in_ep;
goto fail3;
ep->driver_data = ss;
- result = source_sink_start_ep(ss, false, EP_ISOC, speed);
+ result = source_sink_start_ep(ss, false, true, speed);
if (result < 0) {
usb_ep_disable(ep);
ep->driver_data = NULL;
goto fail3;
}
}
-
out:
ss->cur_alt = alt;
if (ss->in_ep->driver_data)
disable_source_sink(ss);
- else if (alt == 2 && ss->int_in_ep->driver_data)
- disable_source_sink(ss);
return enable_source_sink(cdev, ss, alt);
}
isoc_maxpacket = ss_opts->isoc_maxpacket;
isoc_mult = ss_opts->isoc_mult;
isoc_maxburst = ss_opts->isoc_maxburst;
- int_interval = ss_opts->int_interval;
- int_maxpacket = ss_opts->int_maxpacket;
- int_mult = ss_opts->int_mult;
- int_maxburst = ss_opts->int_maxburst;
buflen = ss_opts->bulk_buflen;
ss->function.name = "source/sink";
f_ss_opts_bulk_buflen_show,
f_ss_opts_bulk_buflen_store);
-static ssize_t f_ss_opts_int_interval_show(struct f_ss_opts *opts, char *page)
-{
- int result;
-
- mutex_lock(&opts->lock);
- result = sprintf(page, "%u", opts->int_interval);
- mutex_unlock(&opts->lock);
-
- return result;
-}
-
-static ssize_t f_ss_opts_int_interval_store(struct f_ss_opts *opts,
- const char *page, size_t len)
-{
- int ret;
- u32 num;
-
- mutex_lock(&opts->lock);
- if (opts->refcnt) {
- ret = -EBUSY;
- goto end;
- }
-
- ret = kstrtou32(page, 0, &num);
- if (ret)
- goto end;
-
- if (num > 4096) {
- ret = -EINVAL;
- goto end;
- }
-
- opts->int_interval = num;
- ret = len;
-end:
- mutex_unlock(&opts->lock);
- return ret;
-}
-
-static struct f_ss_opts_attribute f_ss_opts_int_interval =
- __CONFIGFS_ATTR(int_interval, S_IRUGO | S_IWUSR,
- f_ss_opts_int_interval_show,
- f_ss_opts_int_interval_store);
-
-static ssize_t f_ss_opts_int_maxpacket_show(struct f_ss_opts *opts, char *page)
-{
- int result;
-
- mutex_lock(&opts->lock);
- result = sprintf(page, "%u", opts->int_maxpacket);
- mutex_unlock(&opts->lock);
-
- return result;
-}
-
-static ssize_t f_ss_opts_int_maxpacket_store(struct f_ss_opts *opts,
- const char *page, size_t len)
-{
- int ret;
- u16 num;
-
- mutex_lock(&opts->lock);
- if (opts->refcnt) {
- ret = -EBUSY;
- goto end;
- }
-
- ret = kstrtou16(page, 0, &num);
- if (ret)
- goto end;
-
- if (num > 1024) {
- ret = -EINVAL;
- goto end;
- }
-
- opts->int_maxpacket = num;
- ret = len;
-end:
- mutex_unlock(&opts->lock);
- return ret;
-}
-
-static struct f_ss_opts_attribute f_ss_opts_int_maxpacket =
- __CONFIGFS_ATTR(int_maxpacket, S_IRUGO | S_IWUSR,
- f_ss_opts_int_maxpacket_show,
- f_ss_opts_int_maxpacket_store);
-
-static ssize_t f_ss_opts_int_mult_show(struct f_ss_opts *opts, char *page)
-{
- int result;
-
- mutex_lock(&opts->lock);
- result = sprintf(page, "%u", opts->int_mult);
- mutex_unlock(&opts->lock);
-
- return result;
-}
-
-static ssize_t f_ss_opts_int_mult_store(struct f_ss_opts *opts,
- const char *page, size_t len)
-{
- int ret;
- u8 num;
-
- mutex_lock(&opts->lock);
- if (opts->refcnt) {
- ret = -EBUSY;
- goto end;
- }
-
- ret = kstrtou8(page, 0, &num);
- if (ret)
- goto end;
-
- if (num > 2) {
- ret = -EINVAL;
- goto end;
- }
-
- opts->int_mult = num;
- ret = len;
-end:
- mutex_unlock(&opts->lock);
- return ret;
-}
-
-static struct f_ss_opts_attribute f_ss_opts_int_mult =
- __CONFIGFS_ATTR(int_mult, S_IRUGO | S_IWUSR,
- f_ss_opts_int_mult_show,
- f_ss_opts_int_mult_store);
-
-static ssize_t f_ss_opts_int_maxburst_show(struct f_ss_opts *opts, char *page)
-{
- int result;
-
- mutex_lock(&opts->lock);
- result = sprintf(page, "%u", opts->int_maxburst);
- mutex_unlock(&opts->lock);
-
- return result;
-}
-
-static ssize_t f_ss_opts_int_maxburst_store(struct f_ss_opts *opts,
- const char *page, size_t len)
-{
- int ret;
- u8 num;
-
- mutex_lock(&opts->lock);
- if (opts->refcnt) {
- ret = -EBUSY;
- goto end;
- }
-
- ret = kstrtou8(page, 0, &num);
- if (ret)
- goto end;
-
- if (num > 15) {
- ret = -EINVAL;
- goto end;
- }
-
- opts->int_maxburst = num;
- ret = len;
-end:
- mutex_unlock(&opts->lock);
- return ret;
-}
-
-static struct f_ss_opts_attribute f_ss_opts_int_maxburst =
- __CONFIGFS_ATTR(int_maxburst, S_IRUGO | S_IWUSR,
- f_ss_opts_int_maxburst_show,
- f_ss_opts_int_maxburst_store);
-
static struct configfs_attribute *ss_attrs[] = {
&f_ss_opts_pattern.attr,
&f_ss_opts_isoc_interval.attr,
&f_ss_opts_isoc_mult.attr,
&f_ss_opts_isoc_maxburst.attr,
&f_ss_opts_bulk_buflen.attr,
- &f_ss_opts_int_interval.attr,
- &f_ss_opts_int_maxpacket.attr,
- &f_ss_opts_int_mult.attr,
- &f_ss_opts_int_maxburst.attr,
NULL,
};
ss_opts->isoc_interval = GZERO_ISOC_INTERVAL;
ss_opts->isoc_maxpacket = GZERO_ISOC_MAXPACKET;
ss_opts->bulk_buflen = GZERO_BULK_BUFLEN;
- ss_opts->int_interval = GZERO_INT_INTERVAL;
- ss_opts->int_maxpacket = GZERO_INT_MAXPACKET;
config_group_init_type_name(&ss_opts->func_inst.group, "",
&ss_func_type);
#define UNFLW_CTRL 8
#define OVFLW_CTRL 10
-const char *uac2_name = "snd_uac2";
+static const char *uac2_name = "snd_uac2";
struct uac2_req {
struct uac2_rtd_params *pp; /* parent param */
};
/* Clock source for IN traffic */
-struct uac_clock_source_descriptor in_clk_src_desc = {
+static struct uac_clock_source_descriptor in_clk_src_desc = {
.bLength = sizeof in_clk_src_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Clock source for OUT traffic */
-struct uac_clock_source_descriptor out_clk_src_desc = {
+static struct uac_clock_source_descriptor out_clk_src_desc = {
.bLength = sizeof out_clk_src_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Input Terminal for USB_OUT */
-struct uac2_input_terminal_descriptor usb_out_it_desc = {
+static struct uac2_input_terminal_descriptor usb_out_it_desc = {
.bLength = sizeof usb_out_it_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Input Terminal for I/O-In */
-struct uac2_input_terminal_descriptor io_in_it_desc = {
+static struct uac2_input_terminal_descriptor io_in_it_desc = {
.bLength = sizeof io_in_it_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Ouput Terminal for USB_IN */
-struct uac2_output_terminal_descriptor usb_in_ot_desc = {
+static struct uac2_output_terminal_descriptor usb_in_ot_desc = {
.bLength = sizeof usb_in_ot_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Ouput Terminal for I/O-Out */
-struct uac2_output_terminal_descriptor io_out_ot_desc = {
+static struct uac2_output_terminal_descriptor io_out_ot_desc = {
.bLength = sizeof io_out_ot_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
.bmControls = (CONTROL_RDWR << COPY_CTRL),
};
-struct uac2_ac_header_descriptor ac_hdr_desc = {
+static struct uac2_ac_header_descriptor ac_hdr_desc = {
.bLength = sizeof ac_hdr_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Audio Stream OUT Intface Desc */
-struct uac2_as_header_descriptor as_out_hdr_desc = {
+static struct uac2_as_header_descriptor as_out_hdr_desc = {
.bLength = sizeof as_out_hdr_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Audio USB_OUT Format */
-struct uac2_format_type_i_descriptor as_out_fmt1_desc = {
+static struct uac2_format_type_i_descriptor as_out_fmt1_desc = {
.bLength = sizeof as_out_fmt1_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
.bDescriptorSubtype = UAC_FORMAT_TYPE,
};
/* STD AS ISO OUT Endpoint */
-struct usb_endpoint_descriptor fs_epout_desc = {
+static struct usb_endpoint_descriptor fs_epout_desc = {
.bLength = USB_DT_ENDPOINT_SIZE,
.bDescriptorType = USB_DT_ENDPOINT,
.bInterval = 1,
};
-struct usb_endpoint_descriptor hs_epout_desc = {
+static struct usb_endpoint_descriptor hs_epout_desc = {
.bLength = USB_DT_ENDPOINT_SIZE,
.bDescriptorType = USB_DT_ENDPOINT,
};
/* Audio Stream IN Intface Desc */
-struct uac2_as_header_descriptor as_in_hdr_desc = {
+static struct uac2_as_header_descriptor as_in_hdr_desc = {
.bLength = sizeof as_in_hdr_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
};
/* Audio USB_IN Format */
-struct uac2_format_type_i_descriptor as_in_fmt1_desc = {
+static struct uac2_format_type_i_descriptor as_in_fmt1_desc = {
.bLength = sizeof as_in_fmt1_desc,
.bDescriptorType = USB_DT_CS_INTERFACE,
.bDescriptorSubtype = UAC_FORMAT_TYPE,
};
/* STD AS ISO IN Endpoint */
-struct usb_endpoint_descriptor fs_epin_desc = {
+static struct usb_endpoint_descriptor fs_epin_desc = {
.bLength = USB_DT_ENDPOINT_SIZE,
.bDescriptorType = USB_DT_ENDPOINT,
.bInterval = 1,
};
-struct usb_endpoint_descriptor hs_epin_desc = {
+static struct usb_endpoint_descriptor hs_epin_desc = {
.bLength = USB_DT_ENDPOINT_SIZE,
.bDescriptorType = USB_DT_ENDPOINT,
agdev->out_ep->driver_data = NULL;
}
-struct usb_function *afunc_alloc(struct usb_function_instance *fi)
+static struct usb_function *afunc_alloc(struct usb_function_instance *fi)
{
struct audio_dev *agdev;
struct f_uac2_opts *opts;
#define GZERO_QLEN 32
#define GZERO_ISOC_INTERVAL 4
#define GZERO_ISOC_MAXPACKET 1024
-#define GZERO_INT_INTERVAL 1 /* Default interrupt interval = 1 ms */
-#define GZERO_INT_MAXPACKET 1024
struct usb_zero_options {
unsigned pattern;
unsigned isoc_maxpacket;
unsigned isoc_mult;
unsigned isoc_maxburst;
- unsigned int_interval; /* In ms */
- unsigned int_maxpacket;
- unsigned int_mult;
- unsigned int_maxburst;
unsigned bulk_buflen;
unsigned qlen;
};
unsigned isoc_maxpacket;
unsigned isoc_mult;
unsigned isoc_maxburst;
- unsigned int_interval; /* In ms */
- unsigned int_maxpacket;
- unsigned int_mult;
- unsigned int_maxburst;
unsigned bulk_buflen;
/*
void free_ep_req(struct usb_ep *ep, struct usb_request *req);
void disable_endpoints(struct usb_composite_dev *cdev,
struct usb_ep *in, struct usb_ep *out,
- struct usb_ep *iso_in, struct usb_ep *iso_out,
- struct usb_ep *int_in, struct usb_ep *int_out);
+ struct usb_ep *iso_in, struct usb_ep *iso_out);
#endif /* __G_ZERO_H */
#include "uvc.h"
#include "uvc_queue.h"
#include "uvc_video.h"
+#include "uvc_v4l2.h"
/* --------------------------------------------------------------------------
* Requests handling
#include "uvc.h"
#include "uvc_queue.h"
+#include "uvc_video.h"
/* --------------------------------------------------------------------------
* Video codecs
struct usb_configuration c;
int (*eth)(struct usb_configuration *c);
int num;
-} gfs_configurations[] = {
+};
+
+static struct gfs_configuration gfs_configurations[] = {
#ifdef CONFIG_USB_FUNCTIONFS_RNDIS
{
.eth = bind_rndis_config,
if (!try_module_get(THIS_MODULE))
return ERR_PTR(-ENOENT);
- return 0;
+ return NULL;
}
static void functionfs_release_dev(struct ffs_dev *dev)
MODULE_AUTHOR ("David Brownell");
MODULE_LICENSE ("GPL");
+static int ep_open(struct inode *, struct file *);
+
/*----------------------------------------------------------------------*/
* still need dev->lock to use epdata->ep.
*/
static int
-get_ready_ep (unsigned f_flags, struct ep_data *epdata)
+get_ready_ep (unsigned f_flags, struct ep_data *epdata, bool is_write)
{
int val;
if (f_flags & O_NONBLOCK) {
if (!mutex_trylock(&epdata->lock))
goto nonblock;
- if (epdata->state != STATE_EP_ENABLED) {
+ if (epdata->state != STATE_EP_ENABLED &&
+ (!is_write || epdata->state != STATE_EP_READY)) {
mutex_unlock(&epdata->lock);
nonblock:
val = -EAGAIN;
switch (epdata->state) {
case STATE_EP_ENABLED:
+ return 0;
+ case STATE_EP_READY: /* not configured yet */
+ if (is_write)
+ return 0;
+ // FALLTHRU
+ case STATE_EP_UNBOUND: /* clean disconnect */
break;
// case STATE_EP_DISABLED: /* "can't happen" */
- // case STATE_EP_READY: /* "can't happen" */
default: /* error! */
pr_debug ("%s: ep %p not available, state %d\n",
shortname, epdata, epdata->state);
- // FALLTHROUGH
- case STATE_EP_UNBOUND: /* clean disconnect */
- val = -ENODEV;
- mutex_unlock(&epdata->lock);
}
- return val;
+ mutex_unlock(&epdata->lock);
+ return -ENODEV;
}
static ssize_t
return value;
}
-
-/* handle a synchronous OUT bulk/intr/iso transfer */
-static ssize_t
-ep_read (struct file *fd, char __user *buf, size_t len, loff_t *ptr)
-{
- struct ep_data *data = fd->private_data;
- void *kbuf;
- ssize_t value;
-
- if ((value = get_ready_ep (fd->f_flags, data)) < 0)
- return value;
-
- /* halt any endpoint by doing a "wrong direction" i/o call */
- if (usb_endpoint_dir_in(&data->desc)) {
- if (usb_endpoint_xfer_isoc(&data->desc)) {
- mutex_unlock(&data->lock);
- return -EINVAL;
- }
- DBG (data->dev, "%s halt\n", data->name);
- spin_lock_irq (&data->dev->lock);
- if (likely (data->ep != NULL))
- usb_ep_set_halt (data->ep);
- spin_unlock_irq (&data->dev->lock);
- mutex_unlock(&data->lock);
- return -EBADMSG;
- }
-
- /* FIXME readahead for O_NONBLOCK and poll(); careful with ZLPs */
-
- value = -ENOMEM;
- kbuf = kmalloc (len, GFP_KERNEL);
- if (unlikely (!kbuf))
- goto free1;
-
- value = ep_io (data, kbuf, len);
- VDEBUG (data->dev, "%s read %zu OUT, status %d\n",
- data->name, len, (int) value);
- if (value >= 0 && copy_to_user (buf, kbuf, value))
- value = -EFAULT;
-
-free1:
- mutex_unlock(&data->lock);
- kfree (kbuf);
- return value;
-}
-
-/* handle a synchronous IN bulk/intr/iso transfer */
-static ssize_t
-ep_write (struct file *fd, const char __user *buf, size_t len, loff_t *ptr)
-{
- struct ep_data *data = fd->private_data;
- void *kbuf;
- ssize_t value;
-
- if ((value = get_ready_ep (fd->f_flags, data)) < 0)
- return value;
-
- /* halt any endpoint by doing a "wrong direction" i/o call */
- if (!usb_endpoint_dir_in(&data->desc)) {
- if (usb_endpoint_xfer_isoc(&data->desc)) {
- mutex_unlock(&data->lock);
- return -EINVAL;
- }
- DBG (data->dev, "%s halt\n", data->name);
- spin_lock_irq (&data->dev->lock);
- if (likely (data->ep != NULL))
- usb_ep_set_halt (data->ep);
- spin_unlock_irq (&data->dev->lock);
- mutex_unlock(&data->lock);
- return -EBADMSG;
- }
-
- /* FIXME writebehind for O_NONBLOCK and poll(), qlen = 1 */
-
- value = -ENOMEM;
- kbuf = memdup_user(buf, len);
- if (IS_ERR(kbuf)) {
- value = PTR_ERR(kbuf);
- kbuf = NULL;
- goto free1;
- }
-
- value = ep_io (data, kbuf, len);
- VDEBUG (data->dev, "%s write %zu IN, status %d\n",
- data->name, len, (int) value);
-free1:
- mutex_unlock(&data->lock);
- kfree (kbuf);
- return value;
-}
-
static int
ep_release (struct inode *inode, struct file *fd)
{
struct ep_data *data = fd->private_data;
int status;
- if ((status = get_ready_ep (fd->f_flags, data)) < 0)
+ if ((status = get_ready_ep (fd->f_flags, data, false)) < 0)
return status;
spin_lock_irq (&data->dev->lock);
struct mm_struct *mm;
struct work_struct work;
void *buf;
- const struct iovec *iv;
- unsigned long nr_segs;
+ struct iov_iter to;
+ const void *to_free;
unsigned actual;
};
return value;
}
-static ssize_t ep_copy_to_user(struct kiocb_priv *priv)
-{
- ssize_t len, total;
- void *to_copy;
- int i;
-
- /* copy stuff into user buffers */
- total = priv->actual;
- len = 0;
- to_copy = priv->buf;
- for (i=0; i < priv->nr_segs; i++) {
- ssize_t this = min((ssize_t)(priv->iv[i].iov_len), total);
-
- if (copy_to_user(priv->iv[i].iov_base, to_copy, this)) {
- if (len == 0)
- len = -EFAULT;
- break;
- }
-
- total -= this;
- len += this;
- to_copy += this;
- if (total == 0)
- break;
- }
-
- return len;
-}
-
static void ep_user_copy_worker(struct work_struct *work)
{
struct kiocb_priv *priv = container_of(work, struct kiocb_priv, work);
size_t ret;
use_mm(mm);
- ret = ep_copy_to_user(priv);
+ ret = copy_to_iter(priv->buf, priv->actual, &priv->to);
unuse_mm(mm);
+ if (!ret)
+ ret = -EFAULT;
/* completing the iocb can drop the ctx and mm, don't touch mm after */
aio_complete(iocb, ret, ret);
kfree(priv->buf);
+ kfree(priv->to_free);
kfree(priv);
}
* don't need to copy anything to userspace, so we can
* complete the aio request immediately.
*/
- if (priv->iv == NULL || unlikely(req->actual == 0)) {
+ if (priv->to_free == NULL || unlikely(req->actual == 0)) {
kfree(req->buf);
+ kfree(priv->to_free);
kfree(priv);
iocb->private = NULL;
/* aio_complete() reports bytes-transferred _and_ faults */
priv->buf = req->buf;
priv->actual = req->actual;
+ INIT_WORK(&priv->work, ep_user_copy_worker);
schedule_work(&priv->work);
}
spin_unlock(&epdata->dev->lock);
put_ep(epdata);
}
-static ssize_t
-ep_aio_rwtail(
- struct kiocb *iocb,
- char *buf,
- size_t len,
- struct ep_data *epdata,
- const struct iovec *iv,
- unsigned long nr_segs
-)
+static ssize_t ep_aio(struct kiocb *iocb,
+ struct kiocb_priv *priv,
+ struct ep_data *epdata,
+ char *buf,
+ size_t len)
{
- struct kiocb_priv *priv;
- struct usb_request *req;
- ssize_t value;
+ struct usb_request *req;
+ ssize_t value;
- priv = kmalloc(sizeof *priv, GFP_KERNEL);
- if (!priv) {
- value = -ENOMEM;
-fail:
- kfree(buf);
- return value;
- }
iocb->private = priv;
priv->iocb = iocb;
- priv->iv = iv;
- priv->nr_segs = nr_segs;
- INIT_WORK(&priv->work, ep_user_copy_worker);
-
- value = get_ready_ep(iocb->ki_filp->f_flags, epdata);
- if (unlikely(value < 0)) {
- kfree(priv);
- goto fail;
- }
kiocb_set_cancel_fn(iocb, ep_aio_cancel);
get_ep(epdata);
* allocate or submit those if the host disconnected.
*/
spin_lock_irq(&epdata->dev->lock);
- if (likely(epdata->ep)) {
- req = usb_ep_alloc_request(epdata->ep, GFP_ATOMIC);
- if (likely(req)) {
- priv->req = req;
- req->buf = buf;
- req->length = len;
- req->complete = ep_aio_complete;
- req->context = iocb;
- value = usb_ep_queue(epdata->ep, req, GFP_ATOMIC);
- if (unlikely(0 != value))
- usb_ep_free_request(epdata->ep, req);
- } else
- value = -EAGAIN;
- } else
- value = -ENODEV;
- spin_unlock_irq(&epdata->dev->lock);
+ value = -ENODEV;
+ if (unlikely(epdata->ep))
+ goto fail;
- mutex_unlock(&epdata->lock);
+ req = usb_ep_alloc_request(epdata->ep, GFP_ATOMIC);
+ value = -ENOMEM;
+ if (unlikely(!req))
+ goto fail;
- if (unlikely(value)) {
- kfree(priv);
- put_ep(epdata);
- } else
- value = -EIOCBQUEUED;
+ priv->req = req;
+ req->buf = buf;
+ req->length = len;
+ req->complete = ep_aio_complete;
+ req->context = iocb;
+ value = usb_ep_queue(epdata->ep, req, GFP_ATOMIC);
+ if (unlikely(0 != value)) {
+ usb_ep_free_request(epdata->ep, req);
+ goto fail;
+ }
+ spin_unlock_irq(&epdata->dev->lock);
+ return -EIOCBQUEUED;
+
+fail:
+ spin_unlock_irq(&epdata->dev->lock);
+ kfree(priv->to_free);
+ kfree(priv);
+ put_ep(epdata);
return value;
}
static ssize_t
-ep_aio_read(struct kiocb *iocb, const struct iovec *iov,
- unsigned long nr_segs, loff_t o)
+ep_read_iter(struct kiocb *iocb, struct iov_iter *to)
{
- struct ep_data *epdata = iocb->ki_filp->private_data;
- char *buf;
+ struct file *file = iocb->ki_filp;
+ struct ep_data *epdata = file->private_data;
+ size_t len = iov_iter_count(to);
+ ssize_t value;
+ char *buf;
- if (unlikely(usb_endpoint_dir_in(&epdata->desc)))
- return -EINVAL;
+ if ((value = get_ready_ep(file->f_flags, epdata, false)) < 0)
+ return value;
- buf = kmalloc(iocb->ki_nbytes, GFP_KERNEL);
- if (unlikely(!buf))
- return -ENOMEM;
+ /* halt any endpoint by doing a "wrong direction" i/o call */
+ if (usb_endpoint_dir_in(&epdata->desc)) {
+ if (usb_endpoint_xfer_isoc(&epdata->desc) ||
+ !is_sync_kiocb(iocb)) {
+ mutex_unlock(&epdata->lock);
+ return -EINVAL;
+ }
+ DBG (epdata->dev, "%s halt\n", epdata->name);
+ spin_lock_irq(&epdata->dev->lock);
+ if (likely(epdata->ep != NULL))
+ usb_ep_set_halt(epdata->ep);
+ spin_unlock_irq(&epdata->dev->lock);
+ mutex_unlock(&epdata->lock);
+ return -EBADMSG;
+ }
- return ep_aio_rwtail(iocb, buf, iocb->ki_nbytes, epdata, iov, nr_segs);
+ buf = kmalloc(len, GFP_KERNEL);
+ if (unlikely(!buf)) {
+ mutex_unlock(&epdata->lock);
+ return -ENOMEM;
+ }
+ if (is_sync_kiocb(iocb)) {
+ value = ep_io(epdata, buf, len);
+ if (value >= 0 && copy_to_iter(buf, value, to))
+ value = -EFAULT;
+ } else {
+ struct kiocb_priv *priv = kzalloc(sizeof *priv, GFP_KERNEL);
+ value = -ENOMEM;
+ if (!priv)
+ goto fail;
+ priv->to_free = dup_iter(&priv->to, to, GFP_KERNEL);
+ if (!priv->to_free) {
+ kfree(priv);
+ goto fail;
+ }
+ value = ep_aio(iocb, priv, epdata, buf, len);
+ if (value == -EIOCBQUEUED)
+ buf = NULL;
+ }
+fail:
+ kfree(buf);
+ mutex_unlock(&epdata->lock);
+ return value;
}
+static ssize_t ep_config(struct ep_data *, const char *, size_t);
+
static ssize_t
-ep_aio_write(struct kiocb *iocb, const struct iovec *iov,
- unsigned long nr_segs, loff_t o)
+ep_write_iter(struct kiocb *iocb, struct iov_iter *from)
{
- struct ep_data *epdata = iocb->ki_filp->private_data;
- char *buf;
- size_t len = 0;
- int i = 0;
+ struct file *file = iocb->ki_filp;
+ struct ep_data *epdata = file->private_data;
+ size_t len = iov_iter_count(from);
+ bool configured;
+ ssize_t value;
+ char *buf;
+
+ if ((value = get_ready_ep(file->f_flags, epdata, true)) < 0)
+ return value;
- if (unlikely(!usb_endpoint_dir_in(&epdata->desc)))
- return -EINVAL;
+ configured = epdata->state == STATE_EP_ENABLED;
- buf = kmalloc(iocb->ki_nbytes, GFP_KERNEL);
- if (unlikely(!buf))
+ /* halt any endpoint by doing a "wrong direction" i/o call */
+ if (configured && !usb_endpoint_dir_in(&epdata->desc)) {
+ if (usb_endpoint_xfer_isoc(&epdata->desc) ||
+ !is_sync_kiocb(iocb)) {
+ mutex_unlock(&epdata->lock);
+ return -EINVAL;
+ }
+ DBG (epdata->dev, "%s halt\n", epdata->name);
+ spin_lock_irq(&epdata->dev->lock);
+ if (likely(epdata->ep != NULL))
+ usb_ep_set_halt(epdata->ep);
+ spin_unlock_irq(&epdata->dev->lock);
+ mutex_unlock(&epdata->lock);
+ return -EBADMSG;
+ }
+
+ buf = kmalloc(len, GFP_KERNEL);
+ if (unlikely(!buf)) {
+ mutex_unlock(&epdata->lock);
return -ENOMEM;
+ }
- for (i=0; i < nr_segs; i++) {
- if (unlikely(copy_from_user(&buf[len], iov[i].iov_base,
- iov[i].iov_len) != 0)) {
- kfree(buf);
- return -EFAULT;
+ if (unlikely(copy_from_iter(buf, len, from) != len)) {
+ value = -EFAULT;
+ goto out;
+ }
+
+ if (unlikely(!configured)) {
+ value = ep_config(epdata, buf, len);
+ } else if (is_sync_kiocb(iocb)) {
+ value = ep_io(epdata, buf, len);
+ } else {
+ struct kiocb_priv *priv = kzalloc(sizeof *priv, GFP_KERNEL);
+ value = -ENOMEM;
+ if (priv) {
+ value = ep_aio(iocb, priv, epdata, buf, len);
+ if (value == -EIOCBQUEUED)
+ buf = NULL;
}
- len += iov[i].iov_len;
}
- return ep_aio_rwtail(iocb, buf, len, epdata, NULL, 0);
+out:
+ kfree(buf);
+ mutex_unlock(&epdata->lock);
+ return value;
}
/*----------------------------------------------------------------------*/
/* used after endpoint configuration */
static const struct file_operations ep_io_operations = {
.owner = THIS_MODULE,
- .llseek = no_llseek,
- .read = ep_read,
- .write = ep_write,
- .unlocked_ioctl = ep_ioctl,
+ .open = ep_open,
.release = ep_release,
-
- .aio_read = ep_aio_read,
- .aio_write = ep_aio_write,
+ .llseek = no_llseek,
+ .read = new_sync_read,
+ .write = new_sync_write,
+ .unlocked_ioctl = ep_ioctl,
+ .read_iter = ep_read_iter,
+ .write_iter = ep_write_iter,
};
/* ENDPOINT INITIALIZATION
* speed descriptor, then optional high speed descriptor.
*/
static ssize_t
-ep_config (struct file *fd, const char __user *buf, size_t len, loff_t *ptr)
+ep_config (struct ep_data *data, const char *buf, size_t len)
{
- struct ep_data *data = fd->private_data;
struct usb_ep *ep;
u32 tag;
int value, length = len;
- value = mutex_lock_interruptible(&data->lock);
- if (value < 0)
- return value;
-
if (data->state != STATE_EP_READY) {
value = -EL2HLT;
goto fail;
goto fail0;
/* we might need to change message format someday */
- if (copy_from_user (&tag, buf, 4)) {
- goto fail1;
- }
+ memcpy(&tag, buf, 4);
if (tag != 1) {
DBG(data->dev, "config %s, bad tag %d\n", data->name, tag);
goto fail0;
*/
/* full/low speed descriptor, then high speed */
- if (copy_from_user (&data->desc, buf, USB_DT_ENDPOINT_SIZE)) {
- goto fail1;
- }
+ memcpy(&data->desc, buf, USB_DT_ENDPOINT_SIZE);
if (data->desc.bLength != USB_DT_ENDPOINT_SIZE
|| data->desc.bDescriptorType != USB_DT_ENDPOINT)
goto fail0;
if (len != USB_DT_ENDPOINT_SIZE) {
if (len != 2 * USB_DT_ENDPOINT_SIZE)
goto fail0;
- if (copy_from_user (&data->hs_desc, buf + USB_DT_ENDPOINT_SIZE,
- USB_DT_ENDPOINT_SIZE)) {
- goto fail1;
- }
+ memcpy(&data->hs_desc, buf + USB_DT_ENDPOINT_SIZE,
+ USB_DT_ENDPOINT_SIZE);
if (data->hs_desc.bLength != USB_DT_ENDPOINT_SIZE
|| data->hs_desc.bDescriptorType
!= USB_DT_ENDPOINT) {
case USB_SPEED_LOW:
case USB_SPEED_FULL:
ep->desc = &data->desc;
- value = usb_ep_enable(ep);
- if (value == 0)
- data->state = STATE_EP_ENABLED;
break;
case USB_SPEED_HIGH:
/* fails if caller didn't provide that descriptor... */
ep->desc = &data->hs_desc;
- value = usb_ep_enable(ep);
- if (value == 0)
- data->state = STATE_EP_ENABLED;
break;
default:
DBG(data->dev, "unconnected, %s init abandoned\n",
data->name);
value = -EINVAL;
+ goto gone;
}
+ value = usb_ep_enable(ep);
if (value == 0) {
- fd->f_op = &ep_io_operations;
+ data->state = STATE_EP_ENABLED;
value = length;
}
gone:
data->desc.bDescriptorType = 0;
data->hs_desc.bDescriptorType = 0;
}
- mutex_unlock(&data->lock);
return value;
fail0:
value = -EINVAL;
goto fail;
-fail1:
- value = -EFAULT;
- goto fail;
}
static int
return value;
}
-/* used before endpoint configuration */
-static const struct file_operations ep_config_operations = {
- .llseek = no_llseek,
-
- .open = ep_open,
- .write = ep_config,
- .release = ep_release,
-};
-
/*----------------------------------------------------------------------*/
/* EP0 IMPLEMENTATION can be partly in userspace.
enum ep0_state state;
spin_lock_irq (&dev->lock);
+ if (dev->state <= STATE_DEV_OPENED) {
+ retval = -EINVAL;
+ goto done;
+ }
/* report fd mode change before acting on it */
if (dev->setup_abort) {
struct dev_data *dev = fd->private_data;
ssize_t retval = -ESRCH;
- spin_lock_irq (&dev->lock);
-
/* report fd mode change before acting on it */
if (dev->setup_abort) {
dev->setup_abort = 0;
} else
DBG (dev, "fail %s, state %d\n", __func__, dev->state);
- spin_unlock_irq (&dev->lock);
return retval;
}
struct dev_data *dev = fd->private_data;
int mask = 0;
+ if (dev->state <= STATE_DEV_OPENED)
+ return DEFAULT_POLLMASK;
+
poll_wait(fd, &dev->wait, wait);
spin_lock_irq (&dev->lock);
return ret;
}
-/* used after device configuration */
-static const struct file_operations ep0_io_operations = {
- .owner = THIS_MODULE,
- .llseek = no_llseek,
-
- .read = ep0_read,
- .write = ep0_write,
- .fasync = ep0_fasync,
- .poll = ep0_poll,
- .unlocked_ioctl = dev_ioctl,
- .release = dev_release,
-};
-
/*----------------------------------------------------------------------*/
/* The in-kernel gadget driver handles most ep0 issues, in particular
goto enomem1;
data->dentry = gadgetfs_create_file (dev->sb, data->name,
- data, &ep_config_operations);
+ data, &ep_io_operations);
if (!data->dentry)
goto enomem2;
list_add_tail (&data->epfiles, &dev->epfiles);
u32 tag;
char *kbuf;
+ spin_lock_irq(&dev->lock);
+ if (dev->state > STATE_DEV_OPENED) {
+ value = ep0_write(fd, buf, len, ptr);
+ spin_unlock_irq(&dev->lock);
+ return value;
+ }
+ spin_unlock_irq(&dev->lock);
+
if (len < (USB_DT_CONFIG_SIZE + USB_DT_DEVICE_SIZE + 4))
return -EINVAL;
* on, they can work ... except in cleanup paths that
* kick in after the ep0 descriptor is closed.
*/
- fd->f_op = &ep0_io_operations;
value = len;
}
return value;
return value;
}
-static const struct file_operations dev_init_operations = {
+static const struct file_operations ep0_operations = {
.llseek = no_llseek,
.open = dev_open,
+ .read = ep0_read,
.write = dev_config,
.fasync = ep0_fasync,
+ .poll = ep0_poll,
.unlocked_ioctl = dev_ioctl,
.release = dev_release,
};
goto Enomem;
dev->sb = sb;
- dev->dentry = gadgetfs_create_file(sb, CHIP, dev, &dev_init_operations);
+ dev->dentry = gadgetfs_create_file(sb, CHIP, dev, &ep0_operations);
if (!dev->dentry) {
put_dev(dev);
goto Enomem;
goto err_session;
}
/*
- * Now register the TCM vHost virtual I_T Nexus as active with the
- * call to __transport_register_session()
+ * Now register the TCM vHost virtual I_T Nexus as active.
*/
- __transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+ transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
tv_nexus->tvn_se_sess, tv_nexus);
tpg->tpg_nexus = tv_nexus;
mutex_unlock(&tpg->tpg_mutex);
.isoc_maxpacket = GZERO_ISOC_MAXPACKET,
.bulk_buflen = GZERO_BULK_BUFLEN,
.qlen = GZERO_QLEN,
- .int_interval = GZERO_INT_INTERVAL,
- .int_maxpacket = GZERO_INT_MAXPACKET,
};
/*-------------------------------------------------------------------------*/
S_IRUGO|S_IWUSR);
MODULE_PARM_DESC(isoc_maxburst, "0 - 15 (ss only)");
-module_param_named(int_interval, gzero_options.int_interval, uint,
- S_IRUGO|S_IWUSR);
-MODULE_PARM_DESC(int_interval, "1 - 16");
-
-module_param_named(int_maxpacket, gzero_options.int_maxpacket, uint,
- S_IRUGO|S_IWUSR);
-MODULE_PARM_DESC(int_maxpacket, "0 - 1023 (fs), 0 - 1024 (hs/ss)");
-
-module_param_named(int_mult, gzero_options.int_mult, uint, S_IRUGO|S_IWUSR);
-MODULE_PARM_DESC(int_mult, "0 - 2 (hs/ss only)");
-
-module_param_named(int_maxburst, gzero_options.int_maxburst, uint,
- S_IRUGO|S_IWUSR);
-MODULE_PARM_DESC(int_maxburst, "0 - 15 (ss only)");
-
static struct usb_function *func_lb;
static struct usb_function_instance *func_inst_lb;
ss_opts->isoc_maxpacket = gzero_options.isoc_maxpacket;
ss_opts->isoc_mult = gzero_options.isoc_mult;
ss_opts->isoc_maxburst = gzero_options.isoc_maxburst;
- ss_opts->int_interval = gzero_options.int_interval;
- ss_opts->int_maxpacket = gzero_options.int_maxpacket;
- ss_opts->int_mult = gzero_options.int_mult;
- ss_opts->int_maxburst = gzero_options.int_maxburst;
ss_opts->bulk_buflen = gzero_options.bulk_buflen;
func_ss = usb_get_function(func_inst_ss);
struct atmel_ehci_priv {
struct clk *iclk;
- struct clk *fclk;
struct clk *uclk;
bool clocked;
};
{
if (atmel_ehci->clocked)
return;
- if (IS_ENABLED(CONFIG_COMMON_CLK)) {
- clk_set_rate(atmel_ehci->uclk, 48000000);
- clk_prepare_enable(atmel_ehci->uclk);
- }
+
+ clk_prepare_enable(atmel_ehci->uclk);
clk_prepare_enable(atmel_ehci->iclk);
- clk_prepare_enable(atmel_ehci->fclk);
atmel_ehci->clocked = true;
}
{
if (!atmel_ehci->clocked)
return;
- clk_disable_unprepare(atmel_ehci->fclk);
+
clk_disable_unprepare(atmel_ehci->iclk);
- if (IS_ENABLED(CONFIG_COMMON_CLK))
- clk_disable_unprepare(atmel_ehci->uclk);
+ clk_disable_unprepare(atmel_ehci->uclk);
atmel_ehci->clocked = false;
}
retval = -ENOENT;
goto fail_request_resource;
}
- atmel_ehci->fclk = devm_clk_get(&pdev->dev, "uhpck");
- if (IS_ERR(atmel_ehci->fclk)) {
- dev_err(&pdev->dev, "Error getting function clock\n");
- retval = -ENOENT;
+
+ atmel_ehci->uclk = devm_clk_get(&pdev->dev, "usb_clk");
+ if (IS_ERR(atmel_ehci->uclk)) {
+ dev_err(&pdev->dev, "failed to get uclk\n");
+ retval = PTR_ERR(atmel_ehci->uclk);
goto fail_request_resource;
}
- if (IS_ENABLED(CONFIG_COMMON_CLK)) {
- atmel_ehci->uclk = devm_clk_get(&pdev->dev, "usb_clk");
- if (IS_ERR(atmel_ehci->uclk)) {
- dev_err(&pdev->dev, "failed to get uclk\n");
- retval = PTR_ERR(atmel_ehci->uclk);
- goto fail_request_resource;
- }
- }
ehci = hcd_to_ehci(hcd);
/* registers start at offset 0x0 */
status = PORT_PLC;
port_change_bit = "link state";
break;
+ case USB_PORT_FEAT_C_PORT_CONFIG_ERROR:
+ status = PORT_CEC;
+ port_change_bit = "config error";
+ break;
default:
/* Should never happen */
return;
status |= USB_PORT_STAT_C_LINK_STATE << 16;
if ((raw_port_status & PORT_WRC))
status |= USB_PORT_STAT_C_BH_RESET << 16;
+ if ((raw_port_status & PORT_CEC))
+ status |= USB_PORT_STAT_C_CONFIG_ERROR << 16;
}
if (hcd->speed != HCD_USB3) {
case USB_PORT_FEAT_C_OVER_CURRENT:
case USB_PORT_FEAT_C_ENABLE:
case USB_PORT_FEAT_C_PORT_LINK_STATE:
+ case USB_PORT_FEAT_C_PORT_CONFIG_ERROR:
xhci_clear_port_change_bit(xhci, wValue, wIndex,
port_array[wIndex], temp);
break;
*/
status = bus_state->resuming_ports;
- mask = PORT_CSC | PORT_PEC | PORT_OCC | PORT_PLC | PORT_WRC;
+ mask = PORT_CSC | PORT_PEC | PORT_OCC | PORT_PLC | PORT_WRC | PORT_CEC;
spin_lock_irqsave(&xhci->lock, flags);
/* For each port, did anything change? If so, set that bit in buf. */
#define PCI_DEVICE_ID_INTEL_LYNXPOINT_XHCI 0x8c31
#define PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_XHCI 0x9c31
+#define PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI 0x22b5
+#define PCI_DEVICE_ID_INTEL_SUNRISEPOINT_H_XHCI 0xa12f
+#define PCI_DEVICE_ID_INTEL_SUNRISEPOINT_LP_XHCI 0x9d2f
static const char hcd_name[] = "xhci_hcd";
if (pdev->vendor == PCI_VENDOR_ID_INTEL) {
xhci->quirks |= XHCI_LPM_SUPPORT;
xhci->quirks |= XHCI_INTEL_HOST;
+ xhci->quirks |= XHCI_AVOID_BEI;
}
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
pdev->device == PCI_DEVICE_ID_INTEL_PANTHERPOINT_XHCI) {
* PPT chipsets.
*/
xhci->quirks |= XHCI_SPURIOUS_REBOOT;
- xhci->quirks |= XHCI_AVOID_BEI;
}
if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
pdev->device == PCI_DEVICE_ID_INTEL_LYNXPOINT_LP_XHCI) {
xhci->quirks |= XHCI_SPURIOUS_REBOOT;
}
+ if (pdev->vendor == PCI_VENDOR_ID_INTEL &&
+ (pdev->device == PCI_DEVICE_ID_INTEL_SUNRISEPOINT_LP_XHCI ||
+ pdev->device == PCI_DEVICE_ID_INTEL_SUNRISEPOINT_H_XHCI ||
+ pdev->device == PCI_DEVICE_ID_INTEL_CHERRYVIEW_XHCI)) {
+ xhci->quirks |= XHCI_PME_STUCK_QUIRK;
+ }
if (pdev->vendor == PCI_VENDOR_ID_ETRON &&
pdev->device == PCI_DEVICE_ID_EJ168) {
xhci->quirks |= XHCI_RESET_ON_RESUME;
"QUIRK: Resetting on resume");
}
+/*
+ * Make sure PME works on some Intel xHCI controllers by writing 1 to clear
+ * the Internal PME flag bit in vendor specific PMCTRL register at offset 0x80a4
+ */
+static void xhci_pme_quirk(struct xhci_hcd *xhci)
+{
+ u32 val;
+ void __iomem *reg;
+
+ reg = (void __iomem *) xhci->cap_regs + 0x80a4;
+ val = readl(reg);
+ writel(val | BIT(28), reg);
+ readl(reg);
+}
+
/* called during probe() after chip reset completes */
static int xhci_pci_setup(struct usb_hcd *hcd)
{
if (xhci->quirks & XHCI_COMP_MODE_QUIRK)
pdev->no_d3cold = true;
+ if (xhci->quirks & XHCI_PME_STUCK_QUIRK)
+ xhci_pme_quirk(xhci);
+
return xhci_suspend(xhci, do_wakeup);
}
if (pdev->vendor == PCI_VENDOR_ID_INTEL)
usb_enable_intel_xhci_ports(pdev);
+ if (xhci->quirks & XHCI_PME_STUCK_QUIRK)
+ xhci_pme_quirk(xhci);
+
retval = xhci_resume(xhci, hibernated);
return retval;
}
if (irq < 0)
return -ENODEV;
-
- if (of_device_is_compatible(pdev->dev.of_node,
- "marvell,armada-375-xhci") ||
- of_device_is_compatible(pdev->dev.of_node,
- "marvell,armada-380-xhci")) {
- ret = xhci_mvebu_mbus_init_quirk(pdev);
- if (ret)
- return ret;
- }
-
/* Initialize dma_mask and coherent_dma_mask to 32-bits */
ret = dma_set_coherent_mask(&pdev->dev, DMA_BIT_MASK(32));
if (ret)
goto put_hcd;
}
+ if (of_device_is_compatible(pdev->dev.of_node,
+ "marvell,armada-375-xhci") ||
+ of_device_is_compatible(pdev->dev.of_node,
+ "marvell,armada-380-xhci")) {
+ ret = xhci_mvebu_mbus_init_quirk(pdev);
+ if (ret)
+ goto disable_clk;
+ }
+
ret = usb_add_hcd(hcd, irq, IRQF_SHARED);
if (ret)
goto disable_clk;
if (event_trb != ep_ring->dequeue) {
/* The event was for the status stage */
if (event_trb == td->last_trb) {
- if (td->urb->actual_length != 0) {
+ if (td->urb_length_set) {
/* Don't overwrite a previously set error code
*/
if ((*status == -EINPROGRESS || *status == 0) &&
td->urb->transfer_buffer_length;
}
} else {
- /* Maybe the event was for the data stage? */
+ /*
+ * Maybe the event was for the data stage? If so, update
+ * already the actual_length of the URB and flag it as
+ * set, so that it is not overwritten in the event for
+ * the last TRB.
+ */
+ td->urb_length_set = true;
td->urb->actual_length =
td->urb->transfer_buffer_length -
EVENT_TRB_LEN(le32_to_cpu(event->transfer_len));
+
/*
* xHCI host controller driver
*
#define HCS_IST(p) (((p) >> 0) & 0xf)
/* bits 4:7, max number of Event Ring segments */
#define HCS_ERST_MAX(p) (((p) >> 4) & 0xf)
+/* bits 21:25 Hi 5 bits of Scratchpad buffers SW must allocate for the HW */
/* bit 26 Scratchpad restore - for save/restore HW state - not used yet */
-/* bits 27:31 number of Scratchpad buffers SW must allocate for the HW */
-#define HCS_MAX_SCRATCHPAD(p) (((p) >> 27) & 0x1f)
+/* bits 27:31 Lo 5 bits of Scratchpad buffers SW must allocate for the HW */
+#define HCS_MAX_SCRATCHPAD(p) ((((p) >> 16) & 0x3e0) | (((p) >> 27) & 0x1f))
/* HCSPARAMS3 - hcs_params3 - bitmasks */
/* bits 0:7, Max U1 to U0 latency for the roothub ports */
struct xhci_segment *start_seg;
union xhci_trb *first_trb;
union xhci_trb *last_trb;
+ /* actual_length of the URB has already been set */
+ bool urb_length_set;
};
/* xHCI command default timeout value */
#define XHCI_SPURIOUS_WAKEUP (1 << 18)
/* For controllers with a broken beyond repair streams implementation */
#define XHCI_BROKEN_STREAMS (1 << 19)
+#define XHCI_PME_STUCK_QUIRK (1 << 20)
unsigned int num_active_eps;
unsigned int limit_active_eps;
/* There are two roothubs to keep track of bus suspend info for */
}
if (IS_ENABLED(CONFIG_USB_ISP1761_UDC) && !udc_disabled) {
- ret = isp1760_udc_register(isp, irq, irqflags | IRQF_SHARED |
- IRQF_DISABLED);
+ ret = isp1760_udc_register(isp, irq, irqflags);
if (ret < 0) {
isp1760_hcd_unregister(&isp->hcd);
return ret;
for (slot = 0; slot < 32; slot++)
if (priv->atl_slots[slot].qh && time_after(jiffies,
priv->atl_slots[slot].timestamp +
- SLOT_TIMEOUT * HZ / 1000)) {
+ msecs_to_jiffies(SLOT_TIMEOUT))) {
ptd_read(hcd->regs, ATL_PTD_OFFSET, slot, &ptd);
if (!FROM_DW0_VALID(ptd.dw0) &&
!FROM_DW3_ACTIVE(ptd.dw3))
spin_unlock_irqrestore(&priv->lock, spinflags);
- errata2_timer.expires = jiffies + SLOT_CHECK_PERIOD * HZ / 1000;
+ errata2_timer.expires = jiffies + msecs_to_jiffies(SLOT_CHECK_PERIOD);
add_timer(&errata2_timer);
}
return retval;
setup_timer(&errata2_timer, errata2_function, (unsigned long)hcd);
- errata2_timer.expires = jiffies + SLOT_CHECK_PERIOD * HZ / 1000;
+ errata2_timer.expires = jiffies + msecs_to_jiffies(SLOT_CHECK_PERIOD);
add_timer(&errata2_timer);
chipid = reg_read32(hcd->regs, HC_CHIP_ID_REG);
struct usb_gadget_driver *driver)
{
struct isp1760_udc *udc = gadget_to_udc(gadget);
+ unsigned long flags;
/* The hardware doesn't support low speed. */
if (driver->max_speed < USB_SPEED_FULL) {
return -EINVAL;
}
- spin_lock(&udc->lock);
+ spin_lock_irqsave(&udc->lock, flags);
if (udc->driver) {
dev_err(udc->isp->dev, "UDC already has a gadget driver\n");
- spin_unlock(&udc->lock);
+ spin_unlock_irqrestore(&udc->lock, flags);
return -EBUSY;
}
udc->driver = driver;
- spin_unlock(&udc->lock);
+ spin_unlock_irqrestore(&udc->lock, flags);
dev_dbg(udc->isp->dev, "starting UDC with driver %s\n",
driver->function);
static int isp1760_udc_stop(struct usb_gadget *gadget)
{
struct isp1760_udc *udc = gadget_to_udc(gadget);
+ unsigned long flags;
dev_dbg(udc->isp->dev, "%s\n", __func__);
isp1760_udc_write(udc, DC_MODE, 0);
- spin_lock(&udc->lock);
+ spin_lock_irqsave(&udc->lock, flags);
udc->driver = NULL;
- spin_unlock(&udc->lock);
+ spin_unlock_irqrestore(&udc->lock, flags);
return 0;
}
return -ENODEV;
}
- if (chipid != 0x00011582) {
+ if (chipid != 0x00011582 && chipid != 0x00158210) {
dev_err(udc->isp->dev, "udc: invalid chip ID 0x%08x\n", chipid);
return -ENODEV;
}
sprintf(udc->irqname, "%s (udc)", devname);
- ret = request_irq(irq, isp1760_udc_irq, IRQF_SHARED | IRQF_DISABLED |
- irqflags, udc->irqname, udc);
+ ret = request_irq(irq, isp1760_udc_irq, IRQF_SHARED | irqflags,
+ udc->irqname, udc);
if (ret < 0)
goto error;
config USB_MUSB_OMAP2PLUS
tristate "OMAP2430 and onwards"
- depends on ARCH_OMAP2PLUS && USB && OMAP_CONTROL_PHY
+ depends on ARCH_OMAP2PLUS && USB
+ depends on OMAP_CONTROL_PHY || !OMAP_CONTROL_PHY
select GENERIC_PHY
config USB_MUSB_AM35X
goto fail0;
}
- pm_runtime_use_autosuspend(musb->controller);
- pm_runtime_set_autosuspend_delay(musb->controller, 200);
- pm_runtime_enable(musb->controller);
-
spin_lock_init(&musb->lock);
musb->board_set_power = plat->set_power;
musb->min_power = plat->min_power;
musb_readl = musb_default_readl;
musb_writel = musb_default_writel;
+ /* We need musb_read/write functions initialized for PM */
+ pm_runtime_use_autosuspend(musb->controller);
+ pm_runtime_set_autosuspend_delay(musb->controller, 200);
+ pm_runtime_irq_safe(musb->controller);
+ pm_runtime_enable(musb->controller);
+
/* The musb_platform_init() call:
* - adjusts musb->mregs
* - sets the musb->isr
if (IS_ERR(musb->xceiv))
return PTR_ERR(musb->xceiv);
+ musb->phy = devm_phy_get(dev->parent, "usb2-phy");
+
/* Returns zero if e.g. not clocked */
rev = dsps_readl(reg_base, wrp->revision);
if (!rev)
return -ENODEV;
usb_phy_init(musb->xceiv);
+ if (IS_ERR(musb->phy)) {
+ musb->phy = NULL;
+ } else {
+ ret = phy_init(musb->phy);
+ if (ret < 0)
+ return ret;
+ ret = phy_power_on(musb->phy);
+ if (ret) {
+ phy_exit(musb->phy);
+ return ret;
+ }
+ }
+
setup_timer(&glue->timer, otg_timer, (unsigned long) musb);
/* Reset the musb */
del_timer_sync(&glue->timer);
usb_phy_shutdown(musb->xceiv);
+ phy_power_off(musb->phy);
+ phy_exit(musb->phy);
debugfs_remove_recursive(glue->dbgfs_root);
return 0;
struct device *dev = musb->controller;
struct dsps_glue *glue = dev_get_drvdata(dev->parent);
const struct dsps_musb_wrapper *wrp = glue->wrp;
- int session_restart = 0;
+ int session_restart = 0, error;
if (glue->sw_babble_enabled)
session_restart = sw_babble_control(musb);
dsps_writel(musb->ctrl_base, wrp->control, (1 << wrp->reset));
usleep_range(100, 200);
usb_phy_shutdown(musb->xceiv);
+ error = phy_power_off(musb->phy);
+ if (error)
+ dev_err(dev, "phy shutdown failed: %i\n", error);
usleep_range(100, 200);
usb_phy_init(musb->xceiv);
+ error = phy_power_on(musb->phy);
+ if (error)
+ dev_err(dev, "phy powerup failed: %i\n", error);
session_restart = 1;
}
struct musb_hdrc_config *config;
struct platform_device *musb;
struct device_node *dn = parent->dev.of_node;
- int ret;
+ int ret, val;
memset(resources, 0, sizeof(resources));
res = platform_get_resource_byname(parent, IORESOURCE_MEM, "mc");
pdata.mode = get_musb_port_mode(dev);
/* DT keeps this entry in mA, musb expects it as per USB spec */
pdata.power = get_int_prop(dn, "mentor,power") / 2;
- config->multipoint = of_property_read_bool(dn, "mentor,multipoint");
+
+ ret = of_property_read_u32(dn, "mentor,multipoint", &val);
+ if (!ret && val)
+ config->multipoint = true;
ret = platform_device_add_data(musb, &pdata, sizeof(pdata));
if (ret) {
.description = "musb-hcd",
.product_desc = "MUSB HDRC host driver",
.hcd_priv_size = sizeof(struct musb *),
- .flags = HCD_USB2 | HCD_MEMORY,
+ .flags = HCD_USB2 | HCD_MEMORY | HCD_BH,
/* not using irq handler or reset hooks from usbcore, since
* those must be shared with peripheral code for OTG configs
struct omap2430_glue *glue;
struct device_node *np = pdev->dev.of_node;
struct musb_hdrc_config *config;
- int ret = -ENOMEM;
+ int ret = -ENOMEM, val;
glue = devm_kzalloc(&pdev->dev, sizeof(*glue), GFP_KERNEL);
if (!glue)
of_property_read_u32(np, "num-eps", (u32 *)&config->num_eps);
of_property_read_u32(np, "ram-bits", (u32 *)&config->ram_bits);
of_property_read_u32(np, "power", (u32 *)&pdata->power);
- config->multipoint = of_property_read_bool(np, "multipoint");
+
+ ret = of_property_read_u32(np, "multipoint", &val);
+ if (!ret && val)
+ config->multipoint = true;
pdata->board_data = data;
pdata->config = config;
dev_err(ab->dev, "Failed to set the Vintcore to 1.3V, ret=%d\n",
ret);
- ret = regulator_set_optimum_mode(ab->v_ulpi, 28000);
+ ret = regulator_set_load(ab->v_ulpi, 28000);
if (ret < 0)
dev_err(ab->dev, "Failed to set optimum mode (ret=%d)\n",
ret);
ab->saved_v_ulpi, ret);
}
- ret = regulator_set_optimum_mode(ab->v_ulpi, 0);
+ ret = regulator_set_load(ab->v_ulpi, 0);
if (ret < 0)
dev_err(ab->dev, "Failed to set optimum mode (ret=%d)\n",
ret);
return NULL;
dev = bus_find_device(&platform_bus_type, NULL, node, match);
+ if (!dev)
+ return NULL;
+
ctrl_usb = dev_get_drvdata(dev);
if (!ctrl_usb)
return NULL;
int ret = 0;
if (on) {
- ret = regulator_set_optimum_mode(motg->v1p8,
- USB_PHY_1P8_HPM_LOAD);
+ ret = regulator_set_load(motg->v1p8, USB_PHY_1P8_HPM_LOAD);
if (ret < 0) {
pr_err("Could not set HPM for v1p8\n");
return ret;
}
- ret = regulator_set_optimum_mode(motg->v3p3,
- USB_PHY_3P3_HPM_LOAD);
+ ret = regulator_set_load(motg->v3p3, USB_PHY_3P3_HPM_LOAD);
if (ret < 0) {
pr_err("Could not set HPM for v3p3\n");
- regulator_set_optimum_mode(motg->v1p8,
- USB_PHY_1P8_LPM_LOAD);
+ regulator_set_load(motg->v1p8, USB_PHY_1P8_LPM_LOAD);
return ret;
}
} else {
- ret = regulator_set_optimum_mode(motg->v1p8,
- USB_PHY_1P8_LPM_LOAD);
+ ret = regulator_set_load(motg->v1p8, USB_PHY_1P8_LPM_LOAD);
if (ret < 0)
pr_err("Could not set LPM for v1p8\n");
- ret = regulator_set_optimum_mode(motg->v3p3,
- USB_PHY_3P3_LPM_LOAD);
+ ret = regulator_set_load(motg->v3p3, USB_PHY_3P3_LPM_LOAD);
if (ret < 0)
pr_err("Could not set LPM for v3p3\n");
}
tristate 'Renesas USBHS controller'
depends on USB_GADGET
depends on ARCH_SHMOBILE || SUPERH || COMPILE_TEST
+ depends on EXTCON || !EXTCON # if EXTCON=m, USBHS cannot be built-in
default n
help
Renesas USBHS is a discrete USB host and peripheral controller chip
return 0;
}
-static ssize_t port_number_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct usb_serial_port *port = to_usb_serial_port(dev);
-
- return sprintf(buf, "%d\n", port->port_number);
-}
-static DEVICE_ATTR_RO(port_number);
-
static int usb_serial_device_probe(struct device *dev)
{
struct usb_serial_driver *driver;
struct usb_serial_port *port;
+ struct device *tty_dev;
int retval = 0;
int minor;
port = to_usb_serial_port(dev);
- if (!port) {
- retval = -ENODEV;
- goto exit;
- }
+ if (!port)
+ return -ENODEV;
/* make sure suspend/resume doesn't race against port_probe */
retval = usb_autopm_get_interface(port->serial->interface);
if (retval)
- goto exit;
+ return retval;
driver = port->serial->type;
if (driver->port_probe) {
retval = driver->port_probe(port);
if (retval)
- goto exit_with_autopm;
+ goto err_autopm_put;
}
- retval = device_create_file(dev, &dev_attr_port_number);
- if (retval) {
- if (driver->port_remove)
- retval = driver->port_remove(port);
- goto exit_with_autopm;
+ minor = port->minor;
+ tty_dev = tty_register_device(usb_serial_tty_driver, minor, dev);
+ if (IS_ERR(tty_dev)) {
+ retval = PTR_ERR(tty_dev);
+ goto err_port_remove;
}
- minor = port->minor;
- tty_register_device(usb_serial_tty_driver, minor, dev);
+ usb_autopm_put_interface(port->serial->interface);
+
dev_info(&port->serial->dev->dev,
"%s converter now attached to ttyUSB%d\n",
driver->description, minor);
-exit_with_autopm:
+ return 0;
+
+err_port_remove:
+ if (driver->port_remove)
+ driver->port_remove(port);
+err_autopm_put:
usb_autopm_put_interface(port->serial->interface);
-exit:
+
return retval;
}
minor = port->minor;
tty_unregister_device(usb_serial_tty_driver, minor);
- device_remove_file(&port->dev, &dev_attr_port_number);
-
driver = port->serial->type;
if (driver->port_remove)
retval = driver->port_remove(port);
u8 line_status; /* active status of modem control inputs */
};
+static void ch341_set_termios(struct tty_struct *tty,
+ struct usb_serial_port *port,
+ struct ktermios *old_termios);
+
static int ch341_control_out(struct usb_device *dev, u8 request,
u16 value, u16 index)
{
struct ch341_private *priv = usb_get_serial_port_data(port);
int r;
- priv->baud_rate = DEFAULT_BAUD_RATE;
-
r = ch341_configure(serial->dev, priv);
if (r)
goto out;
- r = ch341_set_handshake(serial->dev, priv->line_control);
- if (r)
- goto out;
-
- r = ch341_set_baudrate(serial->dev, priv);
- if (r)
- goto out;
+ if (tty)
+ ch341_set_termios(tty, port, NULL);
dev_dbg(&port->dev, "%s - submitting interrupt urb\n", __func__);
r = usb_submit_urb(port->interrupt_in_urb, GFP_KERNEL);
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
#include <linux/kernel.h>
+#include <linux/module.h>
#include <linux/slab.h>
#include <linux/tty.h>
#include <linux/console.h>
init_ldsem(&tty->ldisc_sem);
INIT_LIST_HEAD(&tty->tty_files);
kref_get(&tty->driver->kref);
+ __module_get(tty->driver->owner);
tty->ops = &usb_console_fake_tty_ops;
if (tty_init_termios(tty)) {
retval = -ENOMEM;
{ USB_DEVICE(0x166A, 0x0305) }, /* Clipsal C-5000CT2 C-Bus Spectrum Colour Touchscreen */
{ USB_DEVICE(0x166A, 0x0401) }, /* Clipsal L51xx C-Bus Architectural Dimmer */
{ USB_DEVICE(0x166A, 0x0101) }, /* Clipsal 5560884 C-Bus Multi-room Audio Matrix Switcher */
+ { USB_DEVICE(0x16C0, 0x09B0) }, /* Lunatico Seletek */
+ { USB_DEVICE(0x16C0, 0x09B1) }, /* Lunatico Seletek */
{ USB_DEVICE(0x16D6, 0x0001) }, /* Jablotron serial interface */
{ USB_DEVICE(0x16DC, 0x0010) }, /* W-IE-NE-R Plein & Baus GmbH PL512 Power Supply */
{ USB_DEVICE(0x16DC, 0x0011) }, /* W-IE-NE-R Plein & Baus GmbH RCM Remote Control for MARATON Power Supply */
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
{ USB_DEVICE(FTDI_VID, FTDI_NT_ORIONLXM_PID),
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
+ { USB_DEVICE(FTDI_VID, FTDI_SYNAPSE_SS200_PID) },
/*
* ELV devices:
*/
{ USB_DEVICE(FTDI_VID, FTDI_ELSTER_UNICOM_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_PROPOX_JTAGCABLEII_PID) },
{ USB_DEVICE(FTDI_VID, FTDI_PROPOX_ISPCABLEIII_PID) },
+ { USB_DEVICE(FTDI_VID, CYBER_CORTEX_AV_PID),
+ .driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
{ USB_DEVICE(OLIMEX_VID, OLIMEX_ARM_USB_OCD_PID),
.driver_info = (kernel_ulong_t)&ftdi_jtag_quirk },
{ USB_DEVICE(OLIMEX_VID, OLIMEX_ARM_USB_OCD_H_PID),
{ USB_DEVICE_INTERFACE_NUMBER(INFINEON_VID, INFINEON_TRIBOARD_PID, 1) },
/* GE Healthcare devices */
{ USB_DEVICE(GE_HEALTHCARE_VID, GE_HEALTHCARE_NEMO_TRACKER_PID) },
+ /* Active Research (Actisense) devices */
+ { USB_DEVICE(FTDI_VID, ACTISENSE_NDC_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_USG_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_NGT_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_NGW_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_D9AC_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_D9AD_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_D9AE_PID) },
+ { USB_DEVICE(FTDI_VID, ACTISENSE_D9AF_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEAGAUGE_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASWITCH_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_NMEA2000_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_ETHERNET_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_WIFI_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_DISPLAY_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_LITE_PID) },
+ { USB_DEVICE(FTDI_VID, CHETCO_SEASMART_ANALOG_PID) },
{ } /* Terminating entry */
};
{
struct usb_device *udev = serial->dev;
- if ((udev->manufacturer && !strcmp(udev->manufacturer, "CALAO Systems")) ||
- (udev->product && !strcmp(udev->product, "BeagleBone/XDS100V2")))
+ if (udev->manufacturer && !strcmp(udev->manufacturer, "CALAO Systems"))
+ return ftdi_jtag_probe(serial);
+
+ if (udev->product &&
+ (!strcmp(udev->product, "BeagleBone/XDS100V2") ||
+ !strcmp(udev->product, "SNAP Connect E10")))
return ftdi_jtag_probe(serial);
return 0;
#define FTDI_LUMEL_PD12_PID 0x6002
+/* Cyber Cortex AV by Fabulous Silicon (http://fabuloussilicon.com) */
+#define CYBER_CORTEX_AV_PID 0x8698
+
/*
* Marvell OpenRD Base, Client
* http://www.open-rd.org
*/
#define FTDI_NT_ORIONLXM_PID 0x7c90 /* OrionLXm Substation Automation Platform */
+/*
+ * Synapse Wireless product ids (FTDI_VID)
+ * http://www.synapse-wireless.com
+ */
+#define FTDI_SYNAPSE_SS200_PID 0x9090 /* SS200 - SNAP Stick 200 */
+
/********************************/
/** third-party VID/PID combos **/
*/
#define GE_HEALTHCARE_VID 0x1901
#define GE_HEALTHCARE_NEMO_TRACKER_PID 0x0015
+
+/*
+ * Active Research (Actisense) devices
+ */
+#define ACTISENSE_NDC_PID 0xD9A8 /* NDC USB Serial Adapter */
+#define ACTISENSE_USG_PID 0xD9A9 /* USG USB Serial Adapter */
+#define ACTISENSE_NGT_PID 0xD9AA /* NGT NMEA2000 Interface */
+#define ACTISENSE_NGW_PID 0xD9AB /* NGW NMEA2000 Gateway */
+#define ACTISENSE_D9AC_PID 0xD9AC /* Actisense Reserved */
+#define ACTISENSE_D9AD_PID 0xD9AD /* Actisense Reserved */
+#define ACTISENSE_D9AE_PID 0xD9AE /* Actisense Reserved */
+#define ACTISENSE_D9AF_PID 0xD9AF /* Actisense Reserved */
+#define CHETCO_SEAGAUGE_PID 0xA548 /* SeaGauge USB Adapter */
+#define CHETCO_SEASWITCH_PID 0xA549 /* SeaSwitch USB Adapter */
+#define CHETCO_SEASMART_NMEA2000_PID 0xA54A /* SeaSmart NMEA2000 Gateway */
+#define CHETCO_SEASMART_ETHERNET_PID 0xA54B /* SeaSmart Ethernet Gateway */
+#define CHETCO_SEASMART_WIFI_PID 0xA5AC /* SeaSmart Wifi Gateway */
+#define CHETCO_SEASMART_DISPLAY_PID 0xA5AD /* SeaSmart NMEA2000 Display */
+#define CHETCO_SEASMART_LITE_PID 0xA5AE /* SeaSmart Lite USB Adapter */
+#define CHETCO_SEASMART_ANALOG_PID 0xA5AF /* SeaSmart Analog Adapter */
* character or at least one jiffy.
*/
period = max_t(unsigned long, (10 * HZ / bps), 1);
- period = min_t(unsigned long, period, timeout);
+ if (timeout)
+ period = min_t(unsigned long, period, timeout);
dev_dbg(&port->dev, "%s - timeout = %u ms, period = %u ms\n",
__func__, jiffies_to_msecs(timeout),
schedule_timeout_interruptible(period);
if (signal_pending(current))
break;
- if (time_after(jiffies, expire))
+ if (timeout && time_after(jiffies, expire))
break;
}
}
/* For Xircom PGSDB9 and older Entrega version of the same device */
#define XIRCOM_VENDOR_ID 0x085a
#define XIRCOM_FAKE_ID 0x8027
+#define XIRCOM_FAKE_ID_2 0x8025 /* "PGMFHUB" serial */
#define ENTREGA_VENDOR_ID 0x1645
#define ENTREGA_FAKE_ID 0x8093
#endif
#ifdef XIRCOM
{ USB_DEVICE(XIRCOM_VENDOR_ID, XIRCOM_FAKE_ID) },
+ { USB_DEVICE(XIRCOM_VENDOR_ID, XIRCOM_FAKE_ID_2) },
{ USB_DEVICE(ENTREGA_VENDOR_ID, ENTREGA_FAKE_ID) },
#endif
{ USB_DEVICE(KEYSPAN_VENDOR_ID, KEYSPAN_PDA_ID) },
#ifdef XIRCOM
static const struct usb_device_id id_table_fake_xircom[] = {
{ USB_DEVICE(XIRCOM_VENDOR_ID, XIRCOM_FAKE_ID) },
+ { USB_DEVICE(XIRCOM_VENDOR_ID, XIRCOM_FAKE_ID_2) },
{ USB_DEVICE(ENTREGA_VENDOR_ID, ENTREGA_FAKE_ID) },
{ }
};
}
/* Initial port termios */
- mxuport_set_termios(tty, port, NULL);
+ if (tty)
+ mxuport_set_termios(tty, port, NULL);
/*
* TODO: use RQ_VENDOR_GET_MSR, once we know what it
#define UART_OVERRUN_ERROR 0x40
#define UART_CTS 0x80
+static void pl2303_set_break(struct usb_serial_port *port, bool enable);
enum pl2303_type {
TYPE_01, /* Type 0 and 1 (difference unknown) */
{
usb_serial_generic_close(port);
usb_kill_urb(port->interrupt_in_urb);
+ pl2303_set_break(port, false);
}
static int pl2303_open(struct tty_struct *tty, struct usb_serial_port *port)
return -ENOIOCTLCMD;
}
-static void pl2303_break_ctl(struct tty_struct *tty, int break_state)
+static void pl2303_set_break(struct usb_serial_port *port, bool enable)
{
- struct usb_serial_port *port = tty->driver_data;
struct usb_serial *serial = port->serial;
u16 state;
int result;
- if (break_state == 0)
- state = BREAK_OFF;
- else
+ if (enable)
state = BREAK_ON;
+ else
+ state = BREAK_OFF;
dev_dbg(&port->dev, "%s - turning break %s\n", __func__,
state == BREAK_OFF ? "off" : "on");
dev_err(&port->dev, "error sending break = %d\n", result);
}
+static void pl2303_break_ctl(struct tty_struct *tty, int state)
+{
+ struct usb_serial_port *port = tty->driver_data;
+
+ pl2303_set_break(port, state);
+}
+
static void pl2303_update_line_status(struct usb_serial_port *port,
unsigned char *data,
unsigned int actual_length)
drv->dtr_rts(p, on);
}
+static ssize_t port_number_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct usb_serial_port *port = to_usb_serial_port(dev);
+
+ return sprintf(buf, "%u\n", port->port_number);
+}
+static DEVICE_ATTR_RO(port_number);
+
+static struct attribute *usb_serial_port_attrs[] = {
+ &dev_attr_port_number.attr,
+ NULL
+};
+ATTRIBUTE_GROUPS(usb_serial_port);
+
static const struct tty_port_operations serial_port_ops = {
.carrier_raised = serial_port_carrier_raised,
.dtr_rts = serial_port_dtr_rts,
port->dev.driver = NULL;
port->dev.bus = &usb_serial_bus_type;
port->dev.release = &usb_serial_port_release;
+ port->dev.groups = usb_serial_port_groups;
device_initialize(&port->dev);
}
port = serial->port[i];
if (kfifo_alloc(&port->write_fifo, PAGE_SIZE, GFP_KERNEL))
goto probe_error;
- buffer_size = max_t(int, serial->type->bulk_out_size,
- usb_endpoint_maxp(endpoint));
+ buffer_size = serial->type->bulk_out_size;
+ if (!buffer_size)
+ buffer_size = usb_endpoint_maxp(endpoint);
port->bulk_out_size = buffer_size;
port->bulk_out_endpointAddress = endpoint->bEndpointAddress;
USB_SC_DEVICE, USB_PR_DEVICE, NULL,
US_FL_NO_ATA_1X),
+/* Reported-by: Benjamin Tissoires <benjamin.tissoires@redhat.com> */
+UNUSUAL_DEV(0x13fd, 0x3940, 0x0000, 0x9999,
+ "Initio Corporation",
+ "",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_NO_ATA_1X),
+
+/* Reported-by: Tom Arild Naess <tanaess@gmail.com> */
+UNUSUAL_DEV(0x152d, 0x0539, 0x0000, 0x9999,
+ "JMicron",
+ "JMS539",
+ USB_SC_DEVICE, USB_PR_DEVICE, NULL,
+ US_FL_NO_REPORT_OPCODES),
+
/* Reported-by: Claudio Bizzarri <claudio.bizzarri@gmail.com> */
UNUSUAL_DEV(0x152d, 0x0567, 0x0000, 0x9999,
"JMicron",
!(us->fflags & US_FL_SCM_MULT_TARG)) {
mutex_lock(&us->dev_mutex);
us->max_lun = usb_stor_Bulk_max_lun(us);
+ /*
+ * Allow proper scanning of devices that present more than 8 LUNs
+ * While not affecting other devices that may need the previous behavior
+ */
+ if (us->max_lun >= 8)
+ us_to_host(us)->max_lun = us->max_lun+1;
mutex_unlock(&us->dev_mutex);
}
scsi_scan_host(us_to_host(us));
func = vfio_pci_set_err_trigger;
break;
}
+ break;
case VFIO_PCI_REQ_IRQ_INDEX:
switch (flags & VFIO_IRQ_SET_ACTION_TYPE_MASK) {
case VFIO_IRQ_SET_ACTION_TRIGGER:
func = vfio_pci_set_req_trigger;
break;
}
+ break;
}
if (!func)
* TODO: support TSO.
*/
iov_iter_advance(&msg.msg_iter, vhost_hlen);
- } else {
- /* It'll come from socket; we'll need to patch
- * ->num_buffers over if VIRTIO_NET_F_MRG_RXBUF
- */
- iov_iter_advance(&fixup, sizeof(hdr));
}
err = sock->ops->recvmsg(NULL, sock, &msg,
sock_len, MSG_DONTWAIT | MSG_TRUNC);
continue;
}
/* Supply virtio_net_hdr if VHOST_NET_F_VIRTIO_NET_HDR */
- if (unlikely(vhost_hlen) &&
- copy_to_iter(&hdr, sizeof(hdr), &fixup) != sizeof(hdr)) {
- vq_err(vq, "Unable to write vnet_hdr at addr %p\n",
- vq->iov->iov_base);
- break;
+ if (unlikely(vhost_hlen)) {
+ if (copy_to_iter(&hdr, sizeof(hdr),
+ &fixup) != sizeof(hdr)) {
+ vq_err(vq, "Unable to write vnet_hdr "
+ "at addr %p\n", vq->iov->iov_base);
+ break;
+ }
+ } else {
+ /* Header came from socket; we'll need to patch
+ * ->num_buffers over if VIRTIO_NET_F_MRG_RXBUF
+ */
+ iov_iter_advance(&fixup, sizeof(hdr));
}
/* TODO: Should check and handle checksum. */
num_buffers = cpu_to_vhost16(vq, headcount);
if (likely(mergeable) &&
- copy_to_iter(&num_buffers, 2, &fixup) != 2) {
+ copy_to_iter(&num_buffers, sizeof num_buffers,
+ &fixup) != sizeof num_buffers) {
vq_err(vq, "Failed num_buffers write");
vhost_discard_vq_desc(vq, headcount);
break;
goto out;
}
/*
- * Now register the TCM vhost virtual I_T Nexus as active with the
- * call to __transport_register_session()
+ * Now register the TCM vhost virtual I_T Nexus as active.
*/
- __transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+ transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
tv_nexus->tvn_se_sess, tv_nexus);
tpg->tpg_nexus = tv_nexus;
len = clcdfb_snprintf_mode(NULL, 0, mode);
name = devm_kzalloc(dev, len + 1, GFP_KERNEL);
+ if (!name)
+ return -ENOMEM;
+
clcdfb_snprintf_mode(name, len + 1, mode);
mode->name = name;
int num = 0, i, first = 1;
int ver, rev;
- ver = edid[EDID_STRUCT_VERSION];
- rev = edid[EDID_STRUCT_REVISION];
-
mode = kzalloc(50 * sizeof(struct fb_videomode), GFP_KERNEL);
if (mode == NULL)
return NULL;
return NULL;
}
+ ver = edid[EDID_STRUCT_VERSION];
+ rev = edid[EDID_STRUCT_REVISION];
+
*dbsize = 0;
DPRINTK(" Detailed Timings\n");
#include <video/omapdss.h>
#include "dss.h"
-static struct omap_dss_device *to_dss_device_sysfs(struct device *dev)
+static ssize_t display_name_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = NULL;
-
- for_each_dss_dev(dssdev) {
- if (dssdev->dev == dev) {
- omap_dss_put_device(dssdev);
- return dssdev;
- }
- }
-
- return NULL;
-}
-
-static ssize_t display_name_show(struct device *dev,
- struct device_attribute *attr, char *buf)
-{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
-
return snprintf(buf, PAGE_SIZE, "%s\n",
dssdev->name ?
dssdev->name : "");
}
-static ssize_t display_enabled_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_enabled_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
-
return snprintf(buf, PAGE_SIZE, "%d\n",
omapdss_device_is_enabled(dssdev));
}
-static ssize_t display_enabled_store(struct device *dev,
- struct device_attribute *attr,
+static ssize_t display_enabled_store(struct omap_dss_device *dssdev,
const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int r;
bool enable;
return size;
}
-static ssize_t display_tear_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_tear_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
return snprintf(buf, PAGE_SIZE, "%d\n",
dssdev->driver->get_te ?
dssdev->driver->get_te(dssdev) : 0);
}
-static ssize_t display_tear_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t size)
+static ssize_t display_tear_store(struct omap_dss_device *dssdev,
+ const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int r;
bool te;
return size;
}
-static ssize_t display_timings_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_timings_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
struct omap_video_timings t;
if (!dssdev->driver->get_timings)
t.y_res, t.vfp, t.vbp, t.vsw);
}
-static ssize_t display_timings_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t size)
+static ssize_t display_timings_store(struct omap_dss_device *dssdev,
+ const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
struct omap_video_timings t = dssdev->panel.timings;
int r, found;
return size;
}
-static ssize_t display_rotate_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_rotate_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int rotate;
if (!dssdev->driver->get_rotate)
return -ENOENT;
return snprintf(buf, PAGE_SIZE, "%u\n", rotate);
}
-static ssize_t display_rotate_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t size)
+static ssize_t display_rotate_store(struct omap_dss_device *dssdev,
+ const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int rot, r;
if (!dssdev->driver->set_rotate || !dssdev->driver->get_rotate)
return size;
}
-static ssize_t display_mirror_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_mirror_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int mirror;
if (!dssdev->driver->get_mirror)
return -ENOENT;
return snprintf(buf, PAGE_SIZE, "%u\n", mirror);
}
-static ssize_t display_mirror_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t size)
+static ssize_t display_mirror_store(struct omap_dss_device *dssdev,
+ const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
int r;
bool mirror;
return size;
}
-static ssize_t display_wss_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static ssize_t display_wss_show(struct omap_dss_device *dssdev, char *buf)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
unsigned int wss;
if (!dssdev->driver->get_wss)
return snprintf(buf, PAGE_SIZE, "0x%05x\n", wss);
}
-static ssize_t display_wss_store(struct device *dev,
- struct device_attribute *attr, const char *buf, size_t size)
+static ssize_t display_wss_store(struct omap_dss_device *dssdev,
+ const char *buf, size_t size)
{
- struct omap_dss_device *dssdev = to_dss_device_sysfs(dev);
u32 wss;
int r;
return size;
}
-static DEVICE_ATTR(display_name, S_IRUGO, display_name_show, NULL);
-static DEVICE_ATTR(enabled, S_IRUGO|S_IWUSR,
+struct display_attribute {
+ struct attribute attr;
+ ssize_t (*show)(struct omap_dss_device *, char *);
+ ssize_t (*store)(struct omap_dss_device *, const char *, size_t);
+};
+
+#define DISPLAY_ATTR(_name, _mode, _show, _store) \
+ struct display_attribute display_attr_##_name = \
+ __ATTR(_name, _mode, _show, _store)
+
+static DISPLAY_ATTR(name, S_IRUGO, display_name_show, NULL);
+static DISPLAY_ATTR(display_name, S_IRUGO, display_name_show, NULL);
+static DISPLAY_ATTR(enabled, S_IRUGO|S_IWUSR,
display_enabled_show, display_enabled_store);
-static DEVICE_ATTR(tear_elim, S_IRUGO|S_IWUSR,
+static DISPLAY_ATTR(tear_elim, S_IRUGO|S_IWUSR,
display_tear_show, display_tear_store);
-static DEVICE_ATTR(timings, S_IRUGO|S_IWUSR,
+static DISPLAY_ATTR(timings, S_IRUGO|S_IWUSR,
display_timings_show, display_timings_store);
-static DEVICE_ATTR(rotate, S_IRUGO|S_IWUSR,
+static DISPLAY_ATTR(rotate, S_IRUGO|S_IWUSR,
display_rotate_show, display_rotate_store);
-static DEVICE_ATTR(mirror, S_IRUGO|S_IWUSR,
+static DISPLAY_ATTR(mirror, S_IRUGO|S_IWUSR,
display_mirror_show, display_mirror_store);
-static DEVICE_ATTR(wss, S_IRUGO|S_IWUSR,
+static DISPLAY_ATTR(wss, S_IRUGO|S_IWUSR,
display_wss_show, display_wss_store);
-static const struct attribute *display_sysfs_attrs[] = {
- &dev_attr_display_name.attr,
- &dev_attr_enabled.attr,
- &dev_attr_tear_elim.attr,
- &dev_attr_timings.attr,
- &dev_attr_rotate.attr,
- &dev_attr_mirror.attr,
- &dev_attr_wss.attr,
+static struct attribute *display_sysfs_attrs[] = {
+ &display_attr_name.attr,
+ &display_attr_display_name.attr,
+ &display_attr_enabled.attr,
+ &display_attr_tear_elim.attr,
+ &display_attr_timings.attr,
+ &display_attr_rotate.attr,
+ &display_attr_mirror.attr,
+ &display_attr_wss.attr,
NULL
};
+static ssize_t display_attr_show(struct kobject *kobj, struct attribute *attr,
+ char *buf)
+{
+ struct omap_dss_device *dssdev;
+ struct display_attribute *display_attr;
+
+ dssdev = container_of(kobj, struct omap_dss_device, kobj);
+ display_attr = container_of(attr, struct display_attribute, attr);
+
+ if (!display_attr->show)
+ return -ENOENT;
+
+ return display_attr->show(dssdev, buf);
+}
+
+static ssize_t display_attr_store(struct kobject *kobj, struct attribute *attr,
+ const char *buf, size_t size)
+{
+ struct omap_dss_device *dssdev;
+ struct display_attribute *display_attr;
+
+ dssdev = container_of(kobj, struct omap_dss_device, kobj);
+ display_attr = container_of(attr, struct display_attribute, attr);
+
+ if (!display_attr->store)
+ return -ENOENT;
+
+ return display_attr->store(dssdev, buf, size);
+}
+
+static const struct sysfs_ops display_sysfs_ops = {
+ .show = display_attr_show,
+ .store = display_attr_store,
+};
+
+static struct kobj_type display_ktype = {
+ .sysfs_ops = &display_sysfs_ops,
+ .default_attrs = display_sysfs_attrs,
+};
+
int display_init_sysfs(struct platform_device *pdev)
{
struct omap_dss_device *dssdev = NULL;
int r;
for_each_dss_dev(dssdev) {
- struct kobject *kobj = &dssdev->dev->kobj;
-
- r = sysfs_create_files(kobj, display_sysfs_attrs);
+ r = kobject_init_and_add(&dssdev->kobj, &display_ktype,
+ &pdev->dev.kobj, dssdev->alias);
if (r) {
DSSERR("failed to create sysfs files\n");
- goto err;
- }
-
- r = sysfs_create_link(&pdev->dev.kobj, kobj, dssdev->alias);
- if (r) {
- sysfs_remove_files(kobj, display_sysfs_attrs);
-
- DSSERR("failed to create sysfs display link\n");
+ omap_dss_put_device(dssdev);
goto err;
}
}
struct omap_dss_device *dssdev = NULL;
for_each_dss_dev(dssdev) {
- sysfs_remove_link(&pdev->dev.kobj, dssdev->alias);
- sysfs_remove_files(&dssdev->dev->kobj,
- display_sysfs_attrs);
+ if (kobject_name(&dssdev->kobj) == NULL)
+ continue;
+
+ kobject_del(&dssdev->kobj);
+ kobject_put(&dssdev->kobj);
+
+ memset(&dssdev->kobj, 0, sizeof(dssdev->kobj));
}
}
#include <linux/module.h>
#include <linux/balloon_compaction.h>
#include <linux/oom.h>
+#include <linux/wait.h>
/*
* Balloon device works in 4K page units. So each page is pointed to by
static int balloon(void *_vballoon)
{
struct virtio_balloon *vb = _vballoon;
+ DEFINE_WAIT_FUNC(wait, woken_wake_function);
set_freezable();
while (!kthread_should_stop()) {
s64 diff;
try_to_freeze();
- wait_event_interruptible(vb->config_change,
- (diff = towards_target(vb)) != 0
- || vb->need_stats_update
- || kthread_should_stop()
- || freezing(current));
+
+ add_wait_queue(&vb->config_change, &wait);
+ for (;;) {
+ if ((diff = towards_target(vb)) != 0 ||
+ vb->need_stats_update ||
+ kthread_should_stop() ||
+ freezing(current))
+ break;
+ wait_woken(&wait, TASK_INTERRUPTIBLE, MAX_SCHEDULE_TIMEOUT);
+ }
+ remove_wait_queue(&vb->config_change, &wait);
+
if (vb->need_stats_update)
stats_handle_request(vb);
if (diff > 0)
if (err < 0)
goto out_oom_notify;
+ virtio_device_ready(vdev);
+
vb->thread = kthread_run(balloon, vb, "vballoon");
if (IS_ERR(vb->thread)) {
err = PTR_ERR(vb->thread);
void *buf, unsigned len)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
- u8 *ptr = buf;
- int i;
+ void __iomem *base = vm_dev->base + VIRTIO_MMIO_CONFIG;
+ u8 b;
+ __le16 w;
+ __le32 l;
- for (i = 0; i < len; i++)
- ptr[i] = readb(vm_dev->base + VIRTIO_MMIO_CONFIG + offset + i);
+ if (vm_dev->version == 1) {
+ u8 *ptr = buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ ptr[i] = readb(base + offset + i);
+ return;
+ }
+
+ switch (len) {
+ case 1:
+ b = readb(base + offset);
+ memcpy(buf, &b, sizeof b);
+ break;
+ case 2:
+ w = cpu_to_le16(readw(base + offset));
+ memcpy(buf, &w, sizeof w);
+ break;
+ case 4:
+ l = cpu_to_le32(readl(base + offset));
+ memcpy(buf, &l, sizeof l);
+ break;
+ case 8:
+ l = cpu_to_le32(readl(base + offset));
+ memcpy(buf, &l, sizeof l);
+ l = cpu_to_le32(ioread32(base + offset + sizeof l));
+ memcpy(buf + sizeof l, &l, sizeof l);
+ break;
+ default:
+ BUG();
+ }
}
static void vm_set(struct virtio_device *vdev, unsigned offset,
const void *buf, unsigned len)
{
struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
- const u8 *ptr = buf;
- int i;
+ void __iomem *base = vm_dev->base + VIRTIO_MMIO_CONFIG;
+ u8 b;
+ __le16 w;
+ __le32 l;
- for (i = 0; i < len; i++)
- writeb(ptr[i], vm_dev->base + VIRTIO_MMIO_CONFIG + offset + i);
+ if (vm_dev->version == 1) {
+ const u8 *ptr = buf;
+ int i;
+
+ for (i = 0; i < len; i++)
+ writeb(ptr[i], base + offset + i);
+
+ return;
+ }
+
+ switch (len) {
+ case 1:
+ memcpy(&b, buf, sizeof b);
+ writeb(b, base + offset);
+ break;
+ case 2:
+ memcpy(&w, buf, sizeof w);
+ writew(le16_to_cpu(w), base + offset);
+ break;
+ case 4:
+ memcpy(&l, buf, sizeof l);
+ writel(le32_to_cpu(l), base + offset);
+ break;
+ case 8:
+ memcpy(&l, buf, sizeof l);
+ writel(le32_to_cpu(l), base + offset);
+ memcpy(&l, buf + sizeof l, sizeof l);
+ writel(le32_to_cpu(l), base + offset + sizeof l);
+ break;
+ default:
+ BUG();
+ }
+}
+
+static u32 vm_generation(struct virtio_device *vdev)
+{
+ struct virtio_mmio_device *vm_dev = to_virtio_mmio_device(vdev);
+
+ if (vm_dev->version == 1)
+ return 0;
+ else
+ return readl(vm_dev->base + VIRTIO_MMIO_CONFIG_GENERATION);
}
static u8 vm_get_status(struct virtio_device *vdev)
static const struct virtio_config_ops virtio_mmio_config_ops = {
.get = vm_get,
.set = vm_set,
+ .generation = vm_generation,
.get_status = vm_get_status,
.set_status = vm_set_status,
.reset = vm_reset,
if ((tmp & AT91_WDT_WDFIEN) && wdt->irq) {
err = request_irq(wdt->irq, wdt_interrupt,
- IRQF_SHARED | IRQF_IRQPOLL,
+ IRQF_SHARED | IRQF_IRQPOLL |
+ IRQF_NO_SUSPEND,
pdev->name, wdt);
if (err)
return err;
#define PDC_WDT_MIN_TIMEOUT 1
#define PDC_WDT_DEF_TIMEOUT 64
-static int heartbeat;
+static int heartbeat = PDC_WDT_DEF_TIMEOUT;
module_param(heartbeat, int, 0);
-MODULE_PARM_DESC(heartbeat, "Watchdog heartbeats in seconds. "
- "(default = " __MODULE_STRING(PDC_WDT_DEF_TIMEOUT) ")");
+MODULE_PARM_DESC(heartbeat, "Watchdog heartbeats in seconds "
+ "(default=" __MODULE_STRING(PDC_WDT_DEF_TIMEOUT) ")");
static bool nowayout = WATCHDOG_NOWAYOUT;
module_param(nowayout, bool, 0);
pdc_wdt->wdt_dev.ops = &pdc_wdt_ops;
pdc_wdt->wdt_dev.max_timeout = 1 << PDC_WDT_CONFIG_DELAY_MASK;
pdc_wdt->wdt_dev.parent = &pdev->dev;
+ watchdog_set_drvdata(&pdc_wdt->wdt_dev, pdc_wdt);
ret = watchdog_init_timeout(&pdc_wdt->wdt_dev, heartbeat, &pdev->dev);
if (ret < 0) {
watchdog_set_nowayout(&pdc_wdt->wdt_dev, nowayout);
platform_set_drvdata(pdev, pdc_wdt);
- watchdog_set_drvdata(&pdc_wdt->wdt_dev, pdc_wdt);
ret = watchdog_register_device(&pdc_wdt->wdt_dev);
if (ret)
u32 reg;
struct mtk_wdt_dev *mtk_wdt = watchdog_get_drvdata(wdt_dev);
void __iomem *wdt_base = mtk_wdt->wdt_base;
- u32 ret;
+ int ret;
ret = mtk_wdt_set_timeout(wdt_dev, wdt_dev->timeout);
if (ret < 0)
In that case step 3 should be omitted.
+config XEN_BALLOON_MEMORY_HOTPLUG_LIMIT
+ int "Hotplugged memory limit (in GiB) for a PV guest"
+ default 512 if X86_64
+ default 4 if X86_32
+ range 0 64 if X86_32
+ depends on XEN_HAVE_PVMMU
+ depends on XEN_BALLOON_MEMORY_HOTPLUG
+ help
+ Maxmium amount of memory (in GiB) that a PV guest can be
+ expanded to when using memory hotplug.
+
+ A PV guest can have more memory than this limit if is
+ started with a larger maximum.
+
+ This value is used to allocate enough space in internal
+ tables needed for physical memory administration.
+
config XEN_SCRUB_PAGES
bool "Scrub pages before returning them to system"
depends on XEN_BALLOON
obj-$(CONFIG_HOTPLUG_CPU) += cpu_hotplug.o
endif
obj-$(CONFIG_X86) += fallback.o
-obj-y += grant-table.o features.o balloon.o manage.o
+obj-y += grant-table.o features.o balloon.o manage.o preempt.o
obj-y += events/
obj-y += xenbus/
balloon_hotplug = round_up(balloon_hotplug, PAGES_PER_SECTION);
nid = memory_add_physaddr_to_nid(hotplug_start_paddr);
+#ifdef CONFIG_XEN_HAVE_PVMMU
+ /*
+ * add_memory() will build page tables for the new memory so
+ * the p2m must contain invalid entries so the correct
+ * non-present PTEs will be written.
+ *
+ * If a failure occurs, the original (identity) p2m entries
+ * are not restored since this region is now known not to
+ * conflict with any devices.
+ */
+ if (!xen_feature(XENFEAT_auto_translated_physmap)) {
+ unsigned long pfn, i;
+
+ pfn = PFN_DOWN(hotplug_start_paddr);
+ for (i = 0; i < balloon_hotplug; i++) {
+ if (!set_phys_to_machine(pfn + i, INVALID_P2M_ENTRY)) {
+ pr_warn("set_phys_to_machine() failed, no memory added\n");
+ return BP_ECANCELED;
+ }
+ }
+ }
+#endif
+
rc = add_memory(nid, hotplug_start_paddr, balloon_hotplug << PAGE_SHIFT);
if (rc) {
pirq_query_unmask(irq);
rc = set_evtchn_to_irq(evtchn, irq);
- if (rc != 0) {
- pr_err("irq%d: Failed to set port to irq mapping (%d)\n",
- irq, rc);
- xen_evtchn_close(evtchn);
- return 0;
- }
+ if (rc)
+ goto err;
+
bind_evtchn_to_cpu(evtchn, 0);
info->evtchn = evtchn;
+ rc = xen_evtchn_port_setup(info);
+ if (rc)
+ goto err;
+
out:
unmask_evtchn(evtchn);
eoi_pirq(irq_get_irq_data(irq));
return 0;
+
+err:
+ pr_err("irq%d: Failed to set port to irq mapping (%d)\n", irq, rc);
+ xen_evtchn_close(evtchn);
+ return 0;
}
static unsigned int startup_pirq(struct irq_data *data)
--- /dev/null
+/*
+ * Preemptible hypercalls
+ *
+ * Copyright (C) 2014 Citrix Systems R&D ltd.
+ *
+ * This source code is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License as
+ * published by the Free Software Foundation; either version 2 of the
+ * License, or (at your option) any later version.
+ */
+
+#include <linux/sched.h>
+#include <xen/xen-ops.h>
+
+#ifndef CONFIG_PREEMPT
+
+/*
+ * Some hypercalls issued by the toolstack can take many 10s of
+ * seconds. Allow tasks running hypercalls via the privcmd driver to
+ * be voluntarily preempted even if full kernel preemption is
+ * disabled.
+ *
+ * Such preemptible hypercalls are bracketed by
+ * xen_preemptible_hcall_begin() and xen_preemptible_hcall_end()
+ * calls.
+ */
+
+DEFINE_PER_CPU(bool, xen_in_preemptible_hcall);
+EXPORT_SYMBOL_GPL(xen_in_preemptible_hcall);
+
+asmlinkage __visible void xen_maybe_preempt_hcall(void)
+{
+ if (unlikely(__this_cpu_read(xen_in_preemptible_hcall)
+ && should_resched())) {
+ /*
+ * Clear flag as we may be rescheduled on a different
+ * cpu.
+ */
+ __this_cpu_write(xen_in_preemptible_hcall, false);
+ _cond_resched();
+ __this_cpu_write(xen_in_preemptible_hcall, true);
+ }
+}
+#endif /* CONFIG_PREEMPT */
if (copy_from_user(&hypercall, udata, sizeof(hypercall)))
return -EFAULT;
+ xen_preemptible_hcall_begin();
ret = privcmd_call(hypercall.op,
hypercall.arg[0], hypercall.arg[1],
hypercall.arg[2], hypercall.arg[3],
hypercall.arg[4]);
+ xen_preemptible_hcall_end();
return ret;
}
#include "conf_space.h"
#include "conf_space_quirks.h"
-static bool permissive;
+bool permissive;
module_param(permissive, bool, 0644);
/* This is where xen_pcibk_read_config_byte, xen_pcibk_read_config_word,
void *data;
};
+extern bool permissive;
+
#define OFFSET(cfg_entry) ((cfg_entry)->base_offset+(cfg_entry)->field->offset)
/* Add fields to a device - the add_fields macro expects to get a pointer to
#include "pciback.h"
#include "conf_space.h"
+struct pci_cmd_info {
+ u16 val;
+};
+
struct pci_bar_info {
u32 val;
u32 len_val;
#define is_enable_cmd(value) ((value)&(PCI_COMMAND_MEMORY|PCI_COMMAND_IO))
#define is_master_cmd(value) ((value)&PCI_COMMAND_MASTER)
-static int command_read(struct pci_dev *dev, int offset, u16 *value, void *data)
+/* Bits guests are allowed to control in permissive mode. */
+#define PCI_COMMAND_GUEST (PCI_COMMAND_MASTER|PCI_COMMAND_SPECIAL| \
+ PCI_COMMAND_INVALIDATE|PCI_COMMAND_VGA_PALETTE| \
+ PCI_COMMAND_WAIT|PCI_COMMAND_FAST_BACK)
+
+static void *command_init(struct pci_dev *dev, int offset)
{
- int i;
- int ret;
-
- ret = xen_pcibk_read_config_word(dev, offset, value, data);
- if (!pci_is_enabled(dev))
- return ret;
-
- for (i = 0; i < PCI_ROM_RESOURCE; i++) {
- if (dev->resource[i].flags & IORESOURCE_IO)
- *value |= PCI_COMMAND_IO;
- if (dev->resource[i].flags & IORESOURCE_MEM)
- *value |= PCI_COMMAND_MEMORY;
+ struct pci_cmd_info *cmd = kmalloc(sizeof(*cmd), GFP_KERNEL);
+ int err;
+
+ if (!cmd)
+ return ERR_PTR(-ENOMEM);
+
+ err = pci_read_config_word(dev, PCI_COMMAND, &cmd->val);
+ if (err) {
+ kfree(cmd);
+ return ERR_PTR(err);
}
+ return cmd;
+}
+
+static int command_read(struct pci_dev *dev, int offset, u16 *value, void *data)
+{
+ int ret = pci_read_config_word(dev, offset, value);
+ const struct pci_cmd_info *cmd = data;
+
+ *value &= PCI_COMMAND_GUEST;
+ *value |= cmd->val & ~PCI_COMMAND_GUEST;
+
return ret;
}
{
struct xen_pcibk_dev_data *dev_data;
int err;
+ u16 val;
+ struct pci_cmd_info *cmd = data;
dev_data = pci_get_drvdata(dev);
if (!pci_is_enabled(dev) && is_enable_cmd(value)) {
}
}
+ cmd->val = value;
+
+ if (!permissive && (!dev_data || !dev_data->permissive))
+ return 0;
+
+ /* Only allow the guest to control certain bits. */
+ err = pci_read_config_word(dev, offset, &val);
+ if (err || val == value)
+ return err;
+
+ value &= PCI_COMMAND_GUEST;
+ value |= val & ~PCI_COMMAND_GUEST;
+
return pci_write_config_word(dev, offset, value);
}
{
.offset = PCI_COMMAND,
.size = 2,
+ .init = command_init,
+ .release = bar_release,
.u.w.read = command_read,
.u.w.write = command_write,
},
static int scsiback_do_cmd_fn(struct vscsibk_info *info)
{
struct vscsiif_back_ring *ring = &info->ring;
- struct vscsiif_request *ring_req;
+ struct vscsiif_request ring_req;
struct vscsibk_pend *pending_req;
RING_IDX rc, rp;
int err, more_to_do;
uint32_t result;
- uint8_t act;
rc = ring->req_cons;
rp = ring->sring->req_prod;
if (!pending_req)
return 1;
- ring_req = RING_GET_REQUEST(ring, rc);
+ ring_req = *RING_GET_REQUEST(ring, rc);
ring->req_cons = ++rc;
- act = ring_req->act;
- err = prepare_pending_reqs(info, ring_req, pending_req);
+ err = prepare_pending_reqs(info, &ring_req, pending_req);
if (err) {
switch (err) {
case -ENODEV:
return 1;
}
- switch (act) {
+ switch (ring_req.act) {
case VSCSIIF_ACT_SCSI_CDB:
- if (scsiback_gnttab_data_map(ring_req, pending_req)) {
+ if (scsiback_gnttab_data_map(&ring_req, pending_req)) {
scsiback_fast_flush_area(pending_req);
scsiback_do_resp_with_sense(NULL,
DRIVER_ERROR << 24, 0, pending_req);
break;
case VSCSIIF_ACT_SCSI_ABORT:
scsiback_device_action(pending_req, TMR_ABORT_TASK,
- ring_req->ref_rqid);
+ ring_req.ref_rqid);
break;
case VSCSIIF_ACT_SCSI_RESET:
scsiback_device_action(pending_req, TMR_LUN_RESET, 0);
name);
goto out;
}
- /*
- * Now register the TCM pvscsi virtual I_T Nexus as active with the
- * call to __transport_register_session()
- */
- __transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
+ /* Now register the TCM pvscsi virtual I_T Nexus as active. */
+ transport_register_session(se_tpg, tv_nexus->tvn_se_sess->se_node_acl,
tv_nexus->tvn_se_sess, tv_nexus);
tpg->tpg_nexus = tv_nexus;
boff = tmp % bsize;
if (boff) {
bh = affs_bread_ino(inode, bidx, 0);
- if (IS_ERR(bh))
- return PTR_ERR(bh);
+ if (IS_ERR(bh)) {
+ written = PTR_ERR(bh);
+ goto err_first_bh;
+ }
tmp = min(bsize - boff, to - from);
BUG_ON(boff + tmp > bsize || tmp > bsize);
memcpy(AFFS_DATA(bh) + boff, data + from, tmp);
bidx++;
} else if (bidx) {
bh = affs_bread_ino(inode, bidx - 1, 0);
- if (IS_ERR(bh))
- return PTR_ERR(bh);
+ if (IS_ERR(bh)) {
+ written = PTR_ERR(bh);
+ goto err_first_bh;
+ }
}
while (from + bsize <= to) {
prev_bh = bh;
bh = affs_getemptyblk_ino(inode, bidx);
if (IS_ERR(bh))
- goto out;
+ goto err_bh;
memcpy(AFFS_DATA(bh), data + from, bsize);
if (buffer_new(bh)) {
AFFS_DATA_HEAD(bh)->ptype = cpu_to_be32(T_DATA);
prev_bh = bh;
bh = affs_bread_ino(inode, bidx, 1);
if (IS_ERR(bh))
- goto out;
+ goto err_bh;
tmp = min(bsize, to - from);
BUG_ON(tmp > bsize);
memcpy(AFFS_DATA(bh), data + from, tmp);
if (tmp > inode->i_size)
inode->i_size = AFFS_I(inode)->mmu_private = tmp;
+err_first_bh:
unlock_page(page);
page_cache_release(page);
return written;
-out:
+err_bh:
bh = prev_bh;
if (!written)
written = PTR_ERR(bh);
return 0;
}
-static void aio_ring_remap(struct file *file, struct vm_area_struct *vma)
+static int aio_ring_remap(struct file *file, struct vm_area_struct *vma)
{
struct mm_struct *mm = vma->vm_mm;
struct kioctx_table *table;
- int i;
+ int i, res = -EINVAL;
spin_lock(&mm->ioctx_lock);
rcu_read_lock();
ctx = table->table[i];
if (ctx && ctx->aio_ring_file == file) {
- ctx->user_id = ctx->mmap_base = vma->vm_start;
+ if (!atomic_read(&ctx->dead)) {
+ ctx->user_id = ctx->mmap_base = vma->vm_start;
+ res = 0;
+ }
break;
}
}
rcu_read_unlock();
spin_unlock(&mm->ioctx_lock);
+ return res;
}
static const struct file_operations aio_ring_fops = {
err_cleanup:
aio_nr_sub(ctx->max_reqs);
err_ctx:
+ atomic_set(&ctx->dead, 1);
+ if (ctx->mmap_size)
+ vm_munmap(ctx->mmap_base, ctx->mmap_size);
aio_free_ring(ctx);
err:
mutex_unlock(&ctx->ring_lock);
{
struct kioctx_table *table;
- if (atomic_xchg(&ctx->dead, 1))
+ spin_lock(&mm->ioctx_lock);
+ if (atomic_xchg(&ctx->dead, 1)) {
+ spin_unlock(&mm->ioctx_lock);
return -EINVAL;
+ }
-
- spin_lock(&mm->ioctx_lock);
table = rcu_dereference_raw(mm->ioctx_table);
WARN_ON(ctx != table->table[ctx->id]);
table->table[ctx->id] = NULL;
parent_nritems = btrfs_header_nritems(parent);
blocksize = root->nodesize;
- end_slot = parent_nritems;
+ end_slot = parent_nritems - 1;
- if (parent_nritems == 1)
+ if (parent_nritems <= 1)
return 0;
btrfs_set_lock_blocking(parent);
- for (i = start_slot; i < end_slot; i++) {
+ for (i = start_slot; i <= end_slot; i++) {
int close = 1;
btrfs_node_key(parent, &disk_key, i);
other = btrfs_node_blockptr(parent, i - 1);
close = close_blocks(blocknr, other, blocksize);
}
- if (!close && i < end_slot - 2) {
+ if (!close && i < end_slot) {
other = btrfs_node_blockptr(parent, i + 1);
close = close_blocks(blocknr, other, blocksize);
}
int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans,
struct btrfs_root *root);
+int btrfs_setup_space_cache(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root);
int btrfs_extent_readonly(struct btrfs_root *root, u64 bytenr);
int btrfs_free_block_groups(struct btrfs_fs_info *info);
int btrfs_read_block_groups(struct btrfs_root *root);
loff_t actual_len, u64 *alloc_hint);
int btrfs_inode_check_errors(struct inode *inode);
extern const struct dentry_operations btrfs_dentry_operations;
+#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
+void btrfs_test_inode_set_ops(struct inode *inode);
+#endif
/* ioctl.c */
long btrfs_ioctl(struct file *file, unsigned int cmd, unsigned long arg);
}
if (btrfs_super_sys_array_size(sb) < sizeof(struct btrfs_disk_key)
+ sizeof(struct btrfs_chunk)) {
- printk(KERN_ERR "BTRFS: system chunk array too small %u < %lu\n",
+ printk(KERN_ERR "BTRFS: system chunk array too small %u < %zu\n",
btrfs_super_sys_array_size(sb),
sizeof(struct btrfs_disk_key)
+ sizeof(struct btrfs_chunk));
return 0;
}
+ if (trans->aborted)
+ return 0;
again:
inode = lookup_free_space_inode(root, block_group, path);
if (IS_ERR(inode) && PTR_ERR(inode) != -ENOENT) {
*/
BTRFS_I(inode)->generation = 0;
ret = btrfs_update_inode(trans, root, inode);
+ if (ret) {
+ /*
+ * So theoretically we could recover from this, simply set the
+ * super cache generation to 0 so we know to invalidate the
+ * cache, but then we'd have to keep track of the block groups
+ * that fail this way so we know we _have_ to reset this cache
+ * before the next commit or risk reading stale cache. So to
+ * limit our exposure to horrible edge cases lets just abort the
+ * transaction, this only happens in really bad situations
+ * anyway.
+ */
+ btrfs_abort_transaction(trans, root, ret);
+ goto out_put;
+ }
WARN_ON(ret);
if (i_size_read(inode) > 0) {
return ret;
}
+int btrfs_setup_space_cache(struct btrfs_trans_handle *trans,
+ struct btrfs_root *root)
+{
+ struct btrfs_block_group_cache *cache, *tmp;
+ struct btrfs_transaction *cur_trans = trans->transaction;
+ struct btrfs_path *path;
+
+ if (list_empty(&cur_trans->dirty_bgs) ||
+ !btrfs_test_opt(root, SPACE_CACHE))
+ return 0;
+
+ path = btrfs_alloc_path();
+ if (!path)
+ return -ENOMEM;
+
+ /* Could add new block groups, use _safe just in case */
+ list_for_each_entry_safe(cache, tmp, &cur_trans->dirty_bgs,
+ dirty_list) {
+ if (cache->disk_cache_state == BTRFS_DC_CLEAR)
+ cache_save_setup(cache, trans, path);
+ }
+
+ btrfs_free_path(path);
+ return 0;
+}
+
int btrfs_write_dirty_block_groups(struct btrfs_trans_handle *trans,
struct btrfs_root *root)
{
num_bytes = ALIGN(num_bytes, root->sectorsize);
spin_lock(&BTRFS_I(inode)->lock);
- BTRFS_I(inode)->outstanding_extents++;
+ nr_extents = (unsigned)div64_u64(num_bytes +
+ BTRFS_MAX_EXTENT_SIZE - 1,
+ BTRFS_MAX_EXTENT_SIZE);
+ BTRFS_I(inode)->outstanding_extents += nr_extents;
+ nr_extents = 0;
if (BTRFS_I(inode)->outstanding_extents >
BTRFS_I(inode)->reserved_extents)
if (dropped > 0)
to_free += btrfs_calc_trans_metadata_size(root, dropped);
+ if (btrfs_test_is_dummy_root(root))
+ return;
+
trace_btrfs_space_reservation(root->fs_info, "delalloc",
btrfs_ino(inode), to_free, 0);
if (root->fs_info->quota_enabled) {
/* Should be safe to release our pages at this point */
btrfs_release_extent_buffer_page(eb);
+#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
+ if (unlikely(test_bit(EXTENT_BUFFER_DUMMY, &eb->bflags))) {
+ __free_extent_buffer(eb);
+ return 1;
+ }
+#endif
call_rcu(&eb->rcu_head, btrfs_release_extent_buffer_rcu);
return 1;
}
mutex_unlock(&inode->i_mutex);
/*
- * we want to make sure fsync finds this change
- * but we haven't joined a transaction running right now.
- *
- * Later on, someone is sure to update the inode and get the
- * real transid recorded.
- *
- * We set last_trans now to the fs_info generation + 1,
- * this will either be one more than the running transaction
- * or the generation used for the next transaction if there isn't
- * one running right now.
- *
* We also have to set last_sub_trans to the current log transid,
* otherwise subsequent syncs to a file that's been synced in this
* transaction will appear to have already occured.
*/
- BTRFS_I(inode)->last_trans = root->fs_info->generation + 1;
BTRFS_I(inode)->last_sub_trans = root->log_transid;
if (num_written > 0) {
err = generic_write_sync(file, pos, num_written);
atomic_inc(&root->log_batch);
/*
- * check the transaction that last modified this inode
- * and see if its already been committed
- */
- if (!BTRFS_I(inode)->last_trans) {
- mutex_unlock(&inode->i_mutex);
- goto out;
- }
-
- /*
- * if the last transaction that changed this file was before
- * the current transaction, we can bail out now without any
- * syncing
+ * If the last transaction that changed this file was before the current
+ * transaction and we have the full sync flag set in our inode, we can
+ * bail out now without any syncing.
+ *
+ * Note that we can't bail out if the full sync flag isn't set. This is
+ * because when the full sync flag is set we start all ordered extents
+ * and wait for them to fully complete - when they complete they update
+ * the inode's last_trans field through:
+ *
+ * btrfs_finish_ordered_io() ->
+ * btrfs_update_inode_fallback() ->
+ * btrfs_update_inode() ->
+ * btrfs_set_inode_last_trans()
+ *
+ * So we are sure that last_trans is up to date and can do this check to
+ * bail out safely. For the fast path, when the full sync flag is not
+ * set in our inode, we can not do it because we start only our ordered
+ * extents and don't wait for them to complete (that is when
+ * btrfs_finish_ordered_io runs), so here at this point their last_trans
+ * value might be less than or equals to fs_info->last_trans_committed,
+ * and setting a speculative last_trans for an inode when a buffered
+ * write is made (such as fs_info->generation + 1 for example) would not
+ * be reliable since after setting the value and before fsync is called
+ * any number of transactions can start and commit (transaction kthread
+ * commits the current transaction periodically), and a transaction
+ * commit does not start nor waits for ordered extents to complete.
*/
smp_mb();
if (btrfs_inode_in_log(inode, root->fs_info->generation) ||
- BTRFS_I(inode)->last_trans <=
- root->fs_info->last_trans_committed) {
- BTRFS_I(inode)->last_trans = 0;
-
+ (full_sync && BTRFS_I(inode)->last_trans <=
+ root->fs_info->last_trans_committed)) {
/*
* We'v had everything committed since the last time we were
* modified so clear this flag in case it was set for whatever
bool same_page;
bool no_holes = btrfs_fs_incompat(root->fs_info, NO_HOLES);
u64 ino_size;
+ bool truncated_page = false;
+ bool updated_inode = false;
ret = btrfs_wait_ordered_range(inode, offset, len);
if (ret)
* entire page.
*/
if (same_page && len < PAGE_CACHE_SIZE) {
- if (offset < ino_size)
+ if (offset < ino_size) {
+ truncated_page = true;
ret = btrfs_truncate_page(inode, offset, len, 0);
+ } else {
+ ret = 0;
+ }
goto out_only_mutex;
}
/* zero back part of the first page */
if (offset < ino_size) {
+ truncated_page = true;
ret = btrfs_truncate_page(inode, offset, 0, 0);
if (ret) {
mutex_unlock(&inode->i_mutex);
if (!ret) {
/* zero the front end of the last page */
if (tail_start + tail_len < ino_size) {
+ truncated_page = true;
ret = btrfs_truncate_page(inode,
tail_start + tail_len, 0, 1);
if (ret)
}
if (lockend < lockstart) {
- mutex_unlock(&inode->i_mutex);
- return 0;
+ ret = 0;
+ goto out_only_mutex;
}
while (1) {
trans->block_rsv = &root->fs_info->trans_block_rsv;
ret = btrfs_update_inode(trans, root, inode);
+ updated_inode = true;
btrfs_end_transaction(trans, root);
btrfs_btree_balance_dirty(root);
out_free:
unlock_extent_cached(&BTRFS_I(inode)->io_tree, lockstart, lockend,
&cached_state, GFP_NOFS);
out_only_mutex:
+ if (!updated_inode && truncated_page && !ret && !err) {
+ /*
+ * If we only end up zeroing part of a page, we still need to
+ * update the inode item, so that all the time fields are
+ * updated as well as the necessary btrfs inode in memory fields
+ * for detecting, at fsync time, if the inode isn't yet in the
+ * log tree or it's there but not up to date.
+ */
+ trans = btrfs_start_transaction(root, 1);
+ if (IS_ERR(trans)) {
+ err = PTR_ERR(trans);
+ } else {
+ err = btrfs_update_inode(trans, root, inode);
+ ret = btrfs_end_transaction(trans, root);
+ }
+ }
mutex_unlock(&inode->i_mutex);
if (ret && !err)
err = ret;
static int btrfs_dirty_inode(struct inode *inode);
+#ifdef CONFIG_BTRFS_FS_RUN_SANITY_TESTS
+void btrfs_test_inode_set_ops(struct inode *inode)
+{
+ BTRFS_I(inode)->io_tree.ops = &btrfs_extent_io_ops;
+}
+#endif
+
static int btrfs_init_inode_security(struct btrfs_trans_handle *trans,
struct inode *inode, struct inode *dir,
const struct qstr *qstr)
u64 new_size;
/*
- * We need the largest size of the remaining extent to see if we
- * need to add a new outstanding extent. Think of the following
- * case
- *
- * [MEAX_EXTENT_SIZEx2 - 4k][4k]
- *
- * The new_size would just be 4k and we'd think we had enough
- * outstanding extents for this if we only took one side of the
- * split, same goes for the other direction. We need to see if
- * the larger size still is the same amount of extents as the
- * original size, because if it is we need to add a new
- * outstanding extent. But if we split up and the larger size
- * is less than the original then we are good to go since we've
- * already accounted for the extra extent in our original
- * accounting.
+ * See the explanation in btrfs_merge_extent_hook, the same
+ * applies here, just in reverse.
*/
new_size = orig->end - split + 1;
- if ((split - orig->start) > new_size)
- new_size = split - orig->start;
-
- num_extents = div64_u64(size + BTRFS_MAX_EXTENT_SIZE - 1,
+ num_extents = div64_u64(new_size + BTRFS_MAX_EXTENT_SIZE - 1,
BTRFS_MAX_EXTENT_SIZE);
- if (div64_u64(new_size + BTRFS_MAX_EXTENT_SIZE - 1,
- BTRFS_MAX_EXTENT_SIZE) < num_extents)
+ new_size = split - orig->start;
+ num_extents += div64_u64(new_size + BTRFS_MAX_EXTENT_SIZE - 1,
+ BTRFS_MAX_EXTENT_SIZE);
+ if (div64_u64(size + BTRFS_MAX_EXTENT_SIZE - 1,
+ BTRFS_MAX_EXTENT_SIZE) >= num_extents)
return;
}
if (!(other->state & EXTENT_DELALLOC))
return;
- old_size = other->end - other->start + 1;
- new_size = old_size + (new->end - new->start + 1);
+ if (new->start > other->start)
+ new_size = new->end - other->start + 1;
+ else
+ new_size = other->end - new->start + 1;
/* we're not bigger than the max, unreserve the space and go */
if (new_size <= BTRFS_MAX_EXTENT_SIZE) {
}
/*
- * If we grew by another max_extent, just return, we want to keep that
- * reserved amount.
+ * We have to add up either side to figure out how many extents were
+ * accounted for before we merged into one big extent. If the number of
+ * extents we accounted for is <= the amount we need for the new range
+ * then we can return, otherwise drop. Think of it like this
+ *
+ * [ 4k][MAX_SIZE]
+ *
+ * So we've grown the extent by a MAX_SIZE extent, this would mean we
+ * need 2 outstanding extents, on one side we have 1 and the other side
+ * we have 1 so they are == and we can return. But in this case
+ *
+ * [MAX_SIZE+4k][MAX_SIZE+4k]
+ *
+ * Each range on their own accounts for 2 extents, but merged together
+ * they are only 3 extents worth of accounting, so we need to drop in
+ * this case.
*/
+ old_size = other->end - other->start + 1;
num_extents = div64_u64(old_size + BTRFS_MAX_EXTENT_SIZE - 1,
BTRFS_MAX_EXTENT_SIZE);
+ old_size = new->end - new->start + 1;
+ num_extents += div64_u64(old_size + BTRFS_MAX_EXTENT_SIZE - 1,
+ BTRFS_MAX_EXTENT_SIZE);
+
if (div64_u64(new_size + BTRFS_MAX_EXTENT_SIZE - 1,
- BTRFS_MAX_EXTENT_SIZE) > num_extents)
+ BTRFS_MAX_EXTENT_SIZE) >= num_extents)
return;
spin_lock(&BTRFS_I(inode)->lock);
spin_unlock(&BTRFS_I(inode)->lock);
}
+ /* For sanity tests */
+ if (btrfs_test_is_dummy_root(root))
+ return;
+
__percpu_counter_add(&root->fs_info->delalloc_bytes, len,
root->fs_info->delalloc_batch);
spin_lock(&BTRFS_I(inode)->lock);
root != root->fs_info->tree_root)
btrfs_delalloc_release_metadata(inode, len);
+ /* For sanity tests. */
+ if (btrfs_test_is_dummy_root(root))
+ return;
+
if (root->root_key.objectid != BTRFS_DATA_RELOC_TREE_OBJECTID
&& do_list && !(state->state & EXTENT_NORESERVE))
btrfs_free_reserved_data_space(inode, len);
u64 start = iblock << inode->i_blkbits;
u64 lockstart, lockend;
u64 len = bh_result->b_size;
- u64 orig_len = len;
+ u64 *outstanding_extents = NULL;
int unlock_bits = EXTENT_LOCKED;
int ret = 0;
lockstart = start;
lockend = start + len - 1;
+ if (current->journal_info) {
+ /*
+ * Need to pull our outstanding extents and set journal_info to NULL so
+ * that anything that needs to check if there's a transction doesn't get
+ * confused.
+ */
+ outstanding_extents = current->journal_info;
+ current->journal_info = NULL;
+ }
+
/*
* If this errors out it's because we couldn't invalidate pagecache for
* this range and we need to fallback to buffered.
((BTRFS_I(inode)->flags & BTRFS_INODE_NODATACOW) &&
em->block_start != EXTENT_MAP_HOLE)) {
int type;
- int ret;
u64 block_start, orig_start, orig_block_len, ram_bytes;
if (test_bit(EXTENT_FLAG_PREALLOC, &em->flags))
if (start + len > i_size_read(inode))
i_size_write(inode, start + len);
- if (len < orig_len) {
+ /*
+ * If we have an outstanding_extents count still set then we're
+ * within our reservation, otherwise we need to adjust our inode
+ * counter appropriately.
+ */
+ if (*outstanding_extents) {
+ (*outstanding_extents)--;
+ } else {
spin_lock(&BTRFS_I(inode)->lock);
BTRFS_I(inode)->outstanding_extents++;
spin_unlock(&BTRFS_I(inode)->lock);
}
+
+ current->journal_info = outstanding_extents;
btrfs_free_reserved_data_space(inode, len);
}
unlock_err:
clear_extent_bit(&BTRFS_I(inode)->io_tree, lockstart, lockend,
unlock_bits, 1, 0, &cached_state, GFP_NOFS);
+ if (outstanding_extents)
+ current->journal_info = outstanding_extents;
return ret;
}
{
struct file *file = iocb->ki_filp;
struct inode *inode = file->f_mapping->host;
+ u64 outstanding_extents = 0;
size_t count = 0;
int flags = 0;
bool wakeup = true;
ret = btrfs_delalloc_reserve_space(inode, count);
if (ret)
goto out;
+ outstanding_extents = div64_u64(count +
+ BTRFS_MAX_EXTENT_SIZE - 1,
+ BTRFS_MAX_EXTENT_SIZE);
+
+ /*
+ * We need to know how many extents we reserved so that we can
+ * do the accounting properly if we go over the number we
+ * originally calculated. Abuse current->journal_info for this.
+ */
+ current->journal_info = &outstanding_extents;
} else if (test_bit(BTRFS_INODE_READDIO_NEED_LOCK,
&BTRFS_I(inode)->runtime_flags)) {
inode_dio_done(inode);
iter, offset, btrfs_get_blocks_direct, NULL,
btrfs_submit_direct, flags);
if (rw & WRITE) {
+ current->journal_info = NULL;
if (ret < 0 && ret != -EIOCBQUEUED)
btrfs_delalloc_release_space(inode, count);
else if (ret >= 0 && (size_t)ret < count)
continue;
if (entry_end(ordered) <= start)
break;
- if (!list_empty(&ordered->log_list))
- continue;
- if (test_bit(BTRFS_ORDERED_LOGGED, &ordered->flags))
+ if (test_and_set_bit(BTRFS_ORDERED_LOGGED, &ordered->flags))
continue;
list_add(&ordered->log_list, logged_list);
atomic_inc(&ordered->refs);
wait_event(ordered->wait, test_bit(BTRFS_ORDERED_IO_DONE,
&ordered->flags));
- if (!test_and_set_bit(BTRFS_ORDERED_LOGGED, &ordered->flags))
- list_add_tail(&ordered->trans_list, &trans->ordered);
+ list_add_tail(&ordered->trans_list, &trans->ordered);
spin_lock_irq(&log->log_extents_lock[index]);
}
spin_unlock_irq(&log->log_extents_lock[index]);
if (oper1->seq < oper2->seq)
return -1;
if (oper1->seq > oper2->seq)
- return -1;
+ return 1;
if (oper1->ref_root < oper2->ref_root)
return -1;
if (oper1->ref_root > oper2->ref_root)
u64 parent_ino;
u64 ino;
u64 gen;
+ bool is_orphan;
struct list_head update_refs;
};
u64 ino_gen,
u64 parent_ino,
struct list_head *new_refs,
- struct list_head *deleted_refs)
+ struct list_head *deleted_refs,
+ const bool is_orphan)
{
struct rb_node **p = &sctx->pending_dir_moves.rb_node;
struct rb_node *parent = NULL;
pm->parent_ino = parent_ino;
pm->ino = ino;
pm->gen = ino_gen;
+ pm->is_orphan = is_orphan;
INIT_LIST_HEAD(&pm->list);
INIT_LIST_HEAD(&pm->update_refs);
RB_CLEAR_NODE(&pm->node);
rmdir_ino = dm->rmdir_ino;
free_waiting_dir_move(sctx, dm);
- ret = get_first_ref(sctx->parent_root, pm->ino,
- &parent_ino, &parent_gen, name);
- if (ret < 0)
- goto out;
-
- ret = get_cur_path(sctx, parent_ino, parent_gen,
- from_path);
- if (ret < 0)
- goto out;
- ret = fs_path_add_path(from_path, name);
+ if (pm->is_orphan) {
+ ret = gen_unique_name(sctx, pm->ino,
+ pm->gen, from_path);
+ } else {
+ ret = get_first_ref(sctx->parent_root, pm->ino,
+ &parent_ino, &parent_gen, name);
+ if (ret < 0)
+ goto out;
+ ret = get_cur_path(sctx, parent_ino, parent_gen,
+ from_path);
+ if (ret < 0)
+ goto out;
+ ret = fs_path_add_path(from_path, name);
+ }
if (ret < 0)
goto out;
LIST_HEAD(deleted_refs);
ASSERT(ancestor > BTRFS_FIRST_FREE_OBJECTID);
ret = add_pending_dir_move(sctx, pm->ino, pm->gen, ancestor,
- &pm->update_refs, &deleted_refs);
+ &pm->update_refs, &deleted_refs,
+ pm->is_orphan);
if (ret < 0)
goto out;
if (rmdir_ino) {
return ret;
}
+/*
+ * We might need to delay a directory rename even when no ancestor directory
+ * (in the send root) with a higher inode number than ours (sctx->cur_ino) was
+ * renamed. This happens when we rename a directory to the old name (the name
+ * in the parent root) of some other unrelated directory that got its rename
+ * delayed due to some ancestor with higher number that got renamed.
+ *
+ * Example:
+ *
+ * Parent snapshot:
+ * . (ino 256)
+ * |---- a/ (ino 257)
+ * | |---- file (ino 260)
+ * |
+ * |---- b/ (ino 258)
+ * |---- c/ (ino 259)
+ *
+ * Send snapshot:
+ * . (ino 256)
+ * |---- a/ (ino 258)
+ * |---- x/ (ino 259)
+ * |---- y/ (ino 257)
+ * |----- file (ino 260)
+ *
+ * Here we can not rename 258 from 'b' to 'a' without the rename of inode 257
+ * from 'a' to 'x/y' happening first, which in turn depends on the rename of
+ * inode 259 from 'c' to 'x'. So the order of rename commands the send stream
+ * must issue is:
+ *
+ * 1 - rename 259 from 'c' to 'x'
+ * 2 - rename 257 from 'a' to 'x/y'
+ * 3 - rename 258 from 'b' to 'a'
+ *
+ * Returns 1 if the rename of sctx->cur_ino needs to be delayed, 0 if it can
+ * be done right away and < 0 on error.
+ */
+static int wait_for_dest_dir_move(struct send_ctx *sctx,
+ struct recorded_ref *parent_ref,
+ const bool is_orphan)
+{
+ struct btrfs_path *path;
+ struct btrfs_key key;
+ struct btrfs_key di_key;
+ struct btrfs_dir_item *di;
+ u64 left_gen;
+ u64 right_gen;
+ int ret = 0;
+
+ if (RB_EMPTY_ROOT(&sctx->waiting_dir_moves))
+ return 0;
+
+ path = alloc_path_for_send();
+ if (!path)
+ return -ENOMEM;
+
+ key.objectid = parent_ref->dir;
+ key.type = BTRFS_DIR_ITEM_KEY;
+ key.offset = btrfs_name_hash(parent_ref->name, parent_ref->name_len);
+
+ ret = btrfs_search_slot(NULL, sctx->parent_root, &key, path, 0, 0);
+ if (ret < 0) {
+ goto out;
+ } else if (ret > 0) {
+ ret = 0;
+ goto out;
+ }
+
+ di = btrfs_match_dir_item_name(sctx->parent_root, path,
+ parent_ref->name, parent_ref->name_len);
+ if (!di) {
+ ret = 0;
+ goto out;
+ }
+ /*
+ * di_key.objectid has the number of the inode that has a dentry in the
+ * parent directory with the same name that sctx->cur_ino is being
+ * renamed to. We need to check if that inode is in the send root as
+ * well and if it is currently marked as an inode with a pending rename,
+ * if it is, we need to delay the rename of sctx->cur_ino as well, so
+ * that it happens after that other inode is renamed.
+ */
+ btrfs_dir_item_key_to_cpu(path->nodes[0], di, &di_key);
+ if (di_key.type != BTRFS_INODE_ITEM_KEY) {
+ ret = 0;
+ goto out;
+ }
+
+ ret = get_inode_info(sctx->parent_root, di_key.objectid, NULL,
+ &left_gen, NULL, NULL, NULL, NULL);
+ if (ret < 0)
+ goto out;
+ ret = get_inode_info(sctx->send_root, di_key.objectid, NULL,
+ &right_gen, NULL, NULL, NULL, NULL);
+ if (ret < 0) {
+ if (ret == -ENOENT)
+ ret = 0;
+ goto out;
+ }
+
+ /* Different inode, no need to delay the rename of sctx->cur_ino */
+ if (right_gen != left_gen) {
+ ret = 0;
+ goto out;
+ }
+
+ if (is_waiting_for_move(sctx, di_key.objectid)) {
+ ret = add_pending_dir_move(sctx,
+ sctx->cur_ino,
+ sctx->cur_inode_gen,
+ di_key.objectid,
+ &sctx->new_refs,
+ &sctx->deleted_refs,
+ is_orphan);
+ if (!ret)
+ ret = 1;
+ }
+out:
+ btrfs_free_path(path);
+ return ret;
+}
+
static int wait_for_parent_move(struct send_ctx *sctx,
struct recorded_ref *parent_ref)
{
sctx->cur_inode_gen,
ino,
&sctx->new_refs,
- &sctx->deleted_refs);
+ &sctx->deleted_refs,
+ false);
if (!ret)
ret = 1;
}
int did_overwrite = 0;
int is_orphan = 0;
u64 last_dir_ino_rm = 0;
+ bool can_rename = true;
verbose_printk("btrfs: process_recorded_refs %llu\n", sctx->cur_ino);
}
}
+ if (S_ISDIR(sctx->cur_inode_mode) && sctx->parent_root) {
+ ret = wait_for_dest_dir_move(sctx, cur, is_orphan);
+ if (ret < 0)
+ goto out;
+ if (ret == 1) {
+ can_rename = false;
+ *pending_move = 1;
+ }
+ }
+
/*
* link/move the ref to the new place. If we have an orphan
* inode, move it and update valid_path. If not, link or move
* it depending on the inode mode.
*/
- if (is_orphan) {
+ if (is_orphan && can_rename) {
ret = send_rename(sctx, valid_path, cur->full_path);
if (ret < 0)
goto out;
ret = fs_path_copy(valid_path, cur->full_path);
if (ret < 0)
goto out;
- } else {
+ } else if (can_rename) {
if (S_ISDIR(sctx->cur_inode_mode)) {
/*
* Dirs can't be linked, so move it. For moved
return ret;
}
+static int test_extent_accounting(void)
+{
+ struct inode *inode = NULL;
+ struct btrfs_root *root = NULL;
+ int ret = -ENOMEM;
+
+ inode = btrfs_new_test_inode();
+ if (!inode) {
+ test_msg("Couldn't allocate inode\n");
+ return ret;
+ }
+
+ root = btrfs_alloc_dummy_root();
+ if (IS_ERR(root)) {
+ test_msg("Couldn't allocate root\n");
+ goto out;
+ }
+
+ root->fs_info = btrfs_alloc_dummy_fs_info();
+ if (!root->fs_info) {
+ test_msg("Couldn't allocate dummy fs info\n");
+ goto out;
+ }
+
+ BTRFS_I(inode)->root = root;
+ btrfs_test_inode_set_ops(inode);
+
+ /* [BTRFS_MAX_EXTENT_SIZE] */
+ BTRFS_I(inode)->outstanding_extents++;
+ ret = btrfs_set_extent_delalloc(inode, 0, BTRFS_MAX_EXTENT_SIZE - 1,
+ NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 1) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 1, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* [BTRFS_MAX_EXTENT_SIZE][4k] */
+ BTRFS_I(inode)->outstanding_extents++;
+ ret = btrfs_set_extent_delalloc(inode, BTRFS_MAX_EXTENT_SIZE,
+ BTRFS_MAX_EXTENT_SIZE + 4095, NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 2) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 2, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* [BTRFS_MAX_EXTENT_SIZE/2][4K HOLE][the rest] */
+ ret = clear_extent_bit(&BTRFS_I(inode)->io_tree,
+ BTRFS_MAX_EXTENT_SIZE >> 1,
+ (BTRFS_MAX_EXTENT_SIZE >> 1) + 4095,
+ EXTENT_DELALLOC | EXTENT_DIRTY |
+ EXTENT_UPTODATE | EXTENT_DO_ACCOUNTING, 0, 0,
+ NULL, GFP_NOFS);
+ if (ret) {
+ test_msg("clear_extent_bit returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 2) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 2, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* [BTRFS_MAX_EXTENT_SIZE][4K] */
+ BTRFS_I(inode)->outstanding_extents++;
+ ret = btrfs_set_extent_delalloc(inode, BTRFS_MAX_EXTENT_SIZE >> 1,
+ (BTRFS_MAX_EXTENT_SIZE >> 1) + 4095,
+ NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 2) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 2, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /*
+ * [BTRFS_MAX_EXTENT_SIZE+4K][4K HOLE][BTRFS_MAX_EXTENT_SIZE+4K]
+ *
+ * I'm artificially adding 2 to outstanding_extents because in the
+ * buffered IO case we'd add things up as we go, but I don't feel like
+ * doing that here, this isn't the interesting case we want to test.
+ */
+ BTRFS_I(inode)->outstanding_extents += 2;
+ ret = btrfs_set_extent_delalloc(inode, BTRFS_MAX_EXTENT_SIZE + 8192,
+ (BTRFS_MAX_EXTENT_SIZE << 1) + 12287,
+ NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 4) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 4, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* [BTRFS_MAX_EXTENT_SIZE+4k][4k][BTRFS_MAX_EXTENT_SIZE+4k] */
+ BTRFS_I(inode)->outstanding_extents++;
+ ret = btrfs_set_extent_delalloc(inode, BTRFS_MAX_EXTENT_SIZE+4096,
+ BTRFS_MAX_EXTENT_SIZE+8191, NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 3) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 3, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* [BTRFS_MAX_EXTENT_SIZE+4k][4K HOLE][BTRFS_MAX_EXTENT_SIZE+4k] */
+ ret = clear_extent_bit(&BTRFS_I(inode)->io_tree,
+ BTRFS_MAX_EXTENT_SIZE+4096,
+ BTRFS_MAX_EXTENT_SIZE+8191,
+ EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_DO_ACCOUNTING | EXTENT_UPTODATE, 0, 0,
+ NULL, GFP_NOFS);
+ if (ret) {
+ test_msg("clear_extent_bit returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 4) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 4, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /*
+ * Refill the hole again just for good measure, because I thought it
+ * might fail and I'd rather satisfy my paranoia at this point.
+ */
+ BTRFS_I(inode)->outstanding_extents++;
+ ret = btrfs_set_extent_delalloc(inode, BTRFS_MAX_EXTENT_SIZE+4096,
+ BTRFS_MAX_EXTENT_SIZE+8191, NULL);
+ if (ret) {
+ test_msg("btrfs_set_extent_delalloc returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents != 3) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 3, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+
+ /* Empty */
+ ret = clear_extent_bit(&BTRFS_I(inode)->io_tree, 0, (u64)-1,
+ EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_DO_ACCOUNTING | EXTENT_UPTODATE, 0, 0,
+ NULL, GFP_NOFS);
+ if (ret) {
+ test_msg("clear_extent_bit returned %d\n", ret);
+ goto out;
+ }
+ if (BTRFS_I(inode)->outstanding_extents) {
+ ret = -EINVAL;
+ test_msg("Miscount, wanted 0, got %u\n",
+ BTRFS_I(inode)->outstanding_extents);
+ goto out;
+ }
+ ret = 0;
+out:
+ if (ret)
+ clear_extent_bit(&BTRFS_I(inode)->io_tree, 0, (u64)-1,
+ EXTENT_DIRTY | EXTENT_DELALLOC |
+ EXTENT_DO_ACCOUNTING | EXTENT_UPTODATE, 0, 0,
+ NULL, GFP_NOFS);
+ iput(inode);
+ btrfs_free_dummy_root(root);
+ return ret;
+}
+
int btrfs_test_inodes(void)
{
int ret;
if (ret)
return ret;
test_msg("Running hole first btrfs_get_extent test\n");
- return test_hole_first();
+ ret = test_hole_first();
+ if (ret)
+ return ret;
+ test_msg("Running outstanding_extents tests\n");
+ return test_extent_accounting();
}
u64 old_root_bytenr;
u64 old_root_used;
struct btrfs_root *tree_root = root->fs_info->tree_root;
- bool extent_root = (root->objectid == BTRFS_EXTENT_TREE_OBJECTID);
old_root_used = btrfs_root_used(&root->root_item);
- btrfs_write_dirty_block_groups(trans, root);
while (1) {
old_root_bytenr = btrfs_root_bytenr(&root->root_item);
if (old_root_bytenr == root->node->start &&
- old_root_used == btrfs_root_used(&root->root_item) &&
- (!extent_root ||
- list_empty(&trans->transaction->dirty_bgs)))
+ old_root_used == btrfs_root_used(&root->root_item))
break;
btrfs_set_root_node(&root->root_item, root->node);
return ret;
old_root_used = btrfs_root_used(&root->root_item);
- if (extent_root) {
- ret = btrfs_write_dirty_block_groups(trans, root);
- if (ret)
- return ret;
- }
- ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
- if (ret)
- return ret;
- ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
- if (ret)
- return ret;
}
return 0;
struct btrfs_root *root)
{
struct btrfs_fs_info *fs_info = root->fs_info;
+ struct list_head *dirty_bgs = &trans->transaction->dirty_bgs;
struct list_head *next;
struct extent_buffer *eb;
int ret;
if (ret)
return ret;
+ ret = btrfs_setup_space_cache(trans, root);
+ if (ret)
+ return ret;
+
/* run_qgroups might have added some more refs */
ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
if (ret)
return ret;
-
+again:
while (!list_empty(&fs_info->dirty_cowonly_roots)) {
next = fs_info->dirty_cowonly_roots.next;
list_del_init(next);
ret = update_cowonly_root(trans, root);
if (ret)
return ret;
+ ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
+ if (ret)
+ return ret;
}
+ while (!list_empty(dirty_bgs)) {
+ ret = btrfs_write_dirty_block_groups(trans, root);
+ if (ret)
+ return ret;
+ ret = btrfs_run_delayed_refs(trans, root, (unsigned long)-1);
+ if (ret)
+ return ret;
+ }
+
+ if (!list_empty(&fs_info->dirty_cowonly_roots))
+ goto again;
+
list_add_tail(&fs_info->extent_root->dirty_list,
&trans->transaction->switch_commits);
btrfs_after_dev_replace_commit(fs_info);
wait_for_commit(root, cur_trans);
+ if (unlikely(cur_trans->aborted))
+ ret = cur_trans->aborted;
+
btrfs_put_transaction(cur_trans);
return ret;
base = btrfs_item_ptr_offset(leaf, path->slots[0]);
while (cur_offset < item_size) {
- extref = (struct btrfs_inode_extref *)base + cur_offset;
+ extref = (struct btrfs_inode_extref *)(base + cur_offset);
victim_name_len = btrfs_inode_extref_name_len(leaf, extref);
static struct btrfs_bio *alloc_btrfs_bio(int total_stripes, int real_stripes)
{
struct btrfs_bio *bbio = kzalloc(
+ /* the size of the btrfs_bio */
sizeof(struct btrfs_bio) +
+ /* plus the variable array for the stripes */
sizeof(struct btrfs_bio_stripe) * (total_stripes) +
+ /* plus the variable array for the tgt dev */
sizeof(int) * (real_stripes) +
- sizeof(u64) * (real_stripes),
+ /*
+ * plus the raid_map, which includes both the tgt dev
+ * and the stripes
+ */
+ sizeof(u64) * (total_stripes),
GFP_NOFS);
if (!bbio)
return NULL;
name, name_len, -1);
if (!di && (flags & XATTR_REPLACE))
ret = -ENODATA;
+ else if (IS_ERR(di))
+ ret = PTR_ERR(di);
else if (di)
ret = btrfs_delete_one_dir_name(trans, root, path, di);
goto out;
ASSERT(mutex_is_locked(&inode->i_mutex));
di = btrfs_lookup_xattr(NULL, root, path, btrfs_ino(inode),
name, name_len, 0);
- if (!di) {
+ if (!di)
ret = -ENODATA;
+ else if (IS_ERR(di))
+ ret = PTR_ERR(di);
+ if (ret)
goto out;
- }
btrfs_release_path(path);
di = NULL;
}
/*
* fs/cifs/cifsencrypt.c
*
+ * Encryption and hashing operations relating to NTLM, NTLMv2. See MS-NLMP
+ * for more detailed information
+ *
* Copyright (C) International Business Machines Corp., 2005,2013
* Author(s): Steve French (sfrench@us.ibm.com)
*
__func__);
return rc;
}
- } else if (ses->serverName) {
+ } else {
+ /* We use ses->serverName if no domain name available */
len = strlen(ses->serverName);
server = kmalloc(2 + (len * 2), GFP_KERNEL);
pr_warn("CIFS: username too long\n");
goto cifs_parse_mount_err;
}
+
+ kfree(vol->username);
vol->username = kstrdup(string, GFP_KERNEL);
if (!vol->username)
goto cifs_parse_mount_err;
goto cifs_parse_mount_err;
}
+ kfree(vol->domainname);
vol->domainname = kstrdup(string, GFP_KERNEL);
if (!vol->domainname) {
pr_warn("CIFS: no memory for domainname\n");
}
if (strncasecmp(string, "default", 7) != 0) {
+ kfree(vol->iocharset);
vol->iocharset = kstrdup(string,
GFP_KERNEL);
if (!vol->iocharset) {
* calling name ends in null (byte 16) from old smb
* convention.
*/
- if (server->workstation_RFC1001_name &&
- server->workstation_RFC1001_name[0] != 0)
+ if (server->workstation_RFC1001_name[0] != 0)
rfc1002mangle(ses_init_buf->trailer.
session_req.calling_name,
server->workstation_RFC1001_name,
#endif /* CIFS_WEAK_PW_HASH */
rc = SMBNTencrypt(tcon->password, ses->server->cryptkey,
bcc_ptr, nls_codepage);
+ if (rc) {
+ cifs_dbg(FYI, "%s Can't generate NTLM rsp. Error: %d\n",
+ __func__, rc);
+ cifs_buf_release(smb_buffer);
+ return rc;
+ }
bcc_ptr += CIFS_AUTH_RESP_SIZE;
if (ses->capabilities & CAP_UNICODE) {
cifsFileInfo_put(inv_file);
spin_lock(&cifs_file_list_lock);
++refind;
+ inv_file = NULL;
goto refind_writable;
}
}
cifs_buf_release(srchinf->ntwrk_buf_start);
}
kfree(srchinf);
+ if (rc)
+ goto cgii_exit;
} else
goto cgii_exit;
/* return pointer to beginning of data area, ie offset from SMB start */
if ((*off != 0) && (*len != 0))
- return hdr->ProtocolId + *off;
+ return (char *)(&hdr->ProtocolId[0]) + *off;
else
return NULL;
}
/* No need to change MaxChunks since already set to 1 */
chunk_sizes_updated = true;
- }
+ } else
+ goto cchunk_out;
}
cchunk_out:
struct smb2_ioctl_req *req;
struct smb2_ioctl_rsp *rsp;
struct TCP_Server_Info *server;
- struct cifs_ses *ses = tcon->ses;
+ struct cifs_ses *ses;
struct kvec iov[2];
int resp_buftype;
int num_iovecs;
if (plen)
*plen = 0;
+ if (tcon)
+ ses = tcon->ses;
+ else
+ return -EIO;
+
if (ses && (ses->server))
server = ses->server;
else
rsp = (struct smb2_ioctl_rsp *)iov[0].iov_base;
if ((rc != 0) && (rc != -EINVAL)) {
- if (tcon)
- cifs_stats_fail_inc(tcon, SMB2_IOCTL_HE);
+ cifs_stats_fail_inc(tcon, SMB2_IOCTL_HE);
goto ioctl_exit;
} else if (rc == -EINVAL) {
if ((opcode != FSCTL_SRV_COPYCHUNK_WRITE) &&
(opcode != FSCTL_SRV_COPYCHUNK)) {
- if (tcon)
- cifs_stats_fail_inc(tcon, SMB2_IOCTL_HE);
+ cifs_stats_fail_inc(tcon, SMB2_IOCTL_HE);
goto ioctl_exit;
}
}
rc = SendReceive2(xid, ses, iov, 1, &resp_buftype, 0);
- if ((rc != 0) && tcon)
+ if (rc != 0)
cifs_stats_fail_inc(tcon, SMB2_FLUSH_HE);
free_rsp_buf(resp_buftype, iov[0].iov_base);
struct kvec iov[2];
int rc = 0;
int len;
- int resp_buftype;
+ int resp_buftype = CIFS_NO_BUFFER;
unsigned char *bufptr;
struct TCP_Server_Info *server;
struct cifs_ses *ses = tcon->ses;
}
#define ECRYPTFS_MAX_KEYSET_SIZE 1024
-#define ECRYPTFS_MAX_CIPHER_NAME_SIZE 32
+#define ECRYPTFS_MAX_CIPHER_NAME_SIZE 31
#define ECRYPTFS_MAX_NUM_ENC_KEYS 64
#define ECRYPTFS_MAX_IV_BYTES 16 /* 128 bits */
#define ECRYPTFS_SALT_BYTES 2
struct crypto_ablkcipher *tfm;
struct crypto_hash *hash_tfm; /* Crypto context for generating
* the initialization vectors */
- unsigned char cipher[ECRYPTFS_MAX_CIPHER_NAME_SIZE];
+ unsigned char cipher[ECRYPTFS_MAX_CIPHER_NAME_SIZE + 1];
unsigned char key[ECRYPTFS_MAX_KEY_BYTES];
unsigned char root_iv[ECRYPTFS_MAX_IV_BYTES];
struct list_head keysig_list;
struct file *lower_file = ecryptfs_file_to_lower(file);
long rc = -ENOTTY;
- if (lower_file->f_op->unlocked_ioctl)
+ if (!lower_file->f_op->unlocked_ioctl)
+ return rc;
+
+ switch (cmd) {
+ case FITRIM:
+ case FS_IOC_GETFLAGS:
+ case FS_IOC_SETFLAGS:
+ case FS_IOC_GETVERSION:
+ case FS_IOC_SETVERSION:
rc = lower_file->f_op->unlocked_ioctl(lower_file, cmd, arg);
- return rc;
+ fsstack_copy_attr_all(file_inode(file), file_inode(lower_file));
+
+ return rc;
+ default:
+ return rc;
+ }
}
#ifdef CONFIG_COMPAT
struct file *lower_file = ecryptfs_file_to_lower(file);
long rc = -ENOIOCTLCMD;
- if (lower_file->f_op->compat_ioctl)
+ if (!lower_file->f_op->compat_ioctl)
+ return rc;
+
+ switch (cmd) {
+ case FITRIM:
+ case FS_IOC32_GETFLAGS:
+ case FS_IOC32_SETFLAGS:
+ case FS_IOC32_GETVERSION:
+ case FS_IOC32_SETVERSION:
rc = lower_file->f_op->compat_ioctl(lower_file, cmd, arg);
- return rc;
+ fsstack_copy_attr_all(file_inode(file), file_inode(lower_file));
+
+ return rc;
+ default:
+ return rc;
+ }
}
#endif
struct blkcipher_desc desc;
char fnek_sig_hex[ECRYPTFS_SIG_SIZE_HEX + 1];
char iv[ECRYPTFS_MAX_IV_BYTES];
- char cipher_string[ECRYPTFS_MAX_CIPHER_NAME_SIZE];
+ char cipher_string[ECRYPTFS_MAX_CIPHER_NAME_SIZE + 1];
};
/**
if (!cipher_name_set) {
int cipher_name_len = strlen(ECRYPTFS_DEFAULT_CIPHER);
- BUG_ON(cipher_name_len >= ECRYPTFS_MAX_CIPHER_NAME_SIZE);
+ BUG_ON(cipher_name_len > ECRYPTFS_MAX_CIPHER_NAME_SIZE);
strcpy(mount_crypt_stat->global_default_cipher_name,
ECRYPTFS_DEFAULT_CIPHER);
}
struct completion *done; /* set if the caller waits */
};
+/*
+ * If an inode is constantly having its pages dirtied, but then the
+ * updates stop dirtytime_expire_interval seconds in the past, it's
+ * possible for the worst case time between when an inode has its
+ * timestamps updated and when they finally get written out to be two
+ * dirtytime_expire_intervals. We set the default to 12 hours (in
+ * seconds), which means most of the time inodes will have their
+ * timestamps written to disk after 12 hours, but in the worst case a
+ * few inodes might not their timestamps updated for 24 hours.
+ */
+unsigned int dirtytime_expire_interval = 12 * 60 * 60;
+
/**
* writeback_in_progress - determine whether there is writeback in progress
* @bdi: the device's backing_dev_info structure.
if ((flags & EXPIRE_DIRTY_ATIME) == 0)
older_than_this = work->older_than_this;
- else if ((work->reason == WB_REASON_SYNC) == 0) {
- expire_time = jiffies - (HZ * 86400);
+ else if (!work->for_sync) {
+ expire_time = jiffies - (dirtytime_expire_interval * HZ);
older_than_this = &expire_time;
}
while (!list_empty(delaying_queue)) {
*/
redirty_tail(inode, wb);
} else if (inode->i_state & I_DIRTY_TIME) {
+ inode->dirtied_when = jiffies;
list_move(&inode->i_wb_list, &wb->b_dirty_time);
} else {
/* The inode is clean. Remove from writeback lists. */
spin_lock(&inode->i_lock);
dirty = inode->i_state & I_DIRTY;
- if (((dirty & (I_DIRTY_SYNC | I_DIRTY_DATASYNC)) &&
- (inode->i_state & I_DIRTY_TIME)) ||
- (inode->i_state & I_DIRTY_TIME_EXPIRED)) {
- dirty |= I_DIRTY_TIME | I_DIRTY_TIME_EXPIRED;
- trace_writeback_lazytime(inode);
- }
+ if (inode->i_state & I_DIRTY_TIME) {
+ if ((dirty & (I_DIRTY_SYNC | I_DIRTY_DATASYNC)) ||
+ unlikely(inode->i_state & I_DIRTY_TIME_EXPIRED) ||
+ unlikely(time_after(jiffies,
+ (inode->dirtied_time_when +
+ dirtytime_expire_interval * HZ)))) {
+ dirty |= I_DIRTY_TIME | I_DIRTY_TIME_EXPIRED;
+ trace_writeback_lazytime(inode);
+ }
+ } else
+ inode->i_state &= ~I_DIRTY_TIME_EXPIRED;
inode->i_state &= ~dirty;
/*
rcu_read_unlock();
}
+/*
+ * Wake up bdi's periodically to make sure dirtytime inodes gets
+ * written back periodically. We deliberately do *not* check the
+ * b_dirtytime list in wb_has_dirty_io(), since this would cause the
+ * kernel to be constantly waking up once there are any dirtytime
+ * inodes on the system. So instead we define a separate delayed work
+ * function which gets called much more rarely. (By default, only
+ * once every 12 hours.)
+ *
+ * If there is any other write activity going on in the file system,
+ * this function won't be necessary. But if the only thing that has
+ * happened on the file system is a dirtytime inode caused by an atime
+ * update, we need this infrastructure below to make sure that inode
+ * eventually gets pushed out to disk.
+ */
+static void wakeup_dirtytime_writeback(struct work_struct *w);
+static DECLARE_DELAYED_WORK(dirtytime_work, wakeup_dirtytime_writeback);
+
+static void wakeup_dirtytime_writeback(struct work_struct *w)
+{
+ struct backing_dev_info *bdi;
+
+ rcu_read_lock();
+ list_for_each_entry_rcu(bdi, &bdi_list, bdi_list) {
+ if (list_empty(&bdi->wb.b_dirty_time))
+ continue;
+ bdi_wakeup_thread(bdi);
+ }
+ rcu_read_unlock();
+ schedule_delayed_work(&dirtytime_work, dirtytime_expire_interval * HZ);
+}
+
+static int __init start_dirtytime_writeback(void)
+{
+ schedule_delayed_work(&dirtytime_work, dirtytime_expire_interval * HZ);
+ return 0;
+}
+__initcall(start_dirtytime_writeback);
+
+int dirtytime_interval_handler(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+ int ret;
+
+ ret = proc_dointvec_minmax(table, write, buffer, lenp, ppos);
+ if (ret == 0 && write)
+ mod_delayed_work(system_wq, &dirtytime_work, 0);
+ return ret;
+}
+
static noinline void block_dump___mark_inode_dirty(struct inode *inode)
{
if (inode->i_ino || strcmp(inode->i_sb->s_id, "bdev")) {
}
inode->dirtied_when = jiffies;
- list_move(&inode->i_wb_list, dirtytime ?
- &bdi->wb.b_dirty_time : &bdi->wb.b_dirty);
+ if (dirtytime)
+ inode->dirtied_time_when = jiffies;
+ if (inode->i_state & (I_DIRTY_INODE | I_DIRTY_PAGES))
+ list_move(&inode->i_wb_list, &bdi->wb.b_dirty);
+ else
+ list_move(&inode->i_wb_list,
+ &bdi->wb.b_dirty_time);
spin_unlock(&bdi->wb.list_lock);
trace_writeback_dirty_inode_enqueue(inode);
newpage = buf->page;
- if (WARN_ON(!PageUptodate(newpage)))
- return -EIO;
+ if (!PageUptodate(newpage))
+ SetPageUptodate(newpage);
ClearPageMappedToDisk(newpage);
return err;
}
+static int fuse_dev_open(struct inode *inode, struct file *file)
+{
+ /*
+ * The fuse device's file's private_data is used to hold
+ * the fuse_conn(ection) when it is mounted, and is used to
+ * keep track of whether the file has been mounted already.
+ */
+ file->private_data = NULL;
+ return 0;
+}
+
static ssize_t fuse_dev_read(struct kiocb *iocb, const struct iovec *iov,
unsigned long nr_segs, loff_t pos)
{
static int fuse_notify(struct fuse_conn *fc, enum fuse_notify_code code,
unsigned int size, struct fuse_copy_state *cs)
{
+ /* Don't try to move pages (yet) */
+ cs->move_pages = 0;
+
switch (code) {
case FUSE_NOTIFY_POLL:
return fuse_notify_poll(fc, size, cs);
const struct file_operations fuse_dev_operations = {
.owner = THIS_MODULE,
+ .open = fuse_dev_open,
.llseek = no_llseek,
.read = do_sync_read,
.aio_read = fuse_dev_read,
hfs_bnode_write(node, entry, data_off + key_len, entry_len);
hfs_bnode_dump(node);
- if (new_node) {
- /* update parent key if we inserted a key
- * at the start of the first node
- */
- if (!rec && new_node != node)
- hfs_brec_update_parent(fd);
+ /*
+ * update parent key if we inserted a key
+ * at the start of the node and it is not the new node
+ */
+ if (!rec && new_node != node) {
+ hfs_bnode_read_key(node, fd->search_key, data_off + size);
+ hfs_brec_update_parent(fd);
+ }
+ if (new_node) {
hfs_bnode_put(fd->bnode);
if (!new_node->parent) {
hfs_btree_inc_height(tree);
goto again;
}
- if (!rec)
- hfs_brec_update_parent(fd);
-
return 0;
}
if (IS_ERR(parent))
return PTR_ERR(parent);
__hfs_brec_find(parent, fd, hfs_find_rec_by_key);
+ if (fd->record < 0)
+ return -ENOENT;
hfs_bnode_dump(parent);
rec = fd->record;
goto out_free;
}
+ of->event = atomic_read(&of->kn->attr.open->event);
ops = kernfs_ops(of->kn);
if (ops->read)
len = ops->read(of, buf, len, *ppos);
int __break_lease(struct inode *inode, unsigned int mode, unsigned int type)
{
int error = 0;
- struct file_lock *new_fl;
struct file_lock_context *ctx = inode->i_flctx;
- struct file_lock *fl;
+ struct file_lock *new_fl, *fl, *tmp;
unsigned long break_time;
int want_write = (mode & O_ACCMODE) != O_RDONLY;
LIST_HEAD(dispose);
break_time++; /* so that 0 means no break time */
}
- list_for_each_entry(fl, &ctx->flc_lease, fl_list) {
+ list_for_each_entry_safe(fl, tmp, &ctx->flc_lease, fl_list) {
if (!leases_conflict(fl, new_fl))
continue;
if (want_write) {
}
if (my_fl != NULL) {
- error = lease->fl_lmops->lm_change(my_fl, arg, &dispose);
+ lease = my_fl;
+ error = lease->fl_lmops->lm_change(lease, arg, &dispose);
if (error)
goto out;
goto out_setup;
break;
}
}
- trace_generic_delete_lease(inode, fl);
+ trace_generic_delete_lease(inode, victim);
if (victim)
error = fl->fl_lmops->lm_change(victim, F_UNLCK, &dispose);
spin_unlock(&ctx->flc_lock);
static bool nfs_client_init_is_complete(const struct nfs_client *clp)
{
- return clp->cl_cons_state != NFS_CS_INITING;
+ return clp->cl_cons_state <= NFS_CS_READY;
}
int nfs_wait_client_init_complete(const struct nfs_client *clp)
clear_bit(NFS_DELEGATION_NEED_RECLAIM,
&delegation->flags);
spin_unlock(&delegation->lock);
- put_rpccred(oldcred);
rcu_read_unlock();
+ put_rpccred(oldcred);
trace_nfs4_reclaim_delegation(inode, res->delegation_type);
} else {
/* We appear to have raced with a delegation return. */
delegation = NULL;
goto out;
}
- freeme = nfs_detach_delegation_locked(nfsi,
+ if (test_and_set_bit(NFS_DELEGATION_RETURNING,
+ &old_delegation->flags))
+ goto out;
+ freeme = nfs_detach_delegation_locked(nfsi,
old_delegation, clp);
if (freeme == NULL)
goto out;
{
bool ret = false;
+ if (test_bit(NFS_DELEGATION_RETURNING, &delegation->flags))
+ goto out;
if (test_and_clear_bit(NFS_DELEGATION_RETURN, &delegation->flags))
ret = true;
if (test_and_clear_bit(NFS_DELEGATION_RETURN_IF_CLOSED, &delegation->flags) && !ret) {
ret = true;
spin_unlock(&delegation->lock);
}
+out:
return ret;
}
super_list) {
if (!nfs_delegation_need_return(delegation))
continue;
- inode = nfs_delegation_grab_inode(delegation);
- if (inode == NULL)
+ if (!nfs_sb_active(server->super))
continue;
+ inode = nfs_delegation_grab_inode(delegation);
+ if (inode == NULL) {
+ rcu_read_unlock();
+ nfs_sb_deactive(server->super);
+ goto restart;
+ }
delegation = nfs_start_delegation_return_locked(NFS_I(inode));
rcu_read_unlock();
err = nfs_end_delegation_return(inode, delegation, 0);
iput(inode);
+ nfs_sb_deactive(server->super);
if (!err)
goto restart;
set_bit(NFS4CLNT_DELEGRETURN, &clp->cl_state);
list_for_each_entry_rcu(server, &clp->cl_superblocks, client_link) {
list_for_each_entry_rcu(delegation, &server->delegations,
super_list) {
+ if (test_bit(NFS_DELEGATION_RETURNING,
+ &delegation->flags))
+ continue;
if (test_bit(NFS_DELEGATION_NEED_RECLAIM,
&delegation->flags) == 0)
continue;
- inode = nfs_delegation_grab_inode(delegation);
- if (inode == NULL)
+ if (!nfs_sb_active(server->super))
continue;
- delegation = nfs_detach_delegation(NFS_I(inode),
- delegation, server);
+ inode = nfs_delegation_grab_inode(delegation);
+ if (inode == NULL) {
+ rcu_read_unlock();
+ nfs_sb_deactive(server->super);
+ goto restart;
+ }
+ delegation = nfs_start_delegation_return_locked(NFS_I(inode));
rcu_read_unlock();
-
- if (delegation != NULL)
- nfs_free_delegation(delegation);
+ if (delegation != NULL) {
+ delegation = nfs_detach_delegation(NFS_I(inode),
+ delegation, server);
+ if (delegation != NULL)
+ nfs_free_delegation(delegation);
+ }
iput(inode);
+ nfs_sb_deactive(server->super);
goto restart;
}
}
return 0;
}
+/* Match file and dirent using either filehandle or fileid
+ * Note: caller is responsible for checking the fsid
+ */
static
int nfs_same_file(struct dentry *dentry, struct nfs_entry *entry)
{
+ struct nfs_inode *nfsi;
+
if (dentry->d_inode == NULL)
goto different;
- if (nfs_compare_fh(entry->fh, NFS_FH(dentry->d_inode)) != 0)
- goto different;
- return 1;
+
+ nfsi = NFS_I(dentry->d_inode);
+ if (entry->fattr->fileid == nfsi->fileid)
+ return 1;
+ if (nfs_compare_fh(entry->fh, &nfsi->fh) == 0)
+ return 1;
different:
return 0;
}
struct inode *inode;
int status;
+ if (!(entry->fattr->valid & NFS_ATTR_FATTR_FILEID))
+ return;
+ if (!(entry->fattr->valid & NFS_ATTR_FATTR_FSID))
+ return;
if (filename.name[0] == '.') {
if (filename.len == 1)
return;
dentry = d_lookup(parent, &filename);
if (dentry != NULL) {
+ /* Is there a mountpoint here? If so, just exit */
+ if (!nfs_fsid_equal(&NFS_SB(dentry->d_sb)->fsid,
+ &entry->fattr->fsid))
+ goto out;
if (nfs_same_file(dentry, entry)) {
nfs_set_verifier(dentry, nfs_save_change_attribute(dir));
status = nfs_refresh_inode(dentry->d_inode, entry->fattr);
iocb->ki_filp,
iov_iter_count(to), (unsigned long) iocb->ki_pos);
- result = nfs_revalidate_mapping(inode, iocb->ki_filp->f_mapping);
+ result = nfs_revalidate_mapping_protected(inode, iocb->ki_filp->f_mapping);
if (!result) {
result = generic_file_read_iter(iocb, to);
if (result > 0)
dprintk("NFS: splice_read(%pD2, %lu@%Lu)\n",
filp, (unsigned long) count, (unsigned long long) *ppos);
- res = nfs_revalidate_mapping(inode, filp->f_mapping);
+ res = nfs_revalidate_mapping_protected(inode, filp->f_mapping);
if (!res) {
res = generic_file_splice_read(filp, ppos, pipe, count, flags);
if (res > 0)
nfs_wait_bit_killable, TASK_KILLABLE);
if (ret)
return ret;
+ /*
+ * Wait for O_DIRECT to complete
+ */
+ nfs_inode_dio_wait(mapping->host);
page = grab_cache_page_write_begin(mapping, index, flags);
if (!page)
/* make sure the cache has finished storing the page */
nfs_fscache_wait_on_page_write(NFS_I(inode), page);
+ wait_on_bit_action(&NFS_I(inode)->flags, NFS_INO_INVALIDATING,
+ nfs_wait_bit_killable, TASK_KILLABLE);
+
lock_page(page);
mapping = page_file_mapping(page);
if (mapping != inode->i_mapping)
* This is a copy of the common vmtruncate, but with the locking
* corrected to take into account the fact that NFS requires
* inode->i_size to be updated under the inode->i_lock.
+ * Note: must be called with inode->i_lock held!
*/
static int nfs_vmtruncate(struct inode * inode, loff_t offset)
{
if (err)
goto out;
- spin_lock(&inode->i_lock);
i_size_write(inode, offset);
/* Optimisation */
if (offset == 0)
NFS_I(inode)->cache_validity &= ~NFS_INO_INVALID_DATA;
- spin_unlock(&inode->i_lock);
+ spin_unlock(&inode->i_lock);
truncate_pagecache(inode, offset);
+ spin_lock(&inode->i_lock);
out:
return err;
}
* Note: we do this in the *proc.c in order to ensure that
* it works for things like exclusive creates too.
*/
-void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr)
+void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr,
+ struct nfs_fattr *fattr)
{
+ /* Barrier: bump the attribute generation count. */
+ nfs_fattr_set_barrier(fattr);
+
+ spin_lock(&inode->i_lock);
+ NFS_I(inode)->attr_gencount = fattr->gencount;
if ((attr->ia_valid & (ATTR_MODE|ATTR_UID|ATTR_GID)) != 0) {
- spin_lock(&inode->i_lock);
if ((attr->ia_valid & ATTR_MODE) != 0) {
int mode = attr->ia_mode & S_IALLUGO;
mode |= inode->i_mode & ~S_IALLUGO;
inode->i_gid = attr->ia_gid;
nfs_set_cache_invalid(inode, NFS_INO_INVALID_ACCESS
| NFS_INO_INVALID_ACL);
- spin_unlock(&inode->i_lock);
}
if ((attr->ia_valid & ATTR_SIZE) != 0) {
nfs_inc_stats(inode, NFSIOS_SETATTRTRUNC);
nfs_vmtruncate(inode, attr->ia_size);
}
+ nfs_update_inode(inode, fattr);
+ spin_unlock(&inode->i_lock);
}
EXPORT_SYMBOL_GPL(nfs_setattr_update_inode);
if (mapping->nrpages != 0) {
if (S_ISREG(inode->i_mode)) {
+ unmap_mapping_range(mapping, 0, 0, 0);
ret = nfs_sync_mapping(mapping);
if (ret < 0)
return ret;
}
/**
- * nfs_revalidate_mapping - Revalidate the pagecache
+ * __nfs_revalidate_mapping - Revalidate the pagecache
* @inode - pointer to host inode
* @mapping - pointer to mapping
+ * @may_lock - take inode->i_mutex?
*/
-int nfs_revalidate_mapping(struct inode *inode, struct address_space *mapping)
+static int __nfs_revalidate_mapping(struct inode *inode,
+ struct address_space *mapping,
+ bool may_lock)
{
struct nfs_inode *nfsi = NFS_I(inode);
unsigned long *bitlock = &nfsi->flags;
nfsi->cache_validity &= ~NFS_INO_INVALID_DATA;
spin_unlock(&inode->i_lock);
trace_nfs_invalidate_mapping_enter(inode);
- ret = nfs_invalidate_mapping(inode, mapping);
+ if (may_lock) {
+ mutex_lock(&inode->i_mutex);
+ ret = nfs_invalidate_mapping(inode, mapping);
+ mutex_unlock(&inode->i_mutex);
+ } else
+ ret = nfs_invalidate_mapping(inode, mapping);
trace_nfs_invalidate_mapping_exit(inode, ret);
clear_bit_unlock(NFS_INO_INVALIDATING, bitlock);
return ret;
}
+/**
+ * nfs_revalidate_mapping - Revalidate the pagecache
+ * @inode - pointer to host inode
+ * @mapping - pointer to mapping
+ */
+int nfs_revalidate_mapping(struct inode *inode, struct address_space *mapping)
+{
+ return __nfs_revalidate_mapping(inode, mapping, false);
+}
+
+/**
+ * nfs_revalidate_mapping_protected - Revalidate the pagecache
+ * @inode - pointer to host inode
+ * @mapping - pointer to mapping
+ *
+ * Differs from nfs_revalidate_mapping() in that it grabs the inode->i_mutex
+ * while invalidating the mapping.
+ */
+int nfs_revalidate_mapping_protected(struct inode *inode, struct address_space *mapping)
+{
+ return __nfs_revalidate_mapping(inode, mapping, true);
+}
+
static unsigned long nfs_wcc_update_inode(struct inode *inode, struct nfs_fattr *fattr)
{
struct nfs_inode *nfsi = NFS_I(inode);
return timespec_compare(&fattr->ctime, &inode->i_ctime) > 0;
}
-static int nfs_size_need_update(const struct inode *inode, const struct nfs_fattr *fattr)
-{
- if (!(fattr->valid & NFS_ATTR_FATTR_SIZE))
- return 0;
- return nfs_size_to_loff_t(fattr->size) > i_size_read(inode);
-}
-
static atomic_long_t nfs_attr_generation_counter;
static unsigned long nfs_read_attr_generation_counter(void)
{
return atomic_long_inc_return(&nfs_attr_generation_counter);
}
+EXPORT_SYMBOL_GPL(nfs_inc_attr_generation_counter);
void nfs_fattr_init(struct nfs_fattr *fattr)
{
}
EXPORT_SYMBOL_GPL(nfs_fattr_init);
+/**
+ * nfs_fattr_set_barrier
+ * @fattr: attributes
+ *
+ * Used to set a barrier after an attribute was updated. This
+ * barrier ensures that older attributes from RPC calls that may
+ * have raced with our update cannot clobber these new values.
+ * Note that you are still responsible for ensuring that other
+ * operations which change the attribute on the server do not
+ * collide.
+ */
+void nfs_fattr_set_barrier(struct nfs_fattr *fattr)
+{
+ fattr->gencount = nfs_inc_attr_generation_counter();
+}
+
struct nfs_fattr *nfs_alloc_fattr(void)
{
struct nfs_fattr *fattr;
return ((long)fattr->gencount - (long)nfsi->attr_gencount) > 0 ||
nfs_ctime_need_update(inode, fattr) ||
- nfs_size_need_update(inode, fattr) ||
((long)nfsi->attr_gencount - (long)nfs_read_attr_generation_counter() > 0);
}
int status;
spin_lock(&inode->i_lock);
+ nfs_fattr_set_barrier(fattr);
status = nfs_post_op_update_inode_locked(inode, fattr);
spin_unlock(&inode->i_lock);
EXPORT_SYMBOL_GPL(nfs_post_op_update_inode);
/**
- * nfs_post_op_update_inode_force_wcc - try to update the inode attribute cache
+ * nfs_post_op_update_inode_force_wcc_locked - update the inode attribute cache
* @inode - pointer to inode
* @fattr - updated attributes
*
*
* This function is mainly designed to be used by the ->write_done() functions.
*/
-int nfs_post_op_update_inode_force_wcc(struct inode *inode, struct nfs_fattr *fattr)
+int nfs_post_op_update_inode_force_wcc_locked(struct inode *inode, struct nfs_fattr *fattr)
{
int status;
- spin_lock(&inode->i_lock);
/* Don't do a WCC update if these attributes are already stale */
if ((fattr->valid & NFS_ATTR_FATTR) == 0 ||
!nfs_inode_attrs_need_update(inode, fattr)) {
}
out_noforce:
status = nfs_post_op_update_inode_locked(inode, fattr);
+ return status;
+}
+
+/**
+ * nfs_post_op_update_inode_force_wcc - try to update the inode attribute cache
+ * @inode - pointer to inode
+ * @fattr - updated attributes
+ *
+ * After an operation that has changed the inode metadata, mark the
+ * attribute cache as being invalid, then try to update it. Fake up
+ * weak cache consistency data, if none exist.
+ *
+ * This function is mainly designed to be used by the ->write_done() functions.
+ */
+int nfs_post_op_update_inode_force_wcc(struct inode *inode, struct nfs_fattr *fattr)
+{
+ int status;
+
+ spin_lock(&inode->i_lock);
+ nfs_fattr_set_barrier(fattr);
+ status = nfs_post_op_update_inode_force_wcc_locked(inode, fattr);
spin_unlock(&inode->i_lock);
return status;
}
nfs_inc_stats(inode, NFSIOS_ATTRINVALIDATE);
nfsi->attrtimeo = NFS_MINATTRTIMEO(inode);
nfsi->attrtimeo_timestamp = now;
+ /* Set barrier to be more recent than all outstanding updates */
nfsi->attr_gencount = nfs_inc_attr_generation_counter();
} else {
if (!time_in_range_open(now, nfsi->attrtimeo_timestamp, nfsi->attrtimeo_timestamp + nfsi->attrtimeo)) {
nfsi->attrtimeo = NFS_MAXATTRTIMEO(inode);
nfsi->attrtimeo_timestamp = now;
}
+ /* Set the barrier to be more recent than this fattr */
+ if ((long)fattr->gencount - (long)nfsi->attr_gencount > 0)
+ nfsi->attr_gencount = fattr->gencount;
}
invalid &= ~NFS_INO_INVALID_ATTR;
/* Don't invalidate the data if we were to blame */
struct nfs_commit_info *cinfo,
u32 ds_commit_idx);
int nfs_write_need_commit(struct nfs_pgio_header *);
+void nfs_writeback_update_inode(struct nfs_pgio_header *hdr);
int nfs_generic_commit_list(struct inode *inode, struct list_head *head,
int how, struct nfs_commit_info *cinfo);
void nfs_retry_commit(struct list_head *page_list,
nfs_fattr_init(fattr);
status = rpc_call_sync(NFS_CLIENT(inode), &msg, 0);
if (status == 0)
- nfs_setattr_update_inode(inode, sattr);
+ nfs_setattr_update_inode(inode, sattr, fattr);
dprintk("NFS reply setattr: %d\n", status);
return status;
}
if (nfs3_async_handle_jukebox(task, inode))
return -EAGAIN;
if (task->tk_status >= 0)
- nfs_post_op_update_inode_force_wcc(inode, hdr->res.fattr);
+ nfs_writeback_update_inode(hdr);
return 0;
}
if (entry->fattr->valid & NFS_ATTR_FATTR_V3)
entry->d_type = nfs_umode_to_dtype(entry->fattr->mode);
+ if (entry->fattr->fileid != entry->ino) {
+ entry->fattr->mounted_on_fileid = entry->ino;
+ entry->fattr->valid |= NFS_ATTR_FATTR_MOUNTED_ON_FILEID;
+ }
+
/* In fact, a post_op_fh3: */
p = xdr_inline_decode(xdr, 4);
if (unlikely(p == NULL))
spin_lock(&nn->nfs_client_lock);
list_for_each_entry(pos, &nn->nfs_client_list, cl_share_link) {
+ if (pos == new)
+ goto found;
+
if (pos->rpc_ops != new->rpc_ops)
continue;
prev = pos;
status = nfs_wait_client_init_complete(pos);
- if (pos->cl_cons_state == NFS_CS_SESSION_INITING) {
- nfs4_schedule_lease_recovery(pos);
- status = nfs4_wait_clnt_recover(pos);
- }
spin_lock(&nn->nfs_client_lock);
if (status < 0)
break;
*/
if (!nfs4_match_client_owner_id(pos, new))
continue;
-
+found:
atomic_inc(&pos->cl_count);
*result = pos;
status = 0;
if (!cinfo->atomic || cinfo->before != dir->i_version)
nfs_force_lookup_revalidate(dir);
dir->i_version = cinfo->after;
+ nfsi->attr_gencount = nfs_inc_attr_generation_counter();
nfs_fscache_invalidate(dir);
spin_unlock(&dir->i_lock);
}
opendata->o_arg.open_flags = 0;
opendata->o_arg.fmode = fmode;
+ opendata->o_arg.share_access = nfs4_map_atomic_open_share(
+ NFS_SB(opendata->dentry->d_sb),
+ fmode, 0);
memset(&opendata->o_res, 0, sizeof(opendata->o_res));
memset(&opendata->c_res, 0, sizeof(opendata->c_res));
nfs4_init_opendata_res(opendata);
opendata->o_res.f_attr, sattr,
state, label, olabel);
if (status == 0) {
- nfs_setattr_update_inode(state->inode, sattr);
- nfs_post_op_update_inode(state->inode, opendata->o_res.f_attr);
+ nfs_setattr_update_inode(state->inode, sattr,
+ opendata->o_res.f_attr);
nfs_setsecurity(state->inode, opendata->o_res.f_attr, olabel);
}
}
case -NFS4ERR_BAD_STATEID:
case -NFS4ERR_EXPIRED:
if (!nfs4_stateid_match(&calldata->arg.stateid,
- &state->stateid)) {
+ &state->open_stateid)) {
rpc_restart_call_prepare(task);
goto out_release;
}
is_rdwr = test_bit(NFS_O_RDWR_STATE, &state->flags);
is_rdonly = test_bit(NFS_O_RDONLY_STATE, &state->flags);
is_wronly = test_bit(NFS_O_WRONLY_STATE, &state->flags);
- nfs4_stateid_copy(&calldata->arg.stateid, &state->stateid);
+ nfs4_stateid_copy(&calldata->arg.stateid, &state->open_stateid);
/* Calculate the change in open mode */
calldata->arg.fmode = 0;
if (state->n_rdwr == 0) {
status = nfs4_do_setattr(inode, cred, fattr, sattr, state, NULL, label);
if (status == 0) {
- nfs_setattr_update_inode(inode, sattr);
+ nfs_setattr_update_inode(inode, sattr, fattr);
nfs_setsecurity(inode, fattr, label);
}
nfs4_label_free(label);
}
if (task->tk_status >= 0) {
renew_lease(NFS_SERVER(inode), hdr->timestamp);
- nfs_post_op_update_inode_force_wcc(inode, &hdr->fattr);
+ nfs_writeback_update_inode(hdr);
}
return 0;
}
if (status == 0) {
clp->cl_clientid = res.clientid;
- clp->cl_exchange_flags = (res.flags & ~EXCHGID4_FLAG_CONFIRMED_R);
- if (!(res.flags & EXCHGID4_FLAG_CONFIRMED_R))
+ clp->cl_exchange_flags = res.flags;
+ /* Client ID is not confirmed */
+ if (!(res.flags & EXCHGID4_FLAG_CONFIRMED_R)) {
+ clear_bit(NFS4_SESSION_ESTABLISHED,
+ &clp->cl_session->session_state);
clp->cl_seqid = res.seqid;
+ }
kfree(clp->cl_serverowner);
clp->cl_serverowner = res.server_owner;
struct nfs41_create_session_res *res)
{
nfs4_copy_sessionid(&session->sess_id, &res->sessionid);
+ /* Mark client id and session as being confirmed */
+ session->clp->cl_exchange_flags |= EXCHGID4_FLAG_CONFIRMED_R;
+ set_bit(NFS4_SESSION_ESTABLISHED, &session->session_state);
session->flags = res->flags;
memcpy(&session->fc_attrs, &res->fc_attrs, sizeof(session->fc_attrs));
if (res->flags & SESSION4_BACK_CHAN)
dprintk("--> nfs4_proc_destroy_session\n");
/* session is still being setup */
- if (session->clp->cl_cons_state != NFS_CS_READY)
- return status;
+ if (!test_and_clear_bit(NFS4_SESSION_ESTABLISHED, &session->session_state))
+ return 0;
status = rpc_call_sync(session->clp->cl_rpcclient, &msg, RPC_TASK_TIMEOUT);
trace_nfs4_destroy_session(session->clp, status);
enum nfs4_session_state {
NFS4_SESSION_INITING,
+ NFS4_SESSION_ESTABLISHED,
};
extern int nfs4_setup_slot_table(struct nfs4_slot_table *tbl,
status = nfs4_proc_exchange_id(clp, cred);
if (status != NFS4_OK)
return status;
- set_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
- return nfs41_walk_client_list(clp, result, cred);
+ status = nfs41_walk_client_list(clp, result, cred);
+ if (status < 0)
+ return status;
+ if (clp != *result)
+ return 0;
+
+ /* Purge state if the client id was established in a prior instance */
+ if (clp->cl_exchange_flags & EXCHGID4_FLAG_CONFIRMED_R)
+ set_bit(NFS4CLNT_PURGE_STATE, &clp->cl_state);
+ else
+ set_bit(NFS4CLNT_LEASE_CONFIRM, &clp->cl_state);
+ nfs4_schedule_state_manager(clp);
+ status = nfs_wait_client_init_complete(clp);
+ if (status < 0)
+ nfs_put_client(clp);
+ return status;
}
#endif /* CONFIG_NFS_V4_1 */
nfs_fattr_init(fattr);
status = rpc_call_sync(NFS_CLIENT(inode), &msg, 0);
if (status == 0)
- nfs_setattr_update_inode(inode, sattr);
+ nfs_setattr_update_inode(inode, sattr, fattr);
dprintk("NFS reply setattr: %d\n", status);
return status;
}
static int nfs_write_done(struct rpc_task *task, struct nfs_pgio_header *hdr)
{
- struct inode *inode = hdr->inode;
-
if (task->tk_status >= 0)
- nfs_post_op_update_inode_force_wcc(inode, hdr->res.fattr);
+ nfs_writeback_update_inode(hdr);
return 0;
}
return 0;
}
+static void nfs_writeback_check_extend(struct nfs_pgio_header *hdr,
+ struct nfs_fattr *fattr)
+{
+ struct nfs_pgio_args *argp = &hdr->args;
+ struct nfs_pgio_res *resp = &hdr->res;
+
+ if (!(fattr->valid & NFS_ATTR_FATTR_SIZE))
+ return;
+ if (argp->offset + resp->count != fattr->size)
+ return;
+ if (nfs_size_to_loff_t(fattr->size) < i_size_read(hdr->inode))
+ return;
+ /* Set attribute barrier */
+ nfs_fattr_set_barrier(fattr);
+}
+
+void nfs_writeback_update_inode(struct nfs_pgio_header *hdr)
+{
+ struct nfs_fattr *fattr = hdr->res.fattr;
+ struct inode *inode = hdr->inode;
+
+ if (fattr == NULL)
+ return;
+ spin_lock(&inode->i_lock);
+ nfs_writeback_check_extend(hdr, fattr);
+ nfs_post_op_update_inode_force_wcc_locked(inode, fattr);
+ spin_unlock(&inode->i_lock);
+}
+EXPORT_SYMBOL_GPL(nfs_writeback_update_inode);
+
/*
* This function is called when the WRITE call is complete.
*/
seg->offset = iomap.offset;
seg->length = iomap.length;
- dprintk("GET: %lld:%lld %d\n", bex->foff, bex->len, bex->es);
+ dprintk("GET: 0x%llx:0x%llx %d\n", bex->foff, bex->len, bex->es);
return 0;
out_error:
p = xdr_decode_hyper(p, &bex.foff);
if (bex.foff & (block_size - 1)) {
- dprintk("%s: unaligned offset %lld\n",
+ dprintk("%s: unaligned offset 0x%llx\n",
__func__, bex.foff);
goto fail;
}
p = xdr_decode_hyper(p, &bex.len);
if (bex.len & (block_size - 1)) {
- dprintk("%s: unaligned length %lld\n",
+ dprintk("%s: unaligned length 0x%llx\n",
__func__, bex.foff);
goto fail;
}
p = xdr_decode_hyper(p, &bex.soff);
if (bex.soff & (block_size - 1)) {
- dprintk("%s: unaligned disk offset %lld\n",
+ dprintk("%s: unaligned disk offset 0x%llx\n",
__func__, bex.soff);
goto fail;
}
{
struct super_block *sb = exp->ex_path.mnt->mnt_sb;
- if (exp->ex_flags & NFSEXP_NOPNFS)
+ if (!(exp->ex_flags & NFSEXP_PNFS))
return;
if (sb->s_export_op->get_uuid &&
list_move_tail(&lp->lo_perstate, reaplist);
return;
}
- end = seg->offset;
+ lo->offset = layout_end(seg);
} else {
/* retain the whole layout segment on a split. */
if (layout_end(seg) < end) {
dprintk("%s: split not supported\n", __func__);
return;
}
-
- lo->offset = layout_end(seg);
+ end = seg->offset;
}
layout_update_len(lo, end);
spin_lock(&clp->cl_lock);
list_for_each_entry_safe(ls, n, &clp->cl_lo_states, ls_perclnt) {
+ if (ls->ls_layout_type != lrp->lr_layout_type)
+ continue;
+
if (lrp->lr_return_type == RETURN_FSID &&
!fh_fsid_match(&ls->ls_stid.sc_file->fi_fhandle,
&cstate->current_fh.fh_handle))
rpc_ntop((struct sockaddr *)&clp->cl_addr, addr_str, sizeof(addr_str));
- nfsd4_cb_layout_fail(ls);
+ trace_layout_recall_fail(&ls->ls_stid.sc_stateid);
printk(KERN_WARNING
"nfsd: client %s failed to respond to layout recall. "
nfserr = ops->proc_getdeviceinfo(exp->ex_path.mnt->mnt_sb, gdp);
gdp->gd_notify_types &= ops->notify_types;
- exp_put(exp);
out:
+ exp_put(exp);
return nfserr;
}
nfs4_put_stid(&dp->dl_stid);
}
while (!list_empty(&clp->cl_revoked)) {
- dp = list_entry(reaplist.next, struct nfs4_delegation, dl_recall_lru);
+ dp = list_entry(clp->cl_revoked.next, struct nfs4_delegation, dl_recall_lru);
list_del_init(&dp->dl_recall_lru);
nfs4_put_stid(&dp->dl_stid);
}
} else
nfs4_free_openowner(&oo->oo_owner);
spin_unlock(&clp->cl_lock);
- return oo;
+ return ret;
}
static void init_open_stateid(struct nfs4_ol_stateid *stp, struct nfs4_file *fp, struct nfsd4_open *open) {
} else
nfs4_free_lockowner(&lo->lo_owner);
spin_unlock(&clp->cl_lock);
- return lo;
+ return ret;
}
static void
p = xdr_decode_hyper(p, &lgp->lg_seg.offset);
p = xdr_decode_hyper(p, &lgp->lg_seg.length);
p = xdr_decode_hyper(p, &lgp->lg_minlength);
- nfsd4_decode_stateid(argp, &lgp->lg_sid);
+
+ status = nfsd4_decode_stateid(argp, &lgp->lg_sid);
+ if (status)
+ return status;
+
READ_BUF(4);
lgp->lg_maxcount = be32_to_cpup(p++);
p = xdr_decode_hyper(p, &lcp->lc_seg.offset);
p = xdr_decode_hyper(p, &lcp->lc_seg.length);
lcp->lc_reclaim = be32_to_cpup(p++);
- nfsd4_decode_stateid(argp, &lcp->lc_sid);
+
+ status = nfsd4_decode_stateid(argp, &lcp->lc_sid);
+ if (status)
+ return status;
+
READ_BUF(4);
lcp->lc_newoffset = be32_to_cpup(p++);
if (lcp->lc_newoffset) {
READ_BUF(16);
p = xdr_decode_hyper(p, &lrp->lr_seg.offset);
p = xdr_decode_hyper(p, &lrp->lr_seg.length);
- nfsd4_decode_stateid(argp, &lrp->lr_sid);
+
+ status = nfsd4_decode_stateid(argp, &lrp->lr_sid);
+ if (status)
+ return status;
+
READ_BUF(4);
lrp->lrf_body_len = be32_to_cpup(p++);
if (lrp->lrf_body_len > 0) {
return nfserr_resource;
*p++ = cpu_to_be32(lrp->lrs_present);
if (lrp->lrs_present)
- nfsd4_encode_stateid(xdr, &lrp->lr_sid);
+ return nfsd4_encode_stateid(xdr, &lrp->lr_sid);
return nfs_ok;
}
#endif /* CONFIG_NFSD_PNFS */
{
unsigned int hashsize;
unsigned int i;
+ int status = 0;
max_drc_entries = nfsd_cache_size_limit();
atomic_set(&num_drc_entries, 0);
hashsize = nfsd_hashsize(max_drc_entries);
maskbits = ilog2(hashsize);
- register_shrinker(&nfsd_reply_cache_shrinker);
+ status = register_shrinker(&nfsd_reply_cache_shrinker);
+ if (status)
+ return status;
+
drc_slab = kmem_cache_create("nfsd_drc", sizeof(struct svc_cacherep),
0, 0, NULL);
if (!drc_slab)
#include "alloc.h"
#include "dat.h"
+static void __nilfs_btree_init(struct nilfs_bmap *bmap);
+
static struct nilfs_btree_path *nilfs_btree_alloc_path(void)
{
struct nilfs_btree_path *path;
return ret;
}
+/**
+ * nilfs_btree_root_broken - verify consistency of btree root node
+ * @node: btree root node to be examined
+ * @ino: inode number
+ *
+ * Return Value: If node is broken, 1 is returned. Otherwise, 0 is returned.
+ */
+static int nilfs_btree_root_broken(const struct nilfs_btree_node *node,
+ unsigned long ino)
+{
+ int level, flags, nchildren;
+ int ret = 0;
+
+ level = nilfs_btree_node_get_level(node);
+ flags = nilfs_btree_node_get_flags(node);
+ nchildren = nilfs_btree_node_get_nchildren(node);
+
+ if (unlikely(level < NILFS_BTREE_LEVEL_NODE_MIN ||
+ level > NILFS_BTREE_LEVEL_MAX ||
+ nchildren < 0 ||
+ nchildren > NILFS_BTREE_ROOT_NCHILDREN_MAX)) {
+ pr_crit("NILFS: bad btree root (inode number=%lu): level = %d, flags = 0x%x, nchildren = %d\n",
+ ino, level, flags, nchildren);
+ ret = 1;
+ }
+ return ret;
+}
+
int nilfs_btree_broken_node_block(struct buffer_head *bh)
{
int ret;
/* convert and insert */
dat = NILFS_BMAP_USE_VBN(btree) ? nilfs_bmap_get_dat(btree) : NULL;
- nilfs_btree_init(btree);
+ __nilfs_btree_init(btree);
if (nreq != NULL) {
nilfs_bmap_commit_alloc_ptr(btree, dreq, dat);
nilfs_bmap_commit_alloc_ptr(btree, nreq, dat);
.bop_gather_data = NULL,
};
-int nilfs_btree_init(struct nilfs_bmap *bmap)
+static void __nilfs_btree_init(struct nilfs_bmap *bmap)
{
bmap->b_ops = &nilfs_btree_ops;
bmap->b_nchildren_per_block =
NILFS_BTREE_NODE_NCHILDREN_MAX(nilfs_btree_node_size(bmap));
- return 0;
+}
+
+int nilfs_btree_init(struct nilfs_bmap *bmap)
+{
+ int ret = 0;
+
+ __nilfs_btree_init(bmap);
+
+ if (nilfs_btree_root_broken(nilfs_btree_get_root(bmap),
+ bmap->b_inode->i_ino))
+ ret = -EIO;
+ return ret;
}
void nilfs_btree_init_gc(struct nilfs_bmap *bmap)
struct the_nilfs *nilfs)
{
struct nilfs_inode_info *ii, *n;
+ int during_mount = !(sci->sc_super->s_flags & MS_ACTIVE);
int defer_iput = false;
spin_lock(&nilfs->ns_inode_lock);
brelse(ii->i_bh);
ii->i_bh = NULL;
list_del_init(&ii->i_dirty);
- if (!ii->vfs_inode.i_nlink) {
+ if (!ii->vfs_inode.i_nlink || during_mount) {
/*
- * Defer calling iput() to avoid a deadlock
- * over I_SYNC flag for inodes with i_nlink == 0
+ * Defer calling iput() to avoid deadlocks if
+ * i_nlink == 0 or mount is not yet finished.
*/
list_add_tail(&ii->i_dirty, &sci->sc_iput_queue);
defer_iput = true;
!(marks_mask & FS_ISDIR & ~marks_ignored_mask))
return false;
- if (event_mask & marks_mask & ~marks_ignored_mask)
+ if (event_mask & FAN_ALL_OUTGOING_EVENTS & marks_mask &
+ ~marks_ignored_mask)
return true;
return false;
/*
* for completing the rest of the request.
*/
- *ppos += written;
count -= written;
written_buffered = generic_perform_write(file, from, *ppos);
/*
goto out_dio;
}
- iocb->ki_pos = *ppos + written_buffered;
/* We need to ensure that the page cache pages are written to
* disk and invalidated to preserve the expected O_DIRECT
* semantics.
ret = filemap_write_and_wait_range(file->f_mapping, *ppos,
endbyte);
if (ret == 0) {
+ iocb->ki_pos = *ppos + written_buffered;
written += written_buffered;
invalidate_mapping_pages(mapping,
*ppos >> PAGE_CACHE_SHIFT,
/* buffered aio wouldn't have proper lock coverage today */
BUG_ON(ret == -EIOCBQUEUED && !(file->f_flags & O_DIRECT));
+ if (unlikely(written <= 0))
+ goto no_sync;
+
if (((file->f_flags & O_DSYNC) && !direct_io) || IS_SYNC(inode) ||
((file->f_flags & O_DIRECT) && !direct_io)) {
- ret = filemap_fdatawrite_range(file->f_mapping, *ppos,
- *ppos + count - 1);
+ ret = filemap_fdatawrite_range(file->f_mapping,
+ iocb->ki_pos - written,
+ iocb->ki_pos - 1);
if (ret < 0)
written = ret;
}
if (!ret)
- ret = filemap_fdatawait_range(file->f_mapping, *ppos,
- *ppos + count - 1);
+ ret = filemap_fdatawait_range(file->f_mapping,
+ iocb->ki_pos - written,
+ iocb->ki_pos - 1);
}
+no_sync:
/*
* deep in g_f_a_w_n()->ocfs2_direct_IO we pass in a ocfs2_dio_end_io
* function pointer which is called when o_direct io completes so that
static inline int ocfs2_supports_append_dio(struct ocfs2_super *osb)
{
- if (osb->s_feature_ro_compat & OCFS2_FEATURE_RO_COMPAT_APPEND_DIO)
+ if (osb->s_feature_incompat & OCFS2_FEATURE_INCOMPAT_APPEND_DIO)
return 1;
return 0;
}
| OCFS2_FEATURE_INCOMPAT_INDEXED_DIRS \
| OCFS2_FEATURE_INCOMPAT_REFCOUNT_TREE \
| OCFS2_FEATURE_INCOMPAT_DISCONTIG_BG \
- | OCFS2_FEATURE_INCOMPAT_CLUSTERINFO)
+ | OCFS2_FEATURE_INCOMPAT_CLUSTERINFO \
+ | OCFS2_FEATURE_INCOMPAT_APPEND_DIO)
#define OCFS2_FEATURE_RO_COMPAT_SUPP (OCFS2_FEATURE_RO_COMPAT_UNWRITTEN \
| OCFS2_FEATURE_RO_COMPAT_USRQUOTA \
- | OCFS2_FEATURE_RO_COMPAT_GRPQUOTA \
- | OCFS2_FEATURE_RO_COMPAT_APPEND_DIO)
+ | OCFS2_FEATURE_RO_COMPAT_GRPQUOTA)
/*
* Heartbeat-only devices are missing journals and other files. The
*/
#define OCFS2_FEATURE_INCOMPAT_CLUSTERINFO 0x4000
+/*
+ * Append Direct IO support
+ */
+#define OCFS2_FEATURE_INCOMPAT_APPEND_DIO 0x8000
+
/*
* backup superblock flag is used to indicate that this volume
* has backup superblocks.
#define OCFS2_FEATURE_RO_COMPAT_USRQUOTA 0x0002
#define OCFS2_FEATURE_RO_COMPAT_GRPQUOTA 0x0004
-/*
- * Append Direct IO support
- */
-#define OCFS2_FEATURE_RO_COMPAT_APPEND_DIO 0x0008
/* The byte offset of the first backup block will be 1G.
* The following will be 4G, 16G, 64G, 256G and 1T.
{
struct ovl_fs *ufs = sb->s_fs_info;
- if (!(*flags & MS_RDONLY) &&
- (!ufs->upper_mnt || (ufs->upper_mnt->mnt_sb->s_flags & MS_RDONLY)))
+ if (!(*flags & MS_RDONLY) && !ufs->upper_mnt)
return -EROFS;
return 0;
break;
default:
+ pr_err("overlayfs: unrecognized mount option \"%s\" or missing value\n", p);
return -EINVAL;
}
}
+
+ /* Workdir is useless in non-upper mount */
+ if (!config->upperdir && config->workdir) {
+ pr_info("overlayfs: option \"workdir=%s\" is useless in a non-upper mount, ignore\n",
+ config->workdir);
+ kfree(config->workdir);
+ config->workdir = NULL;
+ }
+
return 0;
}
sb->s_stack_depth = 0;
if (ufs->config.upperdir) {
- /* FIXME: workdir is not needed for a R/O mount */
if (!ufs->config.workdir) {
pr_err("overlayfs: missing 'workdir'\n");
goto out_free_config;
if (err)
goto out_free_config;
+ /* Upper fs should not be r/o */
+ if (upperpath.mnt->mnt_sb->s_flags & MS_RDONLY) {
+ pr_err("overlayfs: upper fs is r/o, try multi-lower layers mount\n");
+ err = -EINVAL;
+ goto out_put_upperpath;
+ }
+
err = ovl_mount_dir(ufs->config.workdir, &workpath);
if (err)
goto out_put_upperpath;
err = -EINVAL;
stacklen = ovl_split_lowerdirs(lowertmp);
- if (stacklen > OVL_MAX_STACK)
+ if (stacklen > OVL_MAX_STACK) {
+ pr_err("overlayfs: too many lower directries, limit is %d\n",
+ OVL_MAX_STACK);
goto out_free_lowertmp;
+ } else if (!ufs->config.upperdir && stacklen == 1) {
+ pr_err("overlayfs: at least 2 lowerdir are needed while upperdir nonexistent\n");
+ goto out_free_lowertmp;
+ }
stack = kcalloc(stacklen, sizeof(struct path), GFP_KERNEL);
if (!stack)
ufs->numlower++;
}
- /* If the upper fs is r/o or nonexistent, we mark overlayfs r/o too */
- if (!ufs->upper_mnt || (ufs->upper_mnt->mnt_sb->s_flags & MS_RDONLY))
+ /* If the upper fs is nonexistent, we mark overlayfs r/o too */
+ if (!ufs->upper_mnt)
sb->s_flags |= MS_RDONLY;
sb->s_d_op = &ovl_dentry_operations;
static int pagemap_open(struct inode *inode, struct file *file)
{
+ /* do not disclose physical addresses: attack vector */
+ if (!capable(CAP_SYS_ADMIN))
+ return -EPERM;
pr_warn_once("Bits 55-60 of /proc/PID/pagemap entries are about "
"to stop being page-shift some time soon. See the "
"linux/Documentation/vm/pagemap.txt for details.\n");
xfs_zero_last_block(
struct xfs_inode *ip,
xfs_fsize_t offset,
- xfs_fsize_t isize)
+ xfs_fsize_t isize,
+ bool *did_zeroing)
{
struct xfs_mount *mp = ip->i_mount;
xfs_fileoff_t last_fsb = XFS_B_TO_FSBT(mp, isize);
zero_len = mp->m_sb.sb_blocksize - zero_offset;
if (isize + zero_len > offset)
zero_len = offset - isize;
+ *did_zeroing = true;
return xfs_iozero(ip, isize, zero_len);
}
xfs_zero_eof(
struct xfs_inode *ip,
xfs_off_t offset, /* starting I/O offset */
- xfs_fsize_t isize) /* current inode size */
+ xfs_fsize_t isize, /* current inode size */
+ bool *did_zeroing)
{
struct xfs_mount *mp = ip->i_mount;
xfs_fileoff_t start_zero_fsb;
* We only zero a part of that block so it is handled specially.
*/
if (XFS_B_FSB_OFFSET(mp, isize) != 0) {
- error = xfs_zero_last_block(ip, offset, isize);
+ error = xfs_zero_last_block(ip, offset, isize, did_zeroing);
if (error)
return error;
}
if (error)
return error;
+ *did_zeroing = true;
start_zero_fsb = imap.br_startoff + imap.br_blockcount;
ASSERT(start_zero_fsb <= (end_zero_fsb + 1));
}
* having to redo all checks before.
*/
if (*pos > i_size_read(inode)) {
+ bool zero = false;
+
if (*iolock == XFS_IOLOCK_SHARED) {
xfs_rw_iunlock(ip, *iolock);
*iolock = XFS_IOLOCK_EXCL;
xfs_rw_ilock(ip, *iolock);
goto restart;
}
- error = xfs_zero_eof(ip, *pos, i_size_read(inode));
+ error = xfs_zero_eof(ip, *pos, i_size_read(inode), &zero);
if (error)
return error;
}
* Handle RENAME_EXCHANGE flags
*/
if (flags & RENAME_EXCHANGE) {
+ if (target_ip == NULL) {
+ error = -EINVAL;
+ goto error_return;
+ }
error = xfs_cross_rename(tp, src_dp, src_name, src_ip,
target_dp, target_name, target_ip,
&free_list, &first_block, spaceres);
XFS_PREALLOC_INVISIBLE = (1 << 4),
};
-int xfs_update_prealloc_flags(struct xfs_inode *,
- enum xfs_prealloc_flags);
-int xfs_zero_eof(struct xfs_inode *, xfs_off_t, xfs_fsize_t);
-int xfs_iozero(struct xfs_inode *, loff_t, size_t);
+int xfs_update_prealloc_flags(struct xfs_inode *ip,
+ enum xfs_prealloc_flags flags);
+int xfs_zero_eof(struct xfs_inode *ip, xfs_off_t offset,
+ xfs_fsize_t isize, bool *did_zeroing);
+int xfs_iozero(struct xfs_inode *ip, loff_t pos, size_t count);
#define IHOLD(ip) \
int error;
uint lock_flags = 0;
uint commit_flags = 0;
+ bool did_zeroing = false;
trace_xfs_setattr(ip);
return error;
/*
- * Now we can make the changes. Before we join the inode to the
- * transaction, take care of the part of the truncation that must be
- * done without the inode lock. This needs to be done before joining
- * the inode to the transaction, because the inode cannot be unlocked
- * once it is a part of the transaction.
+ * File data changes must be complete before we start the transaction to
+ * modify the inode. This needs to be done before joining the inode to
+ * the transaction because the inode cannot be unlocked once it is a
+ * part of the transaction.
+ *
+ * Start with zeroing any data block beyond EOF that we may expose on
+ * file extension.
*/
if (newsize > oldsize) {
- /*
- * Do the first part of growing a file: zero any data in the
- * last block that is beyond the old EOF. We need to do this
- * before the inode is joined to the transaction to modify
- * i_size.
- */
- error = xfs_zero_eof(ip, newsize, oldsize);
+ error = xfs_zero_eof(ip, newsize, oldsize, &did_zeroing);
if (error)
return error;
}
* any previous writes that are beyond the on disk EOF and the new
* EOF that have not been written out need to be written here. If we
* do not write the data out, we expose ourselves to the null files
- * problem.
- *
- * Only flush from the on disk size to the smaller of the in memory
- * file size or the new size as that's the range we really care about
- * here and prevents waiting for other data not within the range we
- * care about here.
+ * problem. Note that this includes any block zeroing we did above;
+ * otherwise those blocks may not be zeroed after a crash.
*/
- if (oldsize != ip->i_d.di_size && newsize > ip->i_d.di_size) {
+ if (newsize > ip->i_d.di_size &&
+ (oldsize != ip->i_d.di_size || did_zeroing)) {
error = filemap_write_and_wait_range(VFS_I(ip)->i_mapping,
ip->i_d.di_size, newsize);
if (error)
return error;
}
- /*
- * Wait for all direct I/O to complete.
- */
+ /* Now wait for all direct I/O to complete. */
inode_dio_wait(inode);
/*
tp = xfs_trans_alloc(mp, XFS_TRANS_SETATTR_NOT_SIZE);
error = xfs_trans_reserve(tp, &M_RES(mp)->tr_ichange, 0, 0);
- if (error)
+ if (error) {
+ xfs_trans_cancel(tp, 0);
goto out_drop_iolock;
+ }
xfs_ilock(ip, XFS_ILOCK_EXCL);
xfs_trans_ijoin(tp, ip, XFS_ILOCK_EXCL);
*/
xfs_dqcheck(mp, ddq, id+j, type, XFS_QMOPT_DQREPAIR,
"xfs_quotacheck");
+ /*
+ * Reset type in case we are reusing group quota file for
+ * project quotas or vice versa
+ */
+ ddq->d_flags = type;
ddq->d_bcount = 0;
ddq->d_icount = 0;
ddq->d_rtbcount = 0;
unsigned scanned_preceeds_hole : 1;
unsigned allocated : 1;
unsigned long color;
- unsigned long start;
- unsigned long size;
+ u64 start;
+ u64 size;
struct drm_mm *mm;
};
unsigned int scan_check_range : 1;
unsigned scan_alignment;
unsigned long scan_color;
- unsigned long scan_size;
- unsigned long scan_hit_start;
- unsigned long scan_hit_end;
+ u64 scan_size;
+ u64 scan_hit_start;
+ u64 scan_hit_end;
unsigned scanned_blocks;
- unsigned long scan_start;
- unsigned long scan_end;
+ u64 scan_start;
+ u64 scan_end;
struct drm_mm_node *prev_scanned_node;
void (*color_adjust)(struct drm_mm_node *node, unsigned long color,
- unsigned long *start, unsigned long *end);
+ u64 *start, u64 *end);
};
/**
return mm->hole_stack.next;
}
-static inline unsigned long __drm_mm_hole_node_start(struct drm_mm_node *hole_node)
+static inline u64 __drm_mm_hole_node_start(struct drm_mm_node *hole_node)
{
return hole_node->start + hole_node->size;
}
* Returns:
* Start of the subsequent hole.
*/
-static inline unsigned long drm_mm_hole_node_start(struct drm_mm_node *hole_node)
+static inline u64 drm_mm_hole_node_start(struct drm_mm_node *hole_node)
{
BUG_ON(!hole_node->hole_follows);
return __drm_mm_hole_node_start(hole_node);
}
-static inline unsigned long __drm_mm_hole_node_end(struct drm_mm_node *hole_node)
+static inline u64 __drm_mm_hole_node_end(struct drm_mm_node *hole_node)
{
return list_entry(hole_node->node_list.next,
struct drm_mm_node, node_list)->start;
* Returns:
* End of the subsequent hole.
*/
-static inline unsigned long drm_mm_hole_node_end(struct drm_mm_node *hole_node)
+static inline u64 drm_mm_hole_node_end(struct drm_mm_node *hole_node)
{
return __drm_mm_hole_node_end(hole_node);
}
int drm_mm_insert_node_generic(struct drm_mm *mm,
struct drm_mm_node *node,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
enum drm_mm_search_flags sflags,
*/
static inline int drm_mm_insert_node(struct drm_mm *mm,
struct drm_mm_node *node,
- unsigned long size,
+ u64 size,
unsigned alignment,
enum drm_mm_search_flags flags)
{
int drm_mm_insert_node_in_range_generic(struct drm_mm *mm,
struct drm_mm_node *node,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
- unsigned long start,
- unsigned long end,
+ u64 start,
+ u64 end,
enum drm_mm_search_flags sflags,
enum drm_mm_allocator_flags aflags);
/**
*/
static inline int drm_mm_insert_node_in_range(struct drm_mm *mm,
struct drm_mm_node *node,
- unsigned long size,
+ u64 size,
unsigned alignment,
- unsigned long start,
- unsigned long end,
+ u64 start,
+ u64 end,
enum drm_mm_search_flags flags)
{
return drm_mm_insert_node_in_range_generic(mm, node, size, alignment,
void drm_mm_remove_node(struct drm_mm_node *node);
void drm_mm_replace_node(struct drm_mm_node *old, struct drm_mm_node *new);
void drm_mm_init(struct drm_mm *mm,
- unsigned long start,
- unsigned long size);
+ u64 start,
+ u64 size);
void drm_mm_takedown(struct drm_mm *mm);
bool drm_mm_clean(struct drm_mm *mm);
void drm_mm_init_scan(struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color);
void drm_mm_init_scan_with_range(struct drm_mm *mm,
- unsigned long size,
+ u64 size,
unsigned alignment,
unsigned long color,
- unsigned long start,
- unsigned long end);
+ u64 start,
+ u64 end);
bool drm_mm_scan_add_block(struct drm_mm_node *node);
bool drm_mm_scan_remove_block(struct drm_mm_node *node);
INTEL_VGA_DEVICE((((gt) - 1) << 4) | (id), info)
#define _INTEL_BDW_M_IDS(gt, info) \
- _INTEL_BDW_M(gt, 0x1602, info), /* ULT */ \
+ _INTEL_BDW_M(gt, 0x1602, info), /* Halo */ \
_INTEL_BDW_M(gt, 0x1606, info), /* ULT */ \
- _INTEL_BDW_M(gt, 0x160B, info), /* Iris */ \
+ _INTEL_BDW_M(gt, 0x160B, info), /* ULT */ \
_INTEL_BDW_M(gt, 0x160E, info) /* ULX */
#define _INTEL_BDW_D_IDS(gt, info) \
* either of these locks held.
*/
- unsigned long offset;
+ uint64_t offset; /* GPU address space is independent of CPU word size */
uint32_t cur_placement;
struct sg_table *sg;
bool has_type;
bool use_type;
uint32_t flags;
- unsigned long gpu_offset;
+ uint64_t gpu_offset; /* GPU address space is independent of CPU word size */
uint64_t size;
uint32_t available_caching;
uint32_t default_caching;
--- /dev/null
+/*
+ * include/linux/irqchip/irq-st.h
+ *
+ * Copyright (C) 2014 STMicroelectronics – All Rights Reserved
+ *
+ * Author: Lee Jones <lee.jones@linaro.org>
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License version 2 as
+ * published by the Free Software Foundation.
+ */
+
+#ifndef _DT_BINDINGS_INTERRUPT_CONTROLLER_ST_H
+#define _DT_BINDINGS_INTERRUPT_CONTROLLER_ST_H
+
+#define ST_IRQ_SYSCFG_EXT_0 0
+#define ST_IRQ_SYSCFG_EXT_1 1
+#define ST_IRQ_SYSCFG_EXT_2 2
+#define ST_IRQ_SYSCFG_CTI_0 3
+#define ST_IRQ_SYSCFG_CTI_1 4
+#define ST_IRQ_SYSCFG_PMU_0 5
+#define ST_IRQ_SYSCFG_PMU_1 6
+#define ST_IRQ_SYSCFG_pl310_L2 7
+#define ST_IRQ_SYSCFG_DISABLED 0xFFFFFFFF
+
+#define ST_IRQ_SYSCFG_EXT_1_INV 0x1
+#define ST_IRQ_SYSCFG_EXT_2_INV 0x2
+#define ST_IRQ_SYSCFG_EXT_3_INV 0x4
+
+#endif
#define PULL_DISABLE (1 << 3)
#define INPUT_EN (1 << 5)
-#define SLEWCTRL_FAST (1 << 6)
+#define SLEWCTRL_SLOW (1 << 6)
+#define SLEWCTRL_FAST 0
/* update macro depending on INPUT_EN and PULL_ENA */
#undef PIN_OUTPUT
#define PULL_DISABLE (1 << 16)
#define PULL_UP (1 << 17)
#define INPUT_EN (1 << 18)
-#define SLEWCTRL_FAST (1 << 19)
+#define SLEWCTRL_SLOW (1 << 19)
+#define SLEWCTRL_FAST 0
#define DS0_PULL_UP_DOWN_EN (1 << 27)
#define PIN_OUTPUT (PULL_DISABLE)
#include <linux/workqueue.h>
struct arch_timer_kvm {
-#ifdef CONFIG_KVM_ARM_TIMER
/* Is the timer enabled */
bool enabled;
/* Virtual offset */
cycle_t cntvoff;
-#endif
};
struct arch_timer_cpu {
-#ifdef CONFIG_KVM_ARM_TIMER
/* Registers: control register, timer value */
u32 cntv_ctl; /* Saved/restored */
cycle_t cntv_cval; /* Saved/restored */
/* Timer IRQ */
const struct kvm_irq_level *irq;
-#endif
};
-#ifdef CONFIG_KVM_ARM_TIMER
int kvm_timer_hyp_init(void);
void kvm_timer_enable(struct kvm *kvm);
void kvm_timer_init(struct kvm *kvm);
u64 kvm_arm_timer_get_reg(struct kvm_vcpu *, u64 regid);
int kvm_arm_timer_set_reg(struct kvm_vcpu *, u64 regid, u64 value);
-#else
-static inline int kvm_timer_hyp_init(void)
-{
- return 0;
-};
-
-static inline void kvm_timer_enable(struct kvm *kvm) {}
-static inline void kvm_timer_init(struct kvm *kvm) {}
-static inline void kvm_timer_vcpu_reset(struct kvm_vcpu *vcpu,
- const struct kvm_irq_level *irq) {}
-static inline void kvm_timer_vcpu_init(struct kvm_vcpu *vcpu) {}
-static inline void kvm_timer_flush_hwstate(struct kvm_vcpu *vcpu) {}
-static inline void kvm_timer_sync_hwstate(struct kvm_vcpu *vcpu) {}
-static inline void kvm_timer_vcpu_terminate(struct kvm_vcpu *vcpu) {}
-
-static inline int kvm_arm_timer_set_reg(struct kvm_vcpu *vcpu, u64 regid, u64 value)
-{
- return 0;
-}
-
-static inline u64 kvm_arm_timer_get_reg(struct kvm_vcpu *vcpu, u64 regid)
-{
- return 0;
-}
-#endif
+bool kvm_timer_should_fire(struct kvm_vcpu *vcpu);
#endif
#include <linux/irqreturn.h>
#include <linux/spinlock.h>
#include <linux/types.h>
+#include <kvm/iodev.h>
#define VGIC_NR_IRQS_LEGACY 256
#define VGIC_NR_SGIS 16
void (*sync_lr_elrsr)(struct kvm_vcpu *, int, struct vgic_lr);
u64 (*get_elrsr)(const struct kvm_vcpu *vcpu);
u64 (*get_eisr)(const struct kvm_vcpu *vcpu);
+ void (*clear_eisr)(struct kvm_vcpu *vcpu);
u32 (*get_interrupt_status)(const struct kvm_vcpu *vcpu);
void (*enable_underflow)(struct kvm_vcpu *vcpu);
void (*disable_underflow)(struct kvm_vcpu *vcpu);
};
struct vgic_vm_ops {
- bool (*handle_mmio)(struct kvm_vcpu *, struct kvm_run *,
- struct kvm_exit_mmio *);
bool (*queue_sgi)(struct kvm_vcpu *, int irq);
void (*add_sgi_source)(struct kvm_vcpu *, int irq, int source);
int (*init_model)(struct kvm *);
int (*map_resources)(struct kvm *, const struct vgic_params *);
};
+struct vgic_io_device {
+ gpa_t addr;
+ int len;
+ const struct vgic_io_range *reg_ranges;
+ struct kvm_vcpu *redist_vcpu;
+ struct kvm_io_device dev;
+};
+
struct vgic_dist {
-#ifdef CONFIG_KVM_ARM_VGIC
spinlock_t lock;
bool in_kernel;
bool ready;
/* Level-triggered interrupt queued on VCPU interface */
struct vgic_bitmap irq_queued;
+ /* Interrupt was active when unqueue from VCPU interface */
+ struct vgic_bitmap irq_active;
+
/* Interrupt priority. Not used yet. */
struct vgic_bytemap irq_priority;
/* Bitmap indicating which CPU has something pending */
unsigned long *irq_pending_on_cpu;
+ /* Bitmap indicating which CPU has active IRQs */
+ unsigned long *irq_active_on_cpu;
+
struct vgic_vm_ops vm_ops;
-#endif
+ struct vgic_io_device dist_iodev;
+ struct vgic_io_device *redist_iodevs;
};
struct vgic_v2_cpu_if {
};
struct vgic_cpu {
-#ifdef CONFIG_KVM_ARM_VGIC
/* per IRQ to LR mapping */
u8 *vgic_irq_lr_map;
- /* Pending interrupts on this VCPU */
+ /* Pending/active/both interrupts on this VCPU */
DECLARE_BITMAP( pending_percpu, VGIC_NR_PRIVATE_IRQS);
+ DECLARE_BITMAP( active_percpu, VGIC_NR_PRIVATE_IRQS);
+ DECLARE_BITMAP( pend_act_percpu, VGIC_NR_PRIVATE_IRQS);
+
+ /* Pending/active/both shared interrupts, dynamically sized */
unsigned long *pending_shared;
+ unsigned long *active_shared;
+ unsigned long *pend_act_shared;
/* Bitmap of used/free list registers */
DECLARE_BITMAP( lr_used, VGIC_V2_MAX_LRS);
struct vgic_v2_cpu_if vgic_v2;
struct vgic_v3_cpu_if vgic_v3;
};
-#endif
};
#define LR_EMPTY 0xff
struct kvm;
struct kvm_vcpu;
-struct kvm_run;
-struct kvm_exit_mmio;
-#ifdef CONFIG_KVM_ARM_VGIC
int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write);
int kvm_vgic_hyp_init(void);
int kvm_vgic_map_resources(struct kvm *kvm);
bool level);
void vgic_v3_dispatch_sgi(struct kvm_vcpu *vcpu, u64 reg);
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu);
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio);
+int kvm_vgic_vcpu_active_irq(struct kvm_vcpu *vcpu);
#define irqchip_in_kernel(k) (!!((k)->arch.vgic.in_kernel))
#define vgic_initialized(k) (!!((k)->arch.vgic.nr_cpus))
}
#endif
-#else
-static inline int kvm_vgic_hyp_init(void)
-{
- return 0;
-}
-
-static inline int kvm_vgic_set_addr(struct kvm *kvm, unsigned long type, u64 addr)
-{
- return 0;
-}
-
-static inline int kvm_vgic_addr(struct kvm *kvm, unsigned long type, u64 *addr, bool write)
-{
- return -ENXIO;
-}
-
-static inline int kvm_vgic_map_resources(struct kvm *kvm)
-{
- return 0;
-}
-
-static inline int kvm_vgic_create(struct kvm *kvm, u32 type)
-{
- return 0;
-}
-
-static inline void kvm_vgic_destroy(struct kvm *kvm)
-{
-}
-
-static inline void kvm_vgic_vcpu_destroy(struct kvm_vcpu *vcpu)
-{
-}
-
-static inline int kvm_vgic_vcpu_init(struct kvm_vcpu *vcpu)
-{
- return 0;
-}
-
-static inline void kvm_vgic_flush_hwstate(struct kvm_vcpu *vcpu) {}
-static inline void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu) {}
-
-static inline int kvm_vgic_inject_irq(struct kvm *kvm, int cpuid,
- unsigned int irq_num, bool level)
-{
- return 0;
-}
-
-static inline int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
-{
- return 0;
-}
-
-static inline bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
-{
- return false;
-}
-
-static inline int irqchip_in_kernel(struct kvm *kvm)
-{
- return 0;
-}
-
-static inline bool vgic_initialized(struct kvm *kvm)
-{
- return true;
-}
-
-static inline bool vgic_ready(struct kvm *kvm)
-{
- return true;
-}
-
-static inline int kvm_vgic_get_max_vcpus(void)
-{
- return KVM_MAX_VCPUS;
-}
-#endif
-
#endif
* GNU General Public License for more details.
*
* You should have received a copy of the GNU General Public License
- * along with this program; if not, write to the Free Software
- * Foundation, 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301, USA.
+ * along with this program. If not, see <http://www.gnu.org/licenses/>.
*/
#ifndef __KVM_IODEV_H__
#define __KVM_IODEV_H__
#include <linux/kvm_types.h>
-#include <asm/errno.h>
+#include <linux/errno.h>
struct kvm_io_device;
+struct kvm_vcpu;
/**
* kvm_io_device_ops are called under kvm slots_lock.
* or non-zero to have it passed to the next device.
**/
struct kvm_io_device_ops {
- int (*read)(struct kvm_io_device *this,
+ int (*read)(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr,
int len,
void *val);
- int (*write)(struct kvm_io_device *this,
+ int (*write)(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
gpa_t addr,
int len,
const void *val);
dev->ops = ops;
}
-static inline int kvm_iodevice_read(struct kvm_io_device *dev,
- gpa_t addr, int l, void *v)
+static inline int kvm_iodevice_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *dev, gpa_t addr,
+ int l, void *v)
{
- return dev->ops->read ? dev->ops->read(dev, addr, l, v) : -EOPNOTSUPP;
+ return dev->ops->read ? dev->ops->read(vcpu, dev, addr, l, v)
+ : -EOPNOTSUPP;
}
-static inline int kvm_iodevice_write(struct kvm_io_device *dev,
- gpa_t addr, int l, const void *v)
+static inline int kvm_iodevice_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *dev, gpa_t addr,
+ int l, const void *v)
{
- return dev->ops->write ? dev->ops->write(dev, addr, l, v) : -EOPNOTSUPP;
+ return dev->ops->write ? dev->ops->write(vcpu, dev, addr, l, v)
+ : -EOPNOTSUPP;
}
static inline void kvm_iodevice_destructor(struct kvm_io_device *dev)
ATA_ID_SECTOR_SIZE = 106,
ATA_ID_WWN = 108,
ATA_ID_LOGICAL_SECTOR_SIZE = 117, /* and 118 */
+ ATA_ID_COMMAND_SET_3 = 119,
+ ATA_ID_COMMAND_SET_4 = 120,
ATA_ID_LAST_LUN = 126,
ATA_ID_DLF = 128,
ATA_ID_CSFO = 129,
ATA_DSC = (1 << 4), /* drive seek complete */
ATA_DRQ = (1 << 3), /* data request i/o */
ATA_CORR = (1 << 2), /* corrected data error */
- ATA_IDX = (1 << 1), /* index */
+ ATA_SENSE = (1 << 1), /* sense code available */
ATA_ERR = (1 << 0), /* have an error */
ATA_SRST = (1 << 2), /* software reset */
ATA_ICRC = (1 << 7), /* interface CRC error */
SATA_SSP = 0x06, /* Software Settings Preservation */
SATA_DEVSLP = 0x09, /* Device Sleep */
+ SETFEATURE_SENSE_DATA = 0xC3, /* Sense Data Reporting feature */
+
/* feature values for SET_MAX */
ATA_SET_MAX_ADDR = 0x00,
ATA_SET_MAX_PASSWD = 0x01,
#define ata_id_cdb_intr(id) (((id)[ATA_ID_CONFIG] & 0x60) == 0x20)
#define ata_id_has_da(id) ((id)[ATA_ID_SATA_CAPABILITY_2] & (1 << 4))
#define ata_id_has_devslp(id) ((id)[ATA_ID_FEATURE_SUPP] & (1 << 8))
+#define ata_id_has_ncq_autosense(id) \
+ ((id)[ATA_ID_FEATURE_SUPP] & (1 << 7))
static inline bool ata_id_has_hipm(const u16 *id)
{
return id[ATA_ID_CFS_ENABLE_1] & (1 << 5);
}
+static inline bool ata_id_has_read_log_dma_ext(const u16 *id)
+{
+ if (!(id[ATA_ID_CFS_ENABLE_2] & (1 << 15)))
+ return false;
+ return id[ATA_ID_COMMAND_SET_3] & (1 << 3);
+}
+
+static inline bool ata_id_has_sense_reporting(const u16 *id)
+{
+ if (!(id[ATA_ID_CFS_ENABLE_2] & (1 << 15)))
+ return false;
+ return id[ATA_ID_COMMAND_SET_3] & (1 << 6);
+}
+
+static inline bool ata_id_sense_reporting_enabled(const u16 *id)
+{
+ if (!(id[ATA_ID_CFS_ENABLE_2] & (1 << 15)))
+ return false;
+ return id[ATA_ID_COMMAND_SET_4] & (1 << 6);
+}
+
/**
* ata_id_major_version - get ATA level of drive
* @id: Identify data
__REQ_ELVPRIV, /* elevator private data attached */
__REQ_FAILED, /* set if the request failed */
__REQ_QUIET, /* don't worry about errors */
- __REQ_PREEMPT, /* set for "ide_preempt" requests */
+ __REQ_PREEMPT, /* set for "ide_preempt" requests and also
+ for requests for which the SCSI "quiesce"
+ state must be ignored. */
__REQ_ALLOCED, /* request came from our alloc pool */
__REQ_COPY_USER, /* contains copies of user pages */
__REQ_FLUSH_SEQ, /* request for flush sequence */
*/
int clk_get_phase(struct clk *clk);
+/**
+ * clk_is_match - check if two clk's point to the same hardware clock
+ * @p: clk compared against q
+ * @q: clk compared against p
+ *
+ * Returns true if the two struct clk pointers both point to the same hardware
+ * clock node. Put differently, returns true if struct clk *p and struct clk *q
+ * share the same struct clk_core object.
+ *
+ * Returns false otherwise. Note that two NULL clks are treated as matching.
+ */
+bool clk_is_match(const struct clk *p, const struct clk *q);
+
#else
static inline long clk_get_accuracy(struct clk *clk)
return -ENOTSUPP;
}
+static inline bool clk_is_match(const struct clk *p, const struct clk *q)
+{
+ return p == q;
+}
+
#endif
/**
#ifndef _LINUX_CLOCKCHIPS_H
#define _LINUX_CLOCKCHIPS_H
-/* Clock event notification values */
-enum clock_event_nofitiers {
- CLOCK_EVT_NOTIFY_ADD,
- CLOCK_EVT_NOTIFY_BROADCAST_ON,
- CLOCK_EVT_NOTIFY_BROADCAST_OFF,
- CLOCK_EVT_NOTIFY_BROADCAST_FORCE,
- CLOCK_EVT_NOTIFY_BROADCAST_ENTER,
- CLOCK_EVT_NOTIFY_BROADCAST_EXIT,
- CLOCK_EVT_NOTIFY_SUSPEND,
- CLOCK_EVT_NOTIFY_RESUME,
- CLOCK_EVT_NOTIFY_CPU_DYING,
- CLOCK_EVT_NOTIFY_CPU_DEAD,
-};
-
-#ifdef CONFIG_GENERIC_CLOCKEVENTS_BUILD
+#ifdef CONFIG_GENERIC_CLOCKEVENTS
-#include <linux/clocksource.h>
-#include <linux/cpumask.h>
-#include <linux/ktime.h>
-#include <linux/notifier.h>
+# include <linux/clocksource.h>
+# include <linux/cpumask.h>
+# include <linux/ktime.h>
+# include <linux/notifier.h>
struct clock_event_device;
struct module;
-/* Clock event mode commands */
+/* Clock event mode commands for legacy ->set_mode(): OBSOLETE */
enum clock_event_mode {
- CLOCK_EVT_MODE_UNUSED = 0,
+ CLOCK_EVT_MODE_UNUSED,
CLOCK_EVT_MODE_SHUTDOWN,
CLOCK_EVT_MODE_PERIODIC,
CLOCK_EVT_MODE_ONESHOT,
CLOCK_EVT_MODE_RESUME,
};
+/*
+ * Possible states of a clock event device.
+ *
+ * DETACHED: Device is not used by clockevents core. Initial state or can be
+ * reached from SHUTDOWN.
+ * SHUTDOWN: Device is powered-off. Can be reached from PERIODIC or ONESHOT.
+ * PERIODIC: Device is programmed to generate events periodically. Can be
+ * reached from DETACHED or SHUTDOWN.
+ * ONESHOT: Device is programmed to generate event only once. Can be reached
+ * from DETACHED or SHUTDOWN.
+ */
+enum clock_event_state {
+ CLOCK_EVT_STATE_DETACHED,
+ CLOCK_EVT_STATE_SHUTDOWN,
+ CLOCK_EVT_STATE_PERIODIC,
+ CLOCK_EVT_STATE_ONESHOT,
+};
+
/*
* Clock event features
*/
-#define CLOCK_EVT_FEAT_PERIODIC 0x000001
-#define CLOCK_EVT_FEAT_ONESHOT 0x000002
-#define CLOCK_EVT_FEAT_KTIME 0x000004
+# define CLOCK_EVT_FEAT_PERIODIC 0x000001
+# define CLOCK_EVT_FEAT_ONESHOT 0x000002
+# define CLOCK_EVT_FEAT_KTIME 0x000004
+
/*
- * x86(64) specific misfeatures:
+ * x86(64) specific (mis)features:
*
* - Clockevent source stops in C3 State and needs broadcast support.
* - Local APIC timer is used as a dummy device.
*/
-#define CLOCK_EVT_FEAT_C3STOP 0x000008
-#define CLOCK_EVT_FEAT_DUMMY 0x000010
+# define CLOCK_EVT_FEAT_C3STOP 0x000008
+# define CLOCK_EVT_FEAT_DUMMY 0x000010
/*
* Core shall set the interrupt affinity dynamically in broadcast mode
*/
-#define CLOCK_EVT_FEAT_DYNIRQ 0x000020
-#define CLOCK_EVT_FEAT_PERCPU 0x000040
+# define CLOCK_EVT_FEAT_DYNIRQ 0x000020
+# define CLOCK_EVT_FEAT_PERCPU 0x000040
/*
* Clockevent device is based on a hrtimer for broadcast
*/
-#define CLOCK_EVT_FEAT_HRTIMER 0x000080
+# define CLOCK_EVT_FEAT_HRTIMER 0x000080
/**
* struct clock_event_device - clock event device descriptor
* @min_delta_ns: minimum delta value in ns
* @mult: nanosecond to cycles multiplier
* @shift: nanoseconds to cycles divisor (power of two)
- * @mode: operating mode assigned by the management code
+ * @mode: operating mode, relevant only to ->set_mode(), OBSOLETE
+ * @state: current state of the device, assigned by the core code
* @features: features
* @retries: number of forced programming retries
- * @set_mode: set mode function
+ * @set_mode: legacy set mode function, only for modes <= CLOCK_EVT_MODE_RESUME.
+ * @set_state_periodic: switch state to periodic, if !set_mode
+ * @set_state_oneshot: switch state to oneshot, if !set_mode
+ * @set_state_shutdown: switch state to shutdown, if !set_mode
+ * @tick_resume: resume clkevt device, if !set_mode
* @broadcast: function to broadcast events
* @min_delta_ticks: minimum delta value in ticks stored for reconfiguration
* @max_delta_ticks: maximum delta value in ticks stored for reconfiguration
*/
struct clock_event_device {
void (*event_handler)(struct clock_event_device *);
- int (*set_next_event)(unsigned long evt,
- struct clock_event_device *);
- int (*set_next_ktime)(ktime_t expires,
- struct clock_event_device *);
+ int (*set_next_event)(unsigned long evt, struct clock_event_device *);
+ int (*set_next_ktime)(ktime_t expires, struct clock_event_device *);
ktime_t next_event;
u64 max_delta_ns;
u64 min_delta_ns;
u32 mult;
u32 shift;
enum clock_event_mode mode;
+ enum clock_event_state state;
unsigned int features;
unsigned long retries;
+ /*
+ * State transition callback(s): Only one of the two groups should be
+ * defined:
+ * - set_mode(), only for modes <= CLOCK_EVT_MODE_RESUME.
+ * - set_state_{shutdown|periodic|oneshot}(), tick_resume().
+ */
+ void (*set_mode)(enum clock_event_mode mode, struct clock_event_device *);
+ int (*set_state_periodic)(struct clock_event_device *);
+ int (*set_state_oneshot)(struct clock_event_device *);
+ int (*set_state_shutdown)(struct clock_event_device *);
+ int (*tick_resume)(struct clock_event_device *);
+
void (*broadcast)(const struct cpumask *mask);
- void (*set_mode)(enum clock_event_mode mode,
- struct clock_event_device *);
void (*suspend)(struct clock_event_device *);
void (*resume)(struct clock_event_device *);
unsigned long min_delta_ticks;
*
* factor = (clock_ticks << shift) / nanoseconds
*/
-static inline unsigned long div_sc(unsigned long ticks, unsigned long nsec,
- int shift)
+static inline unsigned long
+div_sc(unsigned long ticks, unsigned long nsec, int shift)
{
- uint64_t tmp = ((uint64_t)ticks) << shift;
+ u64 tmp = ((u64)ticks) << shift;
do_div(tmp, nsec);
+
return (unsigned long) tmp;
}
/* Clock event layer functions */
-extern u64 clockevent_delta2ns(unsigned long latch,
- struct clock_event_device *evt);
+extern u64 clockevent_delta2ns(unsigned long latch, struct clock_event_device *evt);
extern void clockevents_register_device(struct clock_event_device *dev);
extern int clockevents_unbind_device(struct clock_event_device *ced, int cpu);
extern int clockevents_update_freq(struct clock_event_device *ce, u32 freq);
-extern void clockevents_exchange_device(struct clock_event_device *old,
- struct clock_event_device *new);
-extern void clockevents_set_mode(struct clock_event_device *dev,
- enum clock_event_mode mode);
-extern int clockevents_program_event(struct clock_event_device *dev,
- ktime_t expires, bool force);
-
-extern void clockevents_handle_noop(struct clock_event_device *dev);
-
static inline void
clockevents_calc_mult_shift(struct clock_event_device *ce, u32 freq, u32 minsec)
{
- return clocks_calc_mult_shift(&ce->mult, &ce->shift, NSEC_PER_SEC,
- freq, minsec);
+ return clocks_calc_mult_shift(&ce->mult, &ce->shift, NSEC_PER_SEC, freq, minsec);
}
extern void clockevents_suspend(void);
extern void clockevents_resume(void);
-#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
-#ifdef CONFIG_ARCH_HAS_TICK_BROADCAST
+# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
+# ifdef CONFIG_ARCH_HAS_TICK_BROADCAST
extern void tick_broadcast(const struct cpumask *mask);
-#else
-#define tick_broadcast NULL
-#endif
+# else
+# define tick_broadcast NULL
+# endif
extern int tick_receive_broadcast(void);
-#endif
+# endif
-#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT)
+# if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT)
extern void tick_setup_hrtimer_broadcast(void);
extern int tick_check_broadcast_expired(void);
-#else
+# else
static inline int tick_check_broadcast_expired(void) { return 0; }
-static inline void tick_setup_hrtimer_broadcast(void) {};
-#endif
+static inline void tick_setup_hrtimer_broadcast(void) { }
+# endif
-#ifdef CONFIG_GENERIC_CLOCKEVENTS
extern int clockevents_notify(unsigned long reason, void *arg);
-#else
-static inline int clockevents_notify(unsigned long reason, void *arg) { return 0; }
-#endif
-
-#else /* CONFIG_GENERIC_CLOCKEVENTS_BUILD */
-static inline void clockevents_suspend(void) {}
-static inline void clockevents_resume(void) {}
+#else /* !CONFIG_GENERIC_CLOCKEVENTS: */
+static inline void clockevents_suspend(void) { }
+static inline void clockevents_resume(void) { }
static inline int clockevents_notify(unsigned long reason, void *arg) { return 0; }
static inline int tick_check_broadcast_expired(void) { return 0; }
-static inline void tick_setup_hrtimer_broadcast(void) {};
+static inline void tick_setup_hrtimer_broadcast(void) { }
-#endif
+#endif /* !CONFIG_GENERIC_CLOCKEVENTS */
-#endif
+#endif /* _LINUX_CLOCKCHIPS_H */
* @shift: cycle to nanosecond divisor (power of two)
* @max_idle_ns: max idle time permitted by the clocksource (nsecs)
* @maxadj: maximum adjustment value to mult (~11%)
+ * @max_cycles: maximum safe cycle value which won't overflow on multiplication
* @flags: flags describing special properties
* @archdata: arch-specific data
* @suspend: suspend function for the clocksource, if necessary
#ifdef CONFIG_ARCH_CLOCKSOURCE_DATA
struct arch_clocksource_data archdata;
#endif
-
+ u64 max_cycles;
const char *name;
struct list_head list;
int rating;
}
-extern int clocksource_register(struct clocksource*);
extern int clocksource_unregister(struct clocksource*);
extern void clocksource_touch_watchdog(void);
extern struct clocksource* clocksource_get_next(void);
extern void clocksource_mark_unstable(struct clocksource *cs);
extern u64
-clocks_calc_max_nsecs(u32 mult, u32 shift, u32 maxadj, u64 mask);
+clocks_calc_max_nsecs(u32 mult, u32 shift, u32 maxadj, u64 mask, u64 *max_cycles);
extern void
clocks_calc_mult_shift(u32 *mult, u32 *shift, u32 from, u32 to, u32 minsec);
extern int
__clocksource_register_scale(struct clocksource *cs, u32 scale, u32 freq);
extern void
-__clocksource_updatefreq_scale(struct clocksource *cs, u32 scale, u32 freq);
+__clocksource_update_freq_scale(struct clocksource *cs, u32 scale, u32 freq);
+
+/*
+ * Don't call this unless you are a default clocksource
+ * (AKA: jiffies) and absolutely have to.
+ */
+static inline int __clocksource_register(struct clocksource *cs)
+{
+ return __clocksource_register_scale(cs, 1, 0);
+}
static inline int clocksource_register_hz(struct clocksource *cs, u32 hz)
{
return __clocksource_register_scale(cs, 1000, khz);
}
-static inline void __clocksource_updatefreq_hz(struct clocksource *cs, u32 hz)
+static inline void __clocksource_update_freq_hz(struct clocksource *cs, u32 hz)
{
- __clocksource_updatefreq_scale(cs, 1, hz);
+ __clocksource_update_freq_scale(cs, 1, hz);
}
-static inline void __clocksource_updatefreq_khz(struct clocksource *cs, u32 khz)
+static inline void __clocksource_update_freq_khz(struct clocksource *cs, u32 khz)
{
- __clocksource_updatefreq_scale(cs, 1000, khz);
+ __clocksource_update_freq_scale(cs, 1000, khz);
}
#include <uapi/linux/types.h>
-static __always_inline void data_access_exceeds_word_size(void)
-#ifdef __compiletime_warning
-__compiletime_warning("data access exceeds word size and won't be atomic")
-#endif
-;
-
-static __always_inline void data_access_exceeds_word_size(void)
-{
-}
-
static __always_inline void __read_once_size(const volatile void *p, void *res, int size)
{
switch (size) {
case 1: *(__u8 *)res = *(volatile __u8 *)p; break;
case 2: *(__u16 *)res = *(volatile __u16 *)p; break;
case 4: *(__u32 *)res = *(volatile __u32 *)p; break;
-#ifdef CONFIG_64BIT
case 8: *(__u64 *)res = *(volatile __u64 *)p; break;
-#endif
default:
barrier();
__builtin_memcpy((void *)res, (const void *)p, size);
- data_access_exceeds_word_size();
barrier();
}
}
case 1: *(volatile __u8 *)p = *(__u8 *)res; break;
case 2: *(volatile __u16 *)p = *(__u16 *)res; break;
case 4: *(volatile __u32 *)p = *(__u32 *)res; break;
-#ifdef CONFIG_64BIT
case 8: *(volatile __u64 *)p = *(__u64 *)res; break;
-#endif
default:
barrier();
__builtin_memcpy((void *)p, (const void *)res, size);
- data_access_exceeds_word_size();
barrier();
}
}
unsigned int cpu;
int last_residency;
- int state_count;
struct cpuidle_state_usage states_usage[CPUIDLE_STATE_MAX];
struct cpuidle_state_kobj *kobjs[CPUIDLE_STATE_MAX];
struct cpuidle_driver_kobj *kobj_driver;
#ifdef CONFIG_CPU_IDLE
extern void disable_cpuidle(void);
+extern bool cpuidle_not_available(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev);
extern int cpuidle_select(struct cpuidle_driver *drv,
struct cpuidle_device *dev);
extern int cpuidle_enable_device(struct cpuidle_device *dev);
extern void cpuidle_disable_device(struct cpuidle_device *dev);
extern int cpuidle_play_dead(void);
-extern void cpuidle_enter_freeze(void);
+extern int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev);
+extern int cpuidle_enter_freeze(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev);
extern struct cpuidle_driver *cpuidle_get_cpu_driver(struct cpuidle_device *dev);
#else
static inline void disable_cpuidle(void) { }
+static inline bool cpuidle_not_available(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev)
+{return true; }
static inline int cpuidle_select(struct cpuidle_driver *drv,
struct cpuidle_device *dev)
{return -ENODEV; }
{return -ENODEV; }
static inline void cpuidle_disable_device(struct cpuidle_device *dev) { }
static inline int cpuidle_play_dead(void) {return -ENODEV; }
-static inline void cpuidle_enter_freeze(void) { }
+static inline int cpuidle_find_deepest_state(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev)
+{return -ENODEV; }
+static inline int cpuidle_enter_freeze(struct cpuidle_driver *drv,
+ struct cpuidle_device *dev)
+{return -ENODEV; }
static inline struct cpuidle_driver *cpuidle_get_cpu_driver(
struct cpuidle_device *dev) {return NULL; }
#endif
*/
struct mapped_device *dm_get_md(dev_t dev);
void dm_get(struct mapped_device *md);
+int dm_hold(struct mapped_device *md);
void dm_put(struct mapped_device *md);
/*
#include <asm/io.h>
#include <asm/scatterlist.h>
+struct device;
+
struct dma_pool *dma_pool_create(const char *name, struct device *dev,
size_t size, size_t align, size_t allocation);
#define EFI_64BIT 5 /* Is the firmware 64-bit? */
#define EFI_PARAVIRT 6 /* Access is via a paravirt interface */
#define EFI_ARCH_1 7 /* First arch-specific bit */
+#define EFI_DBG 8 /* Print additional debug info at runtime */
#ifdef CONFIG_EFI
/*
struct mutex i_mutex;
unsigned long dirtied_when; /* jiffies of first dirtying */
+ unsigned long dirtied_time_when;
struct hlist_node i_hash;
struct list_head i_wb_list; /* backing dev IO list */
long (*unlocked_ioctl) (struct file *, unsigned int, unsigned long);
long (*compat_ioctl) (struct file *, unsigned int, unsigned long);
int (*mmap) (struct file *, struct vm_area_struct *);
- void (*mremap)(struct file *, struct vm_area_struct *);
+ int (*mremap)(struct file *, struct vm_area_struct *);
int (*open) (struct inode *, struct file *);
int (*flush) (struct file *, fl_owner_t id);
int (*release) (struct inode *, struct file *);
extern void synchronize_irq(unsigned int irq);
-extern void synchronize_hardirq(unsigned int irq);
+extern bool synchronize_hardirq(unsigned int irq);
#if defined(CONFIG_TINY_RCU)
* @units: Measurment unit for this attribute.
* @unit_expo: Exponent used in the data.
* @size: Size in bytes for data size.
+ * @logical_minimum: Logical minimum value for this attribute.
+ * @logical_maximum: Logical maximum value for this attribute.
*/
struct hid_sensor_hub_attribute_info {
u32 usage_id;
/**
* sensor_hub_input_attr_get_raw_value() - Synchronous read request
+* @hsdev: Hub device instance.
* @usage_id: Attribute usage id of parent physical device as per spec
* @attr_usage_id: Attribute usage id as per spec
* @report_id: Report id to look for
u32 attr_usage_id, u32 report_id);
/**
* sensor_hub_set_feature() - Feature set request
+* @hsdev: Hub device instance.
* @report_id: Report id to look for
* @field_index: Field index inside a report
* @value: Value to set
/**
* sensor_hub_get_feature() - Feature get request
+* @hsdev: Hub device instance.
* @report_id: Report id to look for
* @field_index: Field index inside a report
* @value: Place holder for return value
#ifdef CONFIG_HID_BATTERY_STRENGTH
/*
* Power supply information for HID devices which report
- * battery strength. power_supply is registered iff
- * battery.name is non-NULL.
+ * battery strength. power_supply was successfully registered if
+ * battery is non-NULL.
*/
- struct power_supply battery;
+ struct power_supply *battery;
__s32 battery_min;
__s32 battery_max;
__s32 battery_report_type;
* obs_kernel_param "array" too far apart in .init.setup.
*/
#define __setup_param(str, unique_id, fn, early) \
- static const char __setup_str_##unique_id[] __initconst \
- __aligned(1) = str; \
- static struct obs_kernel_param __setup_##unique_id \
- __used __section(.init.setup) \
- __attribute__((aligned((sizeof(long))))) \
+ static const char __setup_str_##unique_id[] __initconst \
+ __aligned(1) = str; \
+ static struct obs_kernel_param __setup_##unique_id \
+ __used __section(.init.setup) \
+ __attribute__((aligned((sizeof(long))))) \
= { __setup_str_##unique_id, fn, early }
-#define __setup(str, fn) \
+#define __setup(str, fn) \
__setup_param(str, fn, fn, 0)
-/* NOTE: fn is as per module_param, not __setup! Emits warning if fn
- * returns non-zero. */
-#define early_param(str, fn) \
+/*
+ * NOTE: fn is as per module_param, not __setup!
+ * Emits warning if fn returns non-zero.
+ */
+#define early_param(str, fn) \
__setup_param(str, fn, fn, 1)
+#define early_param_on_off(str_on, str_off, var, config) \
+ \
+ int var = IS_ENABLED(config); \
+ \
+ static int __init parse_##var##_on(char *arg) \
+ { \
+ var = 1; \
+ return 0; \
+ } \
+ __setup_param(str_on, parse_##var##_on, parse_##var##_on, 1); \
+ \
+ static int __init parse_##var##_off(char *arg) \
+ { \
+ var = 0; \
+ return 0; \
+ } \
+ __setup_param(str_off, parse_##var##_off, parse_##var##_off, 1)
+
/* Relies on boot_command_line being set */
void __init parse_early_param(void);
void __init parse_early_options(char *cmdline);
+++ /dev/null
-/*
- * intel_mid_dma.h - Intel MID DMA Drivers
- *
- * Copyright (C) 2008-10 Intel Corp
- * Author: Vinod Koul <vinod.koul@intel.com>
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; version 2 of the License.
- *
- * This program is distributed in the hope that it will be useful, but
- * WITHOUT ANY WARRANTY; without even the implied warranty of
- * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
- * General Public License for more details.
- *
- * You should have received a copy of the GNU General Public License along
- * with this program; if not, write to the Free Software Foundation, Inc.,
- * 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA.
- *
- * ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
- *
- *
- */
-#ifndef __INTEL_MID_DMA_H__
-#define __INTEL_MID_DMA_H__
-
-#include <linux/dmaengine.h>
-
-#define DMA_PREP_CIRCULAR_LIST (1 << 10)
-
-/*DMA mode configurations*/
-enum intel_mid_dma_mode {
- LNW_DMA_PER_TO_MEM = 0, /*periphral to memory configuration*/
- LNW_DMA_MEM_TO_PER, /*memory to periphral configuration*/
- LNW_DMA_MEM_TO_MEM, /*mem to mem confg (testing only)*/
-};
-
-/*DMA handshaking*/
-enum intel_mid_dma_hs_mode {
- LNW_DMA_HW_HS = 0, /*HW Handshaking only*/
- LNW_DMA_SW_HS = 1, /*SW Handshaking not recommended*/
-};
-
-/*Burst size configuration*/
-enum intel_mid_dma_msize {
- LNW_DMA_MSIZE_1 = 0x0,
- LNW_DMA_MSIZE_4 = 0x1,
- LNW_DMA_MSIZE_8 = 0x2,
- LNW_DMA_MSIZE_16 = 0x3,
- LNW_DMA_MSIZE_32 = 0x4,
- LNW_DMA_MSIZE_64 = 0x5,
-};
-
-/**
- * struct intel_mid_dma_slave - DMA slave structure
- *
- * @dirn: DMA trf direction
- * @src_width: tx register width
- * @dst_width: rx register width
- * @hs_mode: HW/SW handshaking mode
- * @cfg_mode: DMA data transfer mode (per-per/mem-per/mem-mem)
- * @src_msize: Source DMA burst size
- * @dst_msize: Dst DMA burst size
- * @per_addr: Periphral address
- * @device_instance: DMA peripheral device instance, we can have multiple
- * peripheral device connected to single DMAC
- */
-struct intel_mid_dma_slave {
- enum intel_mid_dma_hs_mode hs_mode; /*handshaking*/
- enum intel_mid_dma_mode cfg_mode; /*mode configuration*/
- unsigned int device_instance; /*0, 1 for periphral instance*/
- struct dma_slave_config dma_slave;
-};
-
-#endif /*__INTEL_MID_DMA_H__*/
* These flags used only by the kernel as part of the
* irq handling routines.
*
- * IRQF_DISABLED - keep irqs disabled when calling the action handler.
- * DEPRECATED. This flag is a NOOP and scheduled to be removed
* IRQF_SHARED - allow sharing the irq among several devices
* IRQF_PROBE_SHARED - set by callers when they expect sharing mismatches to occur
* IRQF_TIMER - Flag to mark this interrupt as timer interrupt
* IRQF_ONESHOT - Interrupt is not reenabled after the hardirq handler finished.
* Used by threaded interrupts which need to keep the
* irq line disabled until the threaded handler has been run.
- * IRQF_NO_SUSPEND - Do not disable this IRQ during suspend
+ * IRQF_NO_SUSPEND - Do not disable this IRQ during suspend. Does not guarantee
+ * that this interrupt will wake the system from a suspended
+ * state. See Documentation/power/suspend-and-interrupts.txt
* IRQF_FORCE_RESUME - Force enable it on resume even if IRQF_NO_SUSPEND is set
* IRQF_NO_THREAD - Interrupt cannot be threaded
* IRQF_EARLY_RESUME - Resume IRQ early during syscore instead of at device
* resume time.
+ * IRQF_COND_SUSPEND - If the IRQ is shared with a NO_SUSPEND user, execute this
+ * interrupt handler after suspending interrupts. For system
+ * wakeup devices users need to implement wakeup detection in
+ * their interrupt handlers.
*/
-#define IRQF_DISABLED 0x00000020
#define IRQF_SHARED 0x00000080
#define IRQF_PROBE_SHARED 0x00000100
#define __IRQF_TIMER 0x00000200
#define IRQF_FORCE_RESUME 0x00008000
#define IRQF_NO_THREAD 0x00010000
#define IRQF_EARLY_RESUME 0x00020000
+#define IRQF_COND_SUSPEND 0x00040000
#define IRQF_TIMER (__IRQF_TIMER | IRQF_NO_SUSPEND | IRQF_NO_THREAD)
#endif
extern void disable_irq_nosync(unsigned int irq);
+extern bool disable_hardirq(unsigned int irq);
extern void disable_irq(unsigned int irq);
extern void disable_percpu_irq(unsigned int irq);
extern void enable_irq(unsigned int irq);
return irq_set_irq_wake(irq, 0);
}
+/*
+ * irq_get_irqchip_state/irq_set_irqchip_state specific flags
+ */
+enum irqchip_irq_state {
+ IRQCHIP_STATE_PENDING, /* Is interrupt pending? */
+ IRQCHIP_STATE_ACTIVE, /* Is interrupt in progress? */
+ IRQCHIP_STATE_MASKED, /* Is interrupt masked? */
+ IRQCHIP_STATE_LINE_LEVEL, /* Is IRQ line high? */
+};
+
+extern int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
+ bool *state);
+extern int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
+ bool state);
#ifdef CONFIG_IRQ_FORCED_THREADING
extern bool force_irqthreads;
struct seq_file;
struct module;
struct msi_msg;
+enum irqchip_irq_state;
/*
* IRQ line status.
* irq_request_resources
* @irq_compose_msi_msg: optional to compose message content for MSI
* @irq_write_msi_msg: optional to write message content for MSI
+ * @irq_get_irqchip_state: return the internal state of an interrupt
+ * @irq_set_irqchip_state: set the internal state of a interrupt
* @flags: chip specific flags
*/
struct irq_chip {
void (*irq_compose_msi_msg)(struct irq_data *data, struct msi_msg *msg);
void (*irq_write_msi_msg)(struct irq_data *data, struct msi_msg *msg);
+ int (*irq_get_irqchip_state)(struct irq_data *data, enum irqchip_irq_state which, bool *state);
+ int (*irq_set_irqchip_state)(struct irq_data *data, enum irqchip_irq_state which, bool state);
+
unsigned long flags;
};
extern int irq_chip_set_affinity_parent(struct irq_data *data,
const struct cpumask *dest,
bool force);
+extern int irq_chip_set_wake_parent(struct irq_data *data, unsigned int on);
#endif
/* Handling of unhandled and spurious interrupts: */
bool irq_work_queue_on(struct irq_work *work, int cpu);
#endif
-void irq_work_run(void);
void irq_work_tick(void);
void irq_work_sync(struct irq_work *work);
#ifdef CONFIG_IRQ_WORK
#include <asm/irq_work.h>
+void irq_work_run(void);
bool irq_work_needs_cpu(void);
#else
static inline bool irq_work_needs_cpu(void) { return false; }
+static inline void irq_work_run(void) { }
#endif
#endif /* _LINUX_IRQ_WORK_H */
#define GICR_PROPBASER_WaWb (5U << 7)
#define GICR_PROPBASER_RaWaWt (6U << 7)
#define GICR_PROPBASER_RaWaWb (7U << 7)
+#define GICR_PROPBASER_CACHEABILITY_MASK (7U << 7)
#define GICR_PROPBASER_IDBITS_MASK (0x1f)
+#define GICR_PENDBASER_NonShareable (0U << 10)
+#define GICR_PENDBASER_InnerShareable (1U << 10)
+#define GICR_PENDBASER_OuterShareable (2U << 10)
+#define GICR_PENDBASER_SHAREABILITY_MASK (3UL << 10)
+#define GICR_PENDBASER_nCnB (0U << 7)
+#define GICR_PENDBASER_nC (1U << 7)
+#define GICR_PENDBASER_RaWt (2U << 7)
+#define GICR_PENDBASER_RaWb (3U << 7)
+#define GICR_PENDBASER_WaWt (4U << 7)
+#define GICR_PENDBASER_WaWb (5U << 7)
+#define GICR_PENDBASER_RaWaWt (6U << 7)
+#define GICR_PENDBASER_RaWaWb (7U << 7)
+#define GICR_PENDBASER_CACHEABILITY_MASK (7U << 7)
+
/*
* Re-Distributor registers, offsets from SGI_base
*/
#define GITS_TRANSLATER 0x10040
+#define GITS_CTLR_ENABLE (1U << 0)
+#define GITS_CTLR_QUIESCENT (1U << 31)
+
+#define GITS_TYPER_DEVBITS_SHIFT 13
+#define GITS_TYPER_DEVBITS(r) ((((r) >> GITS_TYPER_DEVBITS_SHIFT) & 0x1f) + 1)
#define GITS_TYPER_PTA (1UL << 19)
#define GITS_CBASER_VALID (1UL << 63)
#define GITS_CBASER_WaWb (5UL << 59)
#define GITS_CBASER_RaWaWt (6UL << 59)
#define GITS_CBASER_RaWaWb (7UL << 59)
+#define GITS_CBASER_CACHEABILITY_MASK (7UL << 59)
#define GITS_CBASER_NonShareable (0UL << 10)
#define GITS_CBASER_InnerShareable (1UL << 10)
#define GITS_CBASER_OuterShareable (2UL << 10)
#define GITS_BASER_WaWb (5UL << 59)
#define GITS_BASER_RaWaWt (6UL << 59)
#define GITS_BASER_RaWaWb (7UL << 59)
+#define GITS_BASER_CACHEABILITY_MASK (7UL << 59)
#define GITS_BASER_TYPE_SHIFT (56)
#define GITS_BASER_TYPE(r) (((r) >> GITS_BASER_TYPE_SHIFT) & 7)
#define GITS_BASER_ENTRY_SIZE_SHIFT (48)
extern struct irq_chip gic_arch_extn;
+void gic_set_irqchip_flags(unsigned long flags);
void gic_init_bases(unsigned int, int, void __iomem *, void __iomem *,
u32 offset, struct device_node *);
void gic_cascade_irq(unsigned int gic_nr, unsigned int irq);
void gic_migrate_target(unsigned int new_cpu_id);
unsigned long gic_get_sgir_physaddr(void);
-extern const struct irq_domain_ops *gic_routable_irq_domain_ops;
-static inline void __init register_routable_domain_ops
- (const struct irq_domain_ops *ops)
-{
- gic_routable_irq_domain_ops = ops;
-}
#endif /* __ASSEMBLY */
#endif
+++ /dev/null
-/*
- * drivers/irqchip/irq-crossbar.h
- *
- * Copyright (C) 2013 Texas Instruments Incorporated - http://www.ti.com
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License version 2 as
- * published by the Free Software Foundation.
- *
- */
-int irqcrossbar_init(void);
extern cycle_t gic_read_compare(void);
extern void gic_write_compare(cycle_t cnt);
extern void gic_write_cpu_compare(cycle_t cnt, int cpu);
+extern void gic_start_count(void);
+extern void gic_stop_count(void);
extern void gic_send_ipi(unsigned int intr);
extern unsigned int plat_ipi_call_int_xlate(unsigned int);
extern unsigned int plat_ipi_resched_int_xlate(unsigned int);
#ifdef CONFIG_PM_SLEEP
unsigned int nr_actions;
unsigned int no_suspend_depth;
+ unsigned int cond_suspend_depth;
unsigned int force_resume_depth;
#endif
#ifdef CONFIG_PROC_FS
* The local_irq_*() APIs are equal to the raw_local_irq*()
* if !TRACE_IRQFLAGS.
*/
-#ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT
+#ifdef CONFIG_TRACE_IRQFLAGS
#define local_irq_enable() \
do { trace_hardirqs_on(); raw_local_irq_enable(); } while (0)
#define local_irq_disable() \
raw_local_irq_restore(flags); \
} \
} while (0)
-#define local_save_flags(flags) \
- do { \
- raw_local_save_flags(flags); \
- } while (0)
-
-#define irqs_disabled_flags(flags) \
- ({ \
- raw_irqs_disabled_flags(flags); \
- })
-
-#define irqs_disabled() \
- ({ \
- unsigned long _flags; \
- raw_local_save_flags(_flags); \
- raw_irqs_disabled_flags(_flags); \
- })
#define safe_halt() \
do { \
} while (0)
-#else /* !CONFIG_TRACE_IRQFLAGS_SUPPORT */
+#else /* !CONFIG_TRACE_IRQFLAGS */
#define local_irq_enable() do { raw_local_irq_enable(); } while (0)
#define local_irq_disable() do { raw_local_irq_disable(); } while (0)
raw_local_irq_save(flags); \
} while (0)
#define local_irq_restore(flags) do { raw_local_irq_restore(flags); } while (0)
-#define local_save_flags(flags) do { raw_local_save_flags(flags); } while (0)
-#define irqs_disabled() (raw_irqs_disabled())
-#define irqs_disabled_flags(flags) (raw_irqs_disabled_flags(flags))
#define safe_halt() do { raw_safe_halt(); } while (0)
+#endif /* CONFIG_TRACE_IRQFLAGS */
+
+#define local_save_flags(flags) raw_local_save_flags(flags)
+
+/*
+ * Some architectures don't define arch_irqs_disabled(), so even if either
+ * definition would be fine we need to use different ones for the time being
+ * to avoid build issues.
+ */
+#ifdef CONFIG_TRACE_IRQFLAGS_SUPPORT
+#define irqs_disabled() \
+ ({ \
+ unsigned long _flags; \
+ raw_local_save_flags(_flags); \
+ raw_irqs_disabled_flags(_flags); \
+ })
+#else /* !CONFIG_TRACE_IRQFLAGS_SUPPORT */
+#define irqs_disabled() raw_irqs_disabled()
#endif /* CONFIG_TRACE_IRQFLAGS_SUPPORT */
+#define irqs_disabled_flags(flags) raw_irqs_disabled_flags(flags)
+
#endif
* same as using STATIC_KEY_INIT_FALSE.
*/
+#if defined(CC_HAVE_ASM_GOTO) && defined(CONFIG_JUMP_LABEL)
+# define HAVE_JUMP_LABEL
+#endif
+
+#ifndef __ASSEMBLY__
+
#include <linux/types.h>
#include <linux/compiler.h>
#include <linux/bug.h>
"%s used before call to jump_label_init", \
__func__)
-#if defined(CC_HAVE_ASM_GOTO) && defined(CONFIG_JUMP_LABEL)
+#ifdef HAVE_JUMP_LABEL
struct static_key {
atomic_t enabled;
#endif
};
-# include <asm/jump_label.h>
-# define HAVE_JUMP_LABEL
#else
struct static_key {
atomic_t enabled;
};
-#endif /* CC_HAVE_ASM_GOTO && CONFIG_JUMP_LABEL */
+#endif /* HAVE_JUMP_LABEL */
+#endif /* __ASSEMBLY__ */
+
+#ifdef HAVE_JUMP_LABEL
+#include <asm/jump_label.h>
+#endif
+
+#ifndef __ASSEMBLY__
enum jump_label_type {
JUMP_LABEL_DISABLE = 0,
}
#endif /* _LINUX_JUMP_LABEL_H */
+
+#endif /* __ASSEMBLY__ */
struct kmem_cache;
struct page;
+struct vm_struct;
#ifdef CONFIG_KASAN
void kasan_slab_alloc(struct kmem_cache *s, void *object);
void kasan_slab_free(struct kmem_cache *s, void *object);
-#define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT)
-
int kasan_module_alloc(void *addr, size_t size);
-void kasan_module_free(void *addr);
+void kasan_free_shadow(const struct vm_struct *vm);
#else /* CONFIG_KASAN */
-#define MODULE_ALIGN 1
-
static inline void kasan_unpoison_shadow(const void *address, size_t size) {}
static inline void kasan_enable_current(void) {}
static inline void kasan_slab_free(struct kmem_cache *s, void *object) {}
static inline int kasan_module_alloc(void *addr, size_t size) { return 0; }
-static inline void kasan_module_free(void *addr) {}
+static inline void kasan_free_shadow(const struct vm_struct *vm) {}
#endif /* CONFIG_KASAN */
KVM_NR_BUSES
};
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
int len, const void *val);
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
- int len, const void *val, long cookie);
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr, int len,
- void *val);
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+ gpa_t addr, int len, const void *val, long cookie);
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
+ int len, void *val);
int kvm_io_bus_register_dev(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
int len, struct kvm_io_device *dev);
int kvm_io_bus_unregister_dev(struct kvm *kvm, enum kvm_bus bus_idx,
int kvm_arch_vcpu_should_kick(struct kvm_vcpu *vcpu);
void *kvm_kvzalloc(unsigned long size);
-void kvm_kvfree(const void *addr);
#ifndef __KVM_HAVE_ARCH_VM_ALLOC
static inline struct kvm *kvm_arch_alloc_vm(void)
#endif
}
+#ifdef __KVM_HAVE_ARCH_INTC_INITIALIZED
+/*
+ * returns true if the virtual interrupt controller is initialized and
+ * ready to accept virtual IRQ. On some architectures the virtual interrupt
+ * controller is dynamically instantiated and this is not always true.
+ */
+bool kvm_arch_intc_initialized(struct kvm *kvm);
+#else
+static inline bool kvm_arch_intc_initialized(struct kvm *kvm)
+{
+ return true;
+}
+#endif
+
int kvm_arch_init_vm(struct kvm *kvm, unsigned long type);
void kvm_arch_destroy_vm(struct kvm *kvm);
void kvm_arch_sync_events(struct kvm *kvm);
#endif /* CONFIG_HAVE_KVM_EVENTFD */
#ifdef CONFIG_KVM_APIC_ARCHITECTURE
-static inline bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu)
+static inline bool kvm_vcpu_is_reset_bsp(struct kvm_vcpu *vcpu)
{
return vcpu->kvm->bsp_vcpu_id == vcpu->vcpu_id;
}
+static inline bool kvm_vcpu_is_bsp(struct kvm_vcpu *vcpu)
+{
+ return (vcpu->arch.apic_base & MSR_IA32_APICBASE_BSP) != 0;
+}
+
bool kvm_vcpu_compatible(struct kvm_vcpu *vcpu);
#else
#include <linux/compiler.h>
unsigned long lcm(unsigned long a, unsigned long b) __attribute_const__;
+unsigned long lcm_not_zero(unsigned long a, unsigned long b) __attribute_const__;
#endif /* _LCM_H */
ATA_FLAG_SW_ACTIVITY = (1 << 22), /* driver supports sw activity
* led */
ATA_FLAG_NO_DIPM = (1 << 23), /* host not happy with DIPM */
- ATA_FLAG_LOWTAG = (1 << 24), /* host wants lowest available tag */
+ ATA_FLAG_SAS_HOST = (1 << 24), /* SAS host */
/* bits 24:31 of ap->flags are reserved for LLD specific flags */
#include <linux/power_supply.h>
-#define psy_to_ux500_charger(x) container_of((x), \
- struct ux500_charger, psy)
+/*
+ * Valid only for supplies of type:
+ * - POWER_SUPPLY_TYPE_MAINS,
+ * - POWER_SUPPLY_TYPE_USB,
+ * because only them store as drv_data pointer to struct ux500_charger.
+ */
+#define psy_to_ux500_charger(x) power_supply_get_drvdata(psy)
/* Forward declaration */
struct ux500_charger;
* @power_path USB power path support
*/
struct ux500_charger {
- struct power_supply psy;
+ struct power_supply *psy;
struct ux500_charger_ops ops;
int max_out_volt;
int max_out_curr;
#ifndef __LINUX_MFD_MAX77693_H
#define __LINUX_MFD_MAX77693_H
-/* MAX77686 regulator IDs */
+/* MAX77693 regulator IDs */
enum max77693_regulators {
MAX77693_ESAFEOUT1 = 0,
MAX77693_ESAFEOUT2,
MAX77693_REG_MAX,
};
-struct max77693_regulator_data {
- int id;
- struct regulator_init_data *initdata;
- struct device_node *of_node;
-};
-
struct max77693_reg_data {
u8 addr;
u8 data;
/* MAX77693 */
struct max77693_platform_data {
- /* regulator data */
- struct max77693_regulator_data *regulators;
- int num_regulators;
-
/* muic data */
struct max77693_muic_platform_data *muic_data;
struct max77693_led_platform_data *led_data;
int ldo_begin;
int ldo_end;
int max_reg;
+ bool has_regen3;
struct palmas_regs_info *palmas_regs_info;
struct of_regulator_match *palmas_matches;
struct palmas_sleep_requestor_info *sleep_req_info;
#define PALMAS_GPADC_TRIM15 0x0E
#define PALMAS_GPADC_TRIM16 0x0F
+/* TPS659038 regen2_ctrl offset iss different from palmas */
+#define TPS659038_REGEN2_CTRL 0x12
+
/* TPS65917 Interrupt registers */
/* Registers for function INTERRUPT */
struct i2c_client *client;
struct rt5033_dev *rt5033;
struct regmap *regmap;
- struct power_supply psy;
+ struct power_supply *psy;
};
/* RT5033 charger platform data */
/**
* struct stw481x - state holder for the Stw481x drivers
- * @mutex: mutex to serialize I2C accesses
* @i2c_client: corresponding I2C client
- * @regulator: regulator device for regulator children
* @map: regmap handle to access device registers
*/
struct stw481x {
- struct mutex lock;
struct i2c_client *client;
- struct regulator_dev *vmmc_regulator;
struct regmap *map;
};
struct wm8350_power {
struct platform_device *pdev;
- struct power_supply battery;
- struct power_supply usb;
- struct power_supply ac;
+ struct power_supply *battery;
+ struct power_supply *usb;
+ struct power_supply *ac;
struct wm8350_charger_policy *policy;
int rev_g_coeff;
enum mlx4_update_qp_attr {
MLX4_UPDATE_QP_SMAC = 1 << 0,
- MLX4_UPDATE_QP_VSD = 1 << 2,
+ MLX4_UPDATE_QP_VSD = 1 << 1,
MLX4_UPDATE_QP_SUPPORTED_ATTRS = (1 << 2) - 1
};
#define mmc_cmd_type(cmd) ((cmd)->flags & MMC_CMD_MASK)
unsigned int retries; /* max number of retries */
- unsigned int error; /* command error */
+ int error; /* command error */
/*
* Standard errno values are used for errors, but some have specific
unsigned int timeout_clks; /* data timeout (in clocks) */
unsigned int blksz; /* data block size */
unsigned int blocks; /* number of blocks */
- unsigned int error; /* data error */
+ int error; /* data error */
unsigned int flags;
#define MMC_DATA_WRITE (1 << 8)
* struct dw_mci - MMC controller state shared between all slots
* @lock: Spinlock protecting the queue and associated data.
* @regs: Pointer to MMIO registers.
+ * @fifo_reg: Pointer to MMIO registers for data FIFO
* @sg: Scatterlist entry currently being processed by PIO code, if any.
* @sg_miter: PIO mapping scatterlist iterator.
* @cur_slot: The slot which is currently using the controller.
* @current_speed: Configured rate of the controller.
* @num_slots: Number of slots available.
* @verid: Denote Version ID.
- * @data_offset: Set the offset of DATA register according to VERID.
* @dev: Device associated with the MMC controller.
* @pdata: Platform data associated with the MMC controller.
* @drv_data: Driver specific data for identified variant of the controller
spinlock_t lock;
spinlock_t irq_lock;
void __iomem *regs;
+ void __iomem *fifo_reg;
struct scatterlist *sg;
struct sg_mapping_iter sg_miter;
u32 num_slots;
u32 fifoth_val;
u16 verid;
- u16 data_offset;
struct device *dev;
struct dw_mci_board *pdata;
const struct dw_mci_drv_data *drv_data;
int irq;
int sdio_id0;
+
+ struct timer_list cmd11_timer;
};
/* DMA ops for Internal/External DMAC interface */
};
struct mmc_host_ops {
- /*
- * 'enable' is called when the host is claimed and 'disable' is called
- * when the host is released. 'enable' and 'disable' are deprecated.
- */
- int (*enable)(struct mmc_host *host);
- int (*disable)(struct mmc_host *host);
/*
* It is optional for the host to implement pre_req and post_req in
* order to support double buffering of requests (prepare one
+++ /dev/null
-/*
- * include/linux/mmc/sdhci-spear.h
- *
- * SDHCI declarations specific to ST SPEAr platform
- *
- * Copyright (C) 2010 ST Microelectronics
- * Viresh Kumar <viresh.linux@gmail.com>
- *
- * This file is licensed under the terms of the GNU General Public
- * License version 2. This program is licensed "as is" without any
- * warranty of any kind, whether express or implied.
- */
-
-#ifndef LINUX_MMC_SDHCI_SPEAR_H
-#define LINUX_MMC_SDHCI_SPEAR_H
-
-#include <linux/platform_device.h>
-/*
- * struct sdhci_plat_data: spear sdhci platform data structure
- *
- * card_int_gpio: gpio pin used for card detection
- */
-struct sdhci_plat_data {
- int card_int_gpio;
-};
-
-/* This function is used to set platform_data field of pdev->dev */
-static inline void
-sdhci_set_plat_data(struct platform_device *pdev, struct sdhci_plat_data *data)
-{
- pdev->dev.platform_data = data;
-}
-
-#endif /* LINUX_MMC_SDHCI_SPEAR_H */
+++ /dev/null
-/*
- * linux/include/linux/mmc/sdhci.h - Secure Digital Host Controller Interface
- *
- * Copyright (C) 2005-2008 Pierre Ossman, All Rights Reserved.
- *
- * This program is free software; you can redistribute it and/or modify
- * it under the terms of the GNU General Public License as published by
- * the Free Software Foundation; either version 2 of the License, or (at
- * your option) any later version.
- */
-#ifndef LINUX_MMC_SDHCI_H
-#define LINUX_MMC_SDHCI_H
-
-#include <linux/scatterlist.h>
-#include <linux/compiler.h>
-#include <linux/types.h>
-#include <linux/io.h>
-#include <linux/mmc/host.h>
-
-struct sdhci_host_next {
- unsigned int sg_count;
- s32 cookie;
-};
-
-struct sdhci_host {
- /* Data set by hardware interface driver */
- const char *hw_name; /* Hardware bus name */
-
- unsigned int quirks; /* Deviations from spec. */
-
-/* Controller doesn't honor resets unless we touch the clock register */
-#define SDHCI_QUIRK_CLOCK_BEFORE_RESET (1<<0)
-/* Controller has bad caps bits, but really supports DMA */
-#define SDHCI_QUIRK_FORCE_DMA (1<<1)
-/* Controller doesn't like to be reset when there is no card inserted. */
-#define SDHCI_QUIRK_NO_CARD_NO_RESET (1<<2)
-/* Controller doesn't like clearing the power reg before a change */
-#define SDHCI_QUIRK_SINGLE_POWER_WRITE (1<<3)
-/* Controller has flaky internal state so reset it on each ios change */
-#define SDHCI_QUIRK_RESET_CMD_DATA_ON_IOS (1<<4)
-/* Controller has an unusable DMA engine */
-#define SDHCI_QUIRK_BROKEN_DMA (1<<5)
-/* Controller has an unusable ADMA engine */
-#define SDHCI_QUIRK_BROKEN_ADMA (1<<6)
-/* Controller can only DMA from 32-bit aligned addresses */
-#define SDHCI_QUIRK_32BIT_DMA_ADDR (1<<7)
-/* Controller can only DMA chunk sizes that are a multiple of 32 bits */
-#define SDHCI_QUIRK_32BIT_DMA_SIZE (1<<8)
-/* Controller can only ADMA chunks that are a multiple of 32 bits */
-#define SDHCI_QUIRK_32BIT_ADMA_SIZE (1<<9)
-/* Controller needs to be reset after each request to stay stable */
-#define SDHCI_QUIRK_RESET_AFTER_REQUEST (1<<10)
-/* Controller needs voltage and power writes to happen separately */
-#define SDHCI_QUIRK_NO_SIMULT_VDD_AND_POWER (1<<11)
-/* Controller provides an incorrect timeout value for transfers */
-#define SDHCI_QUIRK_BROKEN_TIMEOUT_VAL (1<<12)
-/* Controller has an issue with buffer bits for small transfers */
-#define SDHCI_QUIRK_BROKEN_SMALL_PIO (1<<13)
-/* Controller does not provide transfer-complete interrupt when not busy */
-#define SDHCI_QUIRK_NO_BUSY_IRQ (1<<14)
-/* Controller has unreliable card detection */
-#define SDHCI_QUIRK_BROKEN_CARD_DETECTION (1<<15)
-/* Controller reports inverted write-protect state */
-#define SDHCI_QUIRK_INVERTED_WRITE_PROTECT (1<<16)
-/* Controller does not like fast PIO transfers */
-#define SDHCI_QUIRK_PIO_NEEDS_DELAY (1<<18)
-/* Controller has to be forced to use block size of 2048 bytes */
-#define SDHCI_QUIRK_FORCE_BLK_SZ_2048 (1<<20)
-/* Controller cannot do multi-block transfers */
-#define SDHCI_QUIRK_NO_MULTIBLOCK (1<<21)
-/* Controller can only handle 1-bit data transfers */
-#define SDHCI_QUIRK_FORCE_1_BIT_DATA (1<<22)
-/* Controller needs 10ms delay between applying power and clock */
-#define SDHCI_QUIRK_DELAY_AFTER_POWER (1<<23)
-/* Controller uses SDCLK instead of TMCLK for data timeouts */
-#define SDHCI_QUIRK_DATA_TIMEOUT_USES_SDCLK (1<<24)
-/* Controller reports wrong base clock capability */
-#define SDHCI_QUIRK_CAP_CLOCK_BASE_BROKEN (1<<25)
-/* Controller cannot support End Attribute in NOP ADMA descriptor */
-#define SDHCI_QUIRK_NO_ENDATTR_IN_NOPDESC (1<<26)
-/* Controller is missing device caps. Use caps provided by host */
-#define SDHCI_QUIRK_MISSING_CAPS (1<<27)
-/* Controller uses Auto CMD12 command to stop the transfer */
-#define SDHCI_QUIRK_MULTIBLOCK_READ_ACMD12 (1<<28)
-/* Controller doesn't have HISPD bit field in HI-SPEED SD card */
-#define SDHCI_QUIRK_NO_HISPD_BIT (1<<29)
-/* Controller treats ADMA descriptors with length 0000h incorrectly */
-#define SDHCI_QUIRK_BROKEN_ADMA_ZEROLEN_DESC (1<<30)
-/* The read-only detection via SDHCI_PRESENT_STATE register is unstable */
-#define SDHCI_QUIRK_UNSTABLE_RO_DETECT (1<<31)
-
- unsigned int quirks2; /* More deviations from spec. */
-
-#define SDHCI_QUIRK2_HOST_OFF_CARD_ON (1<<0)
-#define SDHCI_QUIRK2_HOST_NO_CMD23 (1<<1)
-/* The system physically doesn't support 1.8v, even if the host does */
-#define SDHCI_QUIRK2_NO_1_8_V (1<<2)
-#define SDHCI_QUIRK2_PRESET_VALUE_BROKEN (1<<3)
-#define SDHCI_QUIRK2_CARD_ON_NEEDS_BUS_ON (1<<4)
-/* Controller has a non-standard host control register */
-#define SDHCI_QUIRK2_BROKEN_HOST_CONTROL (1<<5)
-/* Controller does not support HS200 */
-#define SDHCI_QUIRK2_BROKEN_HS200 (1<<6)
-/* Controller does not support DDR50 */
-#define SDHCI_QUIRK2_BROKEN_DDR50 (1<<7)
-/* Stop command (CMD12) can set Transfer Complete when not using MMC_RSP_BUSY */
-#define SDHCI_QUIRK2_STOP_WITH_TC (1<<8)
-/* Controller does not support 64-bit DMA */
-#define SDHCI_QUIRK2_BROKEN_64_BIT_DMA (1<<9)
-/* need clear transfer mode register before send cmd */
-#define SDHCI_QUIRK2_CLEAR_TRANSFERMODE_REG_BEFORE_CMD (1<<10)
-/* Capability register bit-63 indicates HS400 support */
-#define SDHCI_QUIRK2_CAPS_BIT63_FOR_HS400 (1<<11)
-/* forced tuned clock */
-#define SDHCI_QUIRK2_TUNING_WORK_AROUND (1<<12)
-/* disable the block count for single block transactions */
-#define SDHCI_QUIRK2_SUPPORT_SINGLE (1<<13)
-
- int irq; /* Device IRQ */
- void __iomem *ioaddr; /* Mapped address */
-
- const struct sdhci_ops *ops; /* Low level hw interface */
-
- /* Internal data */
- struct mmc_host *mmc; /* MMC structure */
- u64 dma_mask; /* custom DMA mask */
-
-#if defined(CONFIG_LEDS_CLASS) || defined(CONFIG_LEDS_CLASS_MODULE)
- struct led_classdev led; /* LED control */
- char led_name[32];
-#endif
-
- spinlock_t lock; /* Mutex */
-
- int flags; /* Host attributes */
-#define SDHCI_USE_SDMA (1<<0) /* Host is SDMA capable */
-#define SDHCI_USE_ADMA (1<<1) /* Host is ADMA capable */
-#define SDHCI_REQ_USE_DMA (1<<2) /* Use DMA for this req. */
-#define SDHCI_DEVICE_DEAD (1<<3) /* Device unresponsive */
-#define SDHCI_SDR50_NEEDS_TUNING (1<<4) /* SDR50 needs tuning */
-#define SDHCI_NEEDS_RETUNING (1<<5) /* Host needs retuning */
-#define SDHCI_AUTO_CMD12 (1<<6) /* Auto CMD12 support */
-#define SDHCI_AUTO_CMD23 (1<<7) /* Auto CMD23 support */
-#define SDHCI_PV_ENABLED (1<<8) /* Preset value enabled */
-#define SDHCI_SDIO_IRQ_ENABLED (1<<9) /* SDIO irq enabled */
-#define SDHCI_SDR104_NEEDS_TUNING (1<<10) /* SDR104/HS200 needs tuning */
-#define SDHCI_USING_RETUNING_TIMER (1<<11) /* Host is using a retuning timer for the card */
-#define SDHCI_USE_64_BIT_DMA (1<<12) /* Use 64-bit DMA */
-#define SDHCI_HS400_TUNING (1<<13) /* Tuning for HS400 */
-
- unsigned int version; /* SDHCI spec. version */
-
- unsigned int max_clk; /* Max possible freq (MHz) */
- unsigned int timeout_clk; /* Timeout freq (KHz) */
- unsigned int clk_mul; /* Clock Muliplier value */
-
- unsigned int clock; /* Current clock (MHz) */
- u8 pwr; /* Current voltage */
-
- bool runtime_suspended; /* Host is runtime suspended */
- bool bus_on; /* Bus power prevents runtime suspend */
- bool preset_enabled; /* Preset is enabled */
-
- struct mmc_request *mrq; /* Current request */
- struct mmc_command *cmd; /* Current command */
- struct mmc_data *data; /* Current data request */
- unsigned int data_early:1; /* Data finished before cmd */
- unsigned int busy_handle:1; /* Handling the order of Busy-end */
-
- struct sg_mapping_iter sg_miter; /* SG state for PIO */
- unsigned int blocks; /* remaining PIO blocks */
-
- int sg_count; /* Mapped sg entries */
-
- void *adma_table; /* ADMA descriptor table */
- void *align_buffer; /* Bounce buffer */
-
- size_t adma_table_sz; /* ADMA descriptor table size */
- size_t align_buffer_sz; /* Bounce buffer size */
-
- dma_addr_t adma_addr; /* Mapped ADMA descr. table */
- dma_addr_t align_addr; /* Mapped bounce buffer */
-
- unsigned int desc_sz; /* ADMA descriptor size */
- unsigned int align_sz; /* ADMA alignment */
- unsigned int align_mask; /* ADMA alignment mask */
-
- struct tasklet_struct finish_tasklet; /* Tasklet structures */
-
- struct timer_list timer; /* Timer for timeouts */
-
- u32 caps; /* Alternative CAPABILITY_0 */
- u32 caps1; /* Alternative CAPABILITY_1 */
-
- unsigned int ocr_avail_sdio; /* OCR bit masks */
- unsigned int ocr_avail_sd;
- unsigned int ocr_avail_mmc;
- u32 ocr_mask; /* available voltages */
-
- unsigned timing; /* Current timing */
-
- u32 thread_isr;
-
- /* cached registers */
- u32 ier;
-
- wait_queue_head_t buf_ready_int; /* Waitqueue for Buffer Read Ready interrupt */
- unsigned int tuning_done; /* Condition flag set when CMD19 succeeds */
-
- unsigned int tuning_count; /* Timer count for re-tuning */
- unsigned int tuning_mode; /* Re-tuning mode supported by host */
-#define SDHCI_TUNING_MODE_1 0
- struct timer_list tuning_timer; /* Timer for tuning */
-
- struct sdhci_host_next next_data;
- unsigned long private[0] ____cacheline_aligned;
-};
-#endif /* LINUX_MMC_SDHCI_H */
unsigned long wait_table_bits;
ZONE_PADDING(_pad1_)
-
- /* Write-intensive fields used from the page allocator */
- spinlock_t lock;
-
/* free areas of different sizes */
struct free_area free_area[MAX_ORDER];
/* zone flags, see below */
unsigned long flags;
+ /* Write-intensive fields used from the page allocator */
+ spinlock_t lock;
+
ZONE_PADDING(_pad2_)
/* Write-intensive fields used by page reclaim */
unsigned long *ftrace_callsites;
#endif
+#ifdef CONFIG_LIVEPATCH
+ bool klp_alive;
+#endif
+
#ifdef CONFIG_MODULE_UNLOAD
/* What modules depend on me? */
struct list_head source_list;
/* Any cleanup before freeing mod->module_init */
void module_arch_freeing_init(struct module *mod);
+
+#ifdef CONFIG_KASAN
+#include <linux/kasan.h>
+#define MODULE_ALIGN (PAGE_SIZE << KASAN_SHADOW_SCALE_SHIFT)
+#else
+#define MODULE_ALIGN PAGE_SIZE
+#endif
+
#endif
* Used to add FDB entries to dump requests. Implementers should add
* entries to skb and update idx with the number of entries.
*
- * int (*ndo_bridge_setlink)(struct net_device *dev, struct nlmsghdr *nlh)
+ * int (*ndo_bridge_setlink)(struct net_device *dev, struct nlmsghdr *nlh,
+ * u16 flags)
* int (*ndo_bridge_getlink)(struct sk_buff *skb, u32 pid, u32 seq,
* struct net_device *dev, u32 filter_mask)
+ * int (*ndo_bridge_dellink)(struct net_device *dev, struct nlmsghdr *nlh,
+ * u16 flags);
*
* int (*ndo_change_carrier)(struct net_device *dev, bool new_carrier);
* Called to change device carrier. Soft-devices (like dummy, team, etc)
void synchronize_net(void);
int init_dummy_netdev(struct net_device *dev);
+DECLARE_PER_CPU(int, xmit_recursion);
+static inline int dev_recursion_level(void)
+{
+ return this_cpu_read(xmit_recursion);
+}
+
struct net_device *dev_get_by_index(struct net *net, int ifindex);
struct net_device *__dev_get_by_index(struct net *net, int ifindex);
struct net_device *dev_get_by_index_rcu(struct net *net, int ifindex);
static inline void skb_gro_remcsum_init(struct gro_remcsum *grc)
{
+ grc->offset = 0;
grc->delta = 0;
}
extern int nfs_refresh_inode(struct inode *, struct nfs_fattr *);
extern int nfs_post_op_update_inode(struct inode *inode, struct nfs_fattr *fattr);
extern int nfs_post_op_update_inode_force_wcc(struct inode *inode, struct nfs_fattr *fattr);
+extern int nfs_post_op_update_inode_force_wcc_locked(struct inode *inode, struct nfs_fattr *fattr);
extern int nfs_getattr(struct vfsmount *, struct dentry *, struct kstat *);
extern void nfs_access_add_cache(struct inode *, struct nfs_access_entry *);
extern void nfs_access_set_mask(struct nfs_access_entry *, u32);
extern int nfs_revalidate_inode_rcu(struct nfs_server *server, struct inode *inode);
extern int __nfs_revalidate_inode(struct nfs_server *, struct inode *);
extern int nfs_revalidate_mapping(struct inode *inode, struct address_space *mapping);
+extern int nfs_revalidate_mapping_protected(struct inode *inode, struct address_space *mapping);
extern int nfs_setattr(struct dentry *, struct iattr *);
-extern void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr);
+extern void nfs_setattr_update_inode(struct inode *inode, struct iattr *attr, struct nfs_fattr *);
extern void nfs_setsecurity(struct inode *inode, struct nfs_fattr *fattr,
struct nfs4_label *label);
extern struct nfs_open_context *get_nfs_open_context(struct nfs_open_context *ctx);
extern void nfs_put_lock_context(struct nfs_lock_context *l_ctx);
extern u64 nfs_compat_user_ino64(u64 fileid);
extern void nfs_fattr_init(struct nfs_fattr *fattr);
+extern void nfs_fattr_set_barrier(struct nfs_fattr *fattr);
extern unsigned long nfs_inc_attr_generation_counter(void);
extern struct nfs_fattr *nfs_alloc_fattr(void);
return of_node_get(cpu_dev->of_node);
}
+void of_dma_configure(struct device *dev, struct device_node *np);
#else /* CONFIG_OF */
static inline int of_driver_match_device(struct device *dev,
{
return NULL;
}
+static inline void of_dma_configure(struct device *dev, struct device_node *np)
+{}
#endif /* CONFIG_OF */
#endif /* _LINUX_OF_DEVICE_H */
size_t *size);
extern void of_iommu_init(void);
-extern struct iommu_ops *of_iommu_configure(struct device *dev);
+extern struct iommu_ops *of_iommu_configure(struct device *dev,
+ struct device_node *master_np);
#else
}
static inline void of_iommu_init(void) { }
-static inline struct iommu_ops *of_iommu_configure(struct device *dev)
+static inline struct iommu_ops *of_iommu_configure(struct device *dev,
+ struct device_node *master_np)
{
return NULL;
}
int of_irq_parse_and_map_pci(const struct pci_dev *dev, u8 slot, u8 pin);
int of_pci_parse_bus_range(struct device_node *node, struct resource *res);
int of_get_pci_domain_nr(struct device_node *node);
+void of_pci_dma_configure(struct pci_dev *pci_dev);
#else
static inline int of_irq_parse_pci(const struct pci_dev *pdev, struct of_phandle_args *out_irq)
{
{
return -1;
}
+
+static inline void of_pci_dma_configure(struct pci_dev *pci_dev) { }
#endif
#if defined(CONFIG_OF_ADDRESS)
static inline void of_platform_depopulate(struct device *parent) { }
#endif
-#ifdef CONFIG_OF_DYNAMIC
+#if defined(CONFIG_OF_DYNAMIC) && defined(CONFIG_OF_ADDRESS)
extern void of_platform_register_reconfig_notifier(void);
#else
static inline void of_platform_register_reconfig_notifier(void) { }
static inline void acpiphp_check_host_bridge(struct acpi_device *adev) { }
#endif
+extern const u8 pci_acpi_dsm_uuid[];
+#define DEVICE_LABEL_DSM 0x07
+#define RESET_DELAY_DSM 0x08
+#define FUNCTION_DELAY_DSM 0x09
+
#else /* CONFIG_ACPI */
static inline void acpi_pci_add_bus(struct pci_bus *bus) { }
static inline void acpi_pci_remove_bus(struct pci_bus *bus) { }
void pcie_aspm_powersave_config_link(struct pci_dev *pdev);
void pci_disable_link_state(struct pci_dev *pdev, int state);
void pci_disable_link_state_locked(struct pci_dev *pdev, int state);
-void pcie_clear_aspm(struct pci_bus *bus);
void pcie_no_aspm(void);
#else
static inline void pcie_aspm_init_link_state(struct pci_dev *pdev)
static inline void pci_disable_link_state(struct pci_dev *pdev, int state)
{
}
-static inline void pcie_clear_aspm(struct pci_bus *bus)
-{
-}
static inline void pcie_no_aspm(void)
{
}
struct list_head windows; /* resource_entry */
void (*release_fn)(struct pci_host_bridge *);
void *release_data;
+ unsigned int ignore_reset_delay:1; /* for entire hierarchy */
};
#define to_pci_host_bridge(n) container_of(n, struct pci_host_bridge, dev)
return dev->bus->self;
}
+struct device *pci_get_host_bridge_device(struct pci_dev *dev);
+void pci_put_host_bridge_device(struct device *dev);
+
#ifdef CONFIG_PCI_MSI
static inline bool pci_dev_msi_enabled(struct pci_dev *pci_dev)
{
static inline struct pinctrl * __must_check pinctrl_get(struct device *dev)
{
- return ERR_PTR(-ENOSYS);
+ return NULL;
}
static inline void pinctrl_put(struct pinctrl *p)
struct pinctrl *p,
const char *name)
{
- return ERR_PTR(-ENOSYS);
+ return NULL;
}
static inline int pinctrl_select_state(struct pinctrl *p,
static inline struct pinctrl * __must_check devm_pinctrl_get(struct device *dev)
{
- return ERR_PTR(-ENOSYS);
+ return NULL;
}
static inline void devm_pinctrl_put(struct pinctrl *p)
u32 caps; /* Used for the MMC driver on 2430 and later */
u32 pm_caps; /* PM capabilities of the mmc */
- /* switch pin can be for card detect (default) or card cover */
- unsigned cover:1;
-
/* use the internal clock */
unsigned internal_clock:1;
#define HSMMC_HAS_HSPE_SUPPORT (1 << 2)
unsigned features;
- int switch_pin; /* gpio (card detect) */
+ int gpio_cd; /* gpio (card detect) */
+ int gpio_cod; /* gpio (cover detect) */
int gpio_wp; /* gpio (write protect) */
int (*set_power)(struct device *dev, int power_on, int vdd);
int emergency_stop;
char psy_name_buf[PSY_NAME_MAX + 1];
- struct power_supply charger_psy;
+ struct power_supply_desc charger_psy_desc;
+ struct power_supply *charger_psy;
u64 charging_start_time;
u64 charging_end_time;
MAX17047_QRTbl30 = 0x42,
};
-enum max170xx_chip_type {MAX17042, MAX17047};
+enum max170xx_chip_type {
+ MAXIM_DEVICE_TYPE_UNKNOWN = 0,
+ MAXIM_DEVICE_TYPE_MAX17042,
+ MAXIM_DEVICE_TYPE_MAX17047,
+ MAXIM_DEVICE_TYPE_MAX17050,
+
+ MAXIM_DEVICE_TYPE_NUM
+};
/*
* used for setting a register to a desired value
#ifndef __LINUX_POWER_SUPPLY_H__
#define __LINUX_POWER_SUPPLY_H__
+#include <linux/device.h>
#include <linux/workqueue.h>
#include <linux/leds.h>
#include <linux/spinlock.h>
const char *strval;
};
-struct device;
struct device_node;
+struct power_supply;
-struct power_supply {
- const char *name;
- enum power_supply_type type;
- enum power_supply_property *properties;
- size_t num_properties;
+/* Run-time specific power supply configuration */
+struct power_supply_config {
+ struct device_node *of_node;
+ /* Driver private data */
+ void *drv_data;
char **supplied_to;
size_t num_supplicants;
+};
- char **supplied_from;
- size_t num_supplies;
- struct device_node *of_node;
+/* Description of power supply */
+struct power_supply_desc {
+ const char *name;
+ enum power_supply_type type;
+ enum power_supply_property *properties;
+ size_t num_properties;
+ /*
+ * Functions for drivers implementing power supply class.
+ * These shouldn't be called directly by other drivers for accessing
+ * this power supply. Instead use power_supply_*() functions (for
+ * example power_supply_get_property()).
+ */
int (*get_property)(struct power_supply *psy,
enum power_supply_property psp,
union power_supply_propval *val);
bool no_thermal;
/* For APM emulation, think legacy userspace. */
int use_for_apm;
+};
+
+struct power_supply {
+ const struct power_supply_desc *desc;
+
+ char **supplied_to;
+ size_t num_supplicants;
+
+ char **supplied_from;
+ size_t num_supplies;
+ struct device_node *of_node;
+
+ /* Driver private data */
+ void *drv_data;
/* private */
- struct device *dev;
+ struct device dev;
struct work_struct changed_work;
spinlock_t changed_lock;
bool changed;
+ atomic_t use_cnt;
#ifdef CONFIG_THERMAL
struct thermal_zone_device *tzd;
struct thermal_cooling_device *tcd;
extern int power_supply_reg_notifier(struct notifier_block *nb);
extern void power_supply_unreg_notifier(struct notifier_block *nb);
extern struct power_supply *power_supply_get_by_name(const char *name);
+extern void power_supply_put(struct power_supply *psy);
#ifdef CONFIG_OF
extern struct power_supply *power_supply_get_by_phandle(struct device_node *np,
const char *property);
static inline int power_supply_is_system_supplied(void) { return -ENOSYS; }
#endif
-extern int power_supply_register(struct device *parent,
- struct power_supply *psy);
-extern int power_supply_register_no_ws(struct device *parent,
- struct power_supply *psy);
+extern int power_supply_get_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ union power_supply_propval *val);
+extern int power_supply_set_property(struct power_supply *psy,
+ enum power_supply_property psp,
+ const union power_supply_propval *val);
+extern int power_supply_property_is_writeable(struct power_supply *psy,
+ enum power_supply_property psp);
+extern void power_supply_external_power_changed(struct power_supply *psy);
+
+extern struct power_supply *__must_check
+power_supply_register(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg);
+extern struct power_supply *__must_check
+power_supply_register_no_ws(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg);
+extern struct power_supply *__must_check
+devm_power_supply_register(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg);
+extern struct power_supply *__must_check
+devm_power_supply_register_no_ws(struct device *parent,
+ const struct power_supply_desc *desc,
+ const struct power_supply_config *cfg);
extern void power_supply_unregister(struct power_supply *psy);
extern int power_supply_powers(struct power_supply *psy, struct device *dev);
+extern void *power_supply_get_drvdata(struct power_supply *psy);
/* For APM emulation, think legacy userspace. */
extern struct class *power_supply_class;
#include <linux/regulator/machine.h>
+enum {
+ ACT8600_ID_DCDC1,
+ ACT8600_ID_DCDC2,
+ ACT8600_ID_DCDC3,
+ ACT8600_ID_SUDCDC4,
+ ACT8600_ID_LDO5,
+ ACT8600_ID_LDO6,
+ ACT8600_ID_LDO7,
+ ACT8600_ID_LDO8,
+ ACT8600_ID_LDO9,
+ ACT8600_ID_LDO10,
+};
+
enum {
ACT8865_ID_DCDC1,
ACT8865_ID_DCDC2,
};
enum {
+ ACT8600,
ACT8865,
ACT8846,
};
#define REGULATOR_EVENT_OVER_TEMP 0x10
#define REGULATOR_EVENT_FORCE_DISABLE 0x20
#define REGULATOR_EVENT_VOLTAGE_CHANGE 0x40
-#define REGULATOR_EVENT_DISABLE 0x80
+#define REGULATOR_EVENT_DISABLE 0x80
#define REGULATOR_EVENT_PRE_VOLTAGE_CHANGE 0x100
#define REGULATOR_EVENT_ABORT_VOLTAGE_CHANGE 0x200
#define REGULATOR_EVENT_PRE_DISABLE 0x400
int regulator_set_mode(struct regulator *regulator, unsigned int mode);
unsigned int regulator_get_mode(struct regulator *regulator);
-int regulator_set_optimum_mode(struct regulator *regulator, int load_uA);
+int regulator_set_load(struct regulator *regulator, int load_uA);
int regulator_allow_bypass(struct regulator *regulator, bool allow);
/* regulator notifier block */
int regulator_register_notifier(struct regulator *regulator,
struct notifier_block *nb);
+int devm_regulator_register_notifier(struct regulator *regulator,
+ struct notifier_block *nb);
int regulator_unregister_notifier(struct regulator *regulator,
struct notifier_block *nb);
+void devm_regulator_unregister_notifier(struct regulator *regulator,
+ struct notifier_block *nb);
/* driver data - core doesn't touch */
void *regulator_get_drvdata(struct regulator *regulator);
return REGULATOR_MODE_NORMAL;
}
-static inline int regulator_set_optimum_mode(struct regulator *regulator,
- int load_uA)
+static inline int regulator_set_load(struct regulator *regulator, int load_uA)
{
return REGULATOR_MODE_NORMAL;
}
return 0;
}
+static inline int devm_regulator_register_notifier(struct regulator *regulator,
+ struct notifier_block *nb)
+{
+ return 0;
+}
+
static inline int regulator_unregister_notifier(struct regulator *regulator,
struct notifier_block *nb)
{
return 0;
}
+static inline int devm_regulator_unregister_notifier(struct regulator *regulator,
+ struct notifier_block *nb)
+{
+ return 0;
+}
+
static inline void *regulator_get_drvdata(struct regulator *regulator)
{
return NULL;
* REGULATOR_STATUS value (or negative errno)
* @get_optimum_mode: Get the most efficient operating mode for the regulator
* when running with the specified parameters.
+ * @set_load: Set the load for the regulator.
*
* @set_bypass: Set the regulator in bypass mode.
* @get_bypass: Get the regulator bypass mode state.
/* get most efficient regulator operating mode for load */
unsigned int (*get_optimum_mode) (struct regulator_dev *, int input_uV,
int output_uV, int load_uA);
+ /* set the load on the regulator */
+ int (*set_load)(struct regulator_dev *, int load_uA);
/* control and report on bypass mode */
int (*set_bypass)(struct regulator_dev *dev, bool enable);
* @driver_data: private regulator data
* @of_node: OpenFirmware node to parse for device tree bindings (may be
* NULL).
- * @regmap: regmap to use for core regmap helpers if dev_get_regulator() is
+ * @regmap: regmap to use for core regmap helpers if dev_get_regmap() is
* insufficient.
* @ena_gpio_initialized: GPIO controlling regulator enable was properly
* initialized, meaning that >= 0 is a valid gpio
struct device dev;
struct regulation_constraints *constraints;
struct regulator *supply; /* for tree */
+ const char *supply_name;
struct regmap *regmap;
struct delayed_work disable_work;
* @buckets: size * hash buckets
*/
struct bucket_table {
- size_t size;
- unsigned int locks_mask;
- spinlock_t *locks;
- struct rhash_head __rcu *buckets[];
+ size_t size;
+ unsigned int locks_mask;
+ spinlock_t *locks;
+
+ struct rhash_head __rcu *buckets[] ____cacheline_aligned_in_smp;
};
typedef u32 (*rht_hashfn_t)(const void *data, u32 len, u32 seed);
* @locks_mul: Number of bucket locks to allocate per cpu (default: 128)
* @hashfn: Function to hash key
* @obj_hashfn: Function to hash object
- * @grow_decision: If defined, may return true if table should expand
- * @shrink_decision: If defined, may return true if table should shrink
- *
- * Note: when implementing the grow and shrink decision function, min/max
- * shift must be enforced, otherwise, resizing watermarks they set may be
- * useless.
*/
struct rhashtable_params {
size_t nelem_hint;
size_t locks_mul;
rht_hashfn_t hashfn;
rht_obj_hashfn_t obj_hashfn;
- bool (*grow_decision)(const struct rhashtable *ht,
- size_t new_size);
- bool (*shrink_decision)(const struct rhashtable *ht,
- size_t new_size);
};
/**
void rhashtable_insert(struct rhashtable *ht, struct rhash_head *node);
bool rhashtable_remove(struct rhashtable *ht, struct rhash_head *node);
-bool rht_grow_above_75(const struct rhashtable *ht, size_t new_size);
-bool rht_shrink_below_30(const struct rhashtable *ht, size_t new_size);
-
int rhashtable_expand(struct rhashtable *ht);
int rhashtable_shrink(struct rhashtable *ht);
int (*read_alarm)(struct device *, struct rtc_wkalrm *);
int (*set_alarm)(struct device *, struct rtc_wkalrm *);
int (*proc)(struct device *, struct seq_file *);
+ int (*set_mmss64)(struct device *, time64_t secs);
int (*set_mmss)(struct device *, unsigned long secs);
int (*read_callback)(struct device *, int data);
int (*alarm_irq_enable)(struct device *, unsigned int enabled);
extern void calc_global_load(unsigned long ticks);
extern void update_cpu_load_nohz(void);
+/* Notifier for when a task gets migrated to a new CPU */
+struct task_migration_notifier {
+ struct task_struct *task;
+ int from_cpu;
+ int to_cpu;
+};
+extern void register_task_migration_notifier(struct notifier_block *n);
+
extern unsigned long get_parent_ip(unsigned long addr);
extern void dump_cpu_task(int cpu);
};
struct sched_avg {
+ u64 last_runnable_update;
+ s64 decay_count;
+ /*
+ * utilization_avg_contrib describes the amount of time that a
+ * sched_entity is running on a CPU. It is based on running_avg_sum
+ * and is scaled in the range [0..SCHED_LOAD_SCALE].
+ * load_avg_contrib described the amount of time that a sched_entity
+ * is runnable on a rq. It is based on both runnable_avg_sum and the
+ * weight of the task.
+ */
+ unsigned long load_avg_contrib, utilization_avg_contrib;
/*
* These sums represent an infinite geometric series and so are bound
* above by 1024/(1-y). Thus we only need a u32 to store them for all
* choices of y < 1-2^(-32)*1024.
+ * running_avg_sum reflects the time that the sched_entity is
+ * effectively running on the CPU.
+ * runnable_avg_sum represents the amount of time a sched_entity is on
+ * a runqueue which includes the running time that is monitored by
+ * running_avg_sum.
*/
- u32 runnable_avg_sum, runnable_avg_period;
- u64 last_runnable_update;
- s64 decay_count;
- unsigned long load_avg_contrib;
+ u32 runnable_avg_sum, avg_period, running_avg_sum;
};
#ifdef CONFIG_SCHEDSTATS
/*
* numa_faults_locality tracks if faults recorded during the last
- * scan window were remote/local. The task scan period is adapted
- * based on the locality of the faults with different weights
- * depending on whether they were shared or private faults
+ * scan window were remote/local or failed to migrate. The task scan
+ * period is adapted based on the locality of the faults with different
+ * weights depending on whether they were shared or private faults
*/
- unsigned long numa_faults_locality[2];
+ unsigned long numa_faults_locality[3];
unsigned long numa_pages_migrated;
#endif /* CONFIG_NUMA_BALANCING */
#define TNF_NO_GROUP 0x02
#define TNF_SHARED 0x04
#define TNF_FAULT_LOCAL 0x08
+#define TNF_MIGRATE_FAIL 0x10
#ifdef CONFIG_NUMA_BALANCING
extern void task_numa_fault(int last_node, int node, int pages, int flags);
unsigned ret;
repeat:
- ret = ACCESS_ONCE(s->sequence);
+ ret = READ_ONCE(s->sequence);
if (unlikely(ret & 1)) {
cpu_relax();
goto repeat;
*/
static inline unsigned raw_read_seqcount(const seqcount_t *s)
{
- unsigned ret = ACCESS_ONCE(s->sequence);
+ unsigned ret = READ_ONCE(s->sequence);
smp_rmb();
return ret;
}
*/
static inline unsigned raw_seqcount_begin(const seqcount_t *s)
{
- unsigned ret = ACCESS_ONCE(s->sequence);
+ unsigned ret = READ_ONCE(s->sequence);
smp_rmb();
return ret & ~1;
}
unsigned char iotype; /* io access style */
unsigned char unused1;
-#define UPIO_PORT (0) /* 8b I/O port access */
-#define UPIO_HUB6 (1) /* Hub6 ISA card */
-#define UPIO_MEM (2) /* 8b MMIO access */
-#define UPIO_MEM32 (3) /* 32b little endian */
-#define UPIO_MEM32BE (4) /* 32b big endian */
-#define UPIO_AU (5) /* Au1x00 and RT288x type IO */
-#define UPIO_TSI (6) /* Tsi108/109 type IO */
+#define UPIO_PORT (SERIAL_IO_PORT) /* 8b I/O port access */
+#define UPIO_HUB6 (SERIAL_IO_HUB6) /* Hub6 ISA card */
+#define UPIO_MEM (SERIAL_IO_MEM) /* 8b MMIO access */
+#define UPIO_MEM32 (SERIAL_IO_MEM32) /* 32b little endian */
+#define UPIO_AU (SERIAL_IO_AU) /* Au1x00 and RT288x type IO */
+#define UPIO_TSI (SERIAL_IO_TSI) /* Tsi108/109 type IO */
+#define UPIO_MEM32BE (SERIAL_IO_MEM32BE) /* 32b big endian */
unsigned int read_status_mask; /* driver specific */
unsigned int ignore_status_mask; /* driver specific */
to->l4_hash = from->l4_hash;
};
+static inline void skb_sender_cpu_clear(struct sk_buff *skb)
+{
+#ifdef CONFIG_XPS
+ skb->sender_cpu = 0;
+#endif
+}
+
#ifdef NET_SKBUFF_DATA_USES_OFFSET
static inline unsigned char *skb_end_pointer(const struct sk_buff *skb)
{
* @remove: Unbinds this driver from the spi device
* @shutdown: Standard shutdown callback used during system state
* transitions such as powerdown/halt and kexec
- * @suspend: Standard suspend callback used during system state transitions
- * @resume: Standard resume callback used during system state transitions
* @driver: SPI device drivers should initialize the name and owner
* field of this structure.
*
int (*probe)(struct spi_device *spi);
int (*remove)(struct spi_device *spi);
void (*shutdown)(struct spi_device *spi);
- int (*suspend)(struct spi_device *spi, pm_message_t mesg);
- int (*resume)(struct spi_device *spi);
struct device_driver driver;
};
* transfer_one_message are mutually exclusive; when both
* are set, the generic subsystem does not call your
* transfer_one callback.
+ * @handle_err: the subsystem calls the driver to handle an error that occurs
+ * in the generic implementation of transfer_one_message().
* @unprepare_message: undo any work done by prepare_message().
* @cs_gpios: Array of GPIOs to use as chip select lines; one per CS
* number. Any individual value may be -ENOENT for CS lines that
void (*set_cs)(struct spi_device *spi, bool enable);
int (*transfer_one)(struct spi_master *master, struct spi_device *spi,
struct spi_transfer *transfer);
+ void (*handle_err)(struct spi_master *master,
+ struct spi_message *message);
/* gpio chip select */
int *cs_gpios;
* sequence completes. On some systems, many such sequences can execute as
* as single programmed DMA transfer. On all systems, these messages are
* queued, and might complete after transactions to other devices. Messages
- * sent to a given spi_device are alway executed in FIFO order.
+ * sent to a given spi_device are always executed in FIFO order.
*
* The code that submits an spi_message (and its spi_transfers)
* to the lower layers is responsible for managing its memory.
#define offsetof(TYPE, MEMBER) ((size_t) &((TYPE *)0)->MEMBER)
#endif
#endif
+
+/**
+ * offsetofend(TYPE, MEMBER)
+ *
+ * @TYPE: The type of the structure
+ * @MEMBER: The member within the structure to get the end offset of
+ */
+#define offsetofend(TYPE, MEMBER) \
+ (offsetof(TYPE, MEMBER) + sizeof(((TYPE *)0)->MEMBER))
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
void rpc_register_sysctl(void);
void rpc_unregister_sysctl(void);
-int sunrpc_debugfs_init(void);
+void sunrpc_debugfs_init(void);
void sunrpc_debugfs_exit(void);
-int rpc_clnt_debugfs_register(struct rpc_clnt *);
+void rpc_clnt_debugfs_register(struct rpc_clnt *);
void rpc_clnt_debugfs_unregister(struct rpc_clnt *);
-int rpc_xprt_debugfs_register(struct rpc_xprt *);
+void rpc_xprt_debugfs_register(struct rpc_xprt *);
void rpc_xprt_debugfs_unregister(struct rpc_xprt *);
#else
-static inline int
+static inline void
sunrpc_debugfs_init(void)
{
- return 0;
+ return;
}
static inline void
return;
}
-static inline int
+static inline void
rpc_clnt_debugfs_register(struct rpc_clnt *clnt)
{
- return 0;
+ return;
}
static inline void
return;
}
-static inline int
+static inline void
rpc_xprt_debugfs_register(struct rpc_xprt *xprt)
{
- return 0;
+ return;
}
static inline void
}
#endif
+
+#if IS_ENABLED(CONFIG_THERMAL)
struct thermal_zone_device *thermal_zone_device_register(const char *, int, int,
void *, struct thermal_zone_device_ops *,
const struct thermal_zone_params *, int, int);
struct thermal_cooling_device *, int);
void thermal_cdev_update(struct thermal_cooling_device *);
void thermal_notify_framework(struct thermal_zone_device *, int);
-
-#ifdef CONFIG_NET
+#else
+static inline struct thermal_zone_device *thermal_zone_device_register(
+ const char *type, int trips, int mask, void *devdata,
+ struct thermal_zone_device_ops *ops,
+ const struct thermal_zone_params *tzp,
+ int passive_delay, int polling_delay)
+{ return ERR_PTR(-ENODEV); }
+static inline void thermal_zone_device_unregister(
+ struct thermal_zone_device *tz)
+{ }
+static inline int thermal_zone_bind_cooling_device(
+ struct thermal_zone_device *tz, int trip,
+ struct thermal_cooling_device *cdev,
+ unsigned long upper, unsigned long lower)
+{ return -ENODEV; }
+static inline int thermal_zone_unbind_cooling_device(
+ struct thermal_zone_device *tz, int trip,
+ struct thermal_cooling_device *cdev)
+{ return -ENODEV; }
+static inline void thermal_zone_device_update(struct thermal_zone_device *tz)
+{ }
+static inline struct thermal_cooling_device *
+thermal_cooling_device_register(char *type, void *devdata,
+ const struct thermal_cooling_device_ops *ops)
+{ return ERR_PTR(-ENODEV); }
+static inline struct thermal_cooling_device *
+thermal_of_cooling_device_register(struct device_node *np,
+ char *type, void *devdata, const struct thermal_cooling_device_ops *ops)
+{ return ERR_PTR(-ENODEV); }
+static inline void thermal_cooling_device_unregister(
+ struct thermal_cooling_device *cdev)
+{ }
+static inline struct thermal_zone_device *thermal_zone_get_zone_by_name(
+ const char *name)
+{ return ERR_PTR(-ENODEV); }
+static inline int thermal_zone_get_temp(
+ struct thermal_zone_device *tz, unsigned long *temp)
+{ return -ENODEV; }
+static inline int get_tz_trend(struct thermal_zone_device *tz, int trip)
+{ return -ENODEV; }
+static inline struct thermal_instance *
+get_thermal_instance(struct thermal_zone_device *tz,
+ struct thermal_cooling_device *cdev, int trip)
+{ return ERR_PTR(-ENODEV); }
+static inline void thermal_cdev_update(struct thermal_cooling_device *cdev)
+{ }
+static inline void thermal_notify_framework(struct thermal_zone_device *tz,
+ int trip)
+{ }
+#endif /* CONFIG_THERMAL */
+
+#if defined(CONFIG_NET) && IS_ENABLED(CONFIG_THERMAL)
extern int thermal_generate_netlink_event(struct thermal_zone_device *tz,
enum events event);
#else
-/* linux/include/linux/tick.h
- *
- * This file contains the structure definitions for tick related functions
- *
+/*
+ * Tick related global functions
*/
#ifndef _LINUX_TICK_H
#define _LINUX_TICK_H
#include <linux/clockchips.h>
#include <linux/irqflags.h>
#include <linux/percpu.h>
-#include <linux/hrtimer.h>
#include <linux/context_tracking_state.h>
#include <linux/cpumask.h>
#include <linux/sched.h>
#ifdef CONFIG_GENERIC_CLOCKEVENTS
-
-enum tick_device_mode {
- TICKDEV_MODE_PERIODIC,
- TICKDEV_MODE_ONESHOT,
-};
-
-struct tick_device {
- struct clock_event_device *evtdev;
- enum tick_device_mode mode;
-};
-
-enum tick_nohz_mode {
- NOHZ_MODE_INACTIVE,
- NOHZ_MODE_LOWRES,
- NOHZ_MODE_HIGHRES,
-};
-
-/**
- * struct tick_sched - sched tick emulation and no idle tick control/stats
- * @sched_timer: hrtimer to schedule the periodic tick in high
- * resolution mode
- * @last_tick: Store the last tick expiry time when the tick
- * timer is modified for nohz sleeps. This is necessary
- * to resume the tick timer operation in the timeline
- * when the CPU returns from nohz sleep.
- * @tick_stopped: Indicator that the idle tick has been stopped
- * @idle_jiffies: jiffies at the entry to idle for idle time accounting
- * @idle_calls: Total number of idle calls
- * @idle_sleeps: Number of idle calls, where the sched tick was stopped
- * @idle_entrytime: Time when the idle call was entered
- * @idle_waketime: Time when the idle was interrupted
- * @idle_exittime: Time when the idle state was left
- * @idle_sleeptime: Sum of the time slept in idle with sched tick stopped
- * @iowait_sleeptime: Sum of the time slept in idle with sched tick stopped, with IO outstanding
- * @sleep_length: Duration of the current idle sleep
- * @do_timer_lst: CPU was the last one doing do_timer before going idle
- */
-struct tick_sched {
- struct hrtimer sched_timer;
- unsigned long check_clocks;
- enum tick_nohz_mode nohz_mode;
- ktime_t last_tick;
- int inidle;
- int tick_stopped;
- unsigned long idle_jiffies;
- unsigned long idle_calls;
- unsigned long idle_sleeps;
- int idle_active;
- ktime_t idle_entrytime;
- ktime_t idle_waketime;
- ktime_t idle_exittime;
- ktime_t idle_sleeptime;
- ktime_t iowait_sleeptime;
- ktime_t sleep_length;
- unsigned long last_jiffies;
- unsigned long next_jiffies;
- ktime_t idle_expires;
- int do_timer_last;
-};
-
extern void __init tick_init(void);
-extern int tick_is_oneshot_available(void);
-extern struct tick_device *tick_get_device(int cpu);
-
extern void tick_freeze(void);
extern void tick_unfreeze(void);
+/* Should be core only, but ARM BL switcher requires it */
+extern void tick_suspend_local(void);
+/* Should be core only, but XEN resume magic and ARM BL switcher require it */
+extern void tick_resume_local(void);
+extern void tick_handover_do_timer(void);
+extern void tick_cleanup_dead_cpu(int cpu);
+#else /* CONFIG_GENERIC_CLOCKEVENTS */
+static inline void tick_init(void) { }
+static inline void tick_freeze(void) { }
+static inline void tick_unfreeze(void) { }
+static inline void tick_suspend_local(void) { }
+static inline void tick_resume_local(void) { }
+static inline void tick_handover_do_timer(void) { }
+static inline void tick_cleanup_dead_cpu(int cpu) { }
+#endif /* !CONFIG_GENERIC_CLOCKEVENTS */
-# ifdef CONFIG_HIGH_RES_TIMERS
-extern int tick_init_highres(void);
-extern int tick_program_event(ktime_t expires, int force);
-extern void tick_setup_sched_timer(void);
-# endif
-
-# if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS
-extern void tick_cancel_sched_timer(int cpu);
-# else
-static inline void tick_cancel_sched_timer(int cpu) { }
-# endif
-
-# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
-extern struct tick_device *tick_get_broadcast_device(void);
-extern struct cpumask *tick_get_broadcast_mask(void);
-
-# ifdef CONFIG_TICK_ONESHOT
-extern struct cpumask *tick_get_broadcast_oneshot_mask(void);
-# endif
-
-# endif /* BROADCAST */
-
-# ifdef CONFIG_TICK_ONESHOT
-extern void tick_clock_notify(void);
-extern int tick_check_oneshot_change(int allow_nohz);
-extern struct tick_sched *tick_get_tick_sched(int cpu);
+#ifdef CONFIG_TICK_ONESHOT
extern void tick_irq_enter(void);
-extern int tick_oneshot_mode_active(void);
# ifndef arch_needs_cpu
# define arch_needs_cpu() (0)
# endif
# else
-static inline void tick_clock_notify(void) { }
-static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
static inline void tick_irq_enter(void) { }
-static inline int tick_oneshot_mode_active(void) { return 0; }
-# endif
+#endif
-#else /* CONFIG_GENERIC_CLOCKEVENTS */
-static inline void tick_init(void) { }
-static inline void tick_freeze(void) { }
-static inline void tick_unfreeze(void) { }
-static inline void tick_cancel_sched_timer(int cpu) { }
-static inline void tick_clock_notify(void) { }
-static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
-static inline void tick_irq_enter(void) { }
-static inline int tick_oneshot_mode_active(void) { return 0; }
-#endif /* !CONFIG_GENERIC_CLOCKEVENTS */
+#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT)
+extern void hotplug_cpu__broadcast_tick_pull(int dead_cpu);
+#else
+static inline void hotplug_cpu__broadcast_tick_pull(int dead_cpu) { }
+#endif
-# ifdef CONFIG_NO_HZ_COMMON
-DECLARE_PER_CPU(struct tick_sched, tick_cpu_sched);
+enum tick_broadcast_mode {
+ TICK_BROADCAST_OFF,
+ TICK_BROADCAST_ON,
+ TICK_BROADCAST_FORCE,
+};
+
+enum tick_broadcast_state {
+ TICK_BROADCAST_EXIT,
+ TICK_BROADCAST_ENTER,
+};
+
+#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
+extern void tick_broadcast_control(enum tick_broadcast_mode mode);
+#else
+static inline void tick_broadcast_control(enum tick_broadcast_mode mode) { }
+#endif /* BROADCAST */
+
+#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT)
+extern int tick_broadcast_oneshot_control(enum tick_broadcast_state state);
+#else
+static inline int tick_broadcast_oneshot_control(enum tick_broadcast_state state) { return 0; }
+#endif
-static inline int tick_nohz_tick_stopped(void)
+static inline void tick_broadcast_enable(void)
+{
+ tick_broadcast_control(TICK_BROADCAST_ON);
+}
+static inline void tick_broadcast_disable(void)
+{
+ tick_broadcast_control(TICK_BROADCAST_OFF);
+}
+static inline void tick_broadcast_force(void)
+{
+ tick_broadcast_control(TICK_BROADCAST_FORCE);
+}
+static inline int tick_broadcast_enter(void)
{
- return __this_cpu_read(tick_cpu_sched.tick_stopped);
+ return tick_broadcast_oneshot_control(TICK_BROADCAST_ENTER);
+}
+static inline void tick_broadcast_exit(void)
+{
+ tick_broadcast_oneshot_control(TICK_BROADCAST_EXIT);
}
+#ifdef CONFIG_NO_HZ_COMMON
+extern int tick_nohz_tick_stopped(void);
extern void tick_nohz_idle_enter(void);
extern void tick_nohz_idle_exit(void);
extern void tick_nohz_irq_exit(void);
extern ktime_t tick_nohz_get_sleep_length(void);
extern u64 get_cpu_idle_time_us(int cpu, u64 *last_update_time);
extern u64 get_cpu_iowait_time_us(int cpu, u64 *last_update_time);
-
-# else /* !CONFIG_NO_HZ_COMMON */
-static inline int tick_nohz_tick_stopped(void)
-{
- return 0;
-}
-
+#else /* !CONFIG_NO_HZ_COMMON */
+static inline int tick_nohz_tick_stopped(void) { return 0; }
static inline void tick_nohz_idle_enter(void) { }
static inline void tick_nohz_idle_exit(void) { }
}
static inline u64 get_cpu_idle_time_us(int cpu, u64 *unused) { return -1; }
static inline u64 get_cpu_iowait_time_us(int cpu, u64 *unused) { return -1; }
-# endif /* !CONFIG_NO_HZ_COMMON */
+#endif /* !CONFIG_NO_HZ_COMMON */
#ifdef CONFIG_NO_HZ_FULL
extern bool tick_nohz_full_running;
* @read: Read function of @clock
* @mask: Bitmask for two's complement subtraction of non 64bit clocks
* @cycle_last: @clock cycle value at last update
- * @mult: NTP adjusted multiplier for scaled math conversion
+ * @mult: (NTP adjusted) multiplier for scaled math conversion
* @shift: Shift value for scaled math conversion
* @xtime_nsec: Shifted (fractional) nano seconds offset for readout
- * @base_mono: ktime_t (nanoseconds) base time for readout
+ * @base: ktime_t (nanoseconds) base time for readout
*
* This struct has size 56 byte on 64 bit. Together with a seqcount it
* occupies a single 64byte cache line.
*
* The struct is separate from struct timekeeper as it is also used
- * for a fast NMI safe accessor to clock monotonic.
+ * for a fast NMI safe accessors.
*/
struct tk_read_base {
struct clocksource *clock;
u32 mult;
u32 shift;
u64 xtime_nsec;
- ktime_t base_mono;
+ ktime_t base;
};
/**
* struct timekeeper - Structure holding internal timekeeping values.
- * @tkr: The readout base structure
+ * @tkr_mono: The readout base structure for CLOCK_MONOTONIC
+ * @tkr_raw: The readout base structure for CLOCK_MONOTONIC_RAW
* @xtime_sec: Current CLOCK_REALTIME time in seconds
* @ktime_sec: Current CLOCK_MONOTONIC time in seconds
* @wall_to_monotonic: CLOCK_REALTIME to CLOCK_MONOTONIC offset
* @offs_boot: Offset clock monotonic -> clock boottime
* @offs_tai: Offset clock monotonic -> clock tai
* @tai_offset: The current UTC to TAI offset in seconds
- * @base_raw: Monotonic raw base time in ktime_t format
* @raw_time: Monotonic raw base time in timespec64 format
* @cycle_interval: Number of clock cycles in one NTP interval
* @xtime_interval: Number of clock shifted nano seconds in one NTP
* used instead.
*/
struct timekeeper {
- struct tk_read_base tkr;
+ struct tk_read_base tkr_mono;
+ struct tk_read_base tkr_raw;
u64 xtime_sec;
unsigned long ktime_sec;
struct timespec64 wall_to_monotonic;
ktime_t offs_boot;
ktime_t offs_tai;
s32 tai_offset;
- ktime_t base_raw;
struct timespec64 raw_time;
/* The following members are for timekeeping internal use */
return ktime_to_ns(ktime_get_boottime());
}
+static inline u64 ktime_get_tai_ns(void)
+{
+ return ktime_to_ns(ktime_get_clocktai());
+}
+
static inline u64 ktime_get_raw_ns(void)
{
return ktime_to_ns(ktime_get_raw());
}
extern u64 ktime_get_mono_fast_ns(void);
+extern u64 ktime_get_raw_fast_ns(void);
/*
* Timespec interfaces utilizing the ktime based ones
/*
* RTC specific
*/
+extern bool timekeeping_rtc_skipsuspend(void);
+extern bool timekeeping_rtc_skipresume(void);
+
extern void timekeeping_inject_sleeptime64(struct timespec64 *delta);
/*
/*
* Persistent clock related interfaces
*/
-extern bool persistent_clock_exist;
extern int persistent_clock_is_local;
-static inline bool has_persistent_clock(void)
-{
- return persistent_clock_exist;
-}
-
extern void read_persistent_clock(struct timespec *ts);
+extern void read_persistent_clock64(struct timespec64 *ts);
extern void read_boot_clock(struct timespec *ts);
+extern void read_boot_clock64(struct timespec64 *ts);
extern int update_persistent_clock(struct timespec now);
+extern int update_persistent_clock64(struct timespec64 now);
#endif
size_t maxsize, size_t *start);
int iov_iter_npages(const struct iov_iter *i, int maxpages);
+const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags);
+
static inline size_t iov_iter_count(struct iov_iter *i)
{
return i->count;
* @num_ports: the number of different ports this device will have.
* @bulk_in_size: minimum number of bytes to allocate for bulk-in buffer
* (0 = end-point size)
- * @bulk_out_size: minimum number of bytes to allocate for bulk-out buffer
- * (0 = end-point size)
+ * @bulk_out_size: bytes to allocate for bulk-out buffer (0 = end-point size)
* @calc_num_ports: pointer to a function to determine how many ports this
* device has dynamically. It will be called after the probe()
* callback is called, but before attach()
struct urb *urb;
struct usbnet *dev;
enum skb_state state;
- size_t length;
+ long length;
+ unsigned long packets;
};
+/* Drivers that set FLAG_MULTI_PACKET must call this in their
+ * tx_fixup method before returning an skb.
+ */
+static inline void
+usbnet_set_skb_tx_stats(struct sk_buff *skb,
+ unsigned long packets, long bytes_delta)
+{
+ struct skb_data *entry = (struct skb_data *) skb->cb;
+
+ entry->packets = packets;
+ entry->length = bytes_delta;
+}
+
extern int usbnet_open(struct net_device *net);
extern int usbnet_stop(struct net_device *net);
extern netdev_tx_t usbnet_start_xmit(struct sk_buff *skb,
extern void vfio_unregister_iommu_driver(
const struct vfio_iommu_driver_ops *ops);
-/**
- * offsetofend(TYPE, MEMBER)
- *
- * @TYPE: The type of the structure
- * @MEMBER: The member within the structure to get the end offset of
- *
- * Simple helper macro for dealing with variable sized structures passed
- * from user space. This allows us to easily determine if the provided
- * structure is sized to include various fields.
- */
-#define offsetofend(TYPE, MEMBER) \
- (offsetof(TYPE, MEMBER) + sizeof(((TYPE *)0)->MEMBER))
-
/*
* External user API
*/
#define VM_VPAGES 0x00000010 /* buffer for pages was vmalloc'ed */
#define VM_UNINITIALIZED 0x00000020 /* vm_struct is not fully initialized */
#define VM_NO_GUARD 0x00000040 /* don't add guard page */
+#define VM_KASAN 0x00000080 /* has allocated kasan shadow memory */
/* bits [20..32] reserved for arch specific ioremap internals */
/*
/* data contains off-queue information when !WORK_STRUCT_PWQ */
WORK_OFFQ_FLAG_BASE = WORK_STRUCT_COLOR_SHIFT,
- WORK_OFFQ_CANCELING = (1 << WORK_OFFQ_FLAG_BASE),
+ __WORK_OFFQ_CANCELING = WORK_OFFQ_FLAG_BASE,
+ WORK_OFFQ_CANCELING = (1 << __WORK_OFFQ_CANCELING),
/*
* When a work item is off queue, its high bits point to the last
extern unsigned int work_busy(struct work_struct *work);
extern __printf(1, 2) void set_worker_desc(const char *fmt, ...);
extern void print_worker_info(const char *log_lvl, struct task_struct *task);
+extern void show_workqueue_state(void);
/**
* queue_work - queue work on a workqueue
extern unsigned long vm_dirty_bytes;
extern unsigned int dirty_writeback_interval;
extern unsigned int dirty_expire_interval;
+extern unsigned int dirtytime_expire_interval;
extern int vm_highmem_is_dirtyable;
extern int block_dump;
extern int laptop_mode;
extern int dirty_bytes_handler(struct ctl_table *table, int write,
void __user *buffer, size_t *lenp,
loff_t *ppos);
+int dirtytime_interval_handler(struct ctl_table *table, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos);
struct ctl_table;
int dirty_writeback_centisecs_handler(struct ctl_table *, int,
#define ISI_CFG1_FRATE_DIV_MASK (7 << 8)
#define ISI_CFG1_DISCR (1 << 11)
#define ISI_CFG1_FULL_MODE (1 << 12)
+/* Definition for THMASK(ISI_V2) */
+#define ISI_CFG1_THMASK_BEATS_4 (0 << 13)
+#define ISI_CFG1_THMASK_BEATS_8 (1 << 13)
+#define ISI_CFG1_THMASK_BEATS_16 (2 << 13)
/* Bitfields in CFG2 */
#define ISI_CFG2_GRAYSCALE (1 << 13)
* @return Checksum of buffer.
*/
-u16 cfpkt_iterate(struct cfpkt *pkt,
+int cfpkt_iterate(struct cfpkt *pkt,
u16 (*iter_func)(u16 chks, void *buf, u16 len),
u16 data);
enum {
XFRM_LOOKUP_ICMP = 1 << 0,
XFRM_LOOKUP_QUEUE = 1 << 1,
+ XFRM_LOOKUP_KEEP_DST_REF = 1 << 2,
};
struct flowi;
#endif
-static inline int sk_mc_loop(struct sock *sk)
-{
- if (!sk)
- return 1;
- switch (sk->sk_family) {
- case AF_INET:
- return inet_sk(sk)->mc_loop;
-#if IS_ENABLED(CONFIG_IPV6)
- case AF_INET6:
- return inet6_sk(sk)->mc_loop;
-#endif
- }
- WARN_ON(1);
- return 1;
-}
-
bool ip_call_ra_chain(struct sk_buff *skb);
/*
static inline int ip6_skb_dst_mtu(struct sk_buff *skb)
{
- struct ipv6_pinfo *np = skb->sk ? inet6_sk(skb->sk) : NULL;
+ struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ?
+ inet6_sk(skb->sk) : NULL;
return (np && np->pmtudisc >= IPV6_PMTUDISC_PROBE) ?
skb_dst(skb)->dev->mtu : dst_mtu(skb_dst(skb));
const struct nf_loginfo *li,
const char *fmt, ...);
+__printf(8, 9)
+void nf_log_trace(struct net *net,
+ u_int8_t pf,
+ unsigned int hooknum,
+ const struct sk_buff *skb,
+ const struct net_device *in,
+ const struct net_device *out,
+ const struct nf_loginfo *li,
+ const char *fmt, ...);
+
struct nf_log_buf;
struct nf_log_buf *nf_log_buf_open(void);
const struct nft_data *data,
enum nft_data_types type);
+
+/**
+ * struct nft_userdata - user defined data associated with an object
+ *
+ * @len: length of the data
+ * @data: content
+ *
+ * The presence of user data is indicated in an object specific fashion,
+ * so a length of zero can't occur and the value "len" indicates data
+ * of length len + 1.
+ */
+struct nft_userdata {
+ u8 len;
+ unsigned char data[0];
+};
+
/**
* struct nft_set_elem - generic representation of set elements
*
* @handle: rule handle
* @genmask: generation mask
* @dlen: length of expression data
- * @ulen: length of user data (used for comments)
+ * @udata: user data is appended to the rule
* @data: expression data
*/
struct nft_rule {
u64 handle:42,
genmask:2,
dlen:12,
- ulen:8;
+ udata:1;
unsigned char data[]
__attribute__((aligned(__alignof__(struct nft_expr))));
};
return (struct nft_expr *)&rule->data[rule->dlen];
}
-static inline void *nft_userdata(const struct nft_rule *rule)
+static inline struct nft_userdata *nft_userdata(const struct nft_rule *rule)
{
return (void *)&rule->data[rule->dlen];
}
struct dst_entry *sk_dst_check(struct sock *sk, u32 cookie);
+bool sk_mc_loop(struct sock *sk);
+
static inline bool sk_can_gso(const struct sock *sk)
{
return net_gso_ok(sk->sk_route_caps, sk->sk_gso_type);
#define VXLAN_N_VID (1u << 24)
#define VXLAN_VID_MASK (VXLAN_N_VID - 1)
+#define VXLAN_VNI_MASK (VXLAN_VID_MASK << 8)
#define VXLAN_HLEN (sizeof(struct udphdr) + sizeof(struct vxlanhdr))
struct vxlan_metadata {
u64 * info_out);
extern void scsi_build_sense_buffer(int desc, u8 *buf, u8 key, u8 asc, u8 ascq);
+extern void scsi_set_sense_information(u8 *buf, u64 info);
extern int scsi_ioctl_reset(struct scsi_device *, int __user *);
#define AT91_DDRSDRC_UPD_MR (3 << 20) /* Update load mode register and extended mode register */
#define AT91_DDRSDRC_MDR 0x20 /* Memory Device Register */
-#define AT91_DDRSDRC_MD (3 << 0) /* Memory Device Type */
+#define AT91_DDRSDRC_MD (7 << 0) /* Memory Device Type */
#define AT91_DDRSDRC_MD_SDR 0
#define AT91_DDRSDRC_MD_LOW_POWER_SDR 1
#define AT91_DDRSDRC_MD_LOW_POWER_DDR 3
void target_core_setup_sub_cits(struct se_subsystem_api *);
/* attribute helpers from target_core_device.c for backend drivers */
+bool se_dev_check_wce(struct se_device *);
int se_dev_set_max_unmap_lba_count(struct se_device *, u32);
int se_dev_set_max_unmap_block_desc_count(struct se_device *, u32);
int se_dev_set_unmap_granularity(struct se_device *, u32);
--- /dev/null
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM libata
+
+#if !defined(_TRACE_LIBATA_H) || defined(TRACE_HEADER_MULTI_READ)
+#define _TRACE_LIBATA_H
+
+#include <linux/ata.h>
+#include <linux/libata.h>
+#include <linux/tracepoint.h>
+#include <linux/trace_seq.h>
+
+#define ata_opcode_name(opcode) { opcode, #opcode }
+#define show_opcode_name(val) \
+ __print_symbolic(val, \
+ ata_opcode_name(ATA_CMD_DEV_RESET), \
+ ata_opcode_name(ATA_CMD_CHK_POWER), \
+ ata_opcode_name(ATA_CMD_STANDBY), \
+ ata_opcode_name(ATA_CMD_IDLE), \
+ ata_opcode_name(ATA_CMD_EDD), \
+ ata_opcode_name(ATA_CMD_DOWNLOAD_MICRO), \
+ ata_opcode_name(ATA_CMD_DOWNLOAD_MICRO_DMA), \
+ ata_opcode_name(ATA_CMD_NOP), \
+ ata_opcode_name(ATA_CMD_FLUSH), \
+ ata_opcode_name(ATA_CMD_FLUSH_EXT), \
+ ata_opcode_name(ATA_CMD_ID_ATA), \
+ ata_opcode_name(ATA_CMD_ID_ATAPI), \
+ ata_opcode_name(ATA_CMD_SERVICE), \
+ ata_opcode_name(ATA_CMD_READ), \
+ ata_opcode_name(ATA_CMD_READ_EXT), \
+ ata_opcode_name(ATA_CMD_READ_QUEUED), \
+ ata_opcode_name(ATA_CMD_READ_STREAM_EXT), \
+ ata_opcode_name(ATA_CMD_READ_STREAM_DMA_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE), \
+ ata_opcode_name(ATA_CMD_WRITE_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_QUEUED), \
+ ata_opcode_name(ATA_CMD_WRITE_STREAM_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_STREAM_DMA_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_FUA_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_QUEUED_FUA_EXT), \
+ ata_opcode_name(ATA_CMD_FPDMA_READ), \
+ ata_opcode_name(ATA_CMD_FPDMA_WRITE), \
+ ata_opcode_name(ATA_CMD_FPDMA_SEND), \
+ ata_opcode_name(ATA_CMD_FPDMA_RECV), \
+ ata_opcode_name(ATA_CMD_PIO_READ), \
+ ata_opcode_name(ATA_CMD_PIO_READ_EXT), \
+ ata_opcode_name(ATA_CMD_PIO_WRITE), \
+ ata_opcode_name(ATA_CMD_PIO_WRITE_EXT), \
+ ata_opcode_name(ATA_CMD_READ_MULTI), \
+ ata_opcode_name(ATA_CMD_READ_MULTI_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_MULTI), \
+ ata_opcode_name(ATA_CMD_WRITE_MULTI_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_MULTI_FUA_EXT), \
+ ata_opcode_name(ATA_CMD_SET_FEATURES), \
+ ata_opcode_name(ATA_CMD_SET_MULTI), \
+ ata_opcode_name(ATA_CMD_PACKET), \
+ ata_opcode_name(ATA_CMD_VERIFY), \
+ ata_opcode_name(ATA_CMD_VERIFY_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_UNCORR_EXT), \
+ ata_opcode_name(ATA_CMD_STANDBYNOW1), \
+ ata_opcode_name(ATA_CMD_IDLEIMMEDIATE), \
+ ata_opcode_name(ATA_CMD_SLEEP), \
+ ata_opcode_name(ATA_CMD_INIT_DEV_PARAMS), \
+ ata_opcode_name(ATA_CMD_READ_NATIVE_MAX), \
+ ata_opcode_name(ATA_CMD_READ_NATIVE_MAX_EXT), \
+ ata_opcode_name(ATA_CMD_SET_MAX), \
+ ata_opcode_name(ATA_CMD_SET_MAX_EXT), \
+ ata_opcode_name(ATA_CMD_READ_LOG_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_LOG_EXT), \
+ ata_opcode_name(ATA_CMD_READ_LOG_DMA_EXT), \
+ ata_opcode_name(ATA_CMD_WRITE_LOG_DMA_EXT), \
+ ata_opcode_name(ATA_CMD_TRUSTED_NONDATA), \
+ ata_opcode_name(ATA_CMD_TRUSTED_RCV), \
+ ata_opcode_name(ATA_CMD_TRUSTED_RCV_DMA), \
+ ata_opcode_name(ATA_CMD_TRUSTED_SND), \
+ ata_opcode_name(ATA_CMD_TRUSTED_SND_DMA), \
+ ata_opcode_name(ATA_CMD_PMP_READ), \
+ ata_opcode_name(ATA_CMD_PMP_READ_DMA), \
+ ata_opcode_name(ATA_CMD_PMP_WRITE), \
+ ata_opcode_name(ATA_CMD_PMP_WRITE_DMA), \
+ ata_opcode_name(ATA_CMD_CONF_OVERLAY), \
+ ata_opcode_name(ATA_CMD_SEC_SET_PASS), \
+ ata_opcode_name(ATA_CMD_SEC_UNLOCK), \
+ ata_opcode_name(ATA_CMD_SEC_ERASE_PREP), \
+ ata_opcode_name(ATA_CMD_SEC_ERASE_UNIT), \
+ ata_opcode_name(ATA_CMD_SEC_FREEZE_LOCK), \
+ ata_opcode_name(ATA_CMD_SEC_DISABLE_PASS), \
+ ata_opcode_name(ATA_CMD_CONFIG_STREAM), \
+ ata_opcode_name(ATA_CMD_SMART), \
+ ata_opcode_name(ATA_CMD_MEDIA_LOCK), \
+ ata_opcode_name(ATA_CMD_MEDIA_UNLOCK), \
+ ata_opcode_name(ATA_CMD_DSM), \
+ ata_opcode_name(ATA_CMD_CHK_MED_CRD_TYP), \
+ ata_opcode_name(ATA_CMD_CFA_REQ_EXT_ERR), \
+ ata_opcode_name(ATA_CMD_CFA_WRITE_NE), \
+ ata_opcode_name(ATA_CMD_CFA_TRANS_SECT), \
+ ata_opcode_name(ATA_CMD_CFA_ERASE), \
+ ata_opcode_name(ATA_CMD_CFA_WRITE_MULT_NE), \
+ ata_opcode_name(ATA_CMD_REQ_SENSE_DATA), \
+ ata_opcode_name(ATA_CMD_SANITIZE_DEVICE), \
+ ata_opcode_name(ATA_CMD_RESTORE), \
+ ata_opcode_name(ATA_CMD_READ_LONG), \
+ ata_opcode_name(ATA_CMD_READ_LONG_ONCE), \
+ ata_opcode_name(ATA_CMD_WRITE_LONG), \
+ ata_opcode_name(ATA_CMD_WRITE_LONG_ONCE))
+
+#define ata_error_name(result) { result, #result }
+#define show_error_name(val) \
+ __print_symbolic(val, \
+ ata_error_name(ATA_ICRC), \
+ ata_error_name(ATA_UNC), \
+ ata_error_name(ATA_MC), \
+ ata_error_name(ATA_IDNF), \
+ ata_error_name(ATA_MCR), \
+ ata_error_name(ATA_ABORTED), \
+ ata_error_name(ATA_TRK0NF), \
+ ata_error_name(ATA_AMNF))
+
+#define ata_protocol_name(proto) { proto, #proto }
+#define show_protocol_name(val) \
+ __print_symbolic(val, \
+ ata_protocol_name(ATA_PROT_UNKNOWN), \
+ ata_protocol_name(ATA_PROT_NODATA), \
+ ata_protocol_name(ATA_PROT_PIO), \
+ ata_protocol_name(ATA_PROT_DMA), \
+ ata_protocol_name(ATA_PROT_NCQ), \
+ ata_protocol_name(ATAPI_PROT_NODATA), \
+ ata_protocol_name(ATAPI_PROT_PIO), \
+ ata_protocol_name(ATAPI_PROT_DMA))
+
+const char *libata_trace_parse_status(struct trace_seq*, unsigned char);
+#define __parse_status(s) libata_trace_parse_status(p, s)
+
+const char *libata_trace_parse_eh_action(struct trace_seq *, unsigned int);
+#define __parse_eh_action(a) libata_trace_parse_eh_action(p, a)
+
+const char *libata_trace_parse_eh_err_mask(struct trace_seq *, unsigned int);
+#define __parse_eh_err_mask(m) libata_trace_parse_eh_err_mask(p, m)
+
+const char *libata_trace_parse_qc_flags(struct trace_seq *, unsigned int);
+#define __parse_qc_flags(f) libata_trace_parse_qc_flags(p, f)
+
+TRACE_EVENT(ata_qc_issue,
+
+ TP_PROTO(struct ata_queued_cmd *qc),
+
+ TP_ARGS(qc),
+
+ TP_STRUCT__entry(
+ __field( unsigned int, ata_port )
+ __field( unsigned int, ata_dev )
+ __field( unsigned int, tag )
+ __field( unsigned char, cmd )
+ __field( unsigned char, dev )
+ __field( unsigned char, lbal )
+ __field( unsigned char, lbam )
+ __field( unsigned char, lbah )
+ __field( unsigned char, nsect )
+ __field( unsigned char, feature )
+ __field( unsigned char, hob_lbal )
+ __field( unsigned char, hob_lbam )
+ __field( unsigned char, hob_lbah )
+ __field( unsigned char, hob_nsect )
+ __field( unsigned char, hob_feature )
+ __field( unsigned char, ctl )
+ __field( unsigned char, proto )
+ __field( unsigned long, flags )
+ ),
+
+ TP_fast_assign(
+ __entry->ata_port = qc->ap->print_id;
+ __entry->ata_dev = qc->dev->link->pmp + qc->dev->devno;
+ __entry->tag = qc->tag;
+ __entry->proto = qc->tf.protocol;
+ __entry->cmd = qc->tf.command;
+ __entry->dev = qc->tf.device;
+ __entry->lbal = qc->tf.lbal;
+ __entry->lbam = qc->tf.lbam;
+ __entry->lbah = qc->tf.lbah;
+ __entry->hob_lbal = qc->tf.hob_lbal;
+ __entry->hob_lbam = qc->tf.hob_lbam;
+ __entry->hob_lbah = qc->tf.hob_lbah;
+ __entry->feature = qc->tf.feature;
+ __entry->hob_feature = qc->tf.hob_feature;
+ __entry->nsect = qc->tf.nsect;
+ __entry->hob_nsect = qc->tf.hob_nsect;
+ ),
+
+ TP_printk("ata_port=%u ata_dev=%u tag=%d proto=%s cmd=%s " \
+ " tf=(%02x/%02x:%02x:%02x:%02x:%02x/%02x:%02x:%02x:%02x:%02x/%02x)",
+ __entry->ata_port, __entry->ata_dev, __entry->tag,
+ show_protocol_name(__entry->proto),
+ show_opcode_name(__entry->cmd),
+ __entry->cmd, __entry->feature, __entry->nsect,
+ __entry->lbal, __entry->lbam, __entry->lbah,
+ __entry->hob_feature, __entry->hob_nsect,
+ __entry->hob_lbal, __entry->hob_lbam, __entry->hob_lbah,
+ __entry->dev)
+);
+
+DECLARE_EVENT_CLASS(ata_qc_complete_template,
+
+ TP_PROTO(struct ata_queued_cmd *qc),
+
+ TP_ARGS(qc),
+
+ TP_STRUCT__entry(
+ __field( unsigned int, ata_port )
+ __field( unsigned int, ata_dev )
+ __field( unsigned int, tag )
+ __field( unsigned char, status )
+ __field( unsigned char, dev )
+ __field( unsigned char, lbal )
+ __field( unsigned char, lbam )
+ __field( unsigned char, lbah )
+ __field( unsigned char, nsect )
+ __field( unsigned char, error )
+ __field( unsigned char, hob_lbal )
+ __field( unsigned char, hob_lbam )
+ __field( unsigned char, hob_lbah )
+ __field( unsigned char, hob_nsect )
+ __field( unsigned char, hob_feature )
+ __field( unsigned char, ctl )
+ __field( unsigned long, flags )
+ ),
+
+ TP_fast_assign(
+ __entry->ata_port = qc->ap->print_id;
+ __entry->ata_dev = qc->dev->link->pmp + qc->dev->devno;
+ __entry->tag = qc->tag;
+ __entry->status = qc->result_tf.command;
+ __entry->dev = qc->result_tf.device;
+ __entry->lbal = qc->result_tf.lbal;
+ __entry->lbam = qc->result_tf.lbam;
+ __entry->lbah = qc->result_tf.lbah;
+ __entry->hob_lbal = qc->result_tf.hob_lbal;
+ __entry->hob_lbam = qc->result_tf.hob_lbam;
+ __entry->hob_lbah = qc->result_tf.hob_lbah;
+ __entry->error = qc->result_tf.feature;
+ __entry->hob_feature = qc->result_tf.hob_feature;
+ __entry->nsect = qc->result_tf.nsect;
+ __entry->hob_nsect = qc->result_tf.hob_nsect;
+ ),
+
+ TP_printk("ata_port=%u ata_dev=%u tag=%d flags=%s status=%s " \
+ " res=(%02x/%02x:%02x:%02x:%02x:%02x/%02x:%02x:%02x:%02x:%02x/%02x)",
+ __entry->ata_port, __entry->ata_dev, __entry->tag,
+ __parse_qc_flags(__entry->flags),
+ __parse_status(__entry->status),
+ __entry->status, __entry->error, __entry->nsect,
+ __entry->lbal, __entry->lbam, __entry->lbah,
+ __entry->hob_feature, __entry->hob_nsect,
+ __entry->hob_lbal, __entry->hob_lbam, __entry->hob_lbah,
+ __entry->dev)
+);
+
+DEFINE_EVENT(ata_qc_complete_template, ata_qc_complete_internal,
+ TP_PROTO(struct ata_queued_cmd *qc),
+ TP_ARGS(qc));
+
+DEFINE_EVENT(ata_qc_complete_template, ata_qc_complete_failed,
+ TP_PROTO(struct ata_queued_cmd *qc),
+ TP_ARGS(qc));
+
+DEFINE_EVENT(ata_qc_complete_template, ata_qc_complete_done,
+ TP_PROTO(struct ata_queued_cmd *qc),
+ TP_ARGS(qc));
+
+TRACE_EVENT(ata_eh_link_autopsy,
+
+ TP_PROTO(struct ata_device *dev, unsigned int eh_action, unsigned int eh_err_mask),
+
+ TP_ARGS(dev, eh_action, eh_err_mask),
+
+ TP_STRUCT__entry(
+ __field( unsigned int, ata_port )
+ __field( unsigned int, ata_dev )
+ __field( unsigned int, eh_action )
+ __field( unsigned int, eh_err_mask)
+ ),
+
+ TP_fast_assign(
+ __entry->ata_port = dev->link->ap->print_id;
+ __entry->ata_dev = dev->link->pmp + dev->devno;
+ __entry->eh_action = eh_action;
+ __entry->eh_err_mask = eh_err_mask;
+ ),
+
+ TP_printk("ata_port=%u ata_dev=%u eh_action=%s err_mask=%s",
+ __entry->ata_port, __entry->ata_dev,
+ __parse_eh_action(__entry->eh_action),
+ __parse_eh_err_mask(__entry->eh_err_mask))
+);
+
+TRACE_EVENT(ata_eh_link_autopsy_qc,
+
+ TP_PROTO(struct ata_queued_cmd *qc),
+
+ TP_ARGS(qc),
+
+ TP_STRUCT__entry(
+ __field( unsigned int, ata_port )
+ __field( unsigned int, ata_dev )
+ __field( unsigned int, tag )
+ __field( unsigned int, qc_flags )
+ __field( unsigned int, eh_err_mask)
+ ),
+
+ TP_fast_assign(
+ __entry->ata_port = qc->ap->print_id;
+ __entry->ata_dev = qc->dev->link->pmp + qc->dev->devno;
+ __entry->tag = qc->tag;
+ __entry->qc_flags = qc->flags;
+ __entry->eh_err_mask = qc->err_mask;
+ ),
+
+ TP_printk("ata_port=%u ata_dev=%u tag=%d flags=%s err_mask=%s",
+ __entry->ata_port, __entry->ata_dev, __entry->tag,
+ __parse_qc_flags(__entry->qc_flags),
+ __parse_eh_err_mask(__entry->eh_err_mask))
+);
+
+#endif /* _TRACE_LIBATA_H */
+
+/* This part must be outside protection */
+#include <trace/define_trace.h>
# UAPI Header export list
-header-y += hsi_char.h
+header-y += hsi_char.h cs-protocol.h
--- /dev/null
+/*
+ * cmt-speech interface definitions
+ *
+ * Copyright (C) 2008,2009,2010 Nokia Corporation. All rights reserved.
+ *
+ * Contact: Kai Vehmanen <kai.vehmanen@nokia.com>
+ * Original author: Peter Ujfalusi <peter.ujfalusi@nokia.com>
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * version 2 as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * You should have received a copy of the GNU General Public License
+ * along with this program; if not, write to the Free Software
+ * Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA
+ * 02110-1301 USA
+ */
+
+#ifndef _CS_PROTOCOL_H
+#define _CS_PROTOCOL_H
+
+#include <linux/types.h>
+#include <linux/ioctl.h>
+
+/* chardev parameters */
+#define CS_DEV_FILE_NAME "/dev/cmt_speech"
+
+/* user-space API versioning */
+#define CS_IF_VERSION 2
+
+/* APE kernel <-> user space messages */
+#define CS_CMD_SHIFT 28
+#define CS_DOMAIN_SHIFT 24
+
+#define CS_CMD_MASK 0xff000000
+#define CS_PARAM_MASK 0xffffff
+
+#define CS_CMD(id, dom) \
+ (((id) << CS_CMD_SHIFT) | ((dom) << CS_DOMAIN_SHIFT))
+
+#define CS_ERROR CS_CMD(1, 0)
+#define CS_RX_DATA_RECEIVED CS_CMD(2, 0)
+#define CS_TX_DATA_READY CS_CMD(3, 0)
+#define CS_TX_DATA_SENT CS_CMD(4, 0)
+
+/* params to CS_ERROR indication */
+#define CS_ERR_PEER_RESET 0
+
+/* ioctl interface */
+
+/* parameters to CS_CONFIG_BUFS ioctl */
+#define CS_FEAT_TSTAMP_RX_CTRL (1 << 0)
+#define CS_FEAT_ROLLING_RX_COUNTER (2 << 0)
+
+/* parameters to CS_GET_STATE ioctl */
+#define CS_STATE_CLOSED 0
+#define CS_STATE_OPENED 1 /* resource allocated */
+#define CS_STATE_CONFIGURED 2 /* data path active */
+
+/* maximum number of TX/RX buffers */
+#define CS_MAX_BUFFERS_SHIFT 4
+#define CS_MAX_BUFFERS (1 << CS_MAX_BUFFERS_SHIFT)
+
+/* Parameters for setting up the data buffers */
+struct cs_buffer_config {
+ __u32 rx_bufs; /* number of RX buffer slots */
+ __u32 tx_bufs; /* number of TX buffer slots */
+ __u32 buf_size; /* bytes */
+ __u32 flags; /* see CS_FEAT_* */
+ __u32 reserved[4];
+};
+
+/*
+ * Struct describing the layout and contents of the driver mmap area.
+ * This information is meant as read-only information for the application.
+ */
+struct cs_mmap_config_block {
+ __u32 reserved1;
+ __u32 buf_size; /* 0=disabled, otherwise the transfer size */
+ __u32 rx_bufs; /* # of RX buffers */
+ __u32 tx_bufs; /* # of TX buffers */
+ __u32 reserved2;
+ /* array of offsets within the mmap area for each RX and TX buffer */
+ __u32 rx_offsets[CS_MAX_BUFFERS];
+ __u32 tx_offsets[CS_MAX_BUFFERS];
+ __u32 rx_ptr;
+ __u32 rx_ptr_boundary;
+ __u32 reserved3[2];
+ /*
+ * if enabled with CS_FEAT_TSTAMP_RX_CTRL, monotonic
+ * timestamp taken when the last control command was received
+ */
+ struct timespec tstamp_rx_ctrl;
+};
+
+#define CS_IO_MAGIC 'C'
+
+#define CS_IOW(num, dtype) _IOW(CS_IO_MAGIC, num, dtype)
+#define CS_IOR(num, dtype) _IOR(CS_IO_MAGIC, num, dtype)
+#define CS_IOWR(num, dtype) _IOWR(CS_IO_MAGIC, num, dtype)
+#define CS_IO(num) _IO(CS_IO_MAGIC, num)
+
+#define CS_GET_STATE CS_IOR(21, unsigned int)
+#define CS_SET_WAKELINE CS_IOW(23, unsigned int)
+#define CS_GET_IF_VERSION CS_IOR(30, unsigned int)
+#define CS_CONFIG_BUFS CS_IOW(31, struct cs_buffer_config)
+
+#endif /* _CS_PROTOCOL_H */
*/
#define MT_TOOL_FINGER 0
#define MT_TOOL_PEN 1
-#define MT_TOOL_MAX 1
+#define MT_TOOL_PALM 2
+#define MT_TOOL_MAX 2
/*
* Values describing the status of a force-feedback effect
#define KVM_PIT_SPEAKER_DUMMY 1
+struct kvm_s390_skeys {
+ __u64 start_gfn;
+ __u64 count;
+ __u64 skeydata_addr;
+ __u32 flags;
+ __u32 reserved[9];
+};
+#define KVM_S390_GET_SKEYS_NONE 1
+#define KVM_S390_SKEYS_MAX 1048576
+
#define KVM_EXIT_UNKNOWN 0
#define KVM_EXIT_EXCEPTION 1
#define KVM_EXIT_IO 2
#define KVM_EXIT_S390_TSCH 22
#define KVM_EXIT_EPR 23
#define KVM_EXIT_SYSTEM_EVENT 24
+#define KVM_EXIT_S390_STSI 25
/* For KVM_EXIT_INTERNAL_ERROR */
/* Emulate instruction failed. */
__u32 type;
__u64 flags;
} system_event;
+ /* KVM_EXIT_S390_STSI */
+ struct {
+ __u64 addr;
+ __u8 ar;
+ __u8 reserved;
+ __u8 fc;
+ __u8 sel1;
+ __u16 sel2;
+ } s390_stsi;
/* Fix the size of the union. */
char padding[256];
};
__u64 kvm_dirty_regs;
union {
struct kvm_sync_regs regs;
- char padding[1024];
+ char padding[2048];
} s;
};
__u8 pad[5];
};
+/* for KVM_S390_MEM_OP */
+struct kvm_s390_mem_op {
+ /* in */
+ __u64 gaddr; /* the guest address */
+ __u64 flags; /* flags */
+ __u32 size; /* amount of bytes */
+ __u32 op; /* type of operation */
+ __u64 buf; /* buffer in userspace */
+ __u8 ar; /* the access register number */
+ __u8 reserved[31]; /* should be set to 0 */
+};
+/* types for kvm_s390_mem_op->op */
+#define KVM_S390_MEMOP_LOGICAL_READ 0
+#define KVM_S390_MEMOP_LOGICAL_WRITE 1
+/* flags for kvm_s390_mem_op->flags */
+#define KVM_S390_MEMOP_F_CHECK_ONLY (1ULL << 0)
+#define KVM_S390_MEMOP_F_INJECT_EXCEPTION (1ULL << 1)
+
/* for KVM_INTERRUPT */
struct kvm_interrupt {
/* in */
} u;
};
+struct kvm_s390_irq_state {
+ __u64 buf;
+ __u32 flags;
+ __u32 len;
+ __u32 reserved[4];
+};
+
/* for KVM_SET_GUEST_DEBUG */
#define KVM_GUESTDBG_ENABLE 0x00000001
#define KVM_CAP_PPC_ENABLE_HCALL 104
#define KVM_CAP_CHECK_EXTENSION_VM 105
#define KVM_CAP_S390_USER_SIGP 106
+#define KVM_CAP_S390_VECTOR_REGISTERS 107
+#define KVM_CAP_S390_MEM_OP 108
+#define KVM_CAP_S390_USER_STSI 109
+#define KVM_CAP_S390_SKEYS 110
+#define KVM_CAP_MIPS_FPU 111
+#define KVM_CAP_MIPS_MSA 112
+#define KVM_CAP_S390_INJECT_IRQ 113
+#define KVM_CAP_S390_IRQ_STATE 114
#ifdef KVM_CAP_IRQ_ROUTING
#define KVM_ARM_VCPU_INIT _IOW(KVMIO, 0xae, struct kvm_vcpu_init)
#define KVM_ARM_PREFERRED_TARGET _IOR(KVMIO, 0xaf, struct kvm_vcpu_init)
#define KVM_GET_REG_LIST _IOWR(KVMIO, 0xb0, struct kvm_reg_list)
+/* Available with KVM_CAP_S390_MEM_OP */
+#define KVM_S390_MEM_OP _IOW(KVMIO, 0xb1, struct kvm_s390_mem_op)
+/* Available with KVM_CAP_S390_SKEYS */
+#define KVM_S390_GET_SKEYS _IOW(KVMIO, 0xb2, struct kvm_s390_skeys)
+#define KVM_S390_SET_SKEYS _IOW(KVMIO, 0xb3, struct kvm_s390_skeys)
+/* Available with KVM_CAP_S390_INJECT_IRQ */
+#define KVM_S390_IRQ _IOW(KVMIO, 0xb4, struct kvm_s390_irq)
+/* Available with KVM_CAP_S390_IRQ_STATE */
+#define KVM_S390_SET_IRQ_STATE _IOW(KVMIO, 0xb5, struct kvm_s390_irq_state)
+#define KVM_S390_GET_IRQ_STATE _IOW(KVMIO, 0xb6, struct kvm_s390_irq_state)
#define KVM_DEV_ASSIGN_ENABLE_IOMMU (1 << 0)
#define KVM_DEV_ASSIGN_PCI_2_3 (1 << 1)
* exported filesystem.
*/
#define NFSEXP_V4ROOT 0x10000
-#define NFSEXP_NOPNFS 0x20000
+#define NFSEXP_PNFS 0x20000
/* All flags that we claim to support. (Note we don't support NOACL.) */
#define NFSEXP_ALLFLAGS 0x3FE7F
#define SERIAL_IO_PORT 0
#define SERIAL_IO_HUB6 1
#define SERIAL_IO_MEM 2
+#define SERIAL_IO_MEM32 3
+#define SERIAL_IO_AU 4
+#define SERIAL_IO_TSI 5
+#define SERIAL_IO_MEM32BE 6
#define UART_CLEAR_FIFO 0x01
#define UART_USE_FIFO 0x02
header-y += tc_skbedit.h
header-y += tc_vlan.h
header-y += tc_bpf.h
+header-y += tc_connmark.h
__u32 size_max;
/* The maximum number of segments (if VIRTIO_BLK_F_SEG_MAX) */
__u32 seg_max;
- /* geometry the device (if VIRTIO_BLK_F_GEOMETRY) */
+ /* geometry of the device (if VIRTIO_BLK_F_GEOMETRY) */
struct virtio_blk_geometry {
__u16 cylinders;
__u8 heads;
#define VIRTIO_BLK_T_BARRIER 0x80000000
#endif /* !VIRTIO_BLK_NO_LEGACY */
-/* This is the first element of the read scatter-gather list. */
+/*
+ * This comes first in the read scatter-gather list.
+ * For legacy virtio, if VIRTIO_F_ANY_LAYOUT is not negotiated,
+ * this is the first element of the read scatter-gather list.
+ */
struct virtio_blk_outhdr {
/* VIRTIO_BLK_T* */
__virtio32 type;
#include <linux/virtio_types.h>
-#define VIRTIO_SCSI_CDB_SIZE 32
-#define VIRTIO_SCSI_SENSE_SIZE 96
+/* Default values of the CDB and sense data size configuration fields */
+#define VIRTIO_SCSI_CDB_DEFAULT_SIZE 32
+#define VIRTIO_SCSI_SENSE_DEFAULT_SIZE 96
+
+#ifndef VIRTIO_SCSI_CDB_SIZE
+#define VIRTIO_SCSI_CDB_SIZE VIRTIO_SCSI_CDB_DEFAULT_SIZE
+#endif
+#ifndef VIRTIO_SCSI_SENSE_SIZE
+#define VIRTIO_SCSI_SENSE_SIZE VIRTIO_SCSI_SENSE_DEFAULT_SIZE
+#endif
/* SCSI command request, followed by data-out */
struct virtio_scsi_cmd_req {
};
struct omap_dss_device {
+ struct kobject kobj;
struct device *dev;
struct module *owner;
}
#endif
+#ifdef CONFIG_PREEMPT
+
+static inline void xen_preemptible_hcall_begin(void)
+{
+}
+
+static inline void xen_preemptible_hcall_end(void)
+{
+}
+
+#else
+
+DECLARE_PER_CPU(bool, xen_in_preemptible_hcall);
+
+static inline void xen_preemptible_hcall_begin(void)
+{
+ __this_cpu_write(xen_in_preemptible_hcall, true);
+}
+
+static inline void xen_preemptible_hcall_end(void)
+{
+ __this_cpu_write(xen_in_preemptible_hcall, false);
+}
+
+#endif /* CONFIG_PREEMPT */
+
#endif /* INCLUDE_XEN_OPS_H */
const char *mod_name);
#define xenbus_register_frontend(drv) \
- __xenbus_register_frontend(drv, THIS_MODULE, KBUILD_MODNAME);
+ __xenbus_register_frontend(drv, THIS_MODULE, KBUILD_MODNAME)
#define xenbus_register_backend(drv) \
- __xenbus_register_backend(drv, THIS_MODULE, KBUILD_MODNAME);
+ __xenbus_register_backend(drv, THIS_MODULE, KBUILD_MODNAME)
void xenbus_unregister_driver(struct xenbus_driver *drv);
#include <linux/gfp.h>
#include <linux/suspend.h>
#include <linux/lockdep.h>
+#include <linux/tick.h>
#include <trace/events/power.h>
#include "smpboot.h"
return err;
cpu_notify(CPU_DYING | param->mod, param->hcpu);
+ /* Give up timekeeping duties */
+ tick_handover_do_timer();
/* Park the stopper thread */
kthread_park(current);
return 0;
while (!idle_cpu(cpu))
cpu_relax();
+ hotplug_cpu__broadcast_tick_pull(cpu);
/* This actually kills the CPU. */
__cpu_die(cpu);
/* CPU is completely dead: tell everyone. Too late to complain. */
+ tick_cleanup_dead_cpu(cpu);
cpu_notify_nofail(CPU_DEAD | mod, hcpu);
check_for_tasks(cpu);
ctx = perf_event_ctx_lock_nested(event, SINGLE_DEPTH_NESTING);
WARN_ON_ONCE(ctx->parent_ctx);
perf_remove_from_context(event, true);
- mutex_unlock(&ctx->mutex);
+ perf_event_ctx_unlock(event, ctx);
_free_event(event);
}
{
struct perf_event *event = container_of(entry,
struct perf_event, pending);
+ int rctx;
+
+ rctx = perf_swevent_get_recursion_context();
+ /*
+ * If we 'fail' here, that's OK, it means recursion is already disabled
+ * and we won't recurse 'further'.
+ */
if (event->pending_disable) {
event->pending_disable = 0;
event->pending_wakeup = 0;
perf_event_wakeup(event);
}
+
+ if (rctx >= 0)
+ perf_swevent_put_recursion_context(rctx);
}
/*
if (!p)
return -ESRCH;
- if (!p->mm) {
+ if (unlikely(p->flags & PF_KTHREAD)) {
put_task_struct(p);
return -EPERM;
}
return -ENOSYS;
}
+
+/**
+ * irq_chip_set_wake_parent - Set/reset wake-up on the parent interrupt
+ * @data: Pointer to interrupt specific data
+ * @on: Whether to set or reset the wake-up capability of this irq
+ *
+ * Conditional, as the underlying parent chip might not implement it.
+ */
+int irq_chip_set_wake_parent(struct irq_data *data, unsigned int on)
+{
+ data = data->parent_data;
+ if (data->chip->irq_set_wake)
+ return data->chip->irq_set_wake(data, on);
+
+ return -ENOSYS;
+}
#endif
/**
* Do not use this for shutdown scenarios where you must be sure
* that all parts (hardirq and threaded handler) have completed.
*
+ * Returns: false if a threaded handler is active.
+ *
* This function may be called - with care - from IRQ context.
*/
-void synchronize_hardirq(unsigned int irq)
+bool synchronize_hardirq(unsigned int irq)
{
struct irq_desc *desc = irq_to_desc(irq);
- if (desc)
+ if (desc) {
__synchronize_hardirq(desc);
+ return !atomic_read(&desc->threads_active);
+ }
+
+ return true;
}
EXPORT_SYMBOL(synchronize_hardirq);
}
EXPORT_SYMBOL(disable_irq);
+/**
+ * disable_hardirq - disables an irq and waits for hardirq completion
+ * @irq: Interrupt to disable
+ *
+ * Disable the selected interrupt line. Enables and Disables are
+ * nested.
+ * This function waits for any pending hard IRQ handlers for this
+ * interrupt to complete before returning. If you use this function while
+ * holding a resource the hard IRQ handler may need you will deadlock.
+ *
+ * When used to optimistically disable an interrupt from atomic context
+ * the return value must be checked.
+ *
+ * Returns: false if a threaded handler is active.
+ *
+ * This function may be called - with care - from IRQ context.
+ */
+bool disable_hardirq(unsigned int irq)
+{
+ if (!__disable_irq_nosync(irq))
+ return synchronize_hardirq(irq);
+
+ return false;
+}
+EXPORT_SYMBOL_GPL(disable_hardirq);
+
void __enable_irq(struct irq_desc *desc, unsigned int irq)
{
switch (desc->depth) {
* otherwise we'll have trouble later trying to figure out
* which interrupt is which (messes up the interrupt freeing
* logic etc).
+ *
+ * Also IRQF_COND_SUSPEND only makes sense for shared interrupts and
+ * it cannot be set along with IRQF_NO_SUSPEND.
*/
- if ((irqflags & IRQF_SHARED) && !dev_id)
+ if (((irqflags & IRQF_SHARED) && !dev_id) ||
+ (!(irqflags & IRQF_SHARED) && (irqflags & IRQF_COND_SUSPEND)) ||
+ ((irqflags & IRQF_NO_SUSPEND) && (irqflags & IRQF_COND_SUSPEND)))
return -EINVAL;
desc = irq_to_desc(irq);
return retval;
}
+
+/**
+ * irq_get_irqchip_state - returns the irqchip state of a interrupt.
+ * @irq: Interrupt line that is forwarded to a VM
+ * @which: One of IRQCHIP_STATE_* the caller wants to know about
+ * @state: a pointer to a boolean where the state is to be storeed
+ *
+ * This call snapshots the internal irqchip state of an
+ * interrupt, returning into @state the bit corresponding to
+ * stage @which
+ *
+ * This function should be called with preemption disabled if the
+ * interrupt controller has per-cpu registers.
+ */
+int irq_get_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
+ bool *state)
+{
+ struct irq_desc *desc;
+ struct irq_data *data;
+ struct irq_chip *chip;
+ unsigned long flags;
+ int err = -EINVAL;
+
+ desc = irq_get_desc_buslock(irq, &flags, 0);
+ if (!desc)
+ return err;
+
+ data = irq_desc_get_irq_data(desc);
+
+ do {
+ chip = irq_data_get_irq_chip(data);
+ if (chip->irq_get_irqchip_state)
+ break;
+#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
+ data = data->parent_data;
+#else
+ data = NULL;
+#endif
+ } while (data);
+
+ if (data)
+ err = chip->irq_get_irqchip_state(data, which, state);
+
+ irq_put_desc_busunlock(desc, flags);
+ return err;
+}
+
+/**
+ * irq_set_irqchip_state - set the state of a forwarded interrupt.
+ * @irq: Interrupt line that is forwarded to a VM
+ * @which: State to be restored (one of IRQCHIP_STATE_*)
+ * @val: Value corresponding to @which
+ *
+ * This call sets the internal irqchip state of an interrupt,
+ * depending on the value of @which.
+ *
+ * This function should be called with preemption disabled if the
+ * interrupt controller has per-cpu registers.
+ */
+int irq_set_irqchip_state(unsigned int irq, enum irqchip_irq_state which,
+ bool val)
+{
+ struct irq_desc *desc;
+ struct irq_data *data;
+ struct irq_chip *chip;
+ unsigned long flags;
+ int err = -EINVAL;
+
+ desc = irq_get_desc_buslock(irq, &flags, 0);
+ if (!desc)
+ return err;
+
+ data = irq_desc_get_irq_data(desc);
+
+ do {
+ chip = irq_data_get_irq_chip(data);
+ if (chip->irq_set_irqchip_state)
+ break;
+#ifdef CONFIG_IRQ_DOMAIN_HIERARCHY
+ data = data->parent_data;
+#else
+ data = NULL;
+#endif
+ } while (data);
+
+ if (data)
+ err = chip->irq_set_irqchip_state(data, which, val);
+
+ irq_put_desc_busunlock(desc, flags);
+ return err;
+}
struct msi_desc *desc;
for_each_msi_entry(desc, dev) {
- irq_domain_free_irqs(desc->irq, desc->nvec_used);
- desc->irq = 0;
+ /*
+ * We might have failed to allocate an MSI early
+ * enough that there is no IRQ associated to this
+ * entry. If that's the case, don't do anything.
+ */
+ if (desc->irq) {
+ irq_domain_free_irqs(desc->irq, desc->nvec_used);
+ desc->irq = 0;
+ }
}
}
if (action->flags & IRQF_NO_SUSPEND)
desc->no_suspend_depth++;
+ else if (action->flags & IRQF_COND_SUSPEND)
+ desc->cond_suspend_depth++;
WARN_ON_ONCE(desc->no_suspend_depth &&
- desc->no_suspend_depth != desc->nr_actions);
+ (desc->no_suspend_depth +
+ desc->cond_suspend_depth) != desc->nr_actions);
}
/*
if (action->flags & IRQF_NO_SUSPEND)
desc->no_suspend_depth--;
+ else if (action->flags & IRQF_COND_SUSPEND)
+ desc->cond_suspend_depth--;
}
static bool suspend_device_irq(struct irq_desc *desc, int irq)
/* sets obj->mod if object is not vmlinux and module is found */
static void klp_find_object_module(struct klp_object *obj)
{
+ struct module *mod;
+
if (!klp_is_module(obj))
return;
mutex_lock(&module_mutex);
/*
- * We don't need to take a reference on the module here because we have
- * the klp_mutex, which is also taken by the module notifier. This
- * prevents any module from unloading until we release the klp_mutex.
+ * We do not want to block removal of patched modules and therefore
+ * we do not take a reference here. The patches are removed by
+ * a going module handler instead.
+ */
+ mod = find_module(obj->name);
+ /*
+ * Do not mess work of the module coming and going notifiers.
+ * Note that the patch might still be needed before the going handler
+ * is called. Module functions can be called even in the GOING state
+ * until mod->exit() finishes. This is especially important for
+ * patches that modify semantic of the functions.
*/
- obj->mod = find_module(obj->name);
+ if (mod && mod->klp_alive)
+ obj->mod = mod;
+
mutex_unlock(&module_mutex);
}
/* first, check if it's an exported symbol */
preempt_disable();
sym = find_symbol(name, NULL, NULL, true, true);
- preempt_enable();
if (sym) {
*addr = sym->value;
+ preempt_enable();
return 0;
}
+ preempt_enable();
/* otherwise check if it's in another .o within the patch module */
return klp_find_object_symbol(pmod->name, name, addr);
rcu_read_lock();
func = list_first_or_null_rcu(&ops->func_stack, struct klp_func,
stack_node);
- rcu_read_unlock();
-
if (WARN_ON_ONCE(!func))
- return;
+ goto unlock;
klp_arch_set_pc(regs, (unsigned long)func->new_func);
+unlock:
+ rcu_read_unlock();
}
static int klp_disable_func(struct klp_func *func)
func->state = KLP_DISABLED;
return kobject_init_and_add(&func->kobj, &klp_ktype_func,
- obj->kobj, func->old_name);
+ obj->kobj, "%s", func->old_name);
}
/* parts of the initialization that is done only when the object is loaded */
return -EINVAL;
obj->state = KLP_DISABLED;
+ obj->mod = NULL;
klp_find_object_module(obj);
patch->state = KLP_DISABLED;
ret = kobject_init_and_add(&patch->kobj, &klp_ktype_patch,
- klp_root_kobj, patch->mod->name);
+ klp_root_kobj, "%s", patch->mod->name);
if (ret)
goto unlock;
mutex_lock(&klp_mutex);
+ /*
+ * Each module has to know that the notifier has been called.
+ * We never know what module will get patched by a new patch.
+ */
+ if (action == MODULE_STATE_COMING)
+ mod->klp_alive = true;
+ else /* MODULE_STATE_GOING */
+ mod->klp_alive = false;
+
list_for_each_entry(patch, &klp_patches, list) {
for (obj = patch->objs; obj->funcs; obj++) {
if (!klp_is_module(obj) || strcmp(obj->name, mod->name))
if (!new_class->name)
return 0;
- list_for_each_entry(class, &all_lock_classes, lock_entry) {
+ list_for_each_entry_rcu(class, &all_lock_classes, lock_entry) {
if (new_class->key - new_class->subclass == class->key)
return class->name_version;
if (class->name && !strcmp(class->name, new_class->name))
hash_head = classhashentry(key);
/*
- * We can walk the hash lockfree, because the hash only
- * grows, and we are careful when adding entries to the end:
+ * We do an RCU walk of the hash, see lockdep_free_key_range().
*/
- list_for_each_entry(class, hash_head, hash_entry) {
+ if (DEBUG_LOCKS_WARN_ON(!irqs_disabled()))
+ return NULL;
+
+ list_for_each_entry_rcu(class, hash_head, hash_entry) {
if (class->key == key) {
/*
* Huh! same key, different name? Did someone trample
struct lockdep_subclass_key *key;
struct list_head *hash_head;
struct lock_class *class;
- unsigned long flags;
+
+ DEBUG_LOCKS_WARN_ON(!irqs_disabled());
class = look_up_lock_class(lock, subclass);
if (likely(class))
key = lock->key->subkeys + subclass;
hash_head = classhashentry(key);
- raw_local_irq_save(flags);
if (!graph_lock()) {
- raw_local_irq_restore(flags);
return NULL;
}
/*
* We have to do the hash-walk again, to avoid races
* with another CPU:
*/
- list_for_each_entry(class, hash_head, hash_entry)
+ list_for_each_entry_rcu(class, hash_head, hash_entry) {
if (class->key == key)
goto out_unlock_set;
+ }
+
/*
* Allocate a new key from the static array, and add it to
* the hash:
*/
if (nr_lock_classes >= MAX_LOCKDEP_KEYS) {
if (!debug_locks_off_graph_unlock()) {
- raw_local_irq_restore(flags);
return NULL;
}
- raw_local_irq_restore(flags);
print_lockdep_off("BUG: MAX_LOCKDEP_KEYS too low!");
dump_stack();
if (verbose(class)) {
graph_unlock();
- raw_local_irq_restore(flags);
printk("\nnew class %p: %s", class->key, class->name);
if (class->name_version > 1)
printk("\n");
dump_stack();
- raw_local_irq_save(flags);
if (!graph_lock()) {
- raw_local_irq_restore(flags);
return NULL;
}
}
out_unlock_set:
graph_unlock();
- raw_local_irq_restore(flags);
out_set_class_cache:
if (!subclass || force)
entry->distance = distance;
entry->trace = *trace;
/*
- * Since we never remove from the dependency list, the list can
- * be walked lockless by other CPUs, it's only allocation
- * that must be protected by the spinlock. But this also means
- * we must make new entries visible only once writes to the
- * entry become visible - hence the RCU op:
+ * Both allocation and removal are done under the graph lock; but
+ * iteration is under RCU-sched; see look_up_lock_class() and
+ * lockdep_free_key_range().
*/
list_add_tail_rcu(&entry->entry, head);
else
head = &lock->class->locks_before;
- list_for_each_entry(entry, head, entry) {
+ DEBUG_LOCKS_WARN_ON(!irqs_disabled());
+
+ list_for_each_entry_rcu(entry, head, entry) {
if (!lock_accessed(entry)) {
unsigned int cq_depth;
mark_lock_accessed(entry, lock);
* We can walk it lock-free, because entries only get added
* to the hash:
*/
- list_for_each_entry(chain, hash_head, entry) {
+ list_for_each_entry_rcu(chain, hash_head, entry) {
if (chain->chain_key == chain_key) {
cache_hit:
debug_atomic_inc(chain_lookup_hits);
if (unlikely(!debug_locks))
return;
- if (subclass)
+ if (subclass) {
+ unsigned long flags;
+
+ if (DEBUG_LOCKS_WARN_ON(current->lockdep_recursion))
+ return;
+
+ raw_local_irq_save(flags);
+ current->lockdep_recursion = 1;
register_lock_class(lock, subclass, 1);
+ current->lockdep_recursion = 0;
+ raw_local_irq_restore(flags);
+ }
}
EXPORT_SYMBOL_GPL(lockdep_init_map);
return addr >= start && addr < start + size;
}
+/*
+ * Used in module.c to remove lock classes from memory that is going to be
+ * freed; and possibly re-used by other modules.
+ *
+ * We will have had one sync_sched() before getting here, so we're guaranteed
+ * nobody will look up these exact classes -- they're properly dead but still
+ * allocated.
+ */
void lockdep_free_key_range(void *start, unsigned long size)
{
- struct lock_class *class, *next;
+ struct lock_class *class;
struct list_head *head;
unsigned long flags;
int i;
head = classhash_table + i;
if (list_empty(head))
continue;
- list_for_each_entry_safe(class, next, head, hash_entry) {
+ list_for_each_entry_rcu(class, head, hash_entry) {
if (within(class->key, start, size))
zap_class(class);
else if (within(class->name, start, size))
if (locked)
graph_unlock();
raw_local_irq_restore(flags);
+
+ /*
+ * Wait for any possible iterators from look_up_lock_class() to pass
+ * before continuing to free the memory they refer to.
+ *
+ * sync_sched() is sufficient because the read-side is IRQ disable.
+ */
+ synchronize_sched();
+
+ /*
+ * XXX at this point we could return the resources to the pool;
+ * instead we leak them. We would need to change to bitmap allocators
+ * instead of the linear allocators we have now.
+ */
}
void lockdep_reset_lock(struct lockdep_map *lock)
{
- struct lock_class *class, *next;
+ struct lock_class *class;
struct list_head *head;
unsigned long flags;
int i, j;
head = classhash_table + i;
if (list_empty(head))
continue;
- list_for_each_entry_safe(class, next, head, hash_entry) {
+ list_for_each_entry_rcu(class, head, hash_entry) {
int match = 0;
for (j = 0; j < NR_LOCKDEP_CACHING_CLASSES; j++)
*/
return;
}
- ACCESS_ONCE(prev->next) = node;
+ WRITE_ONCE(prev->next, node);
/* Wait until the lock holder passes the lock down. */
arch_mcs_spin_lock_contended(&node->locked);
static inline
void mcs_spin_unlock(struct mcs_spinlock **lock, struct mcs_spinlock *node)
{
- struct mcs_spinlock *next = ACCESS_ONCE(node->next);
+ struct mcs_spinlock *next = READ_ONCE(node->next);
if (likely(!next)) {
/*
if (likely(cmpxchg(lock, node, NULL) == node))
return;
/* Wait until the next pointer is set */
- while (!(next = ACCESS_ONCE(node->next)))
+ while (!(next = READ_ONCE(node->next)))
cpu_relax_lowlatency();
}
#include <linux/spinlock.h>
#include <linux/interrupt.h>
#include <linux/debug_locks.h>
-#include "mcs_spinlock.h"
+#include <linux/osq_lock.h>
/*
* In the DEBUG case we are using the "NULL fastpath" for mutexes,
}
#ifdef CONFIG_MUTEX_SPIN_ON_OWNER
-static inline bool owner_running(struct mutex *lock, struct task_struct *owner)
-{
- if (lock->owner != owner)
- return false;
-
- /*
- * Ensure we emit the owner->on_cpu, dereference _after_ checking
- * lock->owner still matches owner, if that fails, owner might
- * point to free()d memory, if it still matches, the rcu_read_lock()
- * ensures the memory stays valid.
- */
- barrier();
-
- return owner->on_cpu;
-}
-
/*
* Look out! "owner" is an entirely speculative pointer
* access and not reliable.
*/
static noinline
-int mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner)
+bool mutex_spin_on_owner(struct mutex *lock, struct task_struct *owner)
{
+ bool ret = true;
+
rcu_read_lock();
- while (owner_running(lock, owner)) {
- if (need_resched())
+ while (lock->owner == owner) {
+ /*
+ * Ensure we emit the owner->on_cpu, dereference _after_
+ * checking lock->owner still matches owner. If that fails,
+ * owner might point to freed memory. If it still matches,
+ * the rcu_read_lock() ensures the memory stays valid.
+ */
+ barrier();
+
+ if (!owner->on_cpu || need_resched()) {
+ ret = false;
break;
+ }
cpu_relax_lowlatency();
}
rcu_read_unlock();
- /*
- * We break out the loop above on need_resched() and when the
- * owner changed, which is a sign for heavy contention. Return
- * success only when lock->owner is NULL.
- */
- return lock->owner == NULL;
+ return ret;
}
/*
return 0;
rcu_read_lock();
- owner = ACCESS_ONCE(lock->owner);
+ owner = READ_ONCE(lock->owner);
if (owner)
retval = owner->on_cpu;
rcu_read_unlock();
* As such, when deadlock detection needs to be
* performed the optimistic spinning cannot be done.
*/
- if (ACCESS_ONCE(ww->ctx))
+ if (READ_ONCE(ww->ctx))
break;
}
* If there's an owner, wait for it to either
* release the lock or go to sleep.
*/
- owner = ACCESS_ONCE(lock->owner);
+ owner = READ_ONCE(lock->owner);
if (owner && !mutex_spin_on_owner(lock, owner))
break;
__ww_mutex_lock_check_stamp(struct mutex *lock, struct ww_acquire_ctx *ctx)
{
struct ww_mutex *ww = container_of(lock, struct ww_mutex, base);
- struct ww_acquire_ctx *hold_ctx = ACCESS_ONCE(ww->ctx);
+ struct ww_acquire_ctx *hold_ctx = READ_ONCE(ww->ctx);
if (!hold_ctx)
return 0;
prev = decode_cpu(old);
node->prev = prev;
- ACCESS_ONCE(prev->next) = node;
+ WRITE_ONCE(prev->next, node);
/*
* Normally @prev is untouchable after the above store; because at that
* cmpxchg in an attempt to undo our queueing.
*/
- while (!ACCESS_ONCE(node->locked)) {
+ while (!READ_ONCE(node->locked)) {
/*
* If we need to reschedule bail... so we can block.
*/
* Or we race against a concurrent unqueue()'s step-B, in which
* case its step-C will write us a new @node->prev pointer.
*/
- prev = ACCESS_ONCE(node->prev);
+ prev = READ_ONCE(node->prev);
}
/*
* it will wait in Step-A.
*/
- ACCESS_ONCE(next->prev) = prev;
- ACCESS_ONCE(prev->next) = next;
+ WRITE_ONCE(next->prev, prev);
+ WRITE_ONCE(prev->next, next);
return false;
}
node = this_cpu_ptr(&osq_node);
next = xchg(&node->next, NULL);
if (next) {
- ACCESS_ONCE(next->locked) = 1;
+ WRITE_ONCE(next->locked, 1);
return;
}
next = osq_wait_next(lock, node, NULL);
if (next)
- ACCESS_ONCE(next->locked) = 1;
+ WRITE_ONCE(next->locked, 1);
}
*
* @task: the task owning the mutex (owner) for which a chain walk is
* probably needed
- * @deadlock_detect: do we have to carry out deadlock detection?
+ * @chwalk: do we have to carry out deadlock detection?
* @orig_lock: the mutex (can be NULL if we are walking the chain to recheck
* things for a task that has just got its priority adjusted, and
* is waiting on a mutex)
ret = __rt_mutex_slowlock(lock, state, timeout, &waiter);
if (unlikely(ret)) {
+ __set_current_state(TASK_RUNNING);
if (rt_mutex_has_waiters(lock))
remove_waiter(lock, &waiter);
rt_mutex_handle_deadlock(ret, chwalk, &waiter);
list_del(&waiter->list);
tsk = waiter->task;
+ /*
+ * Make sure we do not wakeup the next reader before
+ * setting the nil condition to grant the next reader;
+ * otherwise we could miss the wakeup on the other
+ * side and end up sleeping again. See the pairing
+ * in rwsem_down_read_failed().
+ */
smp_mb();
waiter->task = NULL;
wake_up_process(tsk);
#include <linux/init.h>
#include <linux/export.h>
#include <linux/sched/rt.h>
+#include <linux/osq_lock.h>
-#include "mcs_spinlock.h"
+#include "rwsem.h"
/*
* Guide to the rw_semaphore's count field for common values.
waiter = list_entry(next, struct rwsem_waiter, list);
next = waiter->list.next;
tsk = waiter->task;
+ /*
+ * Make sure we do not wakeup the next reader before
+ * setting the nil condition to grant the next reader;
+ * otherwise we could miss the wakeup on the other
+ * side and end up sleeping again. See the pairing
+ * in rwsem_down_read_failed().
+ */
smp_mb();
waiter->task = NULL;
wake_up_process(tsk);
RWSEM_ACTIVE_WRITE_BIAS) == RWSEM_WAITING_BIAS) {
if (!list_is_singular(&sem->wait_list))
rwsem_atomic_update(RWSEM_WAITING_BIAS, sem);
+ rwsem_set_owner(sem);
return true;
}
*/
static inline bool rwsem_try_write_lock_unqueued(struct rw_semaphore *sem)
{
- long old, count = ACCESS_ONCE(sem->count);
+ long old, count = READ_ONCE(sem->count);
while (true) {
if (!(count == 0 || count == RWSEM_WAITING_BIAS))
return false;
old = cmpxchg(&sem->count, count, count + RWSEM_ACTIVE_WRITE_BIAS);
- if (old == count)
+ if (old == count) {
+ rwsem_set_owner(sem);
return true;
+ }
count = old;
}
static inline bool rwsem_can_spin_on_owner(struct rw_semaphore *sem)
{
struct task_struct *owner;
- bool on_cpu = false;
+ bool ret = true;
if (need_resched())
return false;
rcu_read_lock();
- owner = ACCESS_ONCE(sem->owner);
- if (owner)
- on_cpu = owner->on_cpu;
- rcu_read_unlock();
-
- /*
- * If sem->owner is not set, yet we have just recently entered the
- * slowpath, then there is a possibility reader(s) may have the lock.
- * To be safe, avoid spinning in these situations.
- */
- return on_cpu;
-}
-
-static inline bool owner_running(struct rw_semaphore *sem,
- struct task_struct *owner)
-{
- if (sem->owner != owner)
- return false;
-
- /*
- * Ensure we emit the owner->on_cpu, dereference _after_ checking
- * sem->owner still matches owner, if that fails, owner might
- * point to free()d memory, if it still matches, the rcu_read_lock()
- * ensures the memory stays valid.
- */
- barrier();
+ owner = READ_ONCE(sem->owner);
+ if (!owner) {
+ long count = READ_ONCE(sem->count);
+ /*
+ * If sem->owner is not set, yet we have just recently entered the
+ * slowpath with the lock being active, then there is a possibility
+ * reader(s) may have the lock. To be safe, bail spinning in these
+ * situations.
+ */
+ if (count & RWSEM_ACTIVE_MASK)
+ ret = false;
+ goto done;
+ }
- return owner->on_cpu;
+ ret = owner->on_cpu;
+done:
+ rcu_read_unlock();
+ return ret;
}
static noinline
bool rwsem_spin_on_owner(struct rw_semaphore *sem, struct task_struct *owner)
{
+ long count;
+
rcu_read_lock();
- while (owner_running(sem, owner)) {
- if (need_resched())
- break;
+ while (sem->owner == owner) {
+ /*
+ * Ensure we emit the owner->on_cpu, dereference _after_
+ * checking sem->owner still matches owner, if that fails,
+ * owner might point to free()d memory, if it still matches,
+ * the rcu_read_lock() ensures the memory stays valid.
+ */
+ barrier();
+
+ /* abort spinning when need_resched or owner is not running */
+ if (!owner->on_cpu || need_resched()) {
+ rcu_read_unlock();
+ return false;
+ }
cpu_relax_lowlatency();
}
rcu_read_unlock();
+ if (READ_ONCE(sem->owner))
+ return true; /* new owner, continue spinning */
+
/*
- * We break out the loop above on need_resched() or when the
- * owner changed, which is a sign for heavy contention. Return
- * success only when sem->owner is NULL.
+ * When the owner is not set, the lock could be free or
+ * held by readers. Check the counter to verify the
+ * state.
*/
- return sem->owner == NULL;
+ count = READ_ONCE(sem->count);
+ return (count == 0 || count == RWSEM_WAITING_BIAS);
}
static bool rwsem_optimistic_spin(struct rw_semaphore *sem)
goto done;
while (true) {
- owner = ACCESS_ONCE(sem->owner);
+ owner = READ_ONCE(sem->owner);
if (owner && !rwsem_spin_on_owner(sem, owner))
break;
/* we're now waiting on the lock, but no longer actively locking */
if (waiting) {
- count = ACCESS_ONCE(sem->count);
+ count = READ_ONCE(sem->count);
/*
* If there were already threads queued before us and there are
#include <linux/sched.h>
#include <linux/export.h>
#include <linux/rwsem.h>
-
#include <linux/atomic.h>
-#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
-static inline void rwsem_set_owner(struct rw_semaphore *sem)
-{
- sem->owner = current;
-}
-
-static inline void rwsem_clear_owner(struct rw_semaphore *sem)
-{
- sem->owner = NULL;
-}
-
-#else
-static inline void rwsem_set_owner(struct rw_semaphore *sem)
-{
-}
-
-static inline void rwsem_clear_owner(struct rw_semaphore *sem)
-{
-}
-#endif
+#include "rwsem.h"
/*
* lock for reading
--- /dev/null
+#ifdef CONFIG_RWSEM_SPIN_ON_OWNER
+static inline void rwsem_set_owner(struct rw_semaphore *sem)
+{
+ sem->owner = current;
+}
+
+static inline void rwsem_clear_owner(struct rw_semaphore *sem)
+{
+ sem->owner = NULL;
+}
+
+#else
+static inline void rwsem_set_owner(struct rw_semaphore *sem)
+{
+}
+
+static inline void rwsem_clear_owner(struct rw_semaphore *sem)
+{
+}
+#endif
#include <linux/async.h>
#include <linux/percpu.h>
#include <linux/kmemleak.h>
-#include <linux/kasan.h>
#include <linux/jump_label.h>
#include <linux/pfn.h>
#include <linux/bsearch.h>
void __weak module_memfree(void *module_region)
{
vfree(module_region);
- kasan_module_free(module_region);
}
void __weak module_arch_cleanup(struct module *mod)
kfree(mod->args);
percpu_modfree(mod);
- /* Free lock-classes: */
+ /* Free lock-classes; relies on the preceding sync_rcu(). */
lockdep_free_key_range(mod->module_core, mod->core_size);
/* Finally, free the core (containing the module structure) */
info->symoffs = ALIGN(mod->core_size, symsect->sh_addralign ?: 1);
info->stroffs = mod->core_size = info->symoffs + ndst * sizeof(Elf_Sym);
mod->core_size += strtab_size;
+ mod->core_size = debug_align(mod->core_size);
/* Put string table section at end of init part of module. */
strsect->sh_flags |= SHF_ALLOC;
strsect->sh_entsize = get_offset(mod, &mod->init_size, strsect,
info->index.str) | INIT_OFFSET_MASK;
+ mod->init_size = debug_align(mod->init_size);
pr_debug("\t%s\n", info->secstrings + strsect->sh_name);
}
return 0;
}
+#define COPY_CHUNK_SIZE (16*PAGE_SIZE)
+
+static int copy_chunked_from_user(void *dst, const void __user *usrc, unsigned long len)
+{
+ do {
+ unsigned long n = min(len, COPY_CHUNK_SIZE);
+
+ if (copy_from_user(dst, usrc, n) != 0)
+ return -EFAULT;
+ cond_resched();
+ dst += n;
+ usrc += n;
+ len -= n;
+ } while (len);
+ return 0;
+}
+
/* Sets info->hdr and info->len. */
static int copy_module_from_user(const void __user *umod, unsigned long len,
struct load_info *info)
if (!info->hdr)
return -ENOMEM;
- if (copy_from_user(info->hdr, umod, info->len) != 0) {
+ if (copy_chunked_from_user(info->hdr, umod, info->len) != 0) {
vfree(info->hdr);
return -EFAULT;
}
module_bug_cleanup(mod);
mutex_unlock(&module_mutex);
- /* Free lock-classes: */
- lockdep_free_key_range(mod->module_core, mod->core_size);
-
/* we can't deallocate the module until we clear memory protection */
unset_module_init_ro_nx(mod);
unset_module_core_ro_nx(mod);
synchronize_rcu();
mutex_unlock(&module_mutex);
free_module:
+ /* Free lock-classes; relies on the preceding sync_rcu() */
+ lockdep_free_key_range(mod->module_core, mod->core_size);
+
module_deallocate(mod, info);
free_copy:
free_copy(info);
}
}
-static bool is_nosave_page(unsigned long pfn)
-{
- struct nosave_region *region;
-
- list_for_each_entry(region, &nosave_regions, list) {
- if (pfn >= region->start_pfn && pfn < region->end_pfn) {
- pr_err("PM: %#010llx in e820 nosave region: "
- "[mem %#010llx-%#010llx]\n",
- (unsigned long long) pfn << PAGE_SHIFT,
- (unsigned long long) region->start_pfn << PAGE_SHIFT,
- ((unsigned long long) region->end_pfn << PAGE_SHIFT)
- - 1);
- return true;
- }
- }
-
- return false;
-}
-
/**
* create_basic_memory_bitmaps - create bitmaps needed for marking page
* frames that should not be saved and free page frames. The pointers
do {
pfn = memory_bm_next_pfn(bm);
if (likely(pfn != BM_END_OF_MAP)) {
- if (likely(pfn_valid(pfn)) && !is_nosave_page(pfn))
+ if (likely(pfn_valid(pfn)))
swsusp_set_page_free(pfn_to_page(pfn));
else
return -EFAULT;
struct console_cmdline
{
- char name[8]; /* Name of the driver */
+ char name[16]; /* Name of the driver */
int index; /* Minor dev. to use */
char *options; /* Options for the driver */
#ifdef CONFIG_A11Y_BRAILLE_CONSOLE
for (i = 0, c = console_cmdline;
i < MAX_CMDLINECONSOLES && c->name[0];
i++, c++) {
+ BUILD_BUG_ON(sizeof(c->name) != sizeof(newcon->name));
if (strcmp(c->name, newcon->name) != 0)
continue;
if (newcon->index >= 0 &&
#ifdef CONFIG_NO_HZ_FULL
bool sched_can_stop_tick(void)
{
+ /*
+ * FIFO realtime policy runs the highest priority task. Other runnable
+ * tasks are of a lower priority. The scheduler tick does nothing.
+ */
+ if (current->policy == SCHED_FIFO)
+ return true;
+
+ /*
+ * Round-robin realtime tasks time slice with other tasks at the same
+ * realtime priority. Is this task the only one at this priority?
+ */
+ if (current->policy == SCHED_RR) {
+ struct sched_rt_entity *rt_se = ¤t->rt;
+
+ return rt_se->run_list.prev == rt_se->run_list.next;
+ }
+
/*
* More than one running task need preemption.
* nr_running update is assumed to be visible
rq_clock_skip_update(rq, true);
}
+static ATOMIC_NOTIFIER_HEAD(task_migration_notifier);
+
+void register_task_migration_notifier(struct notifier_block *n)
+{
+ atomic_notifier_chain_register(&task_migration_notifier, n);
+}
+
#ifdef CONFIG_SMP
void set_task_cpu(struct task_struct *p, unsigned int new_cpu)
{
trace_sched_migrate_task(p, new_cpu);
if (task_cpu(p) != new_cpu) {
+ struct task_migration_notifier tmn;
+
if (p->sched_class->migrate_task_rq)
p->sched_class->migrate_task_rq(p, new_cpu);
p->se.nr_migrations++;
perf_sw_event_sched(PERF_COUNT_SW_CPU_MIGRATIONS, 1, 0);
+
+ tmn.task = p;
+ tmn.from_cpu = task_cpu(p);
+ tmn.to_cpu = new_cpu;
+
+ atomic_notifier_call_chain(&task_migration_notifier, 0, &tmn);
}
__set_task_cpu(p, new_cpu);
} else {
if (dl_prio(oldprio))
p->dl.dl_boosted = 0;
+ if (rt_prio(oldprio))
+ p->rt.timeout = 0;
p->sched_class = &fair_sched_class;
}
static int sched_cpu_inactive(struct notifier_block *nfb,
unsigned long action, void *hcpu)
{
- unsigned long flags;
- long cpu = (long)hcpu;
- struct dl_bw *dl_b;
-
switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DOWN_PREPARE:
- set_cpu_active(cpu, false);
-
- /* explicitly allow suspend */
- if (!(action & CPU_TASKS_FROZEN)) {
- bool overflow;
- int cpus;
-
- rcu_read_lock_sched();
- dl_b = dl_bw_of(cpu);
-
- raw_spin_lock_irqsave(&dl_b->lock, flags);
- cpus = dl_bw_cpus(cpu);
- overflow = __dl_overflow(dl_b, cpus, 0, 0);
- raw_spin_unlock_irqrestore(&dl_b->lock, flags);
-
- rcu_read_unlock_sched();
-
- if (overflow)
- return notifier_from_errno(-EBUSY);
- }
+ set_cpu_active((long)hcpu, false);
return NOTIFY_OK;
+ default:
+ return NOTIFY_DONE;
}
-
- return NOTIFY_DONE;
}
static int __init migration_init(void)
break;
}
- /*
- * Even though we initialize ->capacity to something semi-sane,
- * we leave capacity_orig unset. This allows us to detect if
- * domain iteration is still funny without causing /0 traps.
- */
- if (!group->sgc->capacity_orig) {
- printk(KERN_CONT "\n");
- printk(KERN_ERR "ERROR: domain->cpu_capacity not set\n");
- break;
- }
-
if (!cpumask_weight(sched_group_cpus(group))) {
printk(KERN_CONT "\n");
printk(KERN_ERR "ERROR: empty group\n");
* die on a /0 trap.
*/
sg->sgc->capacity = SCHED_CAPACITY_SCALE * cpumask_weight(sg_span);
- sg->sgc->capacity_orig = sg->sgc->capacity;
/*
* Make sure the first group of this domain contains the
*/
if (sd->flags & SD_SHARE_CPUCAPACITY) {
+ sd->flags |= SD_PREFER_SIBLING;
sd->imbalance_pct = 110;
sd->smt_gain = 1178; /* ~15% */
*/
case CPU_ONLINE:
- case CPU_DOWN_FAILED:
cpuset_update_active_cpus(true);
break;
default:
static int cpuset_cpu_inactive(struct notifier_block *nfb, unsigned long action,
void *hcpu)
{
- switch (action) {
+ unsigned long flags;
+ long cpu = (long)hcpu;
+ struct dl_bw *dl_b;
+
+ switch (action & ~CPU_TASKS_FROZEN) {
case CPU_DOWN_PREPARE:
+ /* explicitly allow suspend */
+ if (!(action & CPU_TASKS_FROZEN)) {
+ bool overflow;
+ int cpus;
+
+ rcu_read_lock_sched();
+ dl_b = dl_bw_of(cpu);
+
+ raw_spin_lock_irqsave(&dl_b->lock, flags);
+ cpus = dl_bw_cpus(cpu);
+ overflow = __dl_overflow(dl_b, cpus, 0, 0);
+ raw_spin_unlock_irqrestore(&dl_b->lock, flags);
+
+ rcu_read_unlock_sched();
+
+ if (overflow)
+ return notifier_from_errno(-EBUSY);
+ }
cpuset_update_active_cpus(false);
break;
case CPU_DOWN_PREPARE_FROZEN:
rq->calc_load_active = 0;
rq->calc_load_update = jiffies + LOAD_FREQ;
init_cfs_rq(&rq->cfs);
- init_rt_rq(&rq->rt, rq);
- init_dl_rq(&rq->dl, rq);
+ init_rt_rq(&rq->rt);
+ init_dl_rq(&rq->dl);
#ifdef CONFIG_FAIR_GROUP_SCHED
root_task_group.shares = ROOT_TASK_GROUP_LOAD;
INIT_LIST_HEAD(&rq->leaf_cfs_rq_list);
#ifdef CONFIG_SMP
rq->sd = NULL;
rq->rd = NULL;
- rq->cpu_capacity = SCHED_CAPACITY_SCALE;
+ rq->cpu_capacity = rq->cpu_capacity_orig = SCHED_CAPACITY_SCALE;
rq->post_schedule = 0;
rq->active_balance = 0;
rq->next_balance = jiffies;
}
#endif /* CONFIG_RT_GROUP_SCHED */
-static int sched_dl_global_constraints(void)
+static int sched_dl_global_validate(void)
{
u64 runtime = global_rt_runtime();
u64 period = global_rt_period();
if (ret)
goto undo;
- ret = sched_rt_global_constraints();
+ ret = sched_dl_global_validate();
if (ret)
goto undo;
- ret = sched_dl_global_constraints();
+ ret = sched_rt_global_constraints();
if (ret)
goto undo;
dl_b->total_bw = 0;
}
-void init_dl_rq(struct dl_rq *dl_rq, struct rq *rq)
+void init_dl_rq(struct dl_rq *dl_rq)
{
dl_rq->rb_root = RB_ROOT;
rq->post_schedule = has_pushable_dl_tasks(rq);
}
+static struct rq *find_lock_later_rq(struct task_struct *task, struct rq *rq);
+
+static void dl_task_offline_migration(struct rq *rq, struct task_struct *p)
+{
+ struct rq *later_rq = NULL;
+ bool fallback = false;
+
+ later_rq = find_lock_later_rq(p, rq);
+
+ if (!later_rq) {
+ int cpu;
+
+ /*
+ * If we cannot preempt any rq, fall back to pick any
+ * online cpu.
+ */
+ fallback = true;
+ cpu = cpumask_any_and(cpu_active_mask, tsk_cpus_allowed(p));
+ if (cpu >= nr_cpu_ids) {
+ /*
+ * Fail to find any suitable cpu.
+ * The task will never come back!
+ */
+ BUG_ON(dl_bandwidth_enabled());
+
+ /*
+ * If admission control is disabled we
+ * try a little harder to let the task
+ * run.
+ */
+ cpu = cpumask_any(cpu_active_mask);
+ }
+ later_rq = cpu_rq(cpu);
+ double_lock_balance(rq, later_rq);
+ }
+
+ deactivate_task(rq, p, 0);
+ set_task_cpu(p, later_rq->cpu);
+ activate_task(later_rq, p, ENQUEUE_REPLENISH);
+
+ if (!fallback)
+ resched_curr(later_rq);
+
+ double_unlock_balance(rq, later_rq);
+}
+
#else
static inline
unsigned long flags;
struct rq *rq;
- rq = task_rq_lock(current, &flags);
+ rq = task_rq_lock(p, &flags);
/*
* We need to take care of several possible races here:
sched_clock_tick();
update_rq_clock(rq);
+#ifdef CONFIG_SMP
+ /*
+ * If we find that the rq the task was on is no longer
+ * available, we need to select a new rq.
+ */
+ if (unlikely(!rq->online)) {
+ dl_task_offline_migration(rq, p);
+ goto unlock;
+ }
+#endif
+
/*
* If the throttle happened during sched-out; like:
*
push_dl_task(rq);
#endif
unlock:
- task_rq_unlock(rq, current, &flags);
+ task_rq_unlock(rq, p, &flags);
return HRTIMER_NORESTART;
}
}
update_rq_clock(rq);
update_curr_dl(rq);
+ /*
+ * Tell update_rq_clock() that we've just updated,
+ * so we don't do microscopic update in schedule()
+ * and double the fastpath cost.
+ */
+ rq_clock_skip_update(rq, true);
}
#ifdef CONFIG_SMP
{
int check_resched = 1;
- /*
- * If p is throttled, don't consider the possibility
- * of preempting rq->curr, the check will be done right
- * after its runtime will get replenished.
- */
- if (unlikely(p->dl.dl_throttled))
- return;
-
if (task_on_rq_queued(p) && rq->curr != p) {
#ifdef CONFIG_SMP
if (p->nr_cpus_allowed > 1 && rq->dl.overloaded &&
if (!se) {
struct sched_avg *avg = &cpu_rq(cpu)->avg;
P(avg->runnable_avg_sum);
- P(avg->runnable_avg_period);
+ P(avg->avg_period);
return;
}
P(se->load.weight);
#ifdef CONFIG_SMP
P(se->avg.runnable_avg_sum);
- P(se->avg.runnable_avg_period);
+ P(se->avg.running_avg_sum);
+ P(se->avg.avg_period);
P(se->avg.load_avg_contrib);
+ P(se->avg.utilization_avg_contrib);
P(se->avg.decay_count);
#endif
#undef PN
cfs_rq->runnable_load_avg);
SEQ_printf(m, " .%-30s: %ld\n", "blocked_load_avg",
cfs_rq->blocked_load_avg);
+ SEQ_printf(m, " .%-30s: %ld\n", "utilization_load_avg",
+ cfs_rq->utilization_load_avg);
#ifdef CONFIG_FAIR_GROUP_SCHED
SEQ_printf(m, " .%-30s: %ld\n", "tg_load_contrib",
cfs_rq->tg_load_contrib);
P(se.load.weight);
#ifdef CONFIG_SMP
P(se.avg.runnable_avg_sum);
- P(se.avg.runnable_avg_period);
+ P(se.avg.running_avg_sum);
+ P(se.avg.avg_period);
P(se.avg.load_avg_contrib);
+ P(se.avg.utilization_avg_contrib);
P(se.avg.decay_count);
#endif
P(policy);
static unsigned long task_h_load(struct task_struct *p);
static inline void __update_task_entity_contrib(struct sched_entity *se);
+static inline void __update_task_entity_utilization(struct sched_entity *se);
/* Give new task start runnable values to heavy its load in infant time */
void init_task_runnable_average(struct task_struct *p)
u32 slice;
slice = sched_slice(task_cfs_rq(p), &p->se) >> 10;
- p->se.avg.runnable_avg_sum = slice;
- p->se.avg.runnable_avg_period = slice;
+ p->se.avg.runnable_avg_sum = p->se.avg.running_avg_sum = slice;
+ p->se.avg.avg_period = slice;
__update_task_entity_contrib(&p->se);
+ __update_task_entity_utilization(&p->se);
}
#else
void init_task_runnable_average(struct task_struct *p)
static bool load_too_imbalanced(long src_load, long dst_load,
struct task_numa_env *env)
{
- long imb, old_imb;
- long orig_src_load, orig_dst_load;
long src_capacity, dst_capacity;
+ long orig_src_load;
+ long load_a, load_b;
+ long moved_load;
+ long imb;
/*
* The load is corrected for the CPU capacity available on each node.
dst_capacity = env->dst_stats.compute_capacity;
/* We care about the slope of the imbalance, not the direction. */
- if (dst_load < src_load)
- swap(dst_load, src_load);
+ load_a = dst_load;
+ load_b = src_load;
+ if (load_a < load_b)
+ swap(load_a, load_b);
/* Is the difference below the threshold? */
- imb = dst_load * src_capacity * 100 -
- src_load * dst_capacity * env->imbalance_pct;
+ imb = load_a * src_capacity * 100 -
+ load_b * dst_capacity * env->imbalance_pct;
if (imb <= 0)
return false;
/*
* The imbalance is above the allowed threshold.
- * Compare it with the old imbalance.
+ * Allow a move that brings us closer to a balanced situation,
+ * without moving things past the point of balance.
*/
orig_src_load = env->src_stats.load;
- orig_dst_load = env->dst_stats.load;
- if (orig_dst_load < orig_src_load)
- swap(orig_dst_load, orig_src_load);
-
- old_imb = orig_dst_load * src_capacity * 100 -
- orig_src_load * dst_capacity * env->imbalance_pct;
+ /*
+ * In a task swap, there will be one load moving from src to dst,
+ * and another moving back. This is the net sum of both moves.
+ * A simple task move will always have a positive value.
+ * Allow the move if it brings the system closer to a balanced
+ * situation, without crossing over the balance point.
+ */
+ moved_load = orig_src_load - src_load;
- /* Would this change make things worse? */
- return (imb > old_imb);
+ if (moved_load > 0)
+ /* Moving src -> dst. Did we overshoot balance? */
+ return src_load * dst_capacity < dst_load * src_capacity;
+ else
+ /* Moving dst -> src. Did we overshoot balance? */
+ return dst_load * src_capacity < src_load * dst_capacity;
}
/*
/*
* If there were no record hinting faults then either the task is
* completely idle or all activity is areas that are not of interest
- * to automatic numa balancing. Scan slower
+ * to automatic numa balancing. Related to that, if there were failed
+ * migration then it implies we are migrating too quickly or the local
+ * node is overloaded. In either case, scan slower
*/
- if (local + shared == 0) {
+ if (local + shared == 0 || p->numa_faults_locality[2]) {
p->numa_scan_period = min(p->numa_scan_period_max,
p->numa_scan_period << 1);
*period = now - p->last_task_numa_placement;
} else {
delta = p->se.avg.runnable_avg_sum;
- *period = p->se.avg.runnable_avg_period;
+ *period = p->se.avg.avg_period;
}
p->last_sum_exec_runtime = runtime;
}
}
/* Next round, evaluate the nodes within max_group. */
+ if (!max_faults)
+ break;
nodes = max_group;
}
return nid;
if (migrated)
p->numa_pages_migrated += pages;
+ if (flags & TNF_MIGRATE_FAIL)
+ p->numa_faults_locality[2] += pages;
p->numa_faults[task_faults_idx(NUMA_MEMBUF, mem_node, priv)] += pages;
p->numa_faults[task_faults_idx(NUMA_CPUBUF, cpu_node, priv)] += pages;
vma = mm->mmap;
}
for (; vma; vma = vma->vm_next) {
- if (!vma_migratable(vma) || !vma_policy_mof(vma))
+ if (!vma_migratable(vma) || !vma_policy_mof(vma) ||
+ is_vm_hugetlb_page(vma)) {
continue;
+ }
/*
* Shared library pages mapped by multiple processes are not
* load_avg = u_0` + y*(u_0 + u_1*y + u_2*y^2 + ... )
* = u_0 + u_1*y + u_2*y^2 + ... [re-labeling u_i --> u_{i+1}]
*/
-static __always_inline int __update_entity_runnable_avg(u64 now,
+static __always_inline int __update_entity_runnable_avg(u64 now, int cpu,
struct sched_avg *sa,
- int runnable)
+ int runnable,
+ int running)
{
u64 delta, periods;
u32 runnable_contrib;
int delta_w, decayed = 0;
+ unsigned long scale_freq = arch_scale_freq_capacity(NULL, cpu);
delta = now - sa->last_runnable_update;
/*
sa->last_runnable_update = now;
/* delta_w is the amount already accumulated against our next period */
- delta_w = sa->runnable_avg_period % 1024;
+ delta_w = sa->avg_period % 1024;
if (delta + delta_w >= 1024) {
/* period roll-over */
decayed = 1;
delta_w = 1024 - delta_w;
if (runnable)
sa->runnable_avg_sum += delta_w;
- sa->runnable_avg_period += delta_w;
+ if (running)
+ sa->running_avg_sum += delta_w * scale_freq
+ >> SCHED_CAPACITY_SHIFT;
+ sa->avg_period += delta_w;
delta -= delta_w;
sa->runnable_avg_sum = decay_load(sa->runnable_avg_sum,
periods + 1);
- sa->runnable_avg_period = decay_load(sa->runnable_avg_period,
+ sa->running_avg_sum = decay_load(sa->running_avg_sum,
+ periods + 1);
+ sa->avg_period = decay_load(sa->avg_period,
periods + 1);
/* Efficiently calculate \sum (1..n_period) 1024*y^i */
runnable_contrib = __compute_runnable_contrib(periods);
if (runnable)
sa->runnable_avg_sum += runnable_contrib;
- sa->runnable_avg_period += runnable_contrib;
+ if (running)
+ sa->running_avg_sum += runnable_contrib * scale_freq
+ >> SCHED_CAPACITY_SHIFT;
+ sa->avg_period += runnable_contrib;
}
/* Remainder of delta accrued against u_0` */
if (runnable)
sa->runnable_avg_sum += delta;
- sa->runnable_avg_period += delta;
+ if (running)
+ sa->running_avg_sum += delta * scale_freq
+ >> SCHED_CAPACITY_SHIFT;
+ sa->avg_period += delta;
return decayed;
}
return 0;
se->avg.load_avg_contrib = decay_load(se->avg.load_avg_contrib, decays);
+ se->avg.utilization_avg_contrib =
+ decay_load(se->avg.utilization_avg_contrib, decays);
return decays;
}
/* The fraction of a cpu used by this cfs_rq */
contrib = div_u64((u64)sa->runnable_avg_sum << NICE_0_SHIFT,
- sa->runnable_avg_period + 1);
+ sa->avg_period + 1);
contrib -= cfs_rq->tg_runnable_contrib;
if (abs(contrib) > cfs_rq->tg_runnable_contrib / 64) {
static inline void update_rq_runnable_avg(struct rq *rq, int runnable)
{
- __update_entity_runnable_avg(rq_clock_task(rq), &rq->avg, runnable);
+ __update_entity_runnable_avg(rq_clock_task(rq), cpu_of(rq), &rq->avg,
+ runnable, runnable);
__update_tg_runnable_avg(&rq->avg, &rq->cfs);
}
#else /* CONFIG_FAIR_GROUP_SCHED */
/* avoid overflowing a 32-bit type w/ SCHED_LOAD_SCALE */
contrib = se->avg.runnable_avg_sum * scale_load_down(se->load.weight);
- contrib /= (se->avg.runnable_avg_period + 1);
+ contrib /= (se->avg.avg_period + 1);
se->avg.load_avg_contrib = scale_load(contrib);
}
return se->avg.load_avg_contrib - old_contrib;
}
+
+static inline void __update_task_entity_utilization(struct sched_entity *se)
+{
+ u32 contrib;
+
+ /* avoid overflowing a 32-bit type w/ SCHED_LOAD_SCALE */
+ contrib = se->avg.running_avg_sum * scale_load_down(SCHED_LOAD_SCALE);
+ contrib /= (se->avg.avg_period + 1);
+ se->avg.utilization_avg_contrib = scale_load(contrib);
+}
+
+static long __update_entity_utilization_avg_contrib(struct sched_entity *se)
+{
+ long old_contrib = se->avg.utilization_avg_contrib;
+
+ if (entity_is_task(se))
+ __update_task_entity_utilization(se);
+ else
+ se->avg.utilization_avg_contrib =
+ group_cfs_rq(se)->utilization_load_avg;
+
+ return se->avg.utilization_avg_contrib - old_contrib;
+}
+
static inline void subtract_blocked_load_contrib(struct cfs_rq *cfs_rq,
long load_contrib)
{
int update_cfs_rq)
{
struct cfs_rq *cfs_rq = cfs_rq_of(se);
- long contrib_delta;
+ long contrib_delta, utilization_delta;
+ int cpu = cpu_of(rq_of(cfs_rq));
u64 now;
/*
else
now = cfs_rq_clock_task(group_cfs_rq(se));
- if (!__update_entity_runnable_avg(now, &se->avg, se->on_rq))
+ if (!__update_entity_runnable_avg(now, cpu, &se->avg, se->on_rq,
+ cfs_rq->curr == se))
return;
contrib_delta = __update_entity_load_avg_contrib(se);
+ utilization_delta = __update_entity_utilization_avg_contrib(se);
if (!update_cfs_rq)
return;
- if (se->on_rq)
+ if (se->on_rq) {
cfs_rq->runnable_load_avg += contrib_delta;
- else
+ cfs_rq->utilization_load_avg += utilization_delta;
+ } else {
subtract_blocked_load_contrib(cfs_rq, -contrib_delta);
+ }
}
/*
}
cfs_rq->runnable_load_avg += se->avg.load_avg_contrib;
+ cfs_rq->utilization_load_avg += se->avg.utilization_avg_contrib;
/* we force update consideration on load-balancer moves */
update_cfs_rq_blocked_load(cfs_rq, !wakeup);
}
update_cfs_rq_blocked_load(cfs_rq, !sleep);
cfs_rq->runnable_load_avg -= se->avg.load_avg_contrib;
+ cfs_rq->utilization_load_avg -= se->avg.utilization_avg_contrib;
if (sleep) {
cfs_rq->blocked_load_avg += se->avg.load_avg_contrib;
se->avg.decay_count = atomic64_read(&cfs_rq->decay_counter);
*/
update_stats_wait_end(cfs_rq, se);
__dequeue_entity(cfs_rq, se);
+ update_entity_load_avg(se, 1);
}
update_stats_curr_start(cfs_rq, se);
return cpu_rq(cpu)->cpu_capacity;
}
+static unsigned long capacity_orig_of(int cpu)
+{
+ return cpu_rq(cpu)->cpu_capacity_orig;
+}
+
static unsigned long cpu_avg_load_per_task(int cpu)
{
struct rq *rq = cpu_rq(cpu);
done:
return target;
}
+/*
+ * get_cpu_usage returns the amount of capacity of a CPU that is used by CFS
+ * tasks. The unit of the return value must be the one of capacity so we can
+ * compare the usage with the capacity of the CPU that is available for CFS
+ * task (ie cpu_capacity).
+ * cfs.utilization_load_avg is the sum of running time of runnable tasks on a
+ * CPU. It represents the amount of utilization of a CPU in the range
+ * [0..SCHED_LOAD_SCALE]. The usage of a CPU can't be higher than the full
+ * capacity of the CPU because it's about the running time on this CPU.
+ * Nevertheless, cfs.utilization_load_avg can be higher than SCHED_LOAD_SCALE
+ * because of unfortunate rounding in avg_period and running_load_avg or just
+ * after migrating tasks until the average stabilizes with the new running
+ * time. So we need to check that the usage stays into the range
+ * [0..cpu_capacity_orig] and cap if necessary.
+ * Without capping the usage, a group could be seen as overloaded (CPU0 usage
+ * at 121% + CPU1 usage at 80%) whereas CPU1 has 20% of available capacity
+ */
+static int get_cpu_usage(int cpu)
+{
+ unsigned long usage = cpu_rq(cpu)->cfs.utilization_load_avg;
+ unsigned long capacity = capacity_orig_of(cpu);
+
+ if (usage >= SCHED_LOAD_SCALE)
+ return capacity;
+
+ return (usage * capacity) >> SCHED_LOAD_SHIFT;
+}
/*
* select_task_rq_fair: Select target runqueue for the waking task in domains
unsigned long sum_weighted_load; /* Weighted load of group's tasks */
unsigned long load_per_task;
unsigned long group_capacity;
+ unsigned long group_usage; /* Total usage of the group */
unsigned int sum_nr_running; /* Nr tasks running in the group */
- unsigned int group_capacity_factor;
unsigned int idle_cpus;
unsigned int group_weight;
enum group_type group_type;
- int group_has_free_capacity;
+ int group_no_capacity;
#ifdef CONFIG_NUMA_BALANCING
unsigned int nr_numa_running;
unsigned int nr_preferred_running;
return load_idx;
}
-static unsigned long default_scale_capacity(struct sched_domain *sd, int cpu)
-{
- return SCHED_CAPACITY_SCALE;
-}
-
-unsigned long __weak arch_scale_freq_capacity(struct sched_domain *sd, int cpu)
-{
- return default_scale_capacity(sd, cpu);
-}
-
static unsigned long default_scale_cpu_capacity(struct sched_domain *sd, int cpu)
{
if ((sd->flags & SD_SHARE_CPUCAPACITY) && (sd->span_weight > 1))
static unsigned long scale_rt_capacity(int cpu)
{
struct rq *rq = cpu_rq(cpu);
- u64 total, available, age_stamp, avg;
+ u64 total, used, age_stamp, avg;
s64 delta;
/*
total = sched_avg_period() + delta;
- if (unlikely(total < avg)) {
- /* Ensures that capacity won't end up being negative */
- available = 0;
- } else {
- available = total - avg;
- }
-
- if (unlikely((s64)total < SCHED_CAPACITY_SCALE))
- total = SCHED_CAPACITY_SCALE;
+ used = div_u64(avg, total);
- total >>= SCHED_CAPACITY_SHIFT;
+ if (likely(used < SCHED_CAPACITY_SCALE))
+ return SCHED_CAPACITY_SCALE - used;
- return div_u64(available, total);
+ return 1;
}
static void update_cpu_capacity(struct sched_domain *sd, int cpu)
capacity >>= SCHED_CAPACITY_SHIFT;
- sdg->sgc->capacity_orig = capacity;
-
- if (sched_feat(ARCH_CAPACITY))
- capacity *= arch_scale_freq_capacity(sd, cpu);
- else
- capacity *= default_scale_capacity(sd, cpu);
-
- capacity >>= SCHED_CAPACITY_SHIFT;
+ cpu_rq(cpu)->cpu_capacity_orig = capacity;
capacity *= scale_rt_capacity(cpu);
capacity >>= SCHED_CAPACITY_SHIFT;
{
struct sched_domain *child = sd->child;
struct sched_group *group, *sdg = sd->groups;
- unsigned long capacity, capacity_orig;
+ unsigned long capacity;
unsigned long interval;
interval = msecs_to_jiffies(sd->balance_interval);
return;
}
- capacity_orig = capacity = 0;
+ capacity = 0;
if (child->flags & SD_OVERLAP) {
/*
* Use capacity_of(), which is set irrespective of domains
* in update_cpu_capacity().
*
- * This avoids capacity/capacity_orig from being 0 and
+ * This avoids capacity from being 0 and
* causing divide-by-zero issues on boot.
- *
- * Runtime updates will correct capacity_orig.
*/
if (unlikely(!rq->sd)) {
- capacity_orig += capacity_of(cpu);
capacity += capacity_of(cpu);
continue;
}
sgc = rq->sd->groups->sgc;
- capacity_orig += sgc->capacity_orig;
capacity += sgc->capacity;
}
} else {
group = child->groups;
do {
- capacity_orig += group->sgc->capacity_orig;
capacity += group->sgc->capacity;
group = group->next;
} while (group != child->groups);
}
- sdg->sgc->capacity_orig = capacity_orig;
sdg->sgc->capacity = capacity;
}
/*
- * Try and fix up capacity for tiny siblings, this is needed when
- * things like SD_ASYM_PACKING need f_b_g to select another sibling
- * which on its own isn't powerful enough.
- *
- * See update_sd_pick_busiest() and check_asym_packing().
+ * Check whether the capacity of the rq has been noticeably reduced by side
+ * activity. The imbalance_pct is used for the threshold.
+ * Return true is the capacity is reduced
*/
static inline int
-fix_small_capacity(struct sched_domain *sd, struct sched_group *group)
+check_cpu_capacity(struct rq *rq, struct sched_domain *sd)
{
- /*
- * Only siblings can have significantly less than SCHED_CAPACITY_SCALE
- */
- if (!(sd->flags & SD_SHARE_CPUCAPACITY))
- return 0;
-
- /*
- * If ~90% of the cpu_capacity is still there, we're good.
- */
- if (group->sgc->capacity * 32 > group->sgc->capacity_orig * 29)
- return 1;
-
- return 0;
+ return ((rq->cpu_capacity * sd->imbalance_pct) <
+ (rq->cpu_capacity_orig * 100));
}
/*
}
/*
- * Compute the group capacity factor.
- *
- * Avoid the issue where N*frac(smt_capacity) >= 1 creates 'phantom' cores by
- * first dividing out the smt factor and computing the actual number of cores
- * and limit unit capacity with that.
+ * group_has_capacity returns true if the group has spare capacity that could
+ * be used by some tasks.
+ * We consider that a group has spare capacity if the * number of task is
+ * smaller than the number of CPUs or if the usage is lower than the available
+ * capacity for CFS tasks.
+ * For the latter, we use a threshold to stabilize the state, to take into
+ * account the variance of the tasks' load and to return true if the available
+ * capacity in meaningful for the load balancer.
+ * As an example, an available capacity of 1% can appear but it doesn't make
+ * any benefit for the load balance.
*/
-static inline int sg_capacity_factor(struct lb_env *env, struct sched_group *group)
+static inline bool
+group_has_capacity(struct lb_env *env, struct sg_lb_stats *sgs)
{
- unsigned int capacity_factor, smt, cpus;
- unsigned int capacity, capacity_orig;
+ if (sgs->sum_nr_running < sgs->group_weight)
+ return true;
- capacity = group->sgc->capacity;
- capacity_orig = group->sgc->capacity_orig;
- cpus = group->group_weight;
+ if ((sgs->group_capacity * 100) >
+ (sgs->group_usage * env->sd->imbalance_pct))
+ return true;
+
+ return false;
+}
- /* smt := ceil(cpus / capacity), assumes: 1 < smt_capacity < 2 */
- smt = DIV_ROUND_UP(SCHED_CAPACITY_SCALE * cpus, capacity_orig);
- capacity_factor = cpus / smt; /* cores */
+/*
+ * group_is_overloaded returns true if the group has more tasks than it can
+ * handle.
+ * group_is_overloaded is not equals to !group_has_capacity because a group
+ * with the exact right number of tasks, has no more spare capacity but is not
+ * overloaded so both group_has_capacity and group_is_overloaded return
+ * false.
+ */
+static inline bool
+group_is_overloaded(struct lb_env *env, struct sg_lb_stats *sgs)
+{
+ if (sgs->sum_nr_running <= sgs->group_weight)
+ return false;
- capacity_factor = min_t(unsigned,
- capacity_factor, DIV_ROUND_CLOSEST(capacity, SCHED_CAPACITY_SCALE));
- if (!capacity_factor)
- capacity_factor = fix_small_capacity(env->sd, group);
+ if ((sgs->group_capacity * 100) <
+ (sgs->group_usage * env->sd->imbalance_pct))
+ return true;
- return capacity_factor;
+ return false;
}
-static enum group_type
-group_classify(struct sched_group *group, struct sg_lb_stats *sgs)
+static enum group_type group_classify(struct lb_env *env,
+ struct sched_group *group,
+ struct sg_lb_stats *sgs)
{
- if (sgs->sum_nr_running > sgs->group_capacity_factor)
+ if (sgs->group_no_capacity)
return group_overloaded;
if (sg_imbalanced(group))
load = source_load(i, load_idx);
sgs->group_load += load;
+ sgs->group_usage += get_cpu_usage(i);
sgs->sum_nr_running += rq->cfs.h_nr_running;
if (rq->nr_running > 1)
sgs->load_per_task = sgs->sum_weighted_load / sgs->sum_nr_running;
sgs->group_weight = group->group_weight;
- sgs->group_capacity_factor = sg_capacity_factor(env, group);
- sgs->group_type = group_classify(group, sgs);
- if (sgs->group_capacity_factor > sgs->sum_nr_running)
- sgs->group_has_free_capacity = 1;
+ sgs->group_no_capacity = group_is_overloaded(env, sgs);
+ sgs->group_type = group_classify(env, group, sgs);
}
/**
/*
* In case the child domain prefers tasks go to siblings
- * first, lower the sg capacity factor to one so that we'll try
+ * first, lower the sg capacity so that we'll try
* and move all the excess tasks away. We lower the capacity
* of a group only if the local group has the capacity to fit
- * these excess tasks, i.e. nr_running < group_capacity_factor. The
- * extra check prevents the case where you always pull from the
- * heaviest group when it is already under-utilized (possible
- * with a large weight task outweighs the tasks on the system).
+ * these excess tasks. The extra check prevents the case where
+ * you always pull from the heaviest group when it is already
+ * under-utilized (possible with a large weight task outweighs
+ * the tasks on the system).
*/
if (prefer_sibling && sds->local &&
- sds->local_stat.group_has_free_capacity) {
- sgs->group_capacity_factor = min(sgs->group_capacity_factor, 1U);
- sgs->group_type = group_classify(sg, sgs);
+ group_has_capacity(env, &sds->local_stat) &&
+ (sgs->sum_nr_running > 1)) {
+ sgs->group_no_capacity = 1;
+ sgs->group_type = group_overloaded;
}
if (update_sd_pick_busiest(env, sds, sg, sgs)) {
*/
if (busiest->group_type == group_overloaded &&
local->group_type == group_overloaded) {
- load_above_capacity =
- (busiest->sum_nr_running - busiest->group_capacity_factor);
-
- load_above_capacity *= (SCHED_LOAD_SCALE * SCHED_CAPACITY_SCALE);
- load_above_capacity /= busiest->group_capacity;
+ load_above_capacity = busiest->sum_nr_running *
+ SCHED_LOAD_SCALE;
+ if (load_above_capacity > busiest->group_capacity)
+ load_above_capacity -= busiest->group_capacity;
+ else
+ load_above_capacity = ~0UL;
}
/*
local = &sds.local_stat;
busiest = &sds.busiest_stat;
+ /* ASYM feature bypasses nice load balance check */
if ((env->idle == CPU_IDLE || env->idle == CPU_NEWLY_IDLE) &&
check_asym_packing(env, &sds))
return sds.busiest;
goto force_balance;
/* SD_BALANCE_NEWIDLE trumps SMP nice when underutilized */
- if (env->idle == CPU_NEWLY_IDLE && local->group_has_free_capacity &&
- !busiest->group_has_free_capacity)
+ if (env->idle == CPU_NEWLY_IDLE && group_has_capacity(env, local) &&
+ busiest->group_no_capacity)
goto force_balance;
/*
int i;
for_each_cpu_and(i, sched_group_cpus(group), env->cpus) {
- unsigned long capacity, capacity_factor, wl;
+ unsigned long capacity, wl;
enum fbq_type rt;
rq = cpu_rq(i);
continue;
capacity = capacity_of(i);
- capacity_factor = DIV_ROUND_CLOSEST(capacity, SCHED_CAPACITY_SCALE);
- if (!capacity_factor)
- capacity_factor = fix_small_capacity(env->sd, group);
wl = weighted_cpuload(i);
* When comparing with imbalance, use weighted_cpuload()
* which is not scaled with the cpu capacity.
*/
- if (capacity_factor && rq->nr_running == 1 && wl > env->imbalance)
+
+ if (rq->nr_running == 1 && wl > env->imbalance &&
+ !check_cpu_capacity(rq, env->sd))
continue;
/*
return 1;
}
+ /*
+ * The dst_cpu is idle and the src_cpu CPU has only 1 CFS task.
+ * It's worth migrating the task if the src_cpu's capacity is reduced
+ * because of other sched_class or IRQs if more capacity stays
+ * available on dst_cpu.
+ */
+ if ((env->idle != CPU_NOT_IDLE) &&
+ (env->src_rq->cfs.h_nr_running == 1)) {
+ if ((check_cpu_capacity(env->src_rq, sd)) &&
+ (capacity_of(env->src_cpu)*sd->imbalance_pct < capacity_of(env->dst_cpu)*100))
+ return 1;
+ }
+
return unlikely(sd->nr_balance_failed > sd->cache_nice_tries+2);
}
schedstat_add(sd, lb_imbalance[idle], env.imbalance);
+ env.src_cpu = busiest->cpu;
+ env.src_rq = busiest;
+
ld_moved = 0;
if (busiest->nr_running > 1) {
/*
* correctly treated as an imbalance.
*/
env.flags |= LBF_ALL_PINNED;
- env.src_cpu = busiest->cpu;
- env.src_rq = busiest;
env.loop_max = min(sysctl_sched_nr_migrate, busiest->nr_running);
more_balance:
/*
* Current heuristic for kicking the idle load balancer in the presence
- * of an idle cpu is the system.
+ * of an idle cpu in the system.
* - This rq has more than one task.
- * - At any scheduler domain level, this cpu's scheduler group has multiple
- * busy cpu's exceeding the group's capacity.
+ * - This rq has at least one CFS task and the capacity of the CPU is
+ * significantly reduced because of RT tasks or IRQs.
+ * - At parent of LLC scheduler domain level, this cpu's scheduler group has
+ * multiple busy cpu.
* - For SD_ASYM_PACKING, if the lower numbered cpu's in the scheduler
* domain span are idle.
*/
-static inline int nohz_kick_needed(struct rq *rq)
+static inline bool nohz_kick_needed(struct rq *rq)
{
unsigned long now = jiffies;
struct sched_domain *sd;
struct sched_group_capacity *sgc;
int nr_busy, cpu = rq->cpu;
+ bool kick = false;
if (unlikely(rq->idle_balance))
- return 0;
+ return false;
/*
* We may be recently in ticked or tickless idle mode. At the first
* balancing.
*/
if (likely(!atomic_read(&nohz.nr_cpus)))
- return 0;
+ return false;
if (time_before(now, nohz.next_balance))
- return 0;
+ return false;
if (rq->nr_running >= 2)
- goto need_kick;
+ return true;
rcu_read_lock();
sd = rcu_dereference(per_cpu(sd_busy, cpu));
-
if (sd) {
sgc = sd->groups->sgc;
nr_busy = atomic_read(&sgc->nr_busy_cpus);
- if (nr_busy > 1)
- goto need_kick_unlock;
+ if (nr_busy > 1) {
+ kick = true;
+ goto unlock;
+ }
+
}
- sd = rcu_dereference(per_cpu(sd_asym, cpu));
+ sd = rcu_dereference(rq->sd);
+ if (sd) {
+ if ((rq->cfs.h_nr_running >= 1) &&
+ check_cpu_capacity(rq, sd)) {
+ kick = true;
+ goto unlock;
+ }
+ }
+ sd = rcu_dereference(per_cpu(sd_asym, cpu));
if (sd && (cpumask_first_and(nohz.idle_cpus_mask,
- sched_domain_span(sd)) < cpu))
- goto need_kick_unlock;
-
- rcu_read_unlock();
- return 0;
+ sched_domain_span(sd)) < cpu)) {
+ kick = true;
+ goto unlock;
+ }
-need_kick_unlock:
+unlock:
rcu_read_unlock();
-need_kick:
- return 1;
+ return kick;
}
#else
static void nohz_idle_balance(struct rq *this_rq, enum cpu_idle_type idle) { }
enum cpu_idle_type idle = this_rq->idle_balance ?
CPU_IDLE : CPU_NOT_IDLE;
- rebalance_domains(this_rq, idle);
-
/*
* If this cpu has a pending nohz_balance_kick, then do the
* balancing on behalf of the other idle cpus whose ticks are
- * stopped.
+ * stopped. Do nohz_idle_balance *before* rebalance_domains to
+ * give the idle cpus a chance to load balance. Else we may
+ * load balance only within the local sched_domain hierarchy
+ * and abort nohz_idle_balance altogether if we pull some load.
*/
nohz_idle_balance(this_rq, idle);
+ rebalance_domains(this_rq, idle);
}
/*
*/
SCHED_FEAT(TTWU_QUEUE, true)
+#ifdef HAVE_RT_PUSH_IPI
+/*
+ * In order to avoid a thundering herd attack of CPUs that are
+ * lowering their priorities at the same time, and there being
+ * a single CPU that has an RT task that can migrate and is waiting
+ * to run, where the other CPUs will try to take that CPUs
+ * rq lock and possibly create a large contention, sending an
+ * IPI to that CPU and let that CPU push the RT task to where
+ * it should go may be a better scenario.
+ */
+SCHED_FEAT(RT_PUSH_IPI, true)
+#endif
+
SCHED_FEAT(FORCE_SD_OVERLAP, false)
SCHED_FEAT(RT_RUNTIME_SHARE, true)
SCHED_FEAT(LB_MIN, false)
struct cpuidle_driver *drv = cpuidle_get_cpu_driver(dev);
int next_state, entered_state;
unsigned int broadcast;
+ bool reflect;
/*
* Check if the idle task must be rescheduled. If it is the
*/
rcu_idle_enter();
+ if (cpuidle_not_available(drv, dev))
+ goto use_default;
+
/*
* Suspend-to-idle ("freeze") is a system state in which all user space
* has been frozen, all I/O devices have been suspended and the only
* until a proper wakeup interrupt happens.
*/
if (idle_should_freeze()) {
- cpuidle_enter_freeze();
- local_irq_enable();
- goto exit_idle;
- }
+ entered_state = cpuidle_enter_freeze(drv, dev);
+ if (entered_state >= 0) {
+ local_irq_enable();
+ goto exit_idle;
+ }
- /*
- * Ask the cpuidle framework to choose a convenient idle state.
- * Fall back to the default arch idle method on errors.
- */
- next_state = cpuidle_select(drv, dev);
- if (next_state < 0) {
-use_default:
+ reflect = false;
+ next_state = cpuidle_find_deepest_state(drv, dev);
+ } else {
+ reflect = true;
/*
- * We can't use the cpuidle framework, let's use the default
- * idle routine.
+ * Ask the cpuidle framework to choose a convenient idle state.
*/
- if (current_clr_polling_and_test())
- local_irq_enable();
- else
- arch_cpu_idle();
-
- goto exit_idle;
+ next_state = cpuidle_select(drv, dev);
}
-
+ /* Fall back to the default arch idle method on errors. */
+ if (next_state < 0)
+ goto use_default;
/*
* The idle task must be scheduled, it is pointless to
* is used from another cpu as a broadcast timer, this call may
* fail if it is not available
*/
- if (broadcast &&
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_ENTER, &dev->cpu))
+ if (broadcast && tick_broadcast_enter())
goto use_default;
/* Take note of the planned idle state. */
idle_set_state(this_rq(), NULL);
if (broadcast)
- clockevents_notify(CLOCK_EVT_NOTIFY_BROADCAST_EXIT, &dev->cpu);
+ tick_broadcast_exit();
/*
* Give the governor an opportunity to reflect on the outcome
*/
- cpuidle_reflect(dev, entered_state);
+ if (reflect)
+ cpuidle_reflect(dev, entered_state);
exit_idle:
__current_set_polling();
rcu_idle_exit();
start_critical_timings();
+ return;
+
+use_default:
+ /*
+ * We can't use the cpuidle framework, let's use the default
+ * idle routine.
+ */
+ if (current_clr_polling_and_test())
+ local_irq_enable();
+ else
+ arch_cpu_idle();
+
+ goto exit_idle;
}
/*
#include "sched.h"
#include <linux/slab.h>
+#include <linux/irq_work.h>
int sched_rr_timeslice = RR_TIMESLICE;
raw_spin_unlock(&rt_b->rt_runtime_lock);
}
-void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq)
+#ifdef CONFIG_SMP
+static void push_irq_work_func(struct irq_work *work);
+#endif
+
+void init_rt_rq(struct rt_rq *rt_rq)
{
struct rt_prio_array *array;
int i;
rt_rq->rt_nr_migratory = 0;
rt_rq->overloaded = 0;
plist_head_init(&rt_rq->pushable_tasks);
+
+#ifdef HAVE_RT_PUSH_IPI
+ rt_rq->push_flags = 0;
+ rt_rq->push_cpu = nr_cpu_ids;
+ raw_spin_lock_init(&rt_rq->push_lock);
+ init_irq_work(&rt_rq->push_work, push_irq_work_func);
#endif
+#endif /* CONFIG_SMP */
/* We start is dequeued state, because no RT tasks are queued */
rt_rq->rt_queued = 0;
if (!rt_se)
goto err_free_rq;
- init_rt_rq(rt_rq, cpu_rq(i));
+ init_rt_rq(rt_rq);
rt_rq->rt_runtime = tg->rt_bandwidth.rt_runtime;
init_tg_rt_entry(tg, rt_rq, rt_se, i, parent->rt_se[i]);
}
;
}
+#ifdef HAVE_RT_PUSH_IPI
+/*
+ * The search for the next cpu always starts at rq->cpu and ends
+ * when we reach rq->cpu again. It will never return rq->cpu.
+ * This returns the next cpu to check, or nr_cpu_ids if the loop
+ * is complete.
+ *
+ * rq->rt.push_cpu holds the last cpu returned by this function,
+ * or if this is the first instance, it must hold rq->cpu.
+ */
+static int rto_next_cpu(struct rq *rq)
+{
+ int prev_cpu = rq->rt.push_cpu;
+ int cpu;
+
+ cpu = cpumask_next(prev_cpu, rq->rd->rto_mask);
+
+ /*
+ * If the previous cpu is less than the rq's CPU, then it already
+ * passed the end of the mask, and has started from the beginning.
+ * We end if the next CPU is greater or equal to rq's CPU.
+ */
+ if (prev_cpu < rq->cpu) {
+ if (cpu >= rq->cpu)
+ return nr_cpu_ids;
+
+ } else if (cpu >= nr_cpu_ids) {
+ /*
+ * We passed the end of the mask, start at the beginning.
+ * If the result is greater or equal to the rq's CPU, then
+ * the loop is finished.
+ */
+ cpu = cpumask_first(rq->rd->rto_mask);
+ if (cpu >= rq->cpu)
+ return nr_cpu_ids;
+ }
+ rq->rt.push_cpu = cpu;
+
+ /* Return cpu to let the caller know if the loop is finished or not */
+ return cpu;
+}
+
+static int find_next_push_cpu(struct rq *rq)
+{
+ struct rq *next_rq;
+ int cpu;
+
+ while (1) {
+ cpu = rto_next_cpu(rq);
+ if (cpu >= nr_cpu_ids)
+ break;
+ next_rq = cpu_rq(cpu);
+
+ /* Make sure the next rq can push to this rq */
+ if (next_rq->rt.highest_prio.next < rq->rt.highest_prio.curr)
+ break;
+ }
+
+ return cpu;
+}
+
+#define RT_PUSH_IPI_EXECUTING 1
+#define RT_PUSH_IPI_RESTART 2
+
+static void tell_cpu_to_push(struct rq *rq)
+{
+ int cpu;
+
+ if (rq->rt.push_flags & RT_PUSH_IPI_EXECUTING) {
+ raw_spin_lock(&rq->rt.push_lock);
+ /* Make sure it's still executing */
+ if (rq->rt.push_flags & RT_PUSH_IPI_EXECUTING) {
+ /*
+ * Tell the IPI to restart the loop as things have
+ * changed since it started.
+ */
+ rq->rt.push_flags |= RT_PUSH_IPI_RESTART;
+ raw_spin_unlock(&rq->rt.push_lock);
+ return;
+ }
+ raw_spin_unlock(&rq->rt.push_lock);
+ }
+
+ /* When here, there's no IPI going around */
+
+ rq->rt.push_cpu = rq->cpu;
+ cpu = find_next_push_cpu(rq);
+ if (cpu >= nr_cpu_ids)
+ return;
+
+ rq->rt.push_flags = RT_PUSH_IPI_EXECUTING;
+
+ irq_work_queue_on(&rq->rt.push_work, cpu);
+}
+
+/* Called from hardirq context */
+static void try_to_push_tasks(void *arg)
+{
+ struct rt_rq *rt_rq = arg;
+ struct rq *rq, *src_rq;
+ int this_cpu;
+ int cpu;
+
+ this_cpu = rt_rq->push_cpu;
+
+ /* Paranoid check */
+ BUG_ON(this_cpu != smp_processor_id());
+
+ rq = cpu_rq(this_cpu);
+ src_rq = rq_of_rt_rq(rt_rq);
+
+again:
+ if (has_pushable_tasks(rq)) {
+ raw_spin_lock(&rq->lock);
+ push_rt_task(rq);
+ raw_spin_unlock(&rq->lock);
+ }
+
+ /* Pass the IPI to the next rt overloaded queue */
+ raw_spin_lock(&rt_rq->push_lock);
+ /*
+ * If the source queue changed since the IPI went out,
+ * we need to restart the search from that CPU again.
+ */
+ if (rt_rq->push_flags & RT_PUSH_IPI_RESTART) {
+ rt_rq->push_flags &= ~RT_PUSH_IPI_RESTART;
+ rt_rq->push_cpu = src_rq->cpu;
+ }
+
+ cpu = find_next_push_cpu(src_rq);
+
+ if (cpu >= nr_cpu_ids)
+ rt_rq->push_flags &= ~RT_PUSH_IPI_EXECUTING;
+ raw_spin_unlock(&rt_rq->push_lock);
+
+ if (cpu >= nr_cpu_ids)
+ return;
+
+ /*
+ * It is possible that a restart caused this CPU to be
+ * chosen again. Don't bother with an IPI, just see if we
+ * have more to push.
+ */
+ if (unlikely(cpu == rq->cpu))
+ goto again;
+
+ /* Try the next RT overloaded CPU */
+ irq_work_queue_on(&rt_rq->push_work, cpu);
+}
+
+static void push_irq_work_func(struct irq_work *work)
+{
+ struct rt_rq *rt_rq = container_of(work, struct rt_rq, push_work);
+
+ try_to_push_tasks(rt_rq);
+}
+#endif /* HAVE_RT_PUSH_IPI */
+
static int pull_rt_task(struct rq *this_rq)
{
int this_cpu = this_rq->cpu, ret = 0, cpu;
*/
smp_rmb();
+#ifdef HAVE_RT_PUSH_IPI
+ if (sched_feat(RT_PUSH_IPI)) {
+ tell_cpu_to_push(this_rq);
+ return 0;
+ }
+#endif
+
for_each_cpu(cpu, this_rq->rd->rto_mask) {
if (this_cpu == cpu)
continue;
#include <linux/mutex.h>
#include <linux/spinlock.h>
#include <linux/stop_machine.h>
+#include <linux/irq_work.h>
#include <linux/tick.h>
#include <linux/slab.h>
* Under CFS, load is tracked on a per-entity basis and aggregated up.
* This allows for the description of both thread and group usage (in
* the FAIR_GROUP_SCHED case).
+ * runnable_load_avg is the sum of the load_avg_contrib of the
+ * sched_entities on the rq.
+ * blocked_load_avg is similar to runnable_load_avg except that its
+ * the blocked sched_entities on the rq.
+ * utilization_load_avg is the sum of the average running time of the
+ * sched_entities on the rq.
*/
- unsigned long runnable_load_avg, blocked_load_avg;
+ unsigned long runnable_load_avg, blocked_load_avg, utilization_load_avg;
atomic64_t decay_counter;
u64 last_decay;
atomic_long_t removed_load;
return sysctl_sched_rt_runtime >= 0;
}
+/* RT IPI pull logic requires IRQ_WORK */
+#ifdef CONFIG_IRQ_WORK
+# define HAVE_RT_PUSH_IPI
+#endif
+
/* Real-Time classes' related field in a runqueue: */
struct rt_rq {
struct rt_prio_array active;
unsigned long rt_nr_total;
int overloaded;
struct plist_head pushable_tasks;
+#ifdef HAVE_RT_PUSH_IPI
+ int push_flags;
+ int push_cpu;
+ struct irq_work push_work;
+ raw_spinlock_t push_lock;
#endif
+#endif /* CONFIG_SMP */
int rt_queued;
int rt_throttled;
struct sched_domain *sd;
unsigned long cpu_capacity;
+ unsigned long cpu_capacity_orig;
unsigned char idle_balance;
/* For active balancing */
* CPU capacity of this group, SCHED_LOAD_SCALE being max capacity
* for a single CPU.
*/
- unsigned int capacity, capacity_orig;
+ unsigned int capacity;
unsigned long next_update;
int imbalance; /* XXX unrelated to capacity but shared group state */
/*
#ifdef CONFIG_SMP
extern void sched_avg_update(struct rq *rq);
+
+#ifndef arch_scale_freq_capacity
+static __always_inline
+unsigned long arch_scale_freq_capacity(struct sched_domain *sd, int cpu)
+{
+ return SCHED_CAPACITY_SCALE;
+}
+#endif
+
static inline void sched_rt_avg_update(struct rq *rq, u64 rt_delta)
{
- rq->rt_avg += rt_delta;
+ rq->rt_avg += rt_delta * arch_scale_freq_capacity(NULL, cpu_of(rq));
sched_avg_update(rq);
}
#else
extern void print_dl_stats(struct seq_file *m, int cpu);
extern void init_cfs_rq(struct cfs_rq *cfs_rq);
-extern void init_rt_rq(struct rt_rq *rt_rq, struct rq *rq);
-extern void init_dl_rq(struct dl_rq *dl_rq, struct rq *rq);
+extern void init_rt_rq(struct rt_rq *rt_rq);
+extern void init_dl_rq(struct dl_rq *dl_rq);
extern void cfs_bandwidth_usage_inc(void);
extern void cfs_bandwidth_usage_dec(void);
/*
* Work around broken programs that cannot handle "Linux 3.0".
* Instead we map 3.x to 2.6.40+x, so e.g. 3.0 would be 2.6.40
+ * And we map 4.x to 2.6.60+x, so 4.0 would be 2.6.60.
*/
static int override_release(char __user *release, size_t len)
{
break;
rest++;
}
- v = ((LINUX_VERSION_CODE >> 8) & 0xff) + 40;
+ v = ((LINUX_VERSION_CODE >> 8) & 0xff) + 60;
copy = clamp_t(size_t, len, 1, sizeof(buf));
copy = scnprintf(buf, copy, "2.6.%u%s", v, rest);
ret = copy_to_user(release, buf, copy + 1);
.proc_handler = proc_dointvec_minmax,
.extra1 = &zero,
},
+ {
+ .procname = "dirtytime_expire_seconds",
+ .data = &dirtytime_expire_interval,
+ .maxlen = sizeof(dirty_expire_interval),
+ .mode = 0644,
+ .proc_handler = dirtytime_interval_handler,
+ .extra1 = &zero,
+ },
{
.procname = "nr_pdflush_threads",
.mode = 0444 /* read-only */,
config GENERIC_CLOCKEVENTS
bool
-# Migration helper. Builds, but does not invoke
-config GENERIC_CLOCKEVENTS_BUILD
- bool
- default y
- depends on GENERIC_CLOCKEVENTS
-
# Architecture can handle broadcast in a driver-agnostic way
config ARCH_HAS_TICK_BROADCAST
bool
obj-y += timekeeping.o ntp.o clocksource.o jiffies.o timer_list.o
obj-y += timeconv.o timecounter.o posix-clock.o alarmtimer.o
-obj-$(CONFIG_GENERIC_CLOCKEVENTS_BUILD) += clockevents.o
-obj-$(CONFIG_GENERIC_CLOCKEVENTS) += tick-common.o
+obj-$(CONFIG_GENERIC_CLOCKEVENTS) += clockevents.o tick-common.o
ifeq ($(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST),y)
obj-y += tick-broadcast.o
obj-$(CONFIG_TICK_ONESHOT) += tick-broadcast-hrtimer.o
endif
obj-$(CONFIG_GENERIC_SCHED_CLOCK) += sched_clock.o
-obj-$(CONFIG_TICK_ONESHOT) += tick-oneshot.o
-obj-$(CONFIG_TICK_ONESHOT) += tick-sched.o
+obj-$(CONFIG_TICK_ONESHOT) += tick-oneshot.o tick-sched.o
obj-$(CONFIG_TIMER_STATS) += timer_stats.o
obj-$(CONFIG_DEBUG_FS) += timekeeping_debug.o
obj-$(CONFIG_TEST_UDELAY) += test_udelay.o
}
EXPORT_SYMBOL_GPL(clockevent_delta2ns);
+static int __clockevents_set_state(struct clock_event_device *dev,
+ enum clock_event_state state)
+{
+ /* Transition with legacy set_mode() callback */
+ if (dev->set_mode) {
+ /* Legacy callback doesn't support new modes */
+ if (state > CLOCK_EVT_STATE_ONESHOT)
+ return -ENOSYS;
+ /*
+ * 'clock_event_state' and 'clock_event_mode' have 1-to-1
+ * mapping until *_ONESHOT, and so a simple cast will work.
+ */
+ dev->set_mode((enum clock_event_mode)state, dev);
+ dev->mode = (enum clock_event_mode)state;
+ return 0;
+ }
+
+ if (dev->features & CLOCK_EVT_FEAT_DUMMY)
+ return 0;
+
+ /* Transition with new state-specific callbacks */
+ switch (state) {
+ case CLOCK_EVT_STATE_DETACHED:
+ /*
+ * This is an internal state, which is guaranteed to go from
+ * SHUTDOWN to DETACHED. No driver interaction required.
+ */
+ return 0;
+
+ case CLOCK_EVT_STATE_SHUTDOWN:
+ return dev->set_state_shutdown(dev);
+
+ case CLOCK_EVT_STATE_PERIODIC:
+ /* Core internal bug */
+ if (!(dev->features & CLOCK_EVT_FEAT_PERIODIC))
+ return -ENOSYS;
+ return dev->set_state_periodic(dev);
+
+ case CLOCK_EVT_STATE_ONESHOT:
+ /* Core internal bug */
+ if (!(dev->features & CLOCK_EVT_FEAT_ONESHOT))
+ return -ENOSYS;
+ return dev->set_state_oneshot(dev);
+
+ default:
+ return -ENOSYS;
+ }
+}
+
/**
- * clockevents_set_mode - set the operating mode of a clock event device
+ * clockevents_set_state - set the operating state of a clock event device
* @dev: device to modify
- * @mode: new mode
+ * @state: new state
*
* Must be called with interrupts disabled !
*/
-void clockevents_set_mode(struct clock_event_device *dev,
- enum clock_event_mode mode)
+void clockevents_set_state(struct clock_event_device *dev,
+ enum clock_event_state state)
{
- if (dev->mode != mode) {
- dev->set_mode(mode, dev);
- dev->mode = mode;
+ if (dev->state != state) {
+ if (__clockevents_set_state(dev, state))
+ return;
+
+ dev->state = state;
/*
* A nsec2cyc multiplicator of 0 is invalid and we'd crash
* on it, so fix it up and emit a warning:
*/
- if (mode == CLOCK_EVT_MODE_ONESHOT) {
+ if (state == CLOCK_EVT_STATE_ONESHOT) {
if (unlikely(!dev->mult)) {
dev->mult = 1;
WARN_ON(1);
*/
void clockevents_shutdown(struct clock_event_device *dev)
{
- clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_SHUTDOWN);
dev->next_event.tv64 = KTIME_MAX;
}
+/**
+ * clockevents_tick_resume - Resume the tick device before using it again
+ * @dev: device to resume
+ */
+int clockevents_tick_resume(struct clock_event_device *dev)
+{
+ int ret = 0;
+
+ if (dev->set_mode) {
+ dev->set_mode(CLOCK_EVT_MODE_RESUME, dev);
+ dev->mode = CLOCK_EVT_MODE_RESUME;
+ } else if (dev->tick_resume) {
+ ret = dev->tick_resume(dev);
+ }
+
+ return ret;
+}
+
#ifdef CONFIG_GENERIC_CLOCKEVENTS_MIN_ADJUST
/* Limit min_delta to a jiffie */
delta = dev->min_delta_ns;
dev->next_event = ktime_add_ns(ktime_get(), delta);
- if (dev->mode == CLOCK_EVT_MODE_SHUTDOWN)
+ if (dev->state == CLOCK_EVT_STATE_SHUTDOWN)
return 0;
dev->retries++;
delta = dev->min_delta_ns;
dev->next_event = ktime_add_ns(ktime_get(), delta);
- if (dev->mode == CLOCK_EVT_MODE_SHUTDOWN)
+ if (dev->state == CLOCK_EVT_STATE_SHUTDOWN)
return 0;
dev->retries++;
dev->next_event = expires;
- if (dev->mode == CLOCK_EVT_MODE_SHUTDOWN)
+ if (dev->state == CLOCK_EVT_STATE_SHUTDOWN)
return 0;
/* Shortcut for clockevent devices that can deal with ktime. */
struct clock_event_device *dev, *newdev = NULL;
list_for_each_entry(dev, &clockevent_devices, list) {
- if (dev == ced || dev->mode != CLOCK_EVT_MODE_UNUSED)
+ if (dev == ced || dev->state != CLOCK_EVT_STATE_DETACHED)
continue;
if (!tick_check_replacement(newdev, dev))
static int __clockevents_try_unbind(struct clock_event_device *ced, int cpu)
{
/* Fast track. Device is unused */
- if (ced->mode == CLOCK_EVT_MODE_UNUSED) {
+ if (ced->state == CLOCK_EVT_STATE_DETACHED) {
list_del_init(&ced->list);
return 0;
}
}
EXPORT_SYMBOL_GPL(clockevents_unbind);
+/* Sanity check of state transition callbacks */
+static int clockevents_sanity_check(struct clock_event_device *dev)
+{
+ /* Legacy set_mode() callback */
+ if (dev->set_mode) {
+ /* We shouldn't be supporting new modes now */
+ WARN_ON(dev->set_state_periodic || dev->set_state_oneshot ||
+ dev->set_state_shutdown || dev->tick_resume);
+
+ BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
+ return 0;
+ }
+
+ if (dev->features & CLOCK_EVT_FEAT_DUMMY)
+ return 0;
+
+ /* New state-specific callbacks */
+ if (!dev->set_state_shutdown)
+ return -EINVAL;
+
+ if ((dev->features & CLOCK_EVT_FEAT_PERIODIC) &&
+ !dev->set_state_periodic)
+ return -EINVAL;
+
+ if ((dev->features & CLOCK_EVT_FEAT_ONESHOT) &&
+ !dev->set_state_oneshot)
+ return -EINVAL;
+
+ return 0;
+}
+
/**
* clockevents_register_device - register a clock event device
* @dev: device to register
{
unsigned long flags;
- BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
+ BUG_ON(clockevents_sanity_check(dev));
+
+ /* Initialize state to DETACHED */
+ dev->state = CLOCK_EVT_STATE_DETACHED;
+
if (!dev->cpumask) {
WARN_ON(num_possible_cpus() > 1);
dev->cpumask = cpumask_of(smp_processor_id());
{
clockevents_config(dev, freq);
- if (dev->mode == CLOCK_EVT_MODE_ONESHOT)
+ if (dev->state == CLOCK_EVT_STATE_ONESHOT)
return clockevents_program_event(dev, dev->next_event, false);
- if (dev->mode == CLOCK_EVT_MODE_PERIODIC)
- dev->set_mode(CLOCK_EVT_MODE_PERIODIC, dev);
+ if (dev->state == CLOCK_EVT_STATE_PERIODIC)
+ return __clockevents_set_state(dev, CLOCK_EVT_STATE_PERIODIC);
return 0;
}
* @old: device to release (can be NULL)
* @new: device to request (can be NULL)
*
- * Called from the notifier chain. clockevents_lock is held already
+ * Called from various tick functions with clockevents_lock held and
+ * interrupts disabled.
*/
void clockevents_exchange_device(struct clock_event_device *old,
struct clock_event_device *new)
{
- unsigned long flags;
-
- local_irq_save(flags);
/*
* Caller releases a clock event device. We queue it into the
* released list and do a notify add later.
*/
if (old) {
module_put(old->owner);
- clockevents_set_mode(old, CLOCK_EVT_MODE_UNUSED);
+ clockevents_set_state(old, CLOCK_EVT_STATE_DETACHED);
list_del(&old->list);
list_add(&old->list, &clockevents_released);
}
if (new) {
- BUG_ON(new->mode != CLOCK_EVT_MODE_UNUSED);
+ BUG_ON(new->state != CLOCK_EVT_STATE_DETACHED);
clockevents_shutdown(new);
}
- local_irq_restore(flags);
}
/**
dev->resume(dev);
}
-#ifdef CONFIG_GENERIC_CLOCKEVENTS
+#ifdef CONFIG_HOTPLUG_CPU
/**
- * clockevents_notify - notification about relevant events
- * Returns 0 on success, any other value on error
+ * tick_cleanup_dead_cpu - Cleanup the tick and clockevents of a dead cpu
*/
-int clockevents_notify(unsigned long reason, void *arg)
+void tick_cleanup_dead_cpu(int cpu)
{
struct clock_event_device *dev, *tmp;
unsigned long flags;
- int cpu, ret = 0;
raw_spin_lock_irqsave(&clockevents_lock, flags);
- switch (reason) {
- case CLOCK_EVT_NOTIFY_BROADCAST_ON:
- case CLOCK_EVT_NOTIFY_BROADCAST_OFF:
- case CLOCK_EVT_NOTIFY_BROADCAST_FORCE:
- tick_broadcast_on_off(reason, arg);
- break;
-
- case CLOCK_EVT_NOTIFY_BROADCAST_ENTER:
- case CLOCK_EVT_NOTIFY_BROADCAST_EXIT:
- ret = tick_broadcast_oneshot_control(reason);
- break;
-
- case CLOCK_EVT_NOTIFY_CPU_DYING:
- tick_handover_do_timer(arg);
- break;
-
- case CLOCK_EVT_NOTIFY_SUSPEND:
- tick_suspend();
- tick_suspend_broadcast();
- break;
-
- case CLOCK_EVT_NOTIFY_RESUME:
- tick_resume();
- break;
-
- case CLOCK_EVT_NOTIFY_CPU_DEAD:
- tick_shutdown_broadcast_oneshot(arg);
- tick_shutdown_broadcast(arg);
- tick_shutdown(arg);
- /*
- * Unregister the clock event devices which were
- * released from the users in the notify chain.
- */
- list_for_each_entry_safe(dev, tmp, &clockevents_released, list)
+ tick_shutdown_broadcast_oneshot(cpu);
+ tick_shutdown_broadcast(cpu);
+ tick_shutdown(cpu);
+ /*
+ * Unregister the clock event devices which were
+ * released from the users in the notify chain.
+ */
+ list_for_each_entry_safe(dev, tmp, &clockevents_released, list)
+ list_del(&dev->list);
+ /*
+ * Now check whether the CPU has left unused per cpu devices
+ */
+ list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
+ if (cpumask_test_cpu(cpu, dev->cpumask) &&
+ cpumask_weight(dev->cpumask) == 1 &&
+ !tick_is_broadcast_device(dev)) {
+ BUG_ON(dev->state != CLOCK_EVT_STATE_DETACHED);
list_del(&dev->list);
- /*
- * Now check whether the CPU has left unused per cpu devices
- */
- cpu = *((int *)arg);
- list_for_each_entry_safe(dev, tmp, &clockevent_devices, list) {
- if (cpumask_test_cpu(cpu, dev->cpumask) &&
- cpumask_weight(dev->cpumask) == 1 &&
- !tick_is_broadcast_device(dev)) {
- BUG_ON(dev->mode != CLOCK_EVT_MODE_UNUSED);
- list_del(&dev->list);
- }
}
- break;
- default:
- break;
}
raw_spin_unlock_irqrestore(&clockevents_lock, flags);
- return ret;
}
-EXPORT_SYMBOL_GPL(clockevents_notify);
+#endif
#ifdef CONFIG_SYSFS
struct bus_type clockevents_subsys = {
}
device_initcall(clockevents_init_sysfs);
#endif /* SYSFS */
-
-#endif /* GENERIC_CLOCK_EVENTS */
schedule_work(&watchdog_work);
}
-static void clocksource_unstable(struct clocksource *cs, int64_t delta)
-{
- printk(KERN_WARNING "Clocksource %s unstable (delta = %Ld ns)\n",
- cs->name, delta);
- __clocksource_unstable(cs);
-}
-
/**
* clocksource_mark_unstable - mark clocksource unstable via watchdog
* @cs: clocksource to be marked unstable
static void clocksource_watchdog(unsigned long data)
{
struct clocksource *cs;
- cycle_t csnow, wdnow, delta;
+ cycle_t csnow, wdnow, cslast, wdlast, delta;
int64_t wd_nsec, cs_nsec;
int next_cpu, reset_pending;
delta = clocksource_delta(csnow, cs->cs_last, cs->mask);
cs_nsec = clocksource_cyc2ns(delta, cs->mult, cs->shift);
+ wdlast = cs->wd_last; /* save these in case we print them */
+ cslast = cs->cs_last;
cs->cs_last = csnow;
cs->wd_last = wdnow;
/* Check the deviation from the watchdog clocksource. */
if ((abs(cs_nsec - wd_nsec) > WATCHDOG_THRESHOLD)) {
- clocksource_unstable(cs, cs_nsec - wd_nsec);
+ pr_warn("timekeeping watchdog: Marking clocksource '%s' as unstable, because the skew is too large:\n", cs->name);
+ pr_warn(" '%s' wd_now: %llx wd_last: %llx mask: %llx\n",
+ watchdog->name, wdnow, wdlast, watchdog->mask);
+ pr_warn(" '%s' cs_now: %llx cs_last: %llx mask: %llx\n",
+ cs->name, csnow, cslast, cs->mask);
+ __clocksource_unstable(cs);
continue;
}
* @shift: cycle to nanosecond divisor (power of two)
* @maxadj: maximum adjustment value to mult (~11%)
* @mask: bitmask for two's complement subtraction of non 64 bit counters
+ * @max_cyc: maximum cycle value before potential overflow (does not include
+ * any safety margin)
+ *
+ * NOTE: This function includes a safety margin of 50%, in other words, we
+ * return half the number of nanoseconds the hardware counter can technically
+ * cover. This is done so that we can potentially detect problems caused by
+ * delayed timers or bad hardware, which might result in time intervals that
+ * are larger then what the math used can handle without overflows.
*/
-u64 clocks_calc_max_nsecs(u32 mult, u32 shift, u32 maxadj, u64 mask)
+u64 clocks_calc_max_nsecs(u32 mult, u32 shift, u32 maxadj, u64 mask, u64 *max_cyc)
{
u64 max_nsecs, max_cycles;
/*
* Calculate the maximum number of cycles that we can pass to the
- * cyc2ns function without overflowing a 64-bit signed result. The
- * maximum number of cycles is equal to ULLONG_MAX/(mult+maxadj)
- * which is equivalent to the below.
- * max_cycles < (2^63)/(mult + maxadj)
- * max_cycles < 2^(log2((2^63)/(mult + maxadj)))
- * max_cycles < 2^(log2(2^63) - log2(mult + maxadj))
- * max_cycles < 2^(63 - log2(mult + maxadj))
- * max_cycles < 1 << (63 - log2(mult + maxadj))
- * Please note that we add 1 to the result of the log2 to account for
- * any rounding errors, ensure the above inequality is satisfied and
- * no overflow will occur.
+ * cyc2ns() function without overflowing a 64-bit result.
*/
- max_cycles = 1ULL << (63 - (ilog2(mult + maxadj) + 1));
+ max_cycles = ULLONG_MAX;
+ do_div(max_cycles, mult+maxadj);
/*
* The actual maximum number of cycles we can defer the clocksource is
max_cycles = min(max_cycles, mask);
max_nsecs = clocksource_cyc2ns(max_cycles, mult - maxadj, shift);
+ /* return the max_cycles value as well if requested */
+ if (max_cyc)
+ *max_cyc = max_cycles;
+
+ /* Return 50% of the actual maximum, so we can detect bad values */
+ max_nsecs >>= 1;
+
return max_nsecs;
}
/**
- * clocksource_max_deferment - Returns max time the clocksource can be deferred
- * @cs: Pointer to clocksource
+ * clocksource_update_max_deferment - Updates the clocksource max_idle_ns & max_cycles
+ * @cs: Pointer to clocksource to be updated
*
*/
-static u64 clocksource_max_deferment(struct clocksource *cs)
+static inline void clocksource_update_max_deferment(struct clocksource *cs)
{
- u64 max_nsecs;
-
- max_nsecs = clocks_calc_max_nsecs(cs->mult, cs->shift, cs->maxadj,
- cs->mask);
- /*
- * To ensure that the clocksource does not wrap whilst we are idle,
- * limit the time the clocksource can be deferred by 12.5%. Please
- * note a margin of 12.5% is used because this can be computed with
- * a shift, versus say 10% which would require division.
- */
- return max_nsecs - (max_nsecs >> 3);
+ cs->max_idle_ns = clocks_calc_max_nsecs(cs->mult, cs->shift,
+ cs->maxadj, cs->mask,
+ &cs->max_cycles);
}
#ifndef CONFIG_ARCH_USES_GETTIMEOFFSET
}
/**
- * __clocksource_updatefreq_scale - Used update clocksource with new freq
+ * __clocksource_update_freq_scale - Used update clocksource with new freq
* @cs: clocksource to be registered
* @scale: Scale factor multiplied against freq to get clocksource hz
* @freq: clocksource frequency (cycles per second) divided by scale
* This should only be called from the clocksource->enable() method.
*
* This *SHOULD NOT* be called directly! Please use the
- * clocksource_updatefreq_hz() or clocksource_updatefreq_khz helper functions.
+ * __clocksource_update_freq_hz() or __clocksource_update_freq_khz() helper
+ * functions.
*/
-void __clocksource_updatefreq_scale(struct clocksource *cs, u32 scale, u32 freq)
+void __clocksource_update_freq_scale(struct clocksource *cs, u32 scale, u32 freq)
{
u64 sec;
+
/*
- * Calc the maximum number of seconds which we can run before
- * wrapping around. For clocksources which have a mask > 32bit
- * we need to limit the max sleep time to have a good
- * conversion precision. 10 minutes is still a reasonable
- * amount. That results in a shift value of 24 for a
- * clocksource with mask >= 40bit and f >= 4GHz. That maps to
- * ~ 0.06ppm granularity for NTP. We apply the same 12.5%
- * margin as we do in clocksource_max_deferment()
+ * Default clocksources are *special* and self-define their mult/shift.
+ * But, you're not special, so you should specify a freq value.
*/
- sec = (cs->mask - (cs->mask >> 3));
- do_div(sec, freq);
- do_div(sec, scale);
- if (!sec)
- sec = 1;
- else if (sec > 600 && cs->mask > UINT_MAX)
- sec = 600;
-
- clocks_calc_mult_shift(&cs->mult, &cs->shift, freq,
- NSEC_PER_SEC / scale, sec * scale);
-
+ if (freq) {
+ /*
+ * Calc the maximum number of seconds which we can run before
+ * wrapping around. For clocksources which have a mask > 32-bit
+ * we need to limit the max sleep time to have a good
+ * conversion precision. 10 minutes is still a reasonable
+ * amount. That results in a shift value of 24 for a
+ * clocksource with mask >= 40-bit and f >= 4GHz. That maps to
+ * ~ 0.06ppm granularity for NTP.
+ */
+ sec = cs->mask;
+ do_div(sec, freq);
+ do_div(sec, scale);
+ if (!sec)
+ sec = 1;
+ else if (sec > 600 && cs->mask > UINT_MAX)
+ sec = 600;
+
+ clocks_calc_mult_shift(&cs->mult, &cs->shift, freq,
+ NSEC_PER_SEC / scale, sec * scale);
+ }
/*
- * for clocksources that have large mults, to avoid overflow.
- * Since mult may be adjusted by ntp, add an safety extra margin
- *
+ * Ensure clocksources that have large 'mult' values don't overflow
+ * when adjusted.
*/
cs->maxadj = clocksource_max_adjustment(cs);
- while ((cs->mult + cs->maxadj < cs->mult)
- || (cs->mult - cs->maxadj > cs->mult)) {
+ while (freq && ((cs->mult + cs->maxadj < cs->mult)
+ || (cs->mult - cs->maxadj > cs->mult))) {
cs->mult >>= 1;
cs->shift--;
cs->maxadj = clocksource_max_adjustment(cs);
}
- cs->max_idle_ns = clocksource_max_deferment(cs);
+ /*
+ * Only warn for *special* clocksources that self-define
+ * their mult/shift values and don't specify a freq.
+ */
+ WARN_ONCE(cs->mult + cs->maxadj < cs->mult,
+ "timekeeping: Clocksource %s might overflow on 11%% adjustment\n",
+ cs->name);
+
+ clocksource_update_max_deferment(cs);
+
+ pr_info("clocksource %s: mask: 0x%llx max_cycles: 0x%llx, max_idle_ns: %lld ns\n",
+ cs->name, cs->mask, cs->max_cycles, cs->max_idle_ns);
}
-EXPORT_SYMBOL_GPL(__clocksource_updatefreq_scale);
+EXPORT_SYMBOL_GPL(__clocksource_update_freq_scale);
/**
* __clocksource_register_scale - Used to install new clocksources
{
/* Initialize mult/shift and max_idle_ns */
- __clocksource_updatefreq_scale(cs, scale, freq);
+ __clocksource_update_freq_scale(cs, scale, freq);
/* Add clocksource to the clocksource list */
mutex_lock(&clocksource_mutex);
}
EXPORT_SYMBOL_GPL(__clocksource_register_scale);
-
-/**
- * clocksource_register - Used to install new clocksources
- * @cs: clocksource to be registered
- *
- * Returns -EBUSY if registration fails, zero otherwise.
- */
-int clocksource_register(struct clocksource *cs)
-{
- /* calculate max adjustment for given mult/shift */
- cs->maxadj = clocksource_max_adjustment(cs);
- WARN_ONCE(cs->mult + cs->maxadj < cs->mult,
- "Clocksource %s might overflow on 11%% adjustment\n",
- cs->name);
-
- /* calculate max idle time permitted for this clocksource */
- cs->max_idle_ns = clocksource_max_deferment(cs);
-
- mutex_lock(&clocksource_mutex);
- clocksource_enqueue(cs);
- clocksource_enqueue_watchdog(cs);
- clocksource_select();
- mutex_unlock(&clocksource_mutex);
- return 0;
-}
-EXPORT_SYMBOL(clocksource_register);
-
static void __clocksource_change_rating(struct clocksource *cs, int rating)
{
list_del(&cs->list);
#include <trace/events/timer.h>
-#include "timekeeping.h"
+#include "tick-internal.h"
/*
* The timer bases:
break;
#ifdef CONFIG_HOTPLUG_CPU
- case CPU_DYING:
- case CPU_DYING_FROZEN:
- clockevents_notify(CLOCK_EVT_NOTIFY_CPU_DYING, &scpu);
- break;
case CPU_DEAD:
case CPU_DEAD_FROZEN:
- {
- clockevents_notify(CLOCK_EVT_NOTIFY_CPU_DEAD, &scpu);
migrate_hrtimers(scpu);
break;
- }
#endif
default:
#include <linux/module.h>
#include <linux/init.h>
-#include "tick-internal.h"
+#include "timekeeping.h"
/* The Jiffies based clocksource is the lowest common
* denominator clock source which should function on
.mask = 0xffffffff, /*32bits*/
.mult = NSEC_PER_JIFFY << JIFFIES_SHIFT, /* details above */
.shift = JIFFIES_SHIFT,
+ .max_cycles = 10,
};
__cacheline_aligned_in_smp DEFINE_SEQLOCK(jiffies_lock);
static int __init init_jiffies_clocksource(void)
{
- return clocksource_register(&clocksource_jiffies);
+ return __clocksource_register(&clocksource_jiffies);
}
core_initcall(init_jiffies_clocksource);
refined_jiffies.mult = ((u32)nsec_per_tick) << JIFFIES_SHIFT;
- clocksource_register(&refined_jiffies);
+ __clocksource_register(&refined_jiffies);
return 0;
}
#include <linux/module.h>
#include <linux/rtc.h>
-#include "tick-internal.h"
#include "ntp_internal.h"
/*
return leap;
}
+#ifdef CONFIG_GENERIC_CMOS_UPDATE
+int __weak update_persistent_clock64(struct timespec64 now64)
+{
+ struct timespec now;
+
+ now = timespec64_to_timespec(now64);
+ return update_persistent_clock(now);
+}
+#endif
+
#if defined(CONFIG_GENERIC_CMOS_UPDATE) || defined(CONFIG_RTC_SYSTOHC)
static void sync_cmos_clock(struct work_struct *work);
if (persistent_clock_is_local)
adjust.tv_sec -= (sys_tz.tz_minuteswest * 60);
#ifdef CONFIG_GENERIC_CMOS_UPDATE
- fail = update_persistent_clock(timespec64_to_timespec(adjust));
+ fail = update_persistent_clock64(adjust);
#endif
+
#ifdef CONFIG_RTC_SYSTOHC
if (fail == -ENODEV)
fail = rtc_set_ntp_time(adjust);
/*
- * sched_clock.c: support for extending counters to full 64-bit ns counter
+ * sched_clock.c: Generic sched_clock() support, to extend low level
+ * hardware time counters to full 64-bit ns values.
*
* This program is free software; you can redistribute it and/or modify
* it under the terms of the GNU General Public License version 2 as
#include <linux/seqlock.h>
#include <linux/bitops.h>
-struct clock_data {
- ktime_t wrap_kt;
+/**
+ * struct clock_read_data - data required to read from sched_clock()
+ *
+ * @epoch_ns: sched_clock() value at last update
+ * @epoch_cyc: Clock cycle value at last update.
+ * @sched_clock_mask: Bitmask for two's complement subtraction of non 64bit
+ * clocks.
+ * @read_sched_clock: Current clock source (or dummy source when suspended).
+ * @mult: Multipler for scaled math conversion.
+ * @shift: Shift value for scaled math conversion.
+ *
+ * Care must be taken when updating this structure; it is read by
+ * some very hot code paths. It occupies <=40 bytes and, when combined
+ * with the seqcount used to synchronize access, comfortably fits into
+ * a 64 byte cache line.
+ */
+struct clock_read_data {
u64 epoch_ns;
u64 epoch_cyc;
- seqcount_t seq;
- unsigned long rate;
+ u64 sched_clock_mask;
+ u64 (*read_sched_clock)(void);
u32 mult;
u32 shift;
- bool suspended;
+};
+
+/**
+ * struct clock_data - all data needed for sched_clock() (including
+ * registration of a new clock source)
+ *
+ * @seq: Sequence counter for protecting updates. The lowest
+ * bit is the index for @read_data.
+ * @read_data: Data required to read from sched_clock.
+ * @wrap_kt: Duration for which clock can run before wrapping.
+ * @rate: Tick rate of the registered clock.
+ * @actual_read_sched_clock: Registered hardware level clock read function.
+ *
+ * The ordering of this structure has been chosen to optimize cache
+ * performance. In particular 'seq' and 'read_data[0]' (combined) should fit
+ * into a single 64-byte cache line.
+ */
+struct clock_data {
+ seqcount_t seq;
+ struct clock_read_data read_data[2];
+ ktime_t wrap_kt;
+ unsigned long rate;
+
+ u64 (*actual_read_sched_clock)(void);
};
static struct hrtimer sched_clock_timer;
core_param(irqtime, irqtime, int, 0400);
-static struct clock_data cd = {
- .mult = NSEC_PER_SEC / HZ,
-};
-
-static u64 __read_mostly sched_clock_mask;
-
static u64 notrace jiffy_sched_clock_read(void)
{
/*
return (u64)(jiffies - INITIAL_JIFFIES);
}
-static u64 __read_mostly (*read_sched_clock)(void) = jiffy_sched_clock_read;
+static struct clock_data cd ____cacheline_aligned = {
+ .read_data[0] = { .mult = NSEC_PER_SEC / HZ,
+ .read_sched_clock = jiffy_sched_clock_read, },
+ .actual_read_sched_clock = jiffy_sched_clock_read,
+};
static inline u64 notrace cyc_to_ns(u64 cyc, u32 mult, u32 shift)
{
unsigned long long notrace sched_clock(void)
{
- u64 epoch_ns;
- u64 epoch_cyc;
- u64 cyc;
+ u64 cyc, res;
unsigned long seq;
-
- if (cd.suspended)
- return cd.epoch_ns;
+ struct clock_read_data *rd;
do {
- seq = raw_read_seqcount_begin(&cd.seq);
- epoch_cyc = cd.epoch_cyc;
- epoch_ns = cd.epoch_ns;
+ seq = raw_read_seqcount(&cd.seq);
+ rd = cd.read_data + (seq & 1);
+
+ cyc = (rd->read_sched_clock() - rd->epoch_cyc) &
+ rd->sched_clock_mask;
+ res = rd->epoch_ns + cyc_to_ns(cyc, rd->mult, rd->shift);
} while (read_seqcount_retry(&cd.seq, seq));
- cyc = read_sched_clock();
- cyc = (cyc - epoch_cyc) & sched_clock_mask;
- return epoch_ns + cyc_to_ns(cyc, cd.mult, cd.shift);
+ return res;
+}
+
+/*
+ * Updating the data required to read the clock.
+ *
+ * sched_clock() will never observe mis-matched data even if called from
+ * an NMI. We do this by maintaining an odd/even copy of the data and
+ * steering sched_clock() to one or the other using a sequence counter.
+ * In order to preserve the data cache profile of sched_clock() as much
+ * as possible the system reverts back to the even copy when the update
+ * completes; the odd copy is used *only* during an update.
+ */
+static void update_clock_read_data(struct clock_read_data *rd)
+{
+ /* update the backup (odd) copy with the new data */
+ cd.read_data[1] = *rd;
+
+ /* steer readers towards the odd copy */
+ raw_write_seqcount_latch(&cd.seq);
+
+ /* now its safe for us to update the normal (even) copy */
+ cd.read_data[0] = *rd;
+
+ /* switch readers back to the even copy */
+ raw_write_seqcount_latch(&cd.seq);
}
/*
- * Atomically update the sched_clock epoch.
+ * Atomically update the sched_clock() epoch.
*/
-static void notrace update_sched_clock(void)
+static void update_sched_clock(void)
{
- unsigned long flags;
u64 cyc;
u64 ns;
+ struct clock_read_data rd;
+
+ rd = cd.read_data[0];
+
+ cyc = cd.actual_read_sched_clock();
+ ns = rd.epoch_ns + cyc_to_ns((cyc - rd.epoch_cyc) & rd.sched_clock_mask, rd.mult, rd.shift);
+
+ rd.epoch_ns = ns;
+ rd.epoch_cyc = cyc;
- cyc = read_sched_clock();
- ns = cd.epoch_ns +
- cyc_to_ns((cyc - cd.epoch_cyc) & sched_clock_mask,
- cd.mult, cd.shift);
-
- raw_local_irq_save(flags);
- raw_write_seqcount_begin(&cd.seq);
- cd.epoch_ns = ns;
- cd.epoch_cyc = cyc;
- raw_write_seqcount_end(&cd.seq);
- raw_local_irq_restore(flags);
+ update_clock_read_data(&rd);
}
static enum hrtimer_restart sched_clock_poll(struct hrtimer *hrt)
{
update_sched_clock();
hrtimer_forward_now(hrt, cd.wrap_kt);
+
return HRTIMER_RESTART;
}
-void __init sched_clock_register(u64 (*read)(void), int bits,
- unsigned long rate)
+void __init
+sched_clock_register(u64 (*read)(void), int bits, unsigned long rate)
{
u64 res, wrap, new_mask, new_epoch, cyc, ns;
u32 new_mult, new_shift;
- ktime_t new_wrap_kt;
unsigned long r;
char r_unit;
+ struct clock_read_data rd;
if (cd.rate > rate)
return;
WARN_ON(!irqs_disabled());
- /* calculate the mult/shift to convert counter ticks to ns. */
+ /* Calculate the mult/shift to convert counter ticks to ns. */
clocks_calc_mult_shift(&new_mult, &new_shift, rate, NSEC_PER_SEC, 3600);
new_mask = CLOCKSOURCE_MASK(bits);
+ cd.rate = rate;
+
+ /* Calculate how many nanosecs until we risk wrapping */
+ wrap = clocks_calc_max_nsecs(new_mult, new_shift, 0, new_mask, NULL);
+ cd.wrap_kt = ns_to_ktime(wrap);
- /* calculate how many ns until we wrap */
- wrap = clocks_calc_max_nsecs(new_mult, new_shift, 0, new_mask);
- new_wrap_kt = ns_to_ktime(wrap - (wrap >> 3));
+ rd = cd.read_data[0];
- /* update epoch for new counter and update epoch_ns from old counter*/
+ /* Update epoch for new counter and update 'epoch_ns' from old counter*/
new_epoch = read();
- cyc = read_sched_clock();
- ns = cd.epoch_ns + cyc_to_ns((cyc - cd.epoch_cyc) & sched_clock_mask,
- cd.mult, cd.shift);
+ cyc = cd.actual_read_sched_clock();
+ ns = rd.epoch_ns + cyc_to_ns((cyc - rd.epoch_cyc) & rd.sched_clock_mask, rd.mult, rd.shift);
+ cd.actual_read_sched_clock = read;
- raw_write_seqcount_begin(&cd.seq);
- read_sched_clock = read;
- sched_clock_mask = new_mask;
- cd.rate = rate;
- cd.wrap_kt = new_wrap_kt;
- cd.mult = new_mult;
- cd.shift = new_shift;
- cd.epoch_cyc = new_epoch;
- cd.epoch_ns = ns;
- raw_write_seqcount_end(&cd.seq);
+ rd.read_sched_clock = read;
+ rd.sched_clock_mask = new_mask;
+ rd.mult = new_mult;
+ rd.shift = new_shift;
+ rd.epoch_cyc = new_epoch;
+ rd.epoch_ns = ns;
+
+ update_clock_read_data(&rd);
r = rate;
if (r >= 4000000) {
r /= 1000000;
r_unit = 'M';
- } else if (r >= 1000) {
- r /= 1000;
- r_unit = 'k';
- } else
- r_unit = ' ';
-
- /* calculate the ns resolution of this counter */
+ } else {
+ if (r >= 1000) {
+ r /= 1000;
+ r_unit = 'k';
+ } else {
+ r_unit = ' ';
+ }
+ }
+
+ /* Calculate the ns resolution of this counter */
res = cyc_to_ns(1ULL, new_mult, new_shift);
pr_info("sched_clock: %u bits at %lu%cHz, resolution %lluns, wraps every %lluns\n",
bits, r, r_unit, res, wrap);
- /* Enable IRQ time accounting if we have a fast enough sched_clock */
+ /* Enable IRQ time accounting if we have a fast enough sched_clock() */
if (irqtime > 0 || (irqtime == -1 && rate >= 1000000))
enable_sched_clock_irqtime();
void __init sched_clock_postinit(void)
{
/*
- * If no sched_clock function has been provided at that point,
+ * If no sched_clock() function has been provided at that point,
* make it the final one one.
*/
- if (read_sched_clock == jiffy_sched_clock_read)
+ if (cd.actual_read_sched_clock == jiffy_sched_clock_read)
sched_clock_register(jiffy_sched_clock_read, BITS_PER_LONG, HZ);
update_sched_clock();
hrtimer_start(&sched_clock_timer, cd.wrap_kt, HRTIMER_MODE_REL);
}
+/*
+ * Clock read function for use when the clock is suspended.
+ *
+ * This function makes it appear to sched_clock() as if the clock
+ * stopped counting at its last update.
+ *
+ * This function must only be called from the critical
+ * section in sched_clock(). It relies on the read_seqcount_retry()
+ * at the end of the critical section to be sure we observe the
+ * correct copy of 'epoch_cyc'.
+ */
+static u64 notrace suspended_sched_clock_read(void)
+{
+ unsigned long seq = raw_read_seqcount(&cd.seq);
+
+ return cd.read_data[seq & 1].epoch_cyc;
+}
+
static int sched_clock_suspend(void)
{
+ struct clock_read_data *rd = &cd.read_data[0];
+
update_sched_clock();
hrtimer_cancel(&sched_clock_timer);
- cd.suspended = true;
+ rd->read_sched_clock = suspended_sched_clock_read;
+
return 0;
}
static void sched_clock_resume(void)
{
- cd.epoch_cyc = read_sched_clock();
+ struct clock_read_data *rd = &cd.read_data[0];
+
+ rd->epoch_cyc = cd.actual_read_sched_clock();
hrtimer_start(&sched_clock_timer, cd.wrap_kt, HRTIMER_MODE_REL);
- cd.suspended = false;
+ rd->read_sched_clock = cd.actual_read_sched_clock;
}
static struct syscore_ops sched_clock_ops = {
- .suspend = sched_clock_suspend,
- .resume = sched_clock_resume,
+ .suspend = sched_clock_suspend,
+ .resume = sched_clock_resume,
};
static int __init sched_clock_syscore_init(void)
{
register_syscore_ops(&sched_clock_ops);
+
return 0;
}
device_initcall(sched_clock_syscore_init);
*/
static int bc_set_next(ktime_t expires, struct clock_event_device *bc)
{
+ int bc_moved;
/*
* We try to cancel the timer first. If the callback is on
* flight on some other cpu then we let it handle it. If we
* restart the timer because we are in the callback, but we
* can set the expiry time and let the callback return
* HRTIMER_RESTART.
+ *
+ * Since we are in the idle loop at this point and because
+ * hrtimer_{start/cancel} functions call into tracing,
+ * calls to these functions must be bound within RCU_NONIDLE.
*/
- if (hrtimer_try_to_cancel(&bctimer) >= 0) {
- hrtimer_start(&bctimer, expires, HRTIMER_MODE_ABS_PINNED);
+ RCU_NONIDLE(bc_moved = (hrtimer_try_to_cancel(&bctimer) >= 0) ?
+ !hrtimer_start(&bctimer, expires, HRTIMER_MODE_ABS_PINNED) :
+ 0);
+ if (bc_moved) {
/* Bind the "device" to the cpu */
bc->bound_on = smp_processor_id();
} else if (bc->bound_on == smp_processor_id()) {
static cpumask_var_t tick_broadcast_on;
static cpumask_var_t tmpmask;
static DEFINE_RAW_SPINLOCK(tick_broadcast_lock);
-static int tick_broadcast_force;
+static int tick_broadcast_forced;
#ifdef CONFIG_TICK_ONESHOT
static void tick_broadcast_clear_oneshot(int cpu);
+static void tick_resume_broadcast_oneshot(struct clock_event_device *bc);
#else
static inline void tick_broadcast_clear_oneshot(int cpu) { }
+static inline void tick_resume_broadcast_oneshot(struct clock_event_device *bc) { }
#endif
/*
/*
* The device is in periodic mode. No reprogramming necessary:
*/
- if (dev->mode == CLOCK_EVT_MODE_PERIODIC)
+ if (dev->state == CLOCK_EVT_STATE_PERIODIC)
goto unlock;
/*
raw_spin_unlock(&tick_broadcast_lock);
}
-/*
- * Powerstate information: The system enters/leaves a state, where
- * affected devices might stop
+/**
+ * tick_broadcast_control - Enable/disable or force broadcast mode
+ * @mode: The selected broadcast mode
+ *
+ * Called when the system enters a state where affected tick devices
+ * might stop. Note: TICK_BROADCAST_FORCE cannot be undone.
+ *
+ * Called with interrupts disabled, so clockevents_lock is not
+ * required here because the local clock event device cannot go away
+ * under us.
*/
-static void tick_do_broadcast_on_off(unsigned long *reason)
+void tick_broadcast_control(enum tick_broadcast_mode mode)
{
struct clock_event_device *bc, *dev;
struct tick_device *td;
- unsigned long flags;
int cpu, bc_stopped;
- raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
-
- cpu = smp_processor_id();
- td = &per_cpu(tick_cpu_device, cpu);
+ td = this_cpu_ptr(&tick_cpu_device);
dev = td->evtdev;
- bc = tick_broadcast_device.evtdev;
/*
* Is the device not affected by the powerstate ?
*/
if (!dev || !(dev->features & CLOCK_EVT_FEAT_C3STOP))
- goto out;
+ return;
if (!tick_device_is_functional(dev))
- goto out;
+ return;
+ raw_spin_lock(&tick_broadcast_lock);
+ cpu = smp_processor_id();
+ bc = tick_broadcast_device.evtdev;
bc_stopped = cpumask_empty(tick_broadcast_mask);
- switch (*reason) {
- case CLOCK_EVT_NOTIFY_BROADCAST_ON:
- case CLOCK_EVT_NOTIFY_BROADCAST_FORCE:
+ switch (mode) {
+ case TICK_BROADCAST_FORCE:
+ tick_broadcast_forced = 1;
+ case TICK_BROADCAST_ON:
cpumask_set_cpu(cpu, tick_broadcast_on);
if (!cpumask_test_and_set_cpu(cpu, tick_broadcast_mask)) {
if (tick_broadcast_device.mode ==
TICKDEV_MODE_PERIODIC)
clockevents_shutdown(dev);
}
- if (*reason == CLOCK_EVT_NOTIFY_BROADCAST_FORCE)
- tick_broadcast_force = 1;
break;
- case CLOCK_EVT_NOTIFY_BROADCAST_OFF:
- if (tick_broadcast_force)
+
+ case TICK_BROADCAST_OFF:
+ if (tick_broadcast_forced)
break;
cpumask_clear_cpu(cpu, tick_broadcast_on);
if (!tick_device_is_functional(dev))
else
tick_broadcast_setup_oneshot(bc);
}
-out:
- raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
-}
-
-/*
- * Powerstate information: The system enters/leaves a state, where
- * affected devices might stop.
- */
-void tick_broadcast_on_off(unsigned long reason, int *oncpu)
-{
- if (!cpumask_test_cpu(*oncpu, cpu_online_mask))
- printk(KERN_ERR "tick-broadcast: ignoring broadcast for "
- "offline CPU #%d\n", *oncpu);
- else
- tick_do_broadcast_on_off(&reason);
+ raw_spin_unlock(&tick_broadcast_lock);
}
+EXPORT_SYMBOL_GPL(tick_broadcast_control);
/*
* Set the periodic handler depending on broadcast on/off
dev->event_handler = tick_handle_periodic_broadcast;
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
* Remove a CPU from broadcasting
*/
-void tick_shutdown_broadcast(unsigned int *cpup)
+void tick_shutdown_broadcast(unsigned int cpu)
{
struct clock_event_device *bc;
unsigned long flags;
- unsigned int cpu = *cpup;
raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
+#endif
void tick_suspend_broadcast(void)
{
raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
-int tick_resume_broadcast(void)
+/*
+ * This is called from tick_resume_local() on a resuming CPU. That's
+ * called from the core resume function, tick_unfreeze() and the magic XEN
+ * resume hackery.
+ *
+ * In none of these cases the broadcast device mode can change and the
+ * bit of the resuming CPU in the broadcast mask is safe as well.
+ */
+bool tick_resume_check_broadcast(void)
+{
+ if (tick_broadcast_device.mode == TICKDEV_MODE_ONESHOT)
+ return false;
+ else
+ return cpumask_test_cpu(smp_processor_id(), tick_broadcast_mask);
+}
+
+void tick_resume_broadcast(void)
{
struct clock_event_device *bc;
unsigned long flags;
- int broadcast = 0;
raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
bc = tick_broadcast_device.evtdev;
if (bc) {
- clockevents_set_mode(bc, CLOCK_EVT_MODE_RESUME);
+ clockevents_tick_resume(bc);
switch (tick_broadcast_device.mode) {
case TICKDEV_MODE_PERIODIC:
if (!cpumask_empty(tick_broadcast_mask))
tick_broadcast_start_periodic(bc);
- broadcast = cpumask_test_cpu(smp_processor_id(),
- tick_broadcast_mask);
break;
case TICKDEV_MODE_ONESHOT:
if (!cpumask_empty(tick_broadcast_mask))
- broadcast = tick_resume_broadcast_oneshot(bc);
+ tick_resume_broadcast_oneshot(bc);
break;
}
}
raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
-
- return broadcast;
}
-
#ifdef CONFIG_TICK_ONESHOT
static cpumask_var_t tick_broadcast_oneshot_mask;
{
int ret;
- if (bc->mode != CLOCK_EVT_MODE_ONESHOT)
- clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
+ if (bc->state != CLOCK_EVT_STATE_ONESHOT)
+ clockevents_set_state(bc, CLOCK_EVT_STATE_ONESHOT);
ret = clockevents_program_event(bc, expires, force);
if (!ret)
return ret;
}
-int tick_resume_broadcast_oneshot(struct clock_event_device *bc)
+static void tick_resume_broadcast_oneshot(struct clock_event_device *bc)
{
- clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
- return 0;
+ clockevents_set_state(bc, CLOCK_EVT_STATE_ONESHOT);
}
/*
* switched over, leave the device alone.
*/
if (td->mode == TICKDEV_MODE_ONESHOT) {
- clockevents_set_mode(td->evtdev,
- CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(td->evtdev,
+ CLOCK_EVT_STATE_ONESHOT);
}
}
}
if (dev->next_event.tv64 < bc->next_event.tv64)
return;
}
- clockevents_set_mode(dev, CLOCK_EVT_MODE_SHUTDOWN);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_SHUTDOWN);
}
-static void broadcast_move_bc(int deadcpu)
-{
- struct clock_event_device *bc = tick_broadcast_device.evtdev;
-
- if (!bc || !broadcast_needs_cpu(bc, deadcpu))
- return;
- /* This moves the broadcast assignment to this cpu */
- clockevents_program_event(bc, bc->next_event, 1);
-}
-
-/*
- * Powerstate information: The system enters/leaves a state, where
- * affected devices might stop
+/**
+ * tick_broadcast_oneshot_control - Enter/exit broadcast oneshot mode
+ * @state: The target state (enter/exit)
+ *
+ * The system enters/leaves a state, where affected devices might stop
* Returns 0 on success, -EBUSY if the cpu is used to broadcast wakeups.
+ *
+ * Called with interrupts disabled, so clockevents_lock is not
+ * required here because the local clock event device cannot go away
+ * under us.
*/
-int tick_broadcast_oneshot_control(unsigned long reason)
+int tick_broadcast_oneshot_control(enum tick_broadcast_state state)
{
struct clock_event_device *bc, *dev;
struct tick_device *td;
- unsigned long flags;
- ktime_t now;
int cpu, ret = 0;
+ ktime_t now;
/*
* Periodic mode does not care about the enter/exit of power
* We are called with preemtion disabled from the depth of the
* idle code, so we can't be moved away.
*/
- cpu = smp_processor_id();
- td = &per_cpu(tick_cpu_device, cpu);
+ td = this_cpu_ptr(&tick_cpu_device);
dev = td->evtdev;
if (!(dev->features & CLOCK_EVT_FEAT_C3STOP))
return 0;
+ raw_spin_lock(&tick_broadcast_lock);
bc = tick_broadcast_device.evtdev;
+ cpu = smp_processor_id();
- raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
- if (reason == CLOCK_EVT_NOTIFY_BROADCAST_ENTER) {
+ if (state == TICK_BROADCAST_ENTER) {
if (!cpumask_test_and_set_cpu(cpu, tick_broadcast_oneshot_mask)) {
WARN_ON_ONCE(cpumask_test_cpu(cpu, tick_broadcast_pending_mask));
broadcast_shutdown_local(bc, dev);
cpumask_clear_cpu(cpu, tick_broadcast_oneshot_mask);
} else {
if (cpumask_test_and_clear_cpu(cpu, tick_broadcast_oneshot_mask)) {
- clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_ONESHOT);
/*
* The cpu which was handling the broadcast
* timer marked this cpu in the broadcast
}
}
out:
- raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
+ raw_spin_unlock(&tick_broadcast_lock);
return ret;
}
+EXPORT_SYMBOL_GPL(tick_broadcast_oneshot_control);
/*
* Reset the one shot broadcast for a cpu
/* Set it up only once ! */
if (bc->event_handler != tick_handle_oneshot_broadcast) {
- int was_periodic = bc->mode == CLOCK_EVT_MODE_PERIODIC;
+ int was_periodic = bc->state == CLOCK_EVT_STATE_PERIODIC;
bc->event_handler = tick_handle_oneshot_broadcast;
tick_broadcast_oneshot_mask, tmpmask);
if (was_periodic && !cpumask_empty(tmpmask)) {
- clockevents_set_mode(bc, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(bc, CLOCK_EVT_STATE_ONESHOT);
tick_broadcast_init_next_event(tmpmask,
tick_next_period);
tick_broadcast_set_event(bc, cpu, tick_next_period, 1);
raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
+#ifdef CONFIG_HOTPLUG_CPU
+void hotplug_cpu__broadcast_tick_pull(int deadcpu)
+{
+ struct clock_event_device *bc;
+ unsigned long flags;
+
+ raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
+ bc = tick_broadcast_device.evtdev;
+
+ if (bc && broadcast_needs_cpu(bc, deadcpu)) {
+ /* This moves the broadcast assignment to this CPU: */
+ clockevents_program_event(bc, bc->next_event, 1);
+ }
+ raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
+}
/*
* Remove a dead CPU from broadcasting
*/
-void tick_shutdown_broadcast_oneshot(unsigned int *cpup)
+void tick_shutdown_broadcast_oneshot(unsigned int cpu)
{
unsigned long flags;
- unsigned int cpu = *cpup;
raw_spin_lock_irqsave(&tick_broadcast_lock, flags);
cpumask_clear_cpu(cpu, tick_broadcast_pending_mask);
cpumask_clear_cpu(cpu, tick_broadcast_force_mask);
- broadcast_move_bc(cpu);
-
raw_spin_unlock_irqrestore(&tick_broadcast_lock, flags);
}
+#endif
/*
* Check, whether the broadcast device is in one shot mode
tick_periodic(cpu);
- if (dev->mode != CLOCK_EVT_MODE_ONESHOT)
+ if (dev->state != CLOCK_EVT_STATE_ONESHOT)
return;
for (;;) {
/*
if ((dev->features & CLOCK_EVT_FEAT_PERIODIC) &&
!tick_broadcast_oneshot_active()) {
- clockevents_set_mode(dev, CLOCK_EVT_MODE_PERIODIC);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_PERIODIC);
} else {
unsigned long seq;
ktime_t next;
next = tick_next_period;
} while (read_seqretry(&jiffies_lock, seq));
- clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_ONESHOT);
for (;;) {
if (!clockevents_program_event(dev, next, false))
tick_install_broadcast_device(newdev);
}
+#ifdef CONFIG_HOTPLUG_CPU
/*
* Transfer the do_timer job away from a dying cpu.
*
- * Called with interrupts disabled.
+ * Called with interrupts disabled. Not locking required. If
+ * tick_do_timer_cpu is owned by this cpu, nothing can change it.
*/
-void tick_handover_do_timer(int *cpup)
+void tick_handover_do_timer(void)
{
- if (*cpup == tick_do_timer_cpu) {
+ if (tick_do_timer_cpu == smp_processor_id()) {
int cpu = cpumask_first(cpu_online_mask);
tick_do_timer_cpu = (cpu < nr_cpu_ids) ? cpu :
* access the hardware device itself.
* We just set the mode and remove it from the lists.
*/
-void tick_shutdown(unsigned int *cpup)
+void tick_shutdown(unsigned int cpu)
{
- struct tick_device *td = &per_cpu(tick_cpu_device, *cpup);
+ struct tick_device *td = &per_cpu(tick_cpu_device, cpu);
struct clock_event_device *dev = td->evtdev;
td->mode = TICKDEV_MODE_PERIODIC;
* Prevent that the clock events layer tries to call
* the set mode function!
*/
+ dev->state = CLOCK_EVT_STATE_DETACHED;
dev->mode = CLOCK_EVT_MODE_UNUSED;
clockevents_exchange_device(dev, NULL);
dev->event_handler = clockevents_handle_noop;
td->evtdev = NULL;
}
}
+#endif
-void tick_suspend(void)
+/**
+ * tick_suspend_local - Suspend the local tick device
+ *
+ * Called from the local cpu for freeze with interrupts disabled.
+ *
+ * No locks required. Nothing can change the per cpu device.
+ */
+void tick_suspend_local(void)
{
struct tick_device *td = this_cpu_ptr(&tick_cpu_device);
clockevents_shutdown(td->evtdev);
}
-void tick_resume(void)
+/**
+ * tick_resume_local - Resume the local tick device
+ *
+ * Called from the local CPU for unfreeze or XEN resume magic.
+ *
+ * No locks required. Nothing can change the per cpu device.
+ */
+void tick_resume_local(void)
{
struct tick_device *td = this_cpu_ptr(&tick_cpu_device);
- int broadcast = tick_resume_broadcast();
-
- clockevents_set_mode(td->evtdev, CLOCK_EVT_MODE_RESUME);
+ bool broadcast = tick_resume_check_broadcast();
+ clockevents_tick_resume(td->evtdev);
if (!broadcast) {
if (td->mode == TICKDEV_MODE_PERIODIC)
tick_setup_periodic(td->evtdev, 0);
}
}
+/**
+ * tick_suspend - Suspend the tick and the broadcast device
+ *
+ * Called from syscore_suspend() via timekeeping_suspend with only one
+ * CPU online and interrupts disabled or from tick_unfreeze() under
+ * tick_freeze_lock.
+ *
+ * No locks required. Nothing can change the per cpu device.
+ */
+void tick_suspend(void)
+{
+ tick_suspend_local();
+ tick_suspend_broadcast();
+}
+
+/**
+ * tick_resume - Resume the tick and the broadcast device
+ *
+ * Called from syscore_resume() via timekeeping_resume with only one
+ * CPU online and interrupts disabled.
+ *
+ * No locks required. Nothing can change the per cpu device.
+ */
+void tick_resume(void)
+{
+ tick_resume_broadcast();
+ tick_resume_local();
+}
+
static DEFINE_RAW_SPINLOCK(tick_freeze_lock);
static unsigned int tick_freeze_depth;
raw_spin_lock(&tick_freeze_lock);
tick_freeze_depth++;
- if (tick_freeze_depth == num_online_cpus()) {
+ if (tick_freeze_depth == num_online_cpus())
timekeeping_suspend();
- } else {
- tick_suspend();
- tick_suspend_broadcast();
- }
+ else
+ tick_suspend_local();
raw_spin_unlock(&tick_freeze_lock);
}
if (tick_freeze_depth == num_online_cpus())
timekeeping_resume();
else
- tick_resume();
+ tick_resume_local();
tick_freeze_depth--;
#include <linux/tick.h>
#include "timekeeping.h"
+#include "tick-sched.h"
-extern seqlock_t jiffies_lock;
+#ifdef CONFIG_GENERIC_CLOCKEVENTS
-#define CS_NAME_LEN 32
-
-#ifdef CONFIG_GENERIC_CLOCKEVENTS_BUILD
-
-#define TICK_DO_TIMER_NONE -1
-#define TICK_DO_TIMER_BOOT -2
+# define TICK_DO_TIMER_NONE -1
+# define TICK_DO_TIMER_BOOT -2
DECLARE_PER_CPU(struct tick_device, tick_cpu_device);
extern ktime_t tick_next_period;
extern void tick_setup_periodic(struct clock_event_device *dev, int broadcast);
extern void tick_handle_periodic(struct clock_event_device *dev);
extern void tick_check_new_device(struct clock_event_device *dev);
-extern void tick_handover_do_timer(int *cpup);
-extern void tick_shutdown(unsigned int *cpup);
+extern void tick_shutdown(unsigned int cpu);
extern void tick_suspend(void);
extern void tick_resume(void);
extern bool tick_check_replacement(struct clock_event_device *curdev,
struct clock_event_device *newdev);
extern void tick_install_replacement(struct clock_event_device *dev);
+extern int tick_is_oneshot_available(void);
+extern struct tick_device *tick_get_device(int cpu);
-extern void clockevents_shutdown(struct clock_event_device *dev);
+extern int clockevents_tick_resume(struct clock_event_device *dev);
+/* Check, if the device is functional or a dummy for broadcast */
+static inline int tick_device_is_functional(struct clock_event_device *dev)
+{
+ return !(dev->features & CLOCK_EVT_FEAT_DUMMY);
+}
+extern void clockevents_shutdown(struct clock_event_device *dev);
+extern void clockevents_exchange_device(struct clock_event_device *old,
+ struct clock_event_device *new);
+extern void clockevents_set_state(struct clock_event_device *dev,
+ enum clock_event_state state);
+extern int clockevents_program_event(struct clock_event_device *dev,
+ ktime_t expires, bool force);
+extern void clockevents_handle_noop(struct clock_event_device *dev);
+extern int __clockevents_update_freq(struct clock_event_device *dev, u32 freq);
extern ssize_t sysfs_get_uname(const char *buf, char *dst, size_t cnt);
-/*
- * NO_HZ / high resolution timer shared code
- */
+/* Broadcasting support */
+# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
+extern int tick_device_uses_broadcast(struct clock_event_device *dev, int cpu);
+extern void tick_install_broadcast_device(struct clock_event_device *dev);
+extern int tick_is_broadcast_device(struct clock_event_device *dev);
+extern void tick_shutdown_broadcast(unsigned int cpu);
+extern void tick_suspend_broadcast(void);
+extern void tick_resume_broadcast(void);
+extern bool tick_resume_check_broadcast(void);
+extern void tick_broadcast_init(void);
+extern void tick_set_periodic_handler(struct clock_event_device *dev, int broadcast);
+extern int tick_broadcast_update_freq(struct clock_event_device *dev, u32 freq);
+extern struct tick_device *tick_get_broadcast_device(void);
+extern struct cpumask *tick_get_broadcast_mask(void);
+# else /* !CONFIG_GENERIC_CLOCKEVENTS_BROADCAST: */
+static inline void tick_install_broadcast_device(struct clock_event_device *dev) { }
+static inline int tick_is_broadcast_device(struct clock_event_device *dev) { return 0; }
+static inline int tick_device_uses_broadcast(struct clock_event_device *dev, int cpu) { return 0; }
+static inline void tick_do_periodic_broadcast(struct clock_event_device *d) { }
+static inline void tick_shutdown_broadcast(unsigned int cpu) { }
+static inline void tick_suspend_broadcast(void) { }
+static inline void tick_resume_broadcast(void) { }
+static inline bool tick_resume_check_broadcast(void) { return false; }
+static inline void tick_broadcast_init(void) { }
+static inline int tick_broadcast_update_freq(struct clock_event_device *dev, u32 freq) { return -ENODEV; }
+
+/* Set the periodic handler in non broadcast mode */
+static inline void tick_set_periodic_handler(struct clock_event_device *dev, int broadcast)
+{
+ dev->event_handler = tick_handle_periodic;
+}
+# endif /* !CONFIG_GENERIC_CLOCKEVENTS_BROADCAST */
+
+#else /* !GENERIC_CLOCKEVENTS: */
+static inline void tick_suspend(void) { }
+static inline void tick_resume(void) { }
+#endif /* !GENERIC_CLOCKEVENTS */
+
+/* Oneshot related functions */
#ifdef CONFIG_TICK_ONESHOT
extern void tick_setup_oneshot(struct clock_event_device *newdev,
void (*handler)(struct clock_event_device *),
extern void tick_oneshot_notify(void);
extern int tick_switch_to_oneshot(void (*handler)(struct clock_event_device *));
extern void tick_resume_oneshot(void);
-# ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
+static inline bool tick_oneshot_possible(void) { return true; }
+extern int tick_oneshot_mode_active(void);
+extern void tick_clock_notify(void);
+extern int tick_check_oneshot_change(int allow_nohz);
+extern int tick_init_highres(void);
+#else /* !CONFIG_TICK_ONESHOT: */
+static inline
+void tick_setup_oneshot(struct clock_event_device *newdev,
+ void (*handler)(struct clock_event_device *),
+ ktime_t nextevt) { BUG(); }
+static inline void tick_resume_oneshot(void) { BUG(); }
+static inline int tick_program_event(ktime_t expires, int force) { return 0; }
+static inline void tick_oneshot_notify(void) { }
+static inline bool tick_oneshot_possible(void) { return false; }
+static inline int tick_oneshot_mode_active(void) { return 0; }
+static inline void tick_clock_notify(void) { }
+static inline int tick_check_oneshot_change(int allow_nohz) { return 0; }
+#endif /* !CONFIG_TICK_ONESHOT */
+
+/* Functions related to oneshot broadcasting */
+#if defined(CONFIG_GENERIC_CLOCKEVENTS_BROADCAST) && defined(CONFIG_TICK_ONESHOT)
extern void tick_broadcast_setup_oneshot(struct clock_event_device *bc);
-extern int tick_broadcast_oneshot_control(unsigned long reason);
extern void tick_broadcast_switch_to_oneshot(void);
-extern void tick_shutdown_broadcast_oneshot(unsigned int *cpup);
-extern int tick_resume_broadcast_oneshot(struct clock_event_device *bc);
+extern void tick_shutdown_broadcast_oneshot(unsigned int cpu);
extern int tick_broadcast_oneshot_active(void);
extern void tick_check_oneshot_broadcast_this_cpu(void);
bool tick_broadcast_oneshot_available(void);
-# else /* BROADCAST */
-static inline void tick_broadcast_setup_oneshot(struct clock_event_device *bc)
-{
- BUG();
-}
-static inline int tick_broadcast_oneshot_control(unsigned long reason) { return 0; }
+extern struct cpumask *tick_get_broadcast_oneshot_mask(void);
+#else /* !(BROADCAST && ONESHOT): */
+static inline void tick_broadcast_setup_oneshot(struct clock_event_device *bc) { BUG(); }
static inline void tick_broadcast_switch_to_oneshot(void) { }
-static inline void tick_shutdown_broadcast_oneshot(unsigned int *cpup) { }
+static inline void tick_shutdown_broadcast_oneshot(unsigned int cpu) { }
static inline int tick_broadcast_oneshot_active(void) { return 0; }
static inline void tick_check_oneshot_broadcast_this_cpu(void) { }
-static inline bool tick_broadcast_oneshot_available(void) { return true; }
-# endif /* !BROADCAST */
-
-#else /* !ONESHOT */
-static inline
-void tick_setup_oneshot(struct clock_event_device *newdev,
- void (*handler)(struct clock_event_device *),
- ktime_t nextevt)
-{
- BUG();
-}
-static inline void tick_resume_oneshot(void)
-{
- BUG();
-}
-static inline int tick_program_event(ktime_t expires, int force)
-{
- return 0;
-}
-static inline void tick_oneshot_notify(void) { }
-static inline void tick_broadcast_setup_oneshot(struct clock_event_device *bc)
-{
- BUG();
-}
-static inline int tick_broadcast_oneshot_control(unsigned long reason) { return 0; }
-static inline void tick_shutdown_broadcast_oneshot(unsigned int *cpup) { }
-static inline int tick_resume_broadcast_oneshot(struct clock_event_device *bc)
-{
- return 0;
-}
-static inline int tick_broadcast_oneshot_active(void) { return 0; }
-static inline bool tick_broadcast_oneshot_available(void) { return false; }
-#endif /* !TICK_ONESHOT */
+static inline bool tick_broadcast_oneshot_available(void) { return tick_oneshot_possible(); }
+#endif /* !(BROADCAST && ONESHOT) */
/* NO_HZ_FULL internal */
#ifdef CONFIG_NO_HZ_FULL
# else
static inline void tick_nohz_init(void) { }
#endif
-
-/*
- * Broadcasting support
- */
-#ifdef CONFIG_GENERIC_CLOCKEVENTS_BROADCAST
-extern int tick_device_uses_broadcast(struct clock_event_device *dev, int cpu);
-extern void tick_install_broadcast_device(struct clock_event_device *dev);
-extern int tick_is_broadcast_device(struct clock_event_device *dev);
-extern void tick_broadcast_on_off(unsigned long reason, int *oncpu);
-extern void tick_shutdown_broadcast(unsigned int *cpup);
-extern void tick_suspend_broadcast(void);
-extern int tick_resume_broadcast(void);
-extern void tick_broadcast_init(void);
-extern void
-tick_set_periodic_handler(struct clock_event_device *dev, int broadcast);
-int tick_broadcast_update_freq(struct clock_event_device *dev, u32 freq);
-
-#else /* !BROADCAST */
-
-static inline void tick_install_broadcast_device(struct clock_event_device *dev)
-{
-}
-
-static inline int tick_is_broadcast_device(struct clock_event_device *dev)
-{
- return 0;
-}
-static inline int tick_device_uses_broadcast(struct clock_event_device *dev,
- int cpu)
-{
- return 0;
-}
-static inline void tick_do_periodic_broadcast(struct clock_event_device *d) { }
-static inline void tick_broadcast_on_off(unsigned long reason, int *oncpu) { }
-static inline void tick_shutdown_broadcast(unsigned int *cpup) { }
-static inline void tick_suspend_broadcast(void) { }
-static inline int tick_resume_broadcast(void) { return 0; }
-static inline void tick_broadcast_init(void) { }
-static inline int tick_broadcast_update_freq(struct clock_event_device *dev,
- u32 freq) { return -ENODEV; }
-
-/*
- * Set the periodic handler in non broadcast mode
- */
-static inline void tick_set_periodic_handler(struct clock_event_device *dev,
- int broadcast)
-{
- dev->event_handler = tick_handle_periodic;
-}
-#endif /* !BROADCAST */
-
-/*
- * Check, if the device is functional or a dummy for broadcast
- */
-static inline int tick_device_is_functional(struct clock_event_device *dev)
-{
- return !(dev->features & CLOCK_EVT_FEAT_DUMMY);
-}
-
-int __clockevents_update_freq(struct clock_event_device *dev, u32 freq);
-
-#endif
-
-extern void do_timer(unsigned long ticks);
-extern void update_wall_time(void);
{
struct clock_event_device *dev = __this_cpu_read(tick_cpu_device.evtdev);
- clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_ONESHOT);
clockevents_program_event(dev, ktime_get(), true);
}
ktime_t next_event)
{
newdev->event_handler = handler;
- clockevents_set_mode(newdev, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(newdev, CLOCK_EVT_STATE_ONESHOT);
clockevents_program_event(newdev, next_event, true);
}
td->mode = TICKDEV_MODE_ONESHOT;
dev->event_handler = handler;
- clockevents_set_mode(dev, CLOCK_EVT_MODE_ONESHOT);
+ clockevents_set_state(dev, CLOCK_EVT_STATE_ONESHOT);
tick_broadcast_switch_to_oneshot();
return 0;
}
/*
* Per cpu nohz control structure
*/
-DEFINE_PER_CPU(struct tick_sched, tick_cpu_sched);
+static DEFINE_PER_CPU(struct tick_sched, tick_cpu_sched);
/*
* The time, when the last jiffy update happened. Protected by jiffies_lock.
__setup("nohz=", setup_tick_nohz);
+int tick_nohz_tick_stopped(void)
+{
+ return __this_cpu_read(tick_cpu_sched.tick_stopped);
+}
+
/**
* tick_nohz_update_jiffies - update jiffies when idle was interrupted
*
--- /dev/null
+#ifndef _TICK_SCHED_H
+#define _TICK_SCHED_H
+
+#include <linux/hrtimer.h>
+
+enum tick_device_mode {
+ TICKDEV_MODE_PERIODIC,
+ TICKDEV_MODE_ONESHOT,
+};
+
+struct tick_device {
+ struct clock_event_device *evtdev;
+ enum tick_device_mode mode;
+};
+
+enum tick_nohz_mode {
+ NOHZ_MODE_INACTIVE,
+ NOHZ_MODE_LOWRES,
+ NOHZ_MODE_HIGHRES,
+};
+
+/**
+ * struct tick_sched - sched tick emulation and no idle tick control/stats
+ * @sched_timer: hrtimer to schedule the periodic tick in high
+ * resolution mode
+ * @last_tick: Store the last tick expiry time when the tick
+ * timer is modified for nohz sleeps. This is necessary
+ * to resume the tick timer operation in the timeline
+ * when the CPU returns from nohz sleep.
+ * @tick_stopped: Indicator that the idle tick has been stopped
+ * @idle_jiffies: jiffies at the entry to idle for idle time accounting
+ * @idle_calls: Total number of idle calls
+ * @idle_sleeps: Number of idle calls, where the sched tick was stopped
+ * @idle_entrytime: Time when the idle call was entered
+ * @idle_waketime: Time when the idle was interrupted
+ * @idle_exittime: Time when the idle state was left
+ * @idle_sleeptime: Sum of the time slept in idle with sched tick stopped
+ * @iowait_sleeptime: Sum of the time slept in idle with sched tick stopped, with IO outstanding
+ * @sleep_length: Duration of the current idle sleep
+ * @do_timer_lst: CPU was the last one doing do_timer before going idle
+ */
+struct tick_sched {
+ struct hrtimer sched_timer;
+ unsigned long check_clocks;
+ enum tick_nohz_mode nohz_mode;
+ ktime_t last_tick;
+ int inidle;
+ int tick_stopped;
+ unsigned long idle_jiffies;
+ unsigned long idle_calls;
+ unsigned long idle_sleeps;
+ int idle_active;
+ ktime_t idle_entrytime;
+ ktime_t idle_waketime;
+ ktime_t idle_exittime;
+ ktime_t idle_sleeptime;
+ ktime_t iowait_sleeptime;
+ ktime_t sleep_length;
+ unsigned long last_jiffies;
+ unsigned long next_jiffies;
+ ktime_t idle_expires;
+ int do_timer_last;
+};
+
+extern struct tick_sched *tick_get_tick_sched(int cpu);
+
+extern void tick_setup_sched_timer(void);
+#if defined CONFIG_NO_HZ_COMMON || defined CONFIG_HIGH_RES_TIMERS
+extern void tick_cancel_sched_timer(int cpu);
+#else
+static inline void tick_cancel_sched_timer(int cpu) { }
+#endif
+
+#endif
};
static struct tk_fast tk_fast_mono ____cacheline_aligned;
+static struct tk_fast tk_fast_raw ____cacheline_aligned;
/* flag for if timekeeping is suspended */
int __read_mostly timekeeping_suspended;
-/* Flag for if there is a persistent clock on this platform */
-bool __read_mostly persistent_clock_exist = false;
-
static inline void tk_normalize_xtime(struct timekeeper *tk)
{
- while (tk->tkr.xtime_nsec >= ((u64)NSEC_PER_SEC << tk->tkr.shift)) {
- tk->tkr.xtime_nsec -= (u64)NSEC_PER_SEC << tk->tkr.shift;
+ while (tk->tkr_mono.xtime_nsec >= ((u64)NSEC_PER_SEC << tk->tkr_mono.shift)) {
+ tk->tkr_mono.xtime_nsec -= (u64)NSEC_PER_SEC << tk->tkr_mono.shift;
tk->xtime_sec++;
}
}
struct timespec64 ts;
ts.tv_sec = tk->xtime_sec;
- ts.tv_nsec = (long)(tk->tkr.xtime_nsec >> tk->tkr.shift);
+ ts.tv_nsec = (long)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);
return ts;
}
static void tk_set_xtime(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec = ts->tv_sec;
- tk->tkr.xtime_nsec = (u64)ts->tv_nsec << tk->tkr.shift;
+ tk->tkr_mono.xtime_nsec = (u64)ts->tv_nsec << tk->tkr_mono.shift;
}
static void tk_xtime_add(struct timekeeper *tk, const struct timespec64 *ts)
{
tk->xtime_sec += ts->tv_sec;
- tk->tkr.xtime_nsec += (u64)ts->tv_nsec << tk->tkr.shift;
+ tk->tkr_mono.xtime_nsec += (u64)ts->tv_nsec << tk->tkr_mono.shift;
tk_normalize_xtime(tk);
}
tk->offs_boot = ktime_add(tk->offs_boot, delta);
}
+#ifdef CONFIG_DEBUG_TIMEKEEPING
+#define WARNING_FREQ (HZ*300) /* 5 minute rate-limiting */
+/*
+ * These simple flag variables are managed
+ * without locks, which is racy, but ok since
+ * we don't really care about being super
+ * precise about how many events were seen,
+ * just that a problem was observed.
+ */
+static int timekeeping_underflow_seen;
+static int timekeeping_overflow_seen;
+
+/* last_warning is only modified under the timekeeping lock */
+static long timekeeping_last_warning;
+
+static void timekeeping_check_update(struct timekeeper *tk, cycle_t offset)
+{
+
+ cycle_t max_cycles = tk->tkr_mono.clock->max_cycles;
+ const char *name = tk->tkr_mono.clock->name;
+
+ if (offset > max_cycles) {
+ printk_deferred("WARNING: timekeeping: Cycle offset (%lld) is larger than allowed by the '%s' clock's max_cycles value (%lld): time overflow danger\n",
+ offset, name, max_cycles);
+ printk_deferred(" timekeeping: Your kernel is sick, but tries to cope by capping time updates\n");
+ } else {
+ if (offset > (max_cycles >> 1)) {
+ printk_deferred("INFO: timekeeping: Cycle offset (%lld) is larger than the the '%s' clock's 50%% safety margin (%lld)\n",
+ offset, name, max_cycles >> 1);
+ printk_deferred(" timekeeping: Your kernel is still fine, but is feeling a bit nervous\n");
+ }
+ }
+
+ if (timekeeping_underflow_seen) {
+ if (jiffies - timekeeping_last_warning > WARNING_FREQ) {
+ printk_deferred("WARNING: Underflow in clocksource '%s' observed, time update ignored.\n", name);
+ printk_deferred(" Please report this, consider using a different clocksource, if possible.\n");
+ printk_deferred(" Your kernel is probably still fine.\n");
+ timekeeping_last_warning = jiffies;
+ }
+ timekeeping_underflow_seen = 0;
+ }
+
+ if (timekeeping_overflow_seen) {
+ if (jiffies - timekeeping_last_warning > WARNING_FREQ) {
+ printk_deferred("WARNING: Overflow in clocksource '%s' observed, time update capped.\n", name);
+ printk_deferred(" Please report this, consider using a different clocksource, if possible.\n");
+ printk_deferred(" Your kernel is probably still fine.\n");
+ timekeeping_last_warning = jiffies;
+ }
+ timekeeping_overflow_seen = 0;
+ }
+}
+
+static inline cycle_t timekeeping_get_delta(struct tk_read_base *tkr)
+{
+ cycle_t now, last, mask, max, delta;
+ unsigned int seq;
+
+ /*
+ * Since we're called holding a seqlock, the data may shift
+ * under us while we're doing the calculation. This can cause
+ * false positives, since we'd note a problem but throw the
+ * results away. So nest another seqlock here to atomically
+ * grab the points we are checking with.
+ */
+ do {
+ seq = read_seqcount_begin(&tk_core.seq);
+ now = tkr->read(tkr->clock);
+ last = tkr->cycle_last;
+ mask = tkr->mask;
+ max = tkr->clock->max_cycles;
+ } while (read_seqcount_retry(&tk_core.seq, seq));
+
+ delta = clocksource_delta(now, last, mask);
+
+ /*
+ * Try to catch underflows by checking if we are seeing small
+ * mask-relative negative values.
+ */
+ if (unlikely((~delta & mask) < (mask >> 3))) {
+ timekeeping_underflow_seen = 1;
+ delta = 0;
+ }
+
+ /* Cap delta value to the max_cycles values to avoid mult overflows */
+ if (unlikely(delta > max)) {
+ timekeeping_overflow_seen = 1;
+ delta = tkr->clock->max_cycles;
+ }
+
+ return delta;
+}
+#else
+static inline void timekeeping_check_update(struct timekeeper *tk, cycle_t offset)
+{
+}
+static inline cycle_t timekeeping_get_delta(struct tk_read_base *tkr)
+{
+ cycle_t cycle_now, delta;
+
+ /* read clocksource */
+ cycle_now = tkr->read(tkr->clock);
+
+ /* calculate the delta since the last update_wall_time */
+ delta = clocksource_delta(cycle_now, tkr->cycle_last, tkr->mask);
+
+ return delta;
+}
+#endif
+
/**
* tk_setup_internals - Set up internals to use clocksource clock.
*
u64 tmp, ntpinterval;
struct clocksource *old_clock;
- old_clock = tk->tkr.clock;
- tk->tkr.clock = clock;
- tk->tkr.read = clock->read;
- tk->tkr.mask = clock->mask;
- tk->tkr.cycle_last = tk->tkr.read(clock);
+ old_clock = tk->tkr_mono.clock;
+ tk->tkr_mono.clock = clock;
+ tk->tkr_mono.read = clock->read;
+ tk->tkr_mono.mask = clock->mask;
+ tk->tkr_mono.cycle_last = tk->tkr_mono.read(clock);
+
+ tk->tkr_raw.clock = clock;
+ tk->tkr_raw.read = clock->read;
+ tk->tkr_raw.mask = clock->mask;
+ tk->tkr_raw.cycle_last = tk->tkr_mono.cycle_last;
/* Do the ns -> cycle conversion first, using original mult */
tmp = NTP_INTERVAL_LENGTH;
if (old_clock) {
int shift_change = clock->shift - old_clock->shift;
if (shift_change < 0)
- tk->tkr.xtime_nsec >>= -shift_change;
+ tk->tkr_mono.xtime_nsec >>= -shift_change;
else
- tk->tkr.xtime_nsec <<= shift_change;
+ tk->tkr_mono.xtime_nsec <<= shift_change;
}
- tk->tkr.shift = clock->shift;
+ tk->tkr_raw.xtime_nsec = 0;
+
+ tk->tkr_mono.shift = clock->shift;
+ tk->tkr_raw.shift = clock->shift;
tk->ntp_error = 0;
tk->ntp_error_shift = NTP_SCALE_SHIFT - clock->shift;
* active clocksource. These value will be adjusted via NTP
* to counteract clock drifting.
*/
- tk->tkr.mult = clock->mult;
+ tk->tkr_mono.mult = clock->mult;
+ tk->tkr_raw.mult = clock->mult;
tk->ntp_err_mult = 0;
}
static inline s64 timekeeping_get_ns(struct tk_read_base *tkr)
{
- cycle_t cycle_now, delta;
+ cycle_t delta;
s64 nsec;
- /* read clocksource: */
- cycle_now = tkr->read(tkr->clock);
-
- /* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tkr->cycle_last, tkr->mask);
+ delta = timekeeping_get_delta(tkr);
nsec = delta * tkr->mult + tkr->xtime_nsec;
nsec >>= tkr->shift;
return nsec + arch_gettimeoffset();
}
-static inline s64 timekeeping_get_ns_raw(struct timekeeper *tk)
-{
- struct clocksource *clock = tk->tkr.clock;
- cycle_t cycle_now, delta;
- s64 nsec;
-
- /* read clocksource: */
- cycle_now = tk->tkr.read(clock);
-
- /* calculate the delta since the last update_wall_time: */
- delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
-
- /* convert delta to nanoseconds. */
- nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
-
- /* If arch requires, add in get_arch_timeoffset() */
- return nsec + arch_gettimeoffset();
-}
-
/**
* update_fast_timekeeper - Update the fast and NMI safe monotonic timekeeper.
* @tkr: Timekeeping readout base from which we take the update
* slightly wrong timestamp (a few nanoseconds). See
* @ktime_get_mono_fast_ns.
*/
-static void update_fast_timekeeper(struct tk_read_base *tkr)
+static void update_fast_timekeeper(struct tk_read_base *tkr, struct tk_fast *tkf)
{
- struct tk_read_base *base = tk_fast_mono.base;
+ struct tk_read_base *base = tkf->base;
/* Force readers off to base[1] */
- raw_write_seqcount_latch(&tk_fast_mono.seq);
+ raw_write_seqcount_latch(&tkf->seq);
/* Update base[0] */
memcpy(base, tkr, sizeof(*base));
/* Force readers back to base[0] */
- raw_write_seqcount_latch(&tk_fast_mono.seq);
+ raw_write_seqcount_latch(&tkf->seq);
/* Update base[1] */
memcpy(base + 1, base, sizeof(*base));
* of the following timestamps. Callers need to be aware of that and
* deal with it.
*/
-u64 notrace ktime_get_mono_fast_ns(void)
+static __always_inline u64 __ktime_get_fast_ns(struct tk_fast *tkf)
{
struct tk_read_base *tkr;
unsigned int seq;
u64 now;
do {
- seq = raw_read_seqcount(&tk_fast_mono.seq);
- tkr = tk_fast_mono.base + (seq & 0x01);
- now = ktime_to_ns(tkr->base_mono) + timekeeping_get_ns(tkr);
+ seq = raw_read_seqcount(&tkf->seq);
+ tkr = tkf->base + (seq & 0x01);
+ now = ktime_to_ns(tkr->base) + timekeeping_get_ns(tkr);
+ } while (read_seqcount_retry(&tkf->seq, seq));
- } while (read_seqcount_retry(&tk_fast_mono.seq, seq));
return now;
}
+
+u64 ktime_get_mono_fast_ns(void)
+{
+ return __ktime_get_fast_ns(&tk_fast_mono);
+}
EXPORT_SYMBOL_GPL(ktime_get_mono_fast_ns);
+u64 ktime_get_raw_fast_ns(void)
+{
+ return __ktime_get_fast_ns(&tk_fast_raw);
+}
+EXPORT_SYMBOL_GPL(ktime_get_raw_fast_ns);
+
/* Suspend-time cycles value for halted fast timekeeper. */
static cycle_t cycles_at_suspend;
static void halt_fast_timekeeper(struct timekeeper *tk)
{
static struct tk_read_base tkr_dummy;
- struct tk_read_base *tkr = &tk->tkr;
+ struct tk_read_base *tkr = &tk->tkr_mono;
memcpy(&tkr_dummy, tkr, sizeof(tkr_dummy));
cycles_at_suspend = tkr->read(tkr->clock);
tkr_dummy.read = dummy_clock_read;
- update_fast_timekeeper(&tkr_dummy);
+ update_fast_timekeeper(&tkr_dummy, &tk_fast_mono);
+
+ tkr = &tk->tkr_raw;
+ memcpy(&tkr_dummy, tkr, sizeof(tkr_dummy));
+ tkr_dummy.read = dummy_clock_read;
+ update_fast_timekeeper(&tkr_dummy, &tk_fast_raw);
}
#ifdef CONFIG_GENERIC_TIME_VSYSCALL_OLD
xt = timespec64_to_timespec(tk_xtime(tk));
wm = timespec64_to_timespec(tk->wall_to_monotonic);
- update_vsyscall_old(&xt, &wm, tk->tkr.clock, tk->tkr.mult,
- tk->tkr.cycle_last);
+ update_vsyscall_old(&xt, &wm, tk->tkr_mono.clock, tk->tkr_mono.mult,
+ tk->tkr_mono.cycle_last);
}
static inline void old_vsyscall_fixup(struct timekeeper *tk)
* (shifted nanoseconds), and CONFIG_GENERIC_TIME_VSYSCALL_OLD
* users are removed, this can be killed.
*/
- remainder = tk->tkr.xtime_nsec & ((1ULL << tk->tkr.shift) - 1);
- tk->tkr.xtime_nsec -= remainder;
- tk->tkr.xtime_nsec += 1ULL << tk->tkr.shift;
+ remainder = tk->tkr_mono.xtime_nsec & ((1ULL << tk->tkr_mono.shift) - 1);
+ tk->tkr_mono.xtime_nsec -= remainder;
+ tk->tkr_mono.xtime_nsec += 1ULL << tk->tkr_mono.shift;
tk->ntp_error += remainder << tk->ntp_error_shift;
- tk->ntp_error -= (1ULL << tk->tkr.shift) << tk->ntp_error_shift;
+ tk->ntp_error -= (1ULL << tk->tkr_mono.shift) << tk->ntp_error_shift;
}
#else
#define old_vsyscall_fixup(tk)
*/
seconds = (u64)(tk->xtime_sec + tk->wall_to_monotonic.tv_sec);
nsec = (u32) tk->wall_to_monotonic.tv_nsec;
- tk->tkr.base_mono = ns_to_ktime(seconds * NSEC_PER_SEC + nsec);
+ tk->tkr_mono.base = ns_to_ktime(seconds * NSEC_PER_SEC + nsec);
/* Update the monotonic raw base */
- tk->base_raw = timespec64_to_ktime(tk->raw_time);
+ tk->tkr_raw.base = timespec64_to_ktime(tk->raw_time);
/*
* The sum of the nanoseconds portions of xtime and
* wall_to_monotonic can be greater/equal one second. Take
* this into account before updating tk->ktime_sec.
*/
- nsec += (u32)(tk->tkr.xtime_nsec >> tk->tkr.shift);
+ nsec += (u32)(tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift);
if (nsec >= NSEC_PER_SEC)
seconds++;
tk->ktime_sec = seconds;
memcpy(&shadow_timekeeper, &tk_core.timekeeper,
sizeof(tk_core.timekeeper));
- update_fast_timekeeper(&tk->tkr);
+ update_fast_timekeeper(&tk->tkr_mono, &tk_fast_mono);
+ update_fast_timekeeper(&tk->tkr_raw, &tk_fast_raw);
}
/**
*/
static void timekeeping_forward_now(struct timekeeper *tk)
{
- struct clocksource *clock = tk->tkr.clock;
+ struct clocksource *clock = tk->tkr_mono.clock;
cycle_t cycle_now, delta;
s64 nsec;
- cycle_now = tk->tkr.read(clock);
- delta = clocksource_delta(cycle_now, tk->tkr.cycle_last, tk->tkr.mask);
- tk->tkr.cycle_last = cycle_now;
+ cycle_now = tk->tkr_mono.read(clock);
+ delta = clocksource_delta(cycle_now, tk->tkr_mono.cycle_last, tk->tkr_mono.mask);
+ tk->tkr_mono.cycle_last = cycle_now;
+ tk->tkr_raw.cycle_last = cycle_now;
- tk->tkr.xtime_nsec += delta * tk->tkr.mult;
+ tk->tkr_mono.xtime_nsec += delta * tk->tkr_mono.mult;
/* If arch requires, add in get_arch_timeoffset() */
- tk->tkr.xtime_nsec += (u64)arch_gettimeoffset() << tk->tkr.shift;
+ tk->tkr_mono.xtime_nsec += (u64)arch_gettimeoffset() << tk->tkr_mono.shift;
tk_normalize_xtime(tk);
- nsec = clocksource_cyc2ns(delta, clock->mult, clock->shift);
+ nsec = clocksource_cyc2ns(delta, tk->tkr_raw.mult, tk->tkr_raw.shift);
timespec64_add_ns(&tk->raw_time, nsec);
}
seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
- nsecs = timekeeping_get_ns(&tk->tkr);
+ nsecs = timekeeping_get_ns(&tk->tkr_mono);
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->tkr.base_mono;
- nsecs = timekeeping_get_ns(&tk->tkr);
+ base = tk->tkr_mono.base;
+ nsecs = timekeeping_get_ns(&tk->tkr_mono);
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = ktime_add(tk->tkr.base_mono, *offset);
- nsecs = timekeeping_get_ns(&tk->tkr);
+ base = ktime_add(tk->tkr_mono.base, *offset);
+ nsecs = timekeeping_get_ns(&tk->tkr_mono);
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->base_raw;
- nsecs = timekeeping_get_ns_raw(tk);
+ base = tk->tkr_raw.base;
+ nsecs = timekeeping_get_ns(&tk->tkr_raw);
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
ts->tv_sec = tk->xtime_sec;
- nsec = timekeeping_get_ns(&tk->tkr);
+ nsec = timekeeping_get_ns(&tk->tkr_mono);
tomono = tk->wall_to_monotonic;
} while (read_seqcount_retry(&tk_core.seq, seq));
ts_real->tv_sec = tk->xtime_sec;
ts_real->tv_nsec = 0;
- nsecs_raw = timekeeping_get_ns_raw(tk);
- nsecs_real = timekeeping_get_ns(&tk->tkr);
+ nsecs_raw = timekeeping_get_ns(&tk->tkr_raw);
+ nsecs_real = timekeeping_get_ns(&tk->tkr_mono);
} while (read_seqcount_retry(&tk_core.seq, seq));
*/
if (try_module_get(new->owner)) {
if (!new->enable || new->enable(new) == 0) {
- old = tk->tkr.clock;
+ old = tk->tkr_mono.clock;
tk_setup_internals(tk, new);
if (old->disable)
old->disable(old);
{
struct timekeeper *tk = &tk_core.timekeeper;
- if (tk->tkr.clock == clock)
+ if (tk->tkr_mono.clock == clock)
return 0;
stop_machine(change_clocksource, clock, NULL);
tick_clock_notify();
- return tk->tkr.clock == clock ? 0 : -1;
+ return tk->tkr_mono.clock == clock ? 0 : -1;
}
/**
do {
seq = read_seqcount_begin(&tk_core.seq);
- nsecs = timekeeping_get_ns_raw(tk);
+ nsecs = timekeeping_get_ns(&tk->tkr_raw);
ts64 = tk->raw_time;
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
- ret = tk->tkr.clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
+ ret = tk->tkr_mono.clock->flags & CLOCK_SOURCE_VALID_FOR_HRES;
} while (read_seqcount_retry(&tk_core.seq, seq));
do {
seq = read_seqcount_begin(&tk_core.seq);
- ret = tk->tkr.clock->max_idle_ns;
+ ret = tk->tkr_mono.clock->max_idle_ns;
} while (read_seqcount_retry(&tk_core.seq, seq));
ts->tv_nsec = 0;
}
+void __weak read_persistent_clock64(struct timespec64 *ts64)
+{
+ struct timespec ts;
+
+ read_persistent_clock(&ts);
+ *ts64 = timespec_to_timespec64(ts);
+}
+
/**
* read_boot_clock - Return time of the system start.
*
ts->tv_nsec = 0;
}
+void __weak read_boot_clock64(struct timespec64 *ts64)
+{
+ struct timespec ts;
+
+ read_boot_clock(&ts);
+ *ts64 = timespec_to_timespec64(ts);
+}
+
+/* Flag for if timekeeping_resume() has injected sleeptime */
+static bool sleeptime_injected;
+
+/* Flag for if there is a persistent clock on this platform */
+static bool persistent_clock_exists;
+
/*
* timekeeping_init - Initializes the clocksource and common timekeeping values
*/
struct clocksource *clock;
unsigned long flags;
struct timespec64 now, boot, tmp;
- struct timespec ts;
- read_persistent_clock(&ts);
- now = timespec_to_timespec64(ts);
+ read_persistent_clock64(&now);
if (!timespec64_valid_strict(&now)) {
pr_warn("WARNING: Persistent clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
now.tv_sec = 0;
now.tv_nsec = 0;
} else if (now.tv_sec || now.tv_nsec)
- persistent_clock_exist = true;
+ persistent_clock_exists = true;
- read_boot_clock(&ts);
- boot = timespec_to_timespec64(ts);
+ read_boot_clock64(&boot);
if (!timespec64_valid_strict(&boot)) {
pr_warn("WARNING: Boot clock returned invalid value!\n"
" Check your CMOS/BIOS settings.\n");
tk_set_xtime(tk, &now);
tk->raw_time.tv_sec = 0;
tk->raw_time.tv_nsec = 0;
- tk->base_raw.tv64 = 0;
if (boot.tv_sec == 0 && boot.tv_nsec == 0)
boot = tk_xtime(tk);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
}
-/* time in seconds when suspend began */
+/* time in seconds when suspend began for persistent clock */
static struct timespec64 timekeeping_suspend_time;
/**
tk_debug_account_sleep_time(delta);
}
+#if defined(CONFIG_PM_SLEEP) && defined(CONFIG_RTC_HCTOSYS_DEVICE)
+/**
+ * We have three kinds of time sources to use for sleep time
+ * injection, the preference order is:
+ * 1) non-stop clocksource
+ * 2) persistent clock (ie: RTC accessible when irqs are off)
+ * 3) RTC
+ *
+ * 1) and 2) are used by timekeeping, 3) by RTC subsystem.
+ * If system has neither 1) nor 2), 3) will be used finally.
+ *
+ *
+ * If timekeeping has injected sleeptime via either 1) or 2),
+ * 3) becomes needless, so in this case we don't need to call
+ * rtc_resume(), and this is what timekeeping_rtc_skipresume()
+ * means.
+ */
+bool timekeeping_rtc_skipresume(void)
+{
+ return sleeptime_injected;
+}
+
+/**
+ * 1) can be determined whether to use or not only when doing
+ * timekeeping_resume() which is invoked after rtc_suspend(),
+ * so we can't skip rtc_suspend() surely if system has 1).
+ *
+ * But if system has 2), 2) will definitely be used, so in this
+ * case we don't need to call rtc_suspend(), and this is what
+ * timekeeping_rtc_skipsuspend() means.
+ */
+bool timekeeping_rtc_skipsuspend(void)
+{
+ return persistent_clock_exists;
+}
+
/**
* timekeeping_inject_sleeptime64 - Adds suspend interval to timeekeeping values
* @delta: pointer to a timespec64 delta value
*
- * This hook is for architectures that cannot support read_persistent_clock
+ * This hook is for architectures that cannot support read_persistent_clock64
* because their RTC/persistent clock is only accessible when irqs are enabled.
+ * and also don't have an effective nonstop clocksource.
*
* This function should only be called by rtc_resume(), and allows
* a suspend offset to be injected into the timekeeping values.
struct timekeeper *tk = &tk_core.timekeeper;
unsigned long flags;
- /*
- * Make sure we don't set the clock twice, as timekeeping_resume()
- * already did it
- */
- if (has_persistent_clock())
- return;
-
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&tk_core.seq);
/* signal hrtimers about time change */
clock_was_set();
}
+#endif
/**
* timekeeping_resume - Resumes the generic timekeeping subsystem.
- *
- * This is for the generic clocksource timekeeping.
- * xtime/wall_to_monotonic/jiffies/etc are
- * still managed by arch specific suspend/resume code.
*/
void timekeeping_resume(void)
{
struct timekeeper *tk = &tk_core.timekeeper;
- struct clocksource *clock = tk->tkr.clock;
+ struct clocksource *clock = tk->tkr_mono.clock;
unsigned long flags;
struct timespec64 ts_new, ts_delta;
- struct timespec tmp;
cycle_t cycle_now, cycle_delta;
- bool suspendtime_found = false;
- read_persistent_clock(&tmp);
- ts_new = timespec_to_timespec64(tmp);
+ sleeptime_injected = false;
+ read_persistent_clock64(&ts_new);
clockevents_resume();
clocksource_resume();
* The less preferred source will only be tried if there is no better
* usable source. The rtc part is handled separately in rtc core code.
*/
- cycle_now = tk->tkr.read(clock);
+ cycle_now = tk->tkr_mono.read(clock);
if ((clock->flags & CLOCK_SOURCE_SUSPEND_NONSTOP) &&
- cycle_now > tk->tkr.cycle_last) {
+ cycle_now > tk->tkr_mono.cycle_last) {
u64 num, max = ULLONG_MAX;
u32 mult = clock->mult;
u32 shift = clock->shift;
s64 nsec = 0;
- cycle_delta = clocksource_delta(cycle_now, tk->tkr.cycle_last,
- tk->tkr.mask);
+ cycle_delta = clocksource_delta(cycle_now, tk->tkr_mono.cycle_last,
+ tk->tkr_mono.mask);
/*
* "cycle_delta * mutl" may cause 64 bits overflow, if the
nsec += ((u64) cycle_delta * mult) >> shift;
ts_delta = ns_to_timespec64(nsec);
- suspendtime_found = true;
+ sleeptime_injected = true;
} else if (timespec64_compare(&ts_new, &timekeeping_suspend_time) > 0) {
ts_delta = timespec64_sub(ts_new, timekeeping_suspend_time);
- suspendtime_found = true;
+ sleeptime_injected = true;
}
- if (suspendtime_found)
+ if (sleeptime_injected)
__timekeeping_inject_sleeptime(tk, &ts_delta);
/* Re-base the last cycle value */
- tk->tkr.cycle_last = cycle_now;
+ tk->tkr_mono.cycle_last = cycle_now;
+ tk->tkr_raw.cycle_last = cycle_now;
+
tk->ntp_error = 0;
timekeeping_suspended = 0;
timekeeping_update(tk, TK_MIRROR | TK_CLOCK_WAS_SET);
touch_softlockup_watchdog();
- clockevents_notify(CLOCK_EVT_NOTIFY_RESUME, NULL);
-
- /* Resume hrtimers */
+ tick_resume();
hrtimers_resume();
}
unsigned long flags;
struct timespec64 delta, delta_delta;
static struct timespec64 old_delta;
- struct timespec tmp;
- read_persistent_clock(&tmp);
- timekeeping_suspend_time = timespec_to_timespec64(tmp);
+ read_persistent_clock64(&timekeeping_suspend_time);
/*
* On some systems the persistent_clock can not be detected at
* value returned, update the persistent_clock_exists flag.
*/
if (timekeeping_suspend_time.tv_sec || timekeeping_suspend_time.tv_nsec)
- persistent_clock_exist = true;
+ persistent_clock_exists = true;
raw_spin_lock_irqsave(&timekeeper_lock, flags);
write_seqcount_begin(&tk_core.seq);
timekeeping_forward_now(tk);
timekeeping_suspended = 1;
- /*
- * To avoid drift caused by repeated suspend/resumes,
- * which each can add ~1 second drift error,
- * try to compensate so the difference in system time
- * and persistent_clock time stays close to constant.
- */
- delta = timespec64_sub(tk_xtime(tk), timekeeping_suspend_time);
- delta_delta = timespec64_sub(delta, old_delta);
- if (abs(delta_delta.tv_sec) >= 2) {
+ if (persistent_clock_exists) {
/*
- * if delta_delta is too large, assume time correction
- * has occured and set old_delta to the current delta.
+ * To avoid drift caused by repeated suspend/resumes,
+ * which each can add ~1 second drift error,
+ * try to compensate so the difference in system time
+ * and persistent_clock time stays close to constant.
*/
- old_delta = delta;
- } else {
- /* Otherwise try to adjust old_system to compensate */
- timekeeping_suspend_time =
- timespec64_add(timekeeping_suspend_time, delta_delta);
+ delta = timespec64_sub(tk_xtime(tk), timekeeping_suspend_time);
+ delta_delta = timespec64_sub(delta, old_delta);
+ if (abs(delta_delta.tv_sec) >= 2) {
+ /*
+ * if delta_delta is too large, assume time correction
+ * has occurred and set old_delta to the current delta.
+ */
+ old_delta = delta;
+ } else {
+ /* Otherwise try to adjust old_system to compensate */
+ timekeeping_suspend_time =
+ timespec64_add(timekeeping_suspend_time, delta_delta);
+ }
}
timekeeping_update(tk, TK_MIRROR);
write_seqcount_end(&tk_core.seq);
raw_spin_unlock_irqrestore(&timekeeper_lock, flags);
- clockevents_notify(CLOCK_EVT_NOTIFY_SUSPEND, NULL);
+ tick_suspend();
clocksource_suspend();
clockevents_suspend();
*
* XXX - TODO: Doc ntp_error calculation.
*/
- if ((mult_adj > 0) && (tk->tkr.mult + mult_adj < mult_adj)) {
+ if ((mult_adj > 0) && (tk->tkr_mono.mult + mult_adj < mult_adj)) {
/* NTP adjustment caused clocksource mult overflow */
WARN_ON_ONCE(1);
return;
}
- tk->tkr.mult += mult_adj;
+ tk->tkr_mono.mult += mult_adj;
tk->xtime_interval += interval;
- tk->tkr.xtime_nsec -= offset;
+ tk->tkr_mono.xtime_nsec -= offset;
tk->ntp_error -= (interval - offset) << tk->ntp_error_shift;
}
tk->ntp_err_mult = 0;
}
- if (unlikely(tk->tkr.clock->maxadj &&
- (abs(tk->tkr.mult - tk->tkr.clock->mult)
- > tk->tkr.clock->maxadj))) {
+ if (unlikely(tk->tkr_mono.clock->maxadj &&
+ (abs(tk->tkr_mono.mult - tk->tkr_mono.clock->mult)
+ > tk->tkr_mono.clock->maxadj))) {
printk_once(KERN_WARNING
"Adjusting %s more than 11%% (%ld vs %ld)\n",
- tk->tkr.clock->name, (long)tk->tkr.mult,
- (long)tk->tkr.clock->mult + tk->tkr.clock->maxadj);
+ tk->tkr_mono.clock->name, (long)tk->tkr_mono.mult,
+ (long)tk->tkr_mono.clock->mult + tk->tkr_mono.clock->maxadj);
}
/*
* We'll correct this error next time through this function, when
* xtime_nsec is not as small.
*/
- if (unlikely((s64)tk->tkr.xtime_nsec < 0)) {
- s64 neg = -(s64)tk->tkr.xtime_nsec;
- tk->tkr.xtime_nsec = 0;
+ if (unlikely((s64)tk->tkr_mono.xtime_nsec < 0)) {
+ s64 neg = -(s64)tk->tkr_mono.xtime_nsec;
+ tk->tkr_mono.xtime_nsec = 0;
tk->ntp_error += neg << tk->ntp_error_shift;
}
}
*/
static inline unsigned int accumulate_nsecs_to_secs(struct timekeeper *tk)
{
- u64 nsecps = (u64)NSEC_PER_SEC << tk->tkr.shift;
+ u64 nsecps = (u64)NSEC_PER_SEC << tk->tkr_mono.shift;
unsigned int clock_set = 0;
- while (tk->tkr.xtime_nsec >= nsecps) {
+ while (tk->tkr_mono.xtime_nsec >= nsecps) {
int leap;
- tk->tkr.xtime_nsec -= nsecps;
+ tk->tkr_mono.xtime_nsec -= nsecps;
tk->xtime_sec++;
/* Figure out if its a leap sec and apply if needed */
/* Accumulate one shifted interval */
offset -= interval;
- tk->tkr.cycle_last += interval;
+ tk->tkr_mono.cycle_last += interval;
+ tk->tkr_raw.cycle_last += interval;
- tk->tkr.xtime_nsec += tk->xtime_interval << shift;
+ tk->tkr_mono.xtime_nsec += tk->xtime_interval << shift;
*clock_set |= accumulate_nsecs_to_secs(tk);
/* Accumulate raw time */
#ifdef CONFIG_ARCH_USES_GETTIMEOFFSET
offset = real_tk->cycle_interval;
#else
- offset = clocksource_delta(tk->tkr.read(tk->tkr.clock),
- tk->tkr.cycle_last, tk->tkr.mask);
+ offset = clocksource_delta(tk->tkr_mono.read(tk->tkr_mono.clock),
+ tk->tkr_mono.cycle_last, tk->tkr_mono.mask);
#endif
/* Check if there's really nothing to do */
if (offset < real_tk->cycle_interval)
goto out;
+ /* Do some additional sanity checking */
+ timekeeping_check_update(real_tk, offset);
+
/*
* With NO_HZ we may have to accumulate many cycle_intervals
* (think "ticks") worth of time at once. To do this efficiently,
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->tkr.base_mono;
- nsecs = tk->tkr.xtime_nsec >> tk->tkr.shift;
+ base = tk->tkr_mono.base;
+ nsecs = tk->tkr_mono.xtime_nsec >> tk->tkr_mono.shift;
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
do {
seq = read_seqcount_begin(&tk_core.seq);
- base = tk->tkr.base_mono;
- nsecs = timekeeping_get_ns(&tk->tkr);
+ base = tk->tkr_mono.base;
+ nsecs = timekeeping_get_ns(&tk->tkr_mono);
*offs_real = tk->offs_real;
*offs_boot = tk->offs_boot;
extern int timekeeping_suspend(void);
extern void timekeeping_resume(void);
+extern void do_timer(unsigned long ticks);
+extern void update_wall_time(void);
+
+extern seqlock_t jiffies_lock;
+
+#define CS_NAME_LEN 32
+
#endif
struct tvec tv5;
} ____cacheline_aligned;
+/*
+ * __TIMER_INITIALIZER() needs to set ->base to a valid pointer (because we've
+ * made NULL special, hint: lock_timer_base()) and we cannot get a compile time
+ * pointer to per-cpu entries because we don't know where we'll map the section,
+ * even for the boot cpu.
+ *
+ * And so we use boot_tvec_bases for boot CPU and per-cpu __tvec_bases for the
+ * rest of them.
+ */
struct tvec_base boot_tvec_bases;
EXPORT_SYMBOL(boot_tvec_bases);
+
static DEFINE_PER_CPU(struct tvec_base *, tvec_bases) = &boot_tvec_bases;
/* Functions below help us manage 'deferrable' flag */
EXPORT_SYMBOL(try_to_del_timer_sync);
#ifdef CONFIG_SMP
+static DEFINE_PER_CPU(struct tvec_base, __tvec_bases);
+
/**
* del_timer_sync - deactivate a timer and wait for the handler to finish.
* @timer: the timer to be deactivated
}
EXPORT_SYMBOL(schedule_timeout_uninterruptible);
-static int init_timers_cpu(int cpu)
-{
- int j;
- struct tvec_base *base;
- static char tvec_base_done[NR_CPUS];
-
- if (!tvec_base_done[cpu]) {
- static char boot_done;
-
- if (boot_done) {
- /*
- * The APs use this path later in boot
- */
- base = kzalloc_node(sizeof(*base), GFP_KERNEL,
- cpu_to_node(cpu));
- if (!base)
- return -ENOMEM;
-
- /* Make sure tvec_base has TIMER_FLAG_MASK bits free */
- if (WARN_ON(base != tbase_get_base(base))) {
- kfree(base);
- return -ENOMEM;
- }
- per_cpu(tvec_bases, cpu) = base;
- } else {
- /*
- * This is for the boot CPU - we use compile-time
- * static initialisation because per-cpu memory isn't
- * ready yet and because the memory allocators are not
- * initialised either.
- */
- boot_done = 1;
- base = &boot_tvec_bases;
- }
- spin_lock_init(&base->lock);
- tvec_base_done[cpu] = 1;
- base->cpu = cpu;
- } else {
- base = per_cpu(tvec_bases, cpu);
- }
-
-
- for (j = 0; j < TVN_SIZE; j++) {
- INIT_LIST_HEAD(base->tv5.vec + j);
- INIT_LIST_HEAD(base->tv4.vec + j);
- INIT_LIST_HEAD(base->tv3.vec + j);
- INIT_LIST_HEAD(base->tv2.vec + j);
- }
- for (j = 0; j < TVR_SIZE; j++)
- INIT_LIST_HEAD(base->tv1.vec + j);
-
- base->timer_jiffies = jiffies;
- base->next_timer = base->timer_jiffies;
- base->active_timers = 0;
- base->all_timers = 0;
- return 0;
-}
-
#ifdef CONFIG_HOTPLUG_CPU
static void migrate_timer_list(struct tvec_base *new_base, struct list_head *head)
{
migrate_timer_list(new_base, old_base->tv5.vec + i);
}
+ old_base->active_timers = 0;
+ old_base->all_timers = 0;
+
spin_unlock(&old_base->lock);
spin_unlock_irq(&new_base->lock);
put_cpu_var(tvec_bases);
}
-#endif /* CONFIG_HOTPLUG_CPU */
static int timer_cpu_notify(struct notifier_block *self,
unsigned long action, void *hcpu)
{
- long cpu = (long)hcpu;
- int err;
-
- switch(action) {
- case CPU_UP_PREPARE:
- case CPU_UP_PREPARE_FROZEN:
- err = init_timers_cpu(cpu);
- if (err < 0)
- return notifier_from_errno(err);
- break;
-#ifdef CONFIG_HOTPLUG_CPU
+ switch (action) {
case CPU_DEAD:
case CPU_DEAD_FROZEN:
- migrate_timers(cpu);
+ migrate_timers((long)hcpu);
break;
-#endif
default:
break;
}
+
return NOTIFY_OK;
}
-static struct notifier_block timers_nb = {
- .notifier_call = timer_cpu_notify,
-};
+static inline void timer_register_cpu_notifier(void)
+{
+ cpu_notifier(timer_cpu_notify, 0);
+}
+#else
+static inline void timer_register_cpu_notifier(void) { }
+#endif /* CONFIG_HOTPLUG_CPU */
+static void __init init_timer_cpu(struct tvec_base *base, int cpu)
+{
+ int j;
-void __init init_timers(void)
+ BUG_ON(base != tbase_get_base(base));
+
+ base->cpu = cpu;
+ per_cpu(tvec_bases, cpu) = base;
+ spin_lock_init(&base->lock);
+
+ for (j = 0; j < TVN_SIZE; j++) {
+ INIT_LIST_HEAD(base->tv5.vec + j);
+ INIT_LIST_HEAD(base->tv4.vec + j);
+ INIT_LIST_HEAD(base->tv3.vec + j);
+ INIT_LIST_HEAD(base->tv2.vec + j);
+ }
+ for (j = 0; j < TVR_SIZE; j++)
+ INIT_LIST_HEAD(base->tv1.vec + j);
+
+ base->timer_jiffies = jiffies;
+ base->next_timer = base->timer_jiffies;
+}
+
+static void __init init_timer_cpus(void)
{
- int err;
+ struct tvec_base *base;
+ int local_cpu = smp_processor_id();
+ int cpu;
+ for_each_possible_cpu(cpu) {
+ if (cpu == local_cpu)
+ base = &boot_tvec_bases;
+#ifdef CONFIG_SMP
+ else
+ base = per_cpu_ptr(&__tvec_bases, cpu);
+#endif
+
+ init_timer_cpu(base, cpu);
+ }
+}
+
+void __init init_timers(void)
+{
/* ensure there are enough low bits for flags in timer->base pointer */
BUILD_BUG_ON(__alignof__(struct tvec_base) & TIMER_FLAG_MASK);
- err = timer_cpu_notify(&timers_nb, (unsigned long)CPU_UP_PREPARE,
- (void *)(long)smp_processor_id());
- BUG_ON(err != NOTIFY_OK);
-
+ init_timer_cpus();
init_timer_stats();
- register_cpu_notifier(&timers_nb);
+ timer_register_cpu_notifier();
open_softirq(TIMER_SOFTIRQ, run_timer_softirq);
}
#include <linux/sched.h>
#include <linux/seq_file.h>
#include <linux/kallsyms.h>
-#include <linux/tick.h>
#include <asm/uaccess.h>
+#include "tick-internal.h"
struct timer_list_iter {
int cpu;
print_name_offset(m, dev->set_next_event);
SEQ_printf(m, "\n");
- SEQ_printf(m, " set_mode: ");
- print_name_offset(m, dev->set_mode);
- SEQ_printf(m, "\n");
+ if (dev->set_mode) {
+ SEQ_printf(m, " set_mode: ");
+ print_name_offset(m, dev->set_mode);
+ SEQ_printf(m, "\n");
+ } else {
+ if (dev->set_state_shutdown) {
+ SEQ_printf(m, " shutdown: ");
+ print_name_offset(m, dev->set_state_shutdown);
+ SEQ_printf(m, "\n");
+ }
+
+ if (dev->set_state_periodic) {
+ SEQ_printf(m, " periodic: ");
+ print_name_offset(m, dev->set_state_periodic);
+ SEQ_printf(m, "\n");
+ }
+
+ if (dev->set_state_oneshot) {
+ SEQ_printf(m, " oneshot: ");
+ print_name_offset(m, dev->set_state_oneshot);
+ SEQ_printf(m, "\n");
+ }
+
+ if (dev->tick_resume) {
+ SEQ_printf(m, " resume: ");
+ print_name_offset(m, dev->tick_resume);
+ SEQ_printf(m, "\n");
+ }
+ }
SEQ_printf(m, " event_handler: ");
print_name_offset(m, dev->event_handler);
static struct pid * const ftrace_swapper_pid = &init_struct_pid;
+#ifdef CONFIG_FUNCTION_GRAPH_TRACER
+static int ftrace_graph_active;
+#else
+# define ftrace_graph_active 0
+#endif
+
#ifdef CONFIG_DYNAMIC_FTRACE
static struct ftrace_ops *removed_ops;
if (!ftrace_rec_count(rec))
rec->flags = 0;
else
- /* Just disable the record (keep REGS state) */
- rec->flags &= ~FTRACE_FL_ENABLED;
+ /*
+ * Just disable the record, but keep the ops TRAMP
+ * and REGS states. The _EN flags must be disabled though.
+ */
+ rec->flags &= ~(FTRACE_FL_ENABLED | FTRACE_FL_TRAMP_EN |
+ FTRACE_FL_REGS_EN);
}
return FTRACE_UPDATE_MAKE_NOP;
static void ftrace_startup_sysctl(void)
{
+ int command;
+
if (unlikely(ftrace_disabled))
return;
/* Force update next time */
saved_ftrace_func = NULL;
/* ftrace_start_up is true if we want ftrace running */
- if (ftrace_start_up)
- ftrace_run_update_code(FTRACE_UPDATE_CALLS);
+ if (ftrace_start_up) {
+ command = FTRACE_UPDATE_CALLS;
+ if (ftrace_graph_active)
+ command |= FTRACE_START_FUNC_RET;
+ ftrace_startup_enable(command);
+ }
}
static void ftrace_shutdown_sysctl(void)
{
+ int command;
+
if (unlikely(ftrace_disabled))
return;
/* ftrace_start_up is true if ftrace is running */
- if (ftrace_start_up)
- ftrace_run_update_code(FTRACE_DISABLE_CALLS);
+ if (ftrace_start_up) {
+ command = FTRACE_DISABLE_CALLS;
+ if (ftrace_graph_active)
+ command |= FTRACE_STOP_FUNC_RET;
+ ftrace_run_update_code(command);
+ }
}
static cycle_t ftrace_update_time;
if (ftrace_enabled) {
- ftrace_startup_sysctl();
-
/* we are starting ftrace again */
if (ftrace_ops_list != &ftrace_list_end)
update_ftrace_function();
+ ftrace_startup_sysctl();
+
} else {
/* stopping ftrace calls (just send to ftrace_stub) */
ftrace_trace_function = ftrace_stub;
ASSIGN_OPS_HASH(graph_ops, &global_ops.local_hash)
};
-static int ftrace_graph_active;
-
int ftrace_graph_entry_stub(struct ftrace_graph_ent *trace)
{
return 0;
/* see manage_workers() for details on the two manager mutexes */
struct mutex manager_arb; /* manager arbitration */
+ struct worker *manager; /* L: purely informational */
struct mutex attach_mutex; /* attach/detach exclusion */
struct list_head workers; /* A: attached workers */
struct completion *detach_completion; /* all workers detached */
*/
struct workqueue_struct {
struct list_head pwqs; /* WR: all pwqs of this wq */
- struct list_head list; /* PL: list of all workqueues */
+ struct list_head list; /* PR: list of all workqueues */
struct mutex mutex; /* protects this wq */
int work_color; /* WQ: current work color */
#endif
char name[WQ_NAME_LEN]; /* I: workqueue name */
+ /*
+ * Destruction of workqueue_struct is sched-RCU protected to allow
+ * walking the workqueues list without grabbing wq_pool_mutex.
+ * This is used to dump all workqueues from sysrq.
+ */
+ struct rcu_head rcu;
+
/* hot fields used during command issue, aligned to cacheline */
unsigned int flags ____cacheline_aligned; /* WQ: WQ_* flags */
struct pool_workqueue __percpu *cpu_pwqs; /* I: per-cpu pwqs */
static DEFINE_MUTEX(wq_pool_mutex); /* protects pools and workqueues list */
static DEFINE_SPINLOCK(wq_mayday_lock); /* protects wq->maydays list */
-static LIST_HEAD(workqueues); /* PL: list of all workqueues */
+static LIST_HEAD(workqueues); /* PR: list of all workqueues */
static bool workqueue_freezing; /* PL: have wqs started freezing? */
/* the per-cpu worker pools */
static int worker_thread(void *__worker);
static void copy_workqueue_attrs(struct workqueue_attrs *to,
const struct workqueue_attrs *from);
+static void workqueue_sysfs_unregister(struct workqueue_struct *wq);
#define CREATE_TRACE_POINTS
#include <trace/events/workqueue.h>
*/
if (!mutex_trylock(&pool->manager_arb))
return false;
+ pool->manager = worker;
maybe_create_worker(pool);
+ pool->manager = NULL;
mutex_unlock(&pool->manager_arb);
return true;
}
struct wq_barrier {
struct work_struct work;
struct completion done;
+ struct task_struct *task; /* purely informational */
};
static void wq_barrier_func(struct work_struct *work)
INIT_WORK_ONSTACK(&barr->work, wq_barrier_func);
__set_bit(WORK_STRUCT_PENDING_BIT, work_data_bits(&barr->work));
init_completion(&barr->done);
+ barr->task = current;
/*
* If @target is currently being executed, schedule the
}
EXPORT_SYMBOL_GPL(flush_work);
+struct cwt_wait {
+ wait_queue_t wait;
+ struct work_struct *work;
+};
+
+static int cwt_wakefn(wait_queue_t *wait, unsigned mode, int sync, void *key)
+{
+ struct cwt_wait *cwait = container_of(wait, struct cwt_wait, wait);
+
+ if (cwait->work != key)
+ return 0;
+ return autoremove_wake_function(wait, mode, sync, key);
+}
+
static bool __cancel_work_timer(struct work_struct *work, bool is_dwork)
{
+ static DECLARE_WAIT_QUEUE_HEAD(cancel_waitq);
unsigned long flags;
int ret;
do {
ret = try_to_grab_pending(work, is_dwork, &flags);
/*
- * If someone else is canceling, wait for the same event it
- * would be waiting for before retrying.
+ * If someone else is already canceling, wait for it to
+ * finish. flush_work() doesn't work for PREEMPT_NONE
+ * because we may get scheduled between @work's completion
+ * and the other canceling task resuming and clearing
+ * CANCELING - flush_work() will return false immediately
+ * as @work is no longer busy, try_to_grab_pending() will
+ * return -ENOENT as @work is still being canceled and the
+ * other canceling task won't be able to clear CANCELING as
+ * we're hogging the CPU.
+ *
+ * Let's wait for completion using a waitqueue. As this
+ * may lead to the thundering herd problem, use a custom
+ * wake function which matches @work along with exclusive
+ * wait and wakeup.
*/
- if (unlikely(ret == -ENOENT))
- flush_work(work);
+ if (unlikely(ret == -ENOENT)) {
+ struct cwt_wait cwait;
+
+ init_wait(&cwait.wait);
+ cwait.wait.func = cwt_wakefn;
+ cwait.work = work;
+
+ prepare_to_wait_exclusive(&cancel_waitq, &cwait.wait,
+ TASK_UNINTERRUPTIBLE);
+ if (work_is_canceling(work))
+ schedule();
+ finish_wait(&cancel_waitq, &cwait.wait);
+ }
} while (unlikely(ret < 0));
/* tell other tasks trying to grab @work to back off */
flush_work(work);
clear_work_data(work);
+
+ /*
+ * Paired with prepare_to_wait() above so that either
+ * waitqueue_active() is visible here or !work_is_canceling() is
+ * visible there.
+ */
+ smp_mb();
+ if (waitqueue_active(&cancel_waitq))
+ __wake_up(&cancel_waitq, TASK_NORMAL, 1, work);
+
return ret;
}
}
EXPORT_SYMBOL_GPL(execute_in_process_context);
-#ifdef CONFIG_SYSFS
-/*
- * Workqueues with WQ_SYSFS flag set is visible to userland via
- * /sys/bus/workqueue/devices/WQ_NAME. All visible workqueues have the
- * following attributes.
- *
- * per_cpu RO bool : whether the workqueue is per-cpu or unbound
- * max_active RW int : maximum number of in-flight work items
- *
- * Unbound workqueues have the following extra attributes.
+/**
+ * free_workqueue_attrs - free a workqueue_attrs
+ * @attrs: workqueue_attrs to free
*
- * id RO int : the associated pool ID
- * nice RW int : nice value of the workers
- * cpumask RW mask : bitmask of allowed CPUs for the workers
+ * Undo alloc_workqueue_attrs().
*/
-struct wq_device {
- struct workqueue_struct *wq;
- struct device dev;
-};
-
-static struct workqueue_struct *dev_to_wq(struct device *dev)
+void free_workqueue_attrs(struct workqueue_attrs *attrs)
{
- struct wq_device *wq_dev = container_of(dev, struct wq_device, dev);
-
- return wq_dev->wq;
+ if (attrs) {
+ free_cpumask_var(attrs->cpumask);
+ kfree(attrs);
+ }
}
-static ssize_t per_cpu_show(struct device *dev, struct device_attribute *attr,
- char *buf)
+/**
+ * alloc_workqueue_attrs - allocate a workqueue_attrs
+ * @gfp_mask: allocation mask to use
+ *
+ * Allocate a new workqueue_attrs, initialize with default settings and
+ * return it.
+ *
+ * Return: The allocated new workqueue_attr on success. %NULL on failure.
+ */
+struct workqueue_attrs *alloc_workqueue_attrs(gfp_t gfp_mask)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
+ struct workqueue_attrs *attrs;
- return scnprintf(buf, PAGE_SIZE, "%d\n", (bool)!(wq->flags & WQ_UNBOUND));
+ attrs = kzalloc(sizeof(*attrs), gfp_mask);
+ if (!attrs)
+ goto fail;
+ if (!alloc_cpumask_var(&attrs->cpumask, gfp_mask))
+ goto fail;
+
+ cpumask_copy(attrs->cpumask, cpu_possible_mask);
+ return attrs;
+fail:
+ free_workqueue_attrs(attrs);
+ return NULL;
}
-static DEVICE_ATTR_RO(per_cpu);
-static ssize_t max_active_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static void copy_workqueue_attrs(struct workqueue_attrs *to,
+ const struct workqueue_attrs *from)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
-
- return scnprintf(buf, PAGE_SIZE, "%d\n", wq->saved_max_active);
+ to->nice = from->nice;
+ cpumask_copy(to->cpumask, from->cpumask);
+ /*
+ * Unlike hash and equality test, this function doesn't ignore
+ * ->no_numa as it is used for both pool and wq attrs. Instead,
+ * get_unbound_pool() explicitly clears ->no_numa after copying.
+ */
+ to->no_numa = from->no_numa;
}
-static ssize_t max_active_store(struct device *dev,
- struct device_attribute *attr, const char *buf,
- size_t count)
+/* hash value of the content of @attr */
+static u32 wqattrs_hash(const struct workqueue_attrs *attrs)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- int val;
-
- if (sscanf(buf, "%d", &val) != 1 || val <= 0)
- return -EINVAL;
+ u32 hash = 0;
- workqueue_set_max_active(wq, val);
- return count;
+ hash = jhash_1word(attrs->nice, hash);
+ hash = jhash(cpumask_bits(attrs->cpumask),
+ BITS_TO_LONGS(nr_cpumask_bits) * sizeof(long), hash);
+ return hash;
}
-static DEVICE_ATTR_RW(max_active);
-
-static struct attribute *wq_sysfs_attrs[] = {
- &dev_attr_per_cpu.attr,
- &dev_attr_max_active.attr,
- NULL,
-};
-ATTRIBUTE_GROUPS(wq_sysfs);
-static ssize_t wq_pool_ids_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+/* content equality test */
+static bool wqattrs_equal(const struct workqueue_attrs *a,
+ const struct workqueue_attrs *b)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- const char *delim = "";
- int node, written = 0;
-
- rcu_read_lock_sched();
- for_each_node(node) {
- written += scnprintf(buf + written, PAGE_SIZE - written,
- "%s%d:%d", delim, node,
- unbound_pwq_by_node(wq, node)->pool->id);
- delim = " ";
- }
- written += scnprintf(buf + written, PAGE_SIZE - written, "\n");
- rcu_read_unlock_sched();
-
- return written;
+ if (a->nice != b->nice)
+ return false;
+ if (!cpumask_equal(a->cpumask, b->cpumask))
+ return false;
+ return true;
}
-static ssize_t wq_nice_show(struct device *dev, struct device_attribute *attr,
- char *buf)
+/**
+ * init_worker_pool - initialize a newly zalloc'd worker_pool
+ * @pool: worker_pool to initialize
+ *
+ * Initiailize a newly zalloc'd @pool. It also allocates @pool->attrs.
+ *
+ * Return: 0 on success, -errno on failure. Even on failure, all fields
+ * inside @pool proper are initialized and put_unbound_pool() can be called
+ * on @pool safely to release it.
+ */
+static int init_worker_pool(struct worker_pool *pool)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- int written;
+ spin_lock_init(&pool->lock);
+ pool->id = -1;
+ pool->cpu = -1;
+ pool->node = NUMA_NO_NODE;
+ pool->flags |= POOL_DISASSOCIATED;
+ INIT_LIST_HEAD(&pool->worklist);
+ INIT_LIST_HEAD(&pool->idle_list);
+ hash_init(pool->busy_hash);
- mutex_lock(&wq->mutex);
- written = scnprintf(buf, PAGE_SIZE, "%d\n", wq->unbound_attrs->nice);
- mutex_unlock(&wq->mutex);
+ init_timer_deferrable(&pool->idle_timer);
+ pool->idle_timer.function = idle_worker_timeout;
+ pool->idle_timer.data = (unsigned long)pool;
- return written;
-}
+ setup_timer(&pool->mayday_timer, pool_mayday_timeout,
+ (unsigned long)pool);
-/* prepare workqueue_attrs for sysfs store operations */
-static struct workqueue_attrs *wq_sysfs_prep_attrs(struct workqueue_struct *wq)
-{
- struct workqueue_attrs *attrs;
+ mutex_init(&pool->manager_arb);
+ mutex_init(&pool->attach_mutex);
+ INIT_LIST_HEAD(&pool->workers);
- attrs = alloc_workqueue_attrs(GFP_KERNEL);
- if (!attrs)
- return NULL;
+ ida_init(&pool->worker_ida);
+ INIT_HLIST_NODE(&pool->hash_node);
+ pool->refcnt = 1;
- mutex_lock(&wq->mutex);
- copy_workqueue_attrs(attrs, wq->unbound_attrs);
- mutex_unlock(&wq->mutex);
- return attrs;
+ /* shouldn't fail above this point */
+ pool->attrs = alloc_workqueue_attrs(GFP_KERNEL);
+ if (!pool->attrs)
+ return -ENOMEM;
+ return 0;
}
-static ssize_t wq_nice_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
+static void rcu_free_wq(struct rcu_head *rcu)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- struct workqueue_attrs *attrs;
- int ret;
-
- attrs = wq_sysfs_prep_attrs(wq);
- if (!attrs)
- return -ENOMEM;
+ struct workqueue_struct *wq =
+ container_of(rcu, struct workqueue_struct, rcu);
- if (sscanf(buf, "%d", &attrs->nice) == 1 &&
- attrs->nice >= MIN_NICE && attrs->nice <= MAX_NICE)
- ret = apply_workqueue_attrs(wq, attrs);
+ if (!(wq->flags & WQ_UNBOUND))
+ free_percpu(wq->cpu_pwqs);
else
- ret = -EINVAL;
+ free_workqueue_attrs(wq->unbound_attrs);
- free_workqueue_attrs(attrs);
- return ret ?: count;
+ kfree(wq->rescuer);
+ kfree(wq);
}
-static ssize_t wq_cpumask_show(struct device *dev,
- struct device_attribute *attr, char *buf)
+static void rcu_free_pool(struct rcu_head *rcu)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- int written;
+ struct worker_pool *pool = container_of(rcu, struct worker_pool, rcu);
- mutex_lock(&wq->mutex);
- written = scnprintf(buf, PAGE_SIZE, "%*pb\n",
- cpumask_pr_args(wq->unbound_attrs->cpumask));
- mutex_unlock(&wq->mutex);
- return written;
+ ida_destroy(&pool->worker_ida);
+ free_workqueue_attrs(pool->attrs);
+ kfree(pool);
}
-static ssize_t wq_cpumask_store(struct device *dev,
- struct device_attribute *attr,
- const char *buf, size_t count)
+/**
+ * put_unbound_pool - put a worker_pool
+ * @pool: worker_pool to put
+ *
+ * Put @pool. If its refcnt reaches zero, it gets destroyed in sched-RCU
+ * safe manner. get_unbound_pool() calls this function on its failure path
+ * and this function should be able to release pools which went through,
+ * successfully or not, init_worker_pool().
+ *
+ * Should be called with wq_pool_mutex held.
+ */
+static void put_unbound_pool(struct worker_pool *pool)
{
- struct workqueue_struct *wq = dev_to_wq(dev);
- struct workqueue_attrs *attrs;
- int ret;
+ DECLARE_COMPLETION_ONSTACK(detach_completion);
+ struct worker *worker;
- attrs = wq_sysfs_prep_attrs(wq);
- if (!attrs)
- return -ENOMEM;
+ lockdep_assert_held(&wq_pool_mutex);
- ret = cpumask_parse(buf, attrs->cpumask);
- if (!ret)
- ret = apply_workqueue_attrs(wq, attrs);
-
- free_workqueue_attrs(attrs);
- return ret ?: count;
-}
-
-static ssize_t wq_numa_show(struct device *dev, struct device_attribute *attr,
- char *buf)
-{
- struct workqueue_struct *wq = dev_to_wq(dev);
- int written;
-
- mutex_lock(&wq->mutex);
- written = scnprintf(buf, PAGE_SIZE, "%d\n",
- !wq->unbound_attrs->no_numa);
- mutex_unlock(&wq->mutex);
-
- return written;
-}
+ if (--pool->refcnt)
+ return;
-static ssize_t wq_numa_store(struct device *dev, struct device_attribute *attr,
- const char *buf, size_t count)
-{
- struct workqueue_struct *wq = dev_to_wq(dev);
- struct workqueue_attrs *attrs;
- int v, ret;
+ /* sanity checks */
+ if (WARN_ON(!(pool->cpu < 0)) ||
+ WARN_ON(!list_empty(&pool->worklist)))
+ return;
- attrs = wq_sysfs_prep_attrs(wq);
- if (!attrs)
- return -ENOMEM;
+ /* release id and unhash */
+ if (pool->id >= 0)
+ idr_remove(&worker_pool_idr, pool->id);
+ hash_del(&pool->hash_node);
- ret = -EINVAL;
- if (sscanf(buf, "%d", &v) == 1) {
- attrs->no_numa = !v;
- ret = apply_workqueue_attrs(wq, attrs);
- }
+ /*
+ * Become the manager and destroy all workers. Grabbing
+ * manager_arb prevents @pool's workers from blocking on
+ * attach_mutex.
+ */
+ mutex_lock(&pool->manager_arb);
- free_workqueue_attrs(attrs);
- return ret ?: count;
-}
+ spin_lock_irq(&pool->lock);
+ while ((worker = first_idle_worker(pool)))
+ destroy_worker(worker);
+ WARN_ON(pool->nr_workers || pool->nr_idle);
+ spin_unlock_irq(&pool->lock);
-static struct device_attribute wq_sysfs_unbound_attrs[] = {
- __ATTR(pool_ids, 0444, wq_pool_ids_show, NULL),
- __ATTR(nice, 0644, wq_nice_show, wq_nice_store),
- __ATTR(cpumask, 0644, wq_cpumask_show, wq_cpumask_store),
- __ATTR(numa, 0644, wq_numa_show, wq_numa_store),
- __ATTR_NULL,
-};
+ mutex_lock(&pool->attach_mutex);
+ if (!list_empty(&pool->workers))
+ pool->detach_completion = &detach_completion;
+ mutex_unlock(&pool->attach_mutex);
-static struct bus_type wq_subsys = {
- .name = "workqueue",
- .dev_groups = wq_sysfs_groups,
-};
+ if (pool->detach_completion)
+ wait_for_completion(pool->detach_completion);
-static int __init wq_sysfs_init(void)
-{
- return subsys_virtual_register(&wq_subsys, NULL);
-}
-core_initcall(wq_sysfs_init);
+ mutex_unlock(&pool->manager_arb);
-static void wq_device_release(struct device *dev)
-{
- struct wq_device *wq_dev = container_of(dev, struct wq_device, dev);
+ /* shut down the timers */
+ del_timer_sync(&pool->idle_timer);
+ del_timer_sync(&pool->mayday_timer);
- kfree(wq_dev);
+ /* sched-RCU protected to allow dereferences from get_work_pool() */
+ call_rcu_sched(&pool->rcu, rcu_free_pool);
}
/**
- * workqueue_sysfs_register - make a workqueue visible in sysfs
- * @wq: the workqueue to register
+ * get_unbound_pool - get a worker_pool with the specified attributes
+ * @attrs: the attributes of the worker_pool to get
*
- * Expose @wq in sysfs under /sys/bus/workqueue/devices.
- * alloc_workqueue*() automatically calls this function if WQ_SYSFS is set
- * which is the preferred method.
+ * Obtain a worker_pool which has the same attributes as @attrs, bump the
+ * reference count and return it. If there already is a matching
+ * worker_pool, it will be used; otherwise, this function attempts to
+ * create a new one.
*
- * Workqueue user should use this function directly iff it wants to apply
- * workqueue_attrs before making the workqueue visible in sysfs; otherwise,
- * apply_workqueue_attrs() may race against userland updating the
- * attributes.
+ * Should be called with wq_pool_mutex held.
*
- * Return: 0 on success, -errno on failure.
+ * Return: On success, a worker_pool with the same attributes as @attrs.
+ * On failure, %NULL.
*/
-int workqueue_sysfs_register(struct workqueue_struct *wq)
+static struct worker_pool *get_unbound_pool(const struct workqueue_attrs *attrs)
{
- struct wq_device *wq_dev;
- int ret;
+ u32 hash = wqattrs_hash(attrs);
+ struct worker_pool *pool;
+ int node;
- /*
- * Adjusting max_active or creating new pwqs by applyting
- * attributes breaks ordering guarantee. Disallow exposing ordered
- * workqueues.
- */
- if (WARN_ON(wq->flags & __WQ_ORDERED))
- return -EINVAL;
+ lockdep_assert_held(&wq_pool_mutex);
- wq->wq_dev = wq_dev = kzalloc(sizeof(*wq_dev), GFP_KERNEL);
- if (!wq_dev)
- return -ENOMEM;
+ /* do we already have a matching pool? */
+ hash_for_each_possible(unbound_pool_hash, pool, hash_node, hash) {
+ if (wqattrs_equal(pool->attrs, attrs)) {
+ pool->refcnt++;
+ return pool;
+ }
+ }
- wq_dev->wq = wq;
- wq_dev->dev.bus = &wq_subsys;
- wq_dev->dev.init_name = wq->name;
- wq_dev->dev.release = wq_device_release;
+ /* nope, create a new one */
+ pool = kzalloc(sizeof(*pool), GFP_KERNEL);
+ if (!pool || init_worker_pool(pool) < 0)
+ goto fail;
+
+ lockdep_set_subclass(&pool->lock, 1); /* see put_pwq() */
+ copy_workqueue_attrs(pool->attrs, attrs);
/*
- * unbound_attrs are created separately. Suppress uevent until
- * everything is ready.
+ * no_numa isn't a worker_pool attribute, always clear it. See
+ * 'struct workqueue_attrs' comments for detail.
*/
- dev_set_uevent_suppress(&wq_dev->dev, true);
-
- ret = device_register(&wq_dev->dev);
- if (ret) {
- kfree(wq_dev);
- wq->wq_dev = NULL;
- return ret;
- }
-
- if (wq->flags & WQ_UNBOUND) {
- struct device_attribute *attr;
+ pool->attrs->no_numa = false;
- for (attr = wq_sysfs_unbound_attrs; attr->attr.name; attr++) {
- ret = device_create_file(&wq_dev->dev, attr);
- if (ret) {
- device_unregister(&wq_dev->dev);
- wq->wq_dev = NULL;
- return ret;
+ /* if cpumask is contained inside a NUMA node, we belong to that node */
+ if (wq_numa_enabled) {
+ for_each_node(node) {
+ if (cpumask_subset(pool->attrs->cpumask,
+ wq_numa_possible_cpumask[node])) {
+ pool->node = node;
+ break;
}
}
}
- dev_set_uevent_suppress(&wq_dev->dev, false);
- kobject_uevent(&wq_dev->dev.kobj, KOBJ_ADD);
- return 0;
+ if (worker_pool_assign_id(pool) < 0)
+ goto fail;
+
+ /* create and start the initial worker */
+ if (!create_worker(pool))
+ goto fail;
+
+ /* install */
+ hash_add(unbound_pool_hash, &pool->hash_node, hash);
+
+ return pool;
+fail:
+ if (pool)
+ put_unbound_pool(pool);
+ return NULL;
}
-/**
- * workqueue_sysfs_unregister - undo workqueue_sysfs_register()
- * @wq: the workqueue to unregister
- *
- * If @wq is registered to sysfs by workqueue_sysfs_register(), unregister.
+static void rcu_free_pwq(struct rcu_head *rcu)
+{
+ kmem_cache_free(pwq_cache,
+ container_of(rcu, struct pool_workqueue, rcu));
+}
+
+/*
+ * Scheduled on system_wq by put_pwq() when an unbound pwq hits zero refcnt
+ * and needs to be destroyed.
*/
-static void workqueue_sysfs_unregister(struct workqueue_struct *wq)
+static void pwq_unbound_release_workfn(struct work_struct *work)
{
- struct wq_device *wq_dev = wq->wq_dev;
+ struct pool_workqueue *pwq = container_of(work, struct pool_workqueue,
+ unbound_release_work);
+ struct workqueue_struct *wq = pwq->wq;
+ struct worker_pool *pool = pwq->pool;
+ bool is_last;
- if (!wq->wq_dev)
+ if (WARN_ON_ONCE(!(wq->flags & WQ_UNBOUND)))
return;
- wq->wq_dev = NULL;
- device_unregister(&wq_dev->dev);
+ mutex_lock(&wq->mutex);
+ list_del_rcu(&pwq->pwqs_node);
+ is_last = list_empty(&wq->pwqs);
+ mutex_unlock(&wq->mutex);
+
+ mutex_lock(&wq_pool_mutex);
+ put_unbound_pool(pool);
+ mutex_unlock(&wq_pool_mutex);
+
+ call_rcu_sched(&pwq->rcu, rcu_free_pwq);
+
+ /*
+ * If we're the last pwq going away, @wq is already dead and no one
+ * is gonna access it anymore. Schedule RCU free.
+ */
+ if (is_last)
+ call_rcu_sched(&wq->rcu, rcu_free_wq);
}
-#else /* CONFIG_SYSFS */
-static void workqueue_sysfs_unregister(struct workqueue_struct *wq) { }
-#endif /* CONFIG_SYSFS */
/**
- * free_workqueue_attrs - free a workqueue_attrs
- * @attrs: workqueue_attrs to free
+ * pwq_adjust_max_active - update a pwq's max_active to the current setting
+ * @pwq: target pool_workqueue
*
- * Undo alloc_workqueue_attrs().
+ * If @pwq isn't freezing, set @pwq->max_active to the associated
+ * workqueue's saved_max_active and activate delayed work items
+ * accordingly. If @pwq is freezing, clear @pwq->max_active to zero.
*/
-void free_workqueue_attrs(struct workqueue_attrs *attrs)
+static void pwq_adjust_max_active(struct pool_workqueue *pwq)
{
- if (attrs) {
- free_cpumask_var(attrs->cpumask);
- kfree(attrs);
- }
-}
+ struct workqueue_struct *wq = pwq->wq;
+ bool freezable = wq->flags & WQ_FREEZABLE;
-/**
- * alloc_workqueue_attrs - allocate a workqueue_attrs
- * @gfp_mask: allocation mask to use
- *
- * Allocate a new workqueue_attrs, initialize with default settings and
- * return it.
- *
- * Return: The allocated new workqueue_attr on success. %NULL on failure.
- */
-struct workqueue_attrs *alloc_workqueue_attrs(gfp_t gfp_mask)
-{
- struct workqueue_attrs *attrs;
+ /* for @wq->saved_max_active */
+ lockdep_assert_held(&wq->mutex);
- attrs = kzalloc(sizeof(*attrs), gfp_mask);
- if (!attrs)
- goto fail;
- if (!alloc_cpumask_var(&attrs->cpumask, gfp_mask))
- goto fail;
+ /* fast exit for non-freezable wqs */
+ if (!freezable && pwq->max_active == wq->saved_max_active)
+ return;
- cpumask_copy(attrs->cpumask, cpu_possible_mask);
- return attrs;
-fail:
- free_workqueue_attrs(attrs);
- return NULL;
-}
+ spin_lock_irq(&pwq->pool->lock);
-static void copy_workqueue_attrs(struct workqueue_attrs *to,
- const struct workqueue_attrs *from)
-{
- to->nice = from->nice;
- cpumask_copy(to->cpumask, from->cpumask);
/*
- * Unlike hash and equality test, this function doesn't ignore
- * ->no_numa as it is used for both pool and wq attrs. Instead,
- * get_unbound_pool() explicitly clears ->no_numa after copying.
+ * During [un]freezing, the caller is responsible for ensuring that
+ * this function is called at least once after @workqueue_freezing
+ * is updated and visible.
*/
- to->no_numa = from->no_numa;
+ if (!freezable || !workqueue_freezing) {
+ pwq->max_active = wq->saved_max_active;
+
+ while (!list_empty(&pwq->delayed_works) &&
+ pwq->nr_active < pwq->max_active)
+ pwq_activate_first_delayed(pwq);
+
+ /*
+ * Need to kick a worker after thawed or an unbound wq's
+ * max_active is bumped. It's a slow path. Do it always.
+ */
+ wake_up_worker(pwq->pool);
+ } else {
+ pwq->max_active = 0;
+ }
+
+ spin_unlock_irq(&pwq->pool->lock);
}
-/* hash value of the content of @attr */
-static u32 wqattrs_hash(const struct workqueue_attrs *attrs)
+/* initialize newly alloced @pwq which is associated with @wq and @pool */
+static void init_pwq(struct pool_workqueue *pwq, struct workqueue_struct *wq,
+ struct worker_pool *pool)
{
- u32 hash = 0;
+ BUG_ON((unsigned long)pwq & WORK_STRUCT_FLAG_MASK);
- hash = jhash_1word(attrs->nice, hash);
- hash = jhash(cpumask_bits(attrs->cpumask),
- BITS_TO_LONGS(nr_cpumask_bits) * sizeof(long), hash);
- return hash;
+ memset(pwq, 0, sizeof(*pwq));
+
+ pwq->pool = pool;
+ pwq->wq = wq;
+ pwq->flush_color = -1;
+ pwq->refcnt = 1;
+ INIT_LIST_HEAD(&pwq->delayed_works);
+ INIT_LIST_HEAD(&pwq->pwqs_node);
+ INIT_LIST_HEAD(&pwq->mayday_node);
+ INIT_WORK(&pwq->unbound_release_work, pwq_unbound_release_workfn);
}
-/* content equality test */
-static bool wqattrs_equal(const struct workqueue_attrs *a,
- const struct workqueue_attrs *b)
+/* sync @pwq with the current state of its associated wq and link it */
+static void link_pwq(struct pool_workqueue *pwq)
{
- if (a->nice != b->nice)
- return false;
- if (!cpumask_equal(a->cpumask, b->cpumask))
- return false;
- return true;
+ struct workqueue_struct *wq = pwq->wq;
+
+ lockdep_assert_held(&wq->mutex);
+
+ /* may be called multiple times, ignore if already linked */
+ if (!list_empty(&pwq->pwqs_node))
+ return;
+
+ /* set the matching work_color */
+ pwq->work_color = wq->work_color;
+
+ /* sync max_active to the current setting */
+ pwq_adjust_max_active(pwq);
+
+ /* link in @pwq */
+ list_add_rcu(&pwq->pwqs_node, &wq->pwqs);
}
-/**
- * init_worker_pool - initialize a newly zalloc'd worker_pool
- * @pool: worker_pool to initialize
- *
- * Initiailize a newly zalloc'd @pool. It also allocates @pool->attrs.
- *
- * Return: 0 on success, -errno on failure. Even on failure, all fields
- * inside @pool proper are initialized and put_unbound_pool() can be called
- * on @pool safely to release it.
- */
-static int init_worker_pool(struct worker_pool *pool)
+/* obtain a pool matching @attr and create a pwq associating the pool and @wq */
+static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
+ const struct workqueue_attrs *attrs)
{
- spin_lock_init(&pool->lock);
- pool->id = -1;
- pool->cpu = -1;
- pool->node = NUMA_NO_NODE;
- pool->flags |= POOL_DISASSOCIATED;
- INIT_LIST_HEAD(&pool->worklist);
- INIT_LIST_HEAD(&pool->idle_list);
- hash_init(pool->busy_hash);
-
- init_timer_deferrable(&pool->idle_timer);
- pool->idle_timer.function = idle_worker_timeout;
- pool->idle_timer.data = (unsigned long)pool;
+ struct worker_pool *pool;
+ struct pool_workqueue *pwq;
- setup_timer(&pool->mayday_timer, pool_mayday_timeout,
- (unsigned long)pool);
+ lockdep_assert_held(&wq_pool_mutex);
- mutex_init(&pool->manager_arb);
- mutex_init(&pool->attach_mutex);
- INIT_LIST_HEAD(&pool->workers);
+ pool = get_unbound_pool(attrs);
+ if (!pool)
+ return NULL;
- ida_init(&pool->worker_ida);
- INIT_HLIST_NODE(&pool->hash_node);
- pool->refcnt = 1;
+ pwq = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node);
+ if (!pwq) {
+ put_unbound_pool(pool);
+ return NULL;
+ }
- /* shouldn't fail above this point */
- pool->attrs = alloc_workqueue_attrs(GFP_KERNEL);
- if (!pool->attrs)
- return -ENOMEM;
- return 0;
+ init_pwq(pwq, wq, pool);
+ return pwq;
}
-static void rcu_free_pool(struct rcu_head *rcu)
+/* undo alloc_unbound_pwq(), used only in the error path */
+static void free_unbound_pwq(struct pool_workqueue *pwq)
{
- struct worker_pool *pool = container_of(rcu, struct worker_pool, rcu);
+ lockdep_assert_held(&wq_pool_mutex);
- ida_destroy(&pool->worker_ida);
- free_workqueue_attrs(pool->attrs);
- kfree(pool);
+ if (pwq) {
+ put_unbound_pool(pwq->pool);
+ kmem_cache_free(pwq_cache, pwq);
+ }
}
/**
- * put_unbound_pool - put a worker_pool
- * @pool: worker_pool to put
+ * wq_calc_node_mask - calculate a wq_attrs' cpumask for the specified node
+ * @attrs: the wq_attrs of interest
+ * @node: the target NUMA node
+ * @cpu_going_down: if >= 0, the CPU to consider as offline
+ * @cpumask: outarg, the resulting cpumask
*
- * Put @pool. If its refcnt reaches zero, it gets destroyed in sched-RCU
- * safe manner. get_unbound_pool() calls this function on its failure path
- * and this function should be able to release pools which went through,
- * successfully or not, init_worker_pool().
+ * Calculate the cpumask a workqueue with @attrs should use on @node. If
+ * @cpu_going_down is >= 0, that cpu is considered offline during
+ * calculation. The result is stored in @cpumask.
*
- * Should be called with wq_pool_mutex held.
+ * If NUMA affinity is not enabled, @attrs->cpumask is always used. If
+ * enabled and @node has online CPUs requested by @attrs, the returned
+ * cpumask is the intersection of the possible CPUs of @node and
+ * @attrs->cpumask.
+ *
+ * The caller is responsible for ensuring that the cpumask of @node stays
+ * stable.
+ *
+ * Return: %true if the resulting @cpumask is different from @attrs->cpumask,
+ * %false if equal.
*/
-static void put_unbound_pool(struct worker_pool *pool)
+static bool wq_calc_node_cpumask(const struct workqueue_attrs *attrs, int node,
+ int cpu_going_down, cpumask_t *cpumask)
{
- DECLARE_COMPLETION_ONSTACK(detach_completion);
- struct worker *worker;
-
- lockdep_assert_held(&wq_pool_mutex);
-
- if (--pool->refcnt)
- return;
+ if (!wq_numa_enabled || attrs->no_numa)
+ goto use_dfl;
- /* sanity checks */
- if (WARN_ON(!(pool->cpu < 0)) ||
- WARN_ON(!list_empty(&pool->worklist)))
- return;
+ /* does @node have any online CPUs @attrs wants? */
+ cpumask_and(cpumask, cpumask_of_node(node), attrs->cpumask);
+ if (cpu_going_down >= 0)
+ cpumask_clear_cpu(cpu_going_down, cpumask);
- /* release id and unhash */
- if (pool->id >= 0)
- idr_remove(&worker_pool_idr, pool->id);
- hash_del(&pool->hash_node);
+ if (cpumask_empty(cpumask))
+ goto use_dfl;
- /*
- * Become the manager and destroy all workers. Grabbing
- * manager_arb prevents @pool's workers from blocking on
- * attach_mutex.
- */
- mutex_lock(&pool->manager_arb);
+ /* yeap, return possible CPUs in @node that @attrs wants */
+ cpumask_and(cpumask, attrs->cpumask, wq_numa_possible_cpumask[node]);
+ return !cpumask_equal(cpumask, attrs->cpumask);
- spin_lock_irq(&pool->lock);
- while ((worker = first_idle_worker(pool)))
- destroy_worker(worker);
- WARN_ON(pool->nr_workers || pool->nr_idle);
- spin_unlock_irq(&pool->lock);
+use_dfl:
+ cpumask_copy(cpumask, attrs->cpumask);
+ return false;
+}
- mutex_lock(&pool->attach_mutex);
- if (!list_empty(&pool->workers))
- pool->detach_completion = &detach_completion;
- mutex_unlock(&pool->attach_mutex);
+/* install @pwq into @wq's numa_pwq_tbl[] for @node and return the old pwq */
+static struct pool_workqueue *numa_pwq_tbl_install(struct workqueue_struct *wq,
+ int node,
+ struct pool_workqueue *pwq)
+{
+ struct pool_workqueue *old_pwq;
- if (pool->detach_completion)
- wait_for_completion(pool->detach_completion);
+ lockdep_assert_held(&wq->mutex);
- mutex_unlock(&pool->manager_arb);
+ /* link_pwq() can handle duplicate calls */
+ link_pwq(pwq);
- /* shut down the timers */
- del_timer_sync(&pool->idle_timer);
- del_timer_sync(&pool->mayday_timer);
-
- /* sched-RCU protected to allow dereferences from get_work_pool() */
- call_rcu_sched(&pool->rcu, rcu_free_pool);
-}
+ old_pwq = rcu_access_pointer(wq->numa_pwq_tbl[node]);
+ rcu_assign_pointer(wq->numa_pwq_tbl[node], pwq);
+ return old_pwq;
+}
/**
- * get_unbound_pool - get a worker_pool with the specified attributes
- * @attrs: the attributes of the worker_pool to get
+ * apply_workqueue_attrs - apply new workqueue_attrs to an unbound workqueue
+ * @wq: the target workqueue
+ * @attrs: the workqueue_attrs to apply, allocated with alloc_workqueue_attrs()
*
- * Obtain a worker_pool which has the same attributes as @attrs, bump the
- * reference count and return it. If there already is a matching
- * worker_pool, it will be used; otherwise, this function attempts to
- * create a new one.
+ * Apply @attrs to an unbound workqueue @wq. Unless disabled, on NUMA
+ * machines, this function maps a separate pwq to each NUMA node with
+ * possibles CPUs in @attrs->cpumask so that work items are affine to the
+ * NUMA node it was issued on. Older pwqs are released as in-flight work
+ * items finish. Note that a work item which repeatedly requeues itself
+ * back-to-back will stay on its current pwq.
*
- * Should be called with wq_pool_mutex held.
+ * Performs GFP_KERNEL allocations.
*
- * Return: On success, a worker_pool with the same attributes as @attrs.
- * On failure, %NULL.
+ * Return: 0 on success and -errno on failure.
*/
-static struct worker_pool *get_unbound_pool(const struct workqueue_attrs *attrs)
+int apply_workqueue_attrs(struct workqueue_struct *wq,
+ const struct workqueue_attrs *attrs)
{
- u32 hash = wqattrs_hash(attrs);
- struct worker_pool *pool;
- int node;
+ struct workqueue_attrs *new_attrs, *tmp_attrs;
+ struct pool_workqueue **pwq_tbl, *dfl_pwq;
+ int node, ret;
- lockdep_assert_held(&wq_pool_mutex);
+ /* only unbound workqueues can change attributes */
+ if (WARN_ON(!(wq->flags & WQ_UNBOUND)))
+ return -EINVAL;
- /* do we already have a matching pool? */
- hash_for_each_possible(unbound_pool_hash, pool, hash_node, hash) {
- if (wqattrs_equal(pool->attrs, attrs)) {
- pool->refcnt++;
- return pool;
- }
- }
+ /* creating multiple pwqs breaks ordering guarantee */
+ if (WARN_ON((wq->flags & __WQ_ORDERED) && !list_empty(&wq->pwqs)))
+ return -EINVAL;
- /* nope, create a new one */
- pool = kzalloc(sizeof(*pool), GFP_KERNEL);
- if (!pool || init_worker_pool(pool) < 0)
- goto fail;
+ pwq_tbl = kzalloc(nr_node_ids * sizeof(pwq_tbl[0]), GFP_KERNEL);
+ new_attrs = alloc_workqueue_attrs(GFP_KERNEL);
+ tmp_attrs = alloc_workqueue_attrs(GFP_KERNEL);
+ if (!pwq_tbl || !new_attrs || !tmp_attrs)
+ goto enomem;
- lockdep_set_subclass(&pool->lock, 1); /* see put_pwq() */
- copy_workqueue_attrs(pool->attrs, attrs);
+ /* make a copy of @attrs and sanitize it */
+ copy_workqueue_attrs(new_attrs, attrs);
+ cpumask_and(new_attrs->cpumask, new_attrs->cpumask, cpu_possible_mask);
/*
- * no_numa isn't a worker_pool attribute, always clear it. See
- * 'struct workqueue_attrs' comments for detail.
+ * We may create multiple pwqs with differing cpumasks. Make a
+ * copy of @new_attrs which will be modified and used to obtain
+ * pools.
*/
- pool->attrs->no_numa = false;
+ copy_workqueue_attrs(tmp_attrs, new_attrs);
- /* if cpumask is contained inside a NUMA node, we belong to that node */
- if (wq_numa_enabled) {
- for_each_node(node) {
- if (cpumask_subset(pool->attrs->cpumask,
- wq_numa_possible_cpumask[node])) {
- pool->node = node;
- break;
- }
- }
- }
+ /*
+ * CPUs should stay stable across pwq creations and installations.
+ * Pin CPUs, determine the target cpumask for each node and create
+ * pwqs accordingly.
+ */
+ get_online_cpus();
- if (worker_pool_assign_id(pool) < 0)
- goto fail;
+ mutex_lock(&wq_pool_mutex);
- /* create and start the initial worker */
- if (!create_worker(pool))
- goto fail;
+ /*
+ * If something goes wrong during CPU up/down, we'll fall back to
+ * the default pwq covering whole @attrs->cpumask. Always create
+ * it even if we don't use it immediately.
+ */
+ dfl_pwq = alloc_unbound_pwq(wq, new_attrs);
+ if (!dfl_pwq)
+ goto enomem_pwq;
- /* install */
- hash_add(unbound_pool_hash, &pool->hash_node, hash);
+ for_each_node(node) {
+ if (wq_calc_node_cpumask(attrs, node, -1, tmp_attrs->cpumask)) {
+ pwq_tbl[node] = alloc_unbound_pwq(wq, tmp_attrs);
+ if (!pwq_tbl[node])
+ goto enomem_pwq;
+ } else {
+ dfl_pwq->refcnt++;
+ pwq_tbl[node] = dfl_pwq;
+ }
+ }
- return pool;
-fail:
- if (pool)
- put_unbound_pool(pool);
- return NULL;
-}
+ mutex_unlock(&wq_pool_mutex);
-static void rcu_free_pwq(struct rcu_head *rcu)
-{
- kmem_cache_free(pwq_cache,
- container_of(rcu, struct pool_workqueue, rcu));
-}
+ /* all pwqs have been created successfully, let's install'em */
+ mutex_lock(&wq->mutex);
-/*
- * Scheduled on system_wq by put_pwq() when an unbound pwq hits zero refcnt
- * and needs to be destroyed.
- */
-static void pwq_unbound_release_workfn(struct work_struct *work)
-{
- struct pool_workqueue *pwq = container_of(work, struct pool_workqueue,
- unbound_release_work);
- struct workqueue_struct *wq = pwq->wq;
- struct worker_pool *pool = pwq->pool;
- bool is_last;
+ copy_workqueue_attrs(wq->unbound_attrs, new_attrs);
- if (WARN_ON_ONCE(!(wq->flags & WQ_UNBOUND)))
- return;
+ /* save the previous pwq and install the new one */
+ for_each_node(node)
+ pwq_tbl[node] = numa_pwq_tbl_install(wq, node, pwq_tbl[node]);
+
+ /* @dfl_pwq might not have been used, ensure it's linked */
+ link_pwq(dfl_pwq);
+ swap(wq->dfl_pwq, dfl_pwq);
- mutex_lock(&wq->mutex);
- list_del_rcu(&pwq->pwqs_node);
- is_last = list_empty(&wq->pwqs);
mutex_unlock(&wq->mutex);
- mutex_lock(&wq_pool_mutex);
- put_unbound_pool(pool);
- mutex_unlock(&wq_pool_mutex);
+ /* put the old pwqs */
+ for_each_node(node)
+ put_pwq_unlocked(pwq_tbl[node]);
+ put_pwq_unlocked(dfl_pwq);
- call_rcu_sched(&pwq->rcu, rcu_free_pwq);
+ put_online_cpus();
+ ret = 0;
+ /* fall through */
+out_free:
+ free_workqueue_attrs(tmp_attrs);
+ free_workqueue_attrs(new_attrs);
+ kfree(pwq_tbl);
+ return ret;
- /*
- * If we're the last pwq going away, @wq is already dead and no one
- * is gonna access it anymore. Free it.
- */
- if (is_last) {
- free_workqueue_attrs(wq->unbound_attrs);
- kfree(wq);
- }
+enomem_pwq:
+ free_unbound_pwq(dfl_pwq);
+ for_each_node(node)
+ if (pwq_tbl && pwq_tbl[node] != dfl_pwq)
+ free_unbound_pwq(pwq_tbl[node]);
+ mutex_unlock(&wq_pool_mutex);
+ put_online_cpus();
+enomem:
+ ret = -ENOMEM;
+ goto out_free;
}
/**
- * pwq_adjust_max_active - update a pwq's max_active to the current setting
- * @pwq: target pool_workqueue
+ * wq_update_unbound_numa - update NUMA affinity of a wq for CPU hot[un]plug
+ * @wq: the target workqueue
+ * @cpu: the CPU coming up or going down
+ * @online: whether @cpu is coming up or going down
*
- * If @pwq isn't freezing, set @pwq->max_active to the associated
- * workqueue's saved_max_active and activate delayed work items
- * accordingly. If @pwq is freezing, clear @pwq->max_active to zero.
+ * This function is to be called from %CPU_DOWN_PREPARE, %CPU_ONLINE and
+ * %CPU_DOWN_FAILED. @cpu is being hot[un]plugged, update NUMA affinity of
+ * @wq accordingly.
+ *
+ * If NUMA affinity can't be adjusted due to memory allocation failure, it
+ * falls back to @wq->dfl_pwq which may not be optimal but is always
+ * correct.
+ *
+ * Note that when the last allowed CPU of a NUMA node goes offline for a
+ * workqueue with a cpumask spanning multiple nodes, the workers which were
+ * already executing the work items for the workqueue will lose their CPU
+ * affinity and may execute on any CPU. This is similar to how per-cpu
+ * workqueues behave on CPU_DOWN. If a workqueue user wants strict
+ * affinity, it's the user's responsibility to flush the work item from
+ * CPU_DOWN_PREPARE.
*/
-static void pwq_adjust_max_active(struct pool_workqueue *pwq)
+static void wq_update_unbound_numa(struct workqueue_struct *wq, int cpu,
+ bool online)
{
- struct workqueue_struct *wq = pwq->wq;
- bool freezable = wq->flags & WQ_FREEZABLE;
+ int node = cpu_to_node(cpu);
+ int cpu_off = online ? -1 : cpu;
+ struct pool_workqueue *old_pwq = NULL, *pwq;
+ struct workqueue_attrs *target_attrs;
+ cpumask_t *cpumask;
- /* for @wq->saved_max_active */
- lockdep_assert_held(&wq->mutex);
+ lockdep_assert_held(&wq_pool_mutex);
- /* fast exit for non-freezable wqs */
- if (!freezable && pwq->max_active == wq->saved_max_active)
+ if (!wq_numa_enabled || !(wq->flags & WQ_UNBOUND))
return;
- spin_lock_irq(&pwq->pool->lock);
-
/*
- * During [un]freezing, the caller is responsible for ensuring that
- * this function is called at least once after @workqueue_freezing
- * is updated and visible.
+ * We don't wanna alloc/free wq_attrs for each wq for each CPU.
+ * Let's use a preallocated one. The following buf is protected by
+ * CPU hotplug exclusion.
*/
- if (!freezable || !workqueue_freezing) {
- pwq->max_active = wq->saved_max_active;
+ target_attrs = wq_update_unbound_numa_attrs_buf;
+ cpumask = target_attrs->cpumask;
- while (!list_empty(&pwq->delayed_works) &&
- pwq->nr_active < pwq->max_active)
- pwq_activate_first_delayed(pwq);
+ mutex_lock(&wq->mutex);
+ if (wq->unbound_attrs->no_numa)
+ goto out_unlock;
- /*
- * Need to kick a worker after thawed or an unbound wq's
- * max_active is bumped. It's a slow path. Do it always.
- */
- wake_up_worker(pwq->pool);
+ copy_workqueue_attrs(target_attrs, wq->unbound_attrs);
+ pwq = unbound_pwq_by_node(wq, node);
+
+ /*
+ * Let's determine what needs to be done. If the target cpumask is
+ * different from wq's, we need to compare it to @pwq's and create
+ * a new one if they don't match. If the target cpumask equals
+ * wq's, the default pwq should be used.
+ */
+ if (wq_calc_node_cpumask(wq->unbound_attrs, node, cpu_off, cpumask)) {
+ if (cpumask_equal(cpumask, pwq->pool->attrs->cpumask))
+ goto out_unlock;
} else {
- pwq->max_active = 0;
+ goto use_dfl_pwq;
}
- spin_unlock_irq(&pwq->pool->lock);
-}
+ mutex_unlock(&wq->mutex);
-/* initialize newly alloced @pwq which is associated with @wq and @pool */
-static void init_pwq(struct pool_workqueue *pwq, struct workqueue_struct *wq,
- struct worker_pool *pool)
-{
- BUG_ON((unsigned long)pwq & WORK_STRUCT_FLAG_MASK);
+ /* create a new pwq */
+ pwq = alloc_unbound_pwq(wq, target_attrs);
+ if (!pwq) {
+ pr_warn("workqueue: allocation failed while updating NUMA affinity of \"%s\"\n",
+ wq->name);
+ mutex_lock(&wq->mutex);
+ goto use_dfl_pwq;
+ }
- memset(pwq, 0, sizeof(*pwq));
+ /*
+ * Install the new pwq. As this function is called only from CPU
+ * hotplug callbacks and applying a new attrs is wrapped with
+ * get/put_online_cpus(), @wq->unbound_attrs couldn't have changed
+ * inbetween.
+ */
+ mutex_lock(&wq->mutex);
+ old_pwq = numa_pwq_tbl_install(wq, node, pwq);
+ goto out_unlock;
- pwq->pool = pool;
- pwq->wq = wq;
- pwq->flush_color = -1;
- pwq->refcnt = 1;
- INIT_LIST_HEAD(&pwq->delayed_works);
- INIT_LIST_HEAD(&pwq->pwqs_node);
- INIT_LIST_HEAD(&pwq->mayday_node);
- INIT_WORK(&pwq->unbound_release_work, pwq_unbound_release_workfn);
+use_dfl_pwq:
+ spin_lock_irq(&wq->dfl_pwq->pool->lock);
+ get_pwq(wq->dfl_pwq);
+ spin_unlock_irq(&wq->dfl_pwq->pool->lock);
+ old_pwq = numa_pwq_tbl_install(wq, node, wq->dfl_pwq);
+out_unlock:
+ mutex_unlock(&wq->mutex);
+ put_pwq_unlocked(old_pwq);
}
-/* sync @pwq with the current state of its associated wq and link it */
-static void link_pwq(struct pool_workqueue *pwq)
+static int alloc_and_link_pwqs(struct workqueue_struct *wq)
{
- struct workqueue_struct *wq = pwq->wq;
+ bool highpri = wq->flags & WQ_HIGHPRI;
+ int cpu, ret;
- lockdep_assert_held(&wq->mutex);
+ if (!(wq->flags & WQ_UNBOUND)) {
+ wq->cpu_pwqs = alloc_percpu(struct pool_workqueue);
+ if (!wq->cpu_pwqs)
+ return -ENOMEM;
- /* may be called multiple times, ignore if already linked */
- if (!list_empty(&pwq->pwqs_node))
- return;
+ for_each_possible_cpu(cpu) {
+ struct pool_workqueue *pwq =
+ per_cpu_ptr(wq->cpu_pwqs, cpu);
+ struct worker_pool *cpu_pools =
+ per_cpu(cpu_worker_pools, cpu);
- /* set the matching work_color */
- pwq->work_color = wq->work_color;
+ init_pwq(pwq, wq, &cpu_pools[highpri]);
- /* sync max_active to the current setting */
- pwq_adjust_max_active(pwq);
+ mutex_lock(&wq->mutex);
+ link_pwq(pwq);
+ mutex_unlock(&wq->mutex);
+ }
+ return 0;
+ } else if (wq->flags & __WQ_ORDERED) {
+ ret = apply_workqueue_attrs(wq, ordered_wq_attrs[highpri]);
+ /* there should only be single pwq for ordering guarantee */
+ WARN(!ret && (wq->pwqs.next != &wq->dfl_pwq->pwqs_node ||
+ wq->pwqs.prev != &wq->dfl_pwq->pwqs_node),
+ "ordering guarantee broken for workqueue %s\n", wq->name);
+ return ret;
+ } else {
+ return apply_workqueue_attrs(wq, unbound_std_wq_attrs[highpri]);
+ }
+}
- /* link in @pwq */
- list_add_rcu(&pwq->pwqs_node, &wq->pwqs);
+static int wq_clamp_max_active(int max_active, unsigned int flags,
+ const char *name)
+{
+ int lim = flags & WQ_UNBOUND ? WQ_UNBOUND_MAX_ACTIVE : WQ_MAX_ACTIVE;
+
+ if (max_active < 1 || max_active > lim)
+ pr_warn("workqueue: max_active %d requested for %s is out of range, clamping between %d and %d\n",
+ max_active, name, 1, lim);
+
+ return clamp_val(max_active, 1, lim);
}
-/* obtain a pool matching @attr and create a pwq associating the pool and @wq */
-static struct pool_workqueue *alloc_unbound_pwq(struct workqueue_struct *wq,
- const struct workqueue_attrs *attrs)
+struct workqueue_struct *__alloc_workqueue_key(const char *fmt,
+ unsigned int flags,
+ int max_active,
+ struct lock_class_key *key,
+ const char *lock_name, ...)
{
- struct worker_pool *pool;
+ size_t tbl_size = 0;
+ va_list args;
+ struct workqueue_struct *wq;
struct pool_workqueue *pwq;
- lockdep_assert_held(&wq_pool_mutex);
+ /* see the comment above the definition of WQ_POWER_EFFICIENT */
+ if ((flags & WQ_POWER_EFFICIENT) && wq_power_efficient)
+ flags |= WQ_UNBOUND;
- pool = get_unbound_pool(attrs);
- if (!pool)
- return NULL;
+ /* allocate wq and format name */
+ if (flags & WQ_UNBOUND)
+ tbl_size = nr_node_ids * sizeof(wq->numa_pwq_tbl[0]);
- pwq = kmem_cache_alloc_node(pwq_cache, GFP_KERNEL, pool->node);
- if (!pwq) {
- put_unbound_pool(pool);
+ wq = kzalloc(sizeof(*wq) + tbl_size, GFP_KERNEL);
+ if (!wq)
return NULL;
+
+ if (flags & WQ_UNBOUND) {
+ wq->unbound_attrs = alloc_workqueue_attrs(GFP_KERNEL);
+ if (!wq->unbound_attrs)
+ goto err_free_wq;
}
- init_pwq(pwq, wq, pool);
- return pwq;
-}
+ va_start(args, lock_name);
+ vsnprintf(wq->name, sizeof(wq->name), fmt, args);
+ va_end(args);
-/* undo alloc_unbound_pwq(), used only in the error path */
-static void free_unbound_pwq(struct pool_workqueue *pwq)
-{
- lockdep_assert_held(&wq_pool_mutex);
+ max_active = max_active ?: WQ_DFL_ACTIVE;
+ max_active = wq_clamp_max_active(max_active, flags, wq->name);
- if (pwq) {
- put_unbound_pool(pwq->pool);
- kmem_cache_free(pwq_cache, pwq);
- }
-}
+ /* init wq */
+ wq->flags = flags;
+ wq->saved_max_active = max_active;
+ mutex_init(&wq->mutex);
+ atomic_set(&wq->nr_pwqs_to_flush, 0);
+ INIT_LIST_HEAD(&wq->pwqs);
+ INIT_LIST_HEAD(&wq->flusher_queue);
+ INIT_LIST_HEAD(&wq->flusher_overflow);
+ INIT_LIST_HEAD(&wq->maydays);
-/**
- * wq_calc_node_mask - calculate a wq_attrs' cpumask for the specified node
- * @attrs: the wq_attrs of interest
- * @node: the target NUMA node
- * @cpu_going_down: if >= 0, the CPU to consider as offline
- * @cpumask: outarg, the resulting cpumask
- *
- * Calculate the cpumask a workqueue with @attrs should use on @node. If
- * @cpu_going_down is >= 0, that cpu is considered offline during
- * calculation. The result is stored in @cpumask.
- *
- * If NUMA affinity is not enabled, @attrs->cpumask is always used. If
- * enabled and @node has online CPUs requested by @attrs, the returned
- * cpumask is the intersection of the possible CPUs of @node and
- * @attrs->cpumask.
- *
- * The caller is responsible for ensuring that the cpumask of @node stays
- * stable.
- *
- * Return: %true if the resulting @cpumask is different from @attrs->cpumask,
- * %false if equal.
- */
-static bool wq_calc_node_cpumask(const struct workqueue_attrs *attrs, int node,
- int cpu_going_down, cpumask_t *cpumask)
-{
- if (!wq_numa_enabled || attrs->no_numa)
- goto use_dfl;
+ lockdep_init_map(&wq->lockdep_map, lock_name, key, 0);
+ INIT_LIST_HEAD(&wq->list);
- /* does @node have any online CPUs @attrs wants? */
- cpumask_and(cpumask, cpumask_of_node(node), attrs->cpumask);
- if (cpu_going_down >= 0)
- cpumask_clear_cpu(cpu_going_down, cpumask);
+ if (alloc_and_link_pwqs(wq) < 0)
+ goto err_free_wq;
- if (cpumask_empty(cpumask))
- goto use_dfl;
+ /*
+ * Workqueues which may be used during memory reclaim should
+ * have a rescuer to guarantee forward progress.
+ */
+ if (flags & WQ_MEM_RECLAIM) {
+ struct worker *rescuer;
- /* yeap, return possible CPUs in @node that @attrs wants */
- cpumask_and(cpumask, attrs->cpumask, wq_numa_possible_cpumask[node]);
- return !cpumask_equal(cpumask, attrs->cpumask);
+ rescuer = alloc_worker(NUMA_NO_NODE);
+ if (!rescuer)
+ goto err_destroy;
-use_dfl:
- cpumask_copy(cpumask, attrs->cpumask);
- return false;
-}
+ rescuer->rescue_wq = wq;
+ rescuer->task = kthread_create(rescuer_thread, rescuer, "%s",
+ wq->name);
+ if (IS_ERR(rescuer->task)) {
+ kfree(rescuer);
+ goto err_destroy;
+ }
-/* install @pwq into @wq's numa_pwq_tbl[] for @node and return the old pwq */
-static struct pool_workqueue *numa_pwq_tbl_install(struct workqueue_struct *wq,
- int node,
- struct pool_workqueue *pwq)
-{
- struct pool_workqueue *old_pwq;
+ wq->rescuer = rescuer;
+ rescuer->task->flags |= PF_NO_SETAFFINITY;
+ wake_up_process(rescuer->task);
+ }
- lockdep_assert_held(&wq->mutex);
+ if ((wq->flags & WQ_SYSFS) && workqueue_sysfs_register(wq))
+ goto err_destroy;
- /* link_pwq() can handle duplicate calls */
- link_pwq(pwq);
+ /*
+ * wq_pool_mutex protects global freeze state and workqueues list.
+ * Grab it, adjust max_active and add the new @wq to workqueues
+ * list.
+ */
+ mutex_lock(&wq_pool_mutex);
- old_pwq = rcu_access_pointer(wq->numa_pwq_tbl[node]);
- rcu_assign_pointer(wq->numa_pwq_tbl[node], pwq);
- return old_pwq;
+ mutex_lock(&wq->mutex);
+ for_each_pwq(pwq, wq)
+ pwq_adjust_max_active(pwq);
+ mutex_unlock(&wq->mutex);
+
+ list_add_tail_rcu(&wq->list, &workqueues);
+
+ mutex_unlock(&wq_pool_mutex);
+
+ return wq;
+
+err_free_wq:
+ free_workqueue_attrs(wq->unbound_attrs);
+ kfree(wq);
+ return NULL;
+err_destroy:
+ destroy_workqueue(wq);
+ return NULL;
}
+EXPORT_SYMBOL_GPL(__alloc_workqueue_key);
/**
- * apply_workqueue_attrs - apply new workqueue_attrs to an unbound workqueue
- * @wq: the target workqueue
- * @attrs: the workqueue_attrs to apply, allocated with alloc_workqueue_attrs()
- *
- * Apply @attrs to an unbound workqueue @wq. Unless disabled, on NUMA
- * machines, this function maps a separate pwq to each NUMA node with
- * possibles CPUs in @attrs->cpumask so that work items are affine to the
- * NUMA node it was issued on. Older pwqs are released as in-flight work
- * items finish. Note that a work item which repeatedly requeues itself
- * back-to-back will stay on its current pwq.
- *
- * Performs GFP_KERNEL allocations.
+ * destroy_workqueue - safely terminate a workqueue
+ * @wq: target workqueue
*
- * Return: 0 on success and -errno on failure.
+ * Safely destroy a workqueue. All work currently pending will be done first.
*/
-int apply_workqueue_attrs(struct workqueue_struct *wq,
- const struct workqueue_attrs *attrs)
+void destroy_workqueue(struct workqueue_struct *wq)
{
- struct workqueue_attrs *new_attrs, *tmp_attrs;
- struct pool_workqueue **pwq_tbl, *dfl_pwq;
- int node, ret;
-
- /* only unbound workqueues can change attributes */
- if (WARN_ON(!(wq->flags & WQ_UNBOUND)))
- return -EINVAL;
+ struct pool_workqueue *pwq;
+ int node;
- /* creating multiple pwqs breaks ordering guarantee */
- if (WARN_ON((wq->flags & __WQ_ORDERED) && !list_empty(&wq->pwqs)))
- return -EINVAL;
+ /* drain it before proceeding with destruction */
+ drain_workqueue(wq);
- pwq_tbl = kzalloc(nr_node_ids * sizeof(pwq_tbl[0]), GFP_KERNEL);
- new_attrs = alloc_workqueue_attrs(GFP_KERNEL);
- tmp_attrs = alloc_workqueue_attrs(GFP_KERNEL);
- if (!pwq_tbl || !new_attrs || !tmp_attrs)
- goto enomem;
+ /* sanity checks */
+ mutex_lock(&wq->mutex);
+ for_each_pwq(pwq, wq) {
+ int i;
- /* make a copy of @attrs and sanitize it */
- copy_workqueue_attrs(new_attrs, attrs);
- cpumask_and(new_attrs->cpumask, new_attrs->cpumask, cpu_possible_mask);
+ for (i = 0; i < WORK_NR_COLORS; i++) {
+ if (WARN_ON(pwq->nr_in_flight[i])) {
+ mutex_unlock(&wq->mutex);
+ return;
+ }
+ }
- /*
- * We may create multiple pwqs with differing cpumasks. Make a
- * copy of @new_attrs which will be modified and used to obtain
- * pools.
- */
- copy_workqueue_attrs(tmp_attrs, new_attrs);
+ if (WARN_ON((pwq != wq->dfl_pwq) && (pwq->refcnt > 1)) ||
+ WARN_ON(pwq->nr_active) ||
+ WARN_ON(!list_empty(&pwq->delayed_works))) {
+ mutex_unlock(&wq->mutex);
+ return;
+ }
+ }
+ mutex_unlock(&wq->mutex);
/*
- * CPUs should stay stable across pwq creations and installations.
- * Pin CPUs, determine the target cpumask for each node and create
- * pwqs accordingly.
+ * wq list is used to freeze wq, remove from list after
+ * flushing is complete in case freeze races us.
*/
- get_online_cpus();
-
mutex_lock(&wq_pool_mutex);
+ list_del_rcu(&wq->list);
+ mutex_unlock(&wq_pool_mutex);
- /*
- * If something goes wrong during CPU up/down, we'll fall back to
- * the default pwq covering whole @attrs->cpumask. Always create
- * it even if we don't use it immediately.
- */
- dfl_pwq = alloc_unbound_pwq(wq, new_attrs);
- if (!dfl_pwq)
- goto enomem_pwq;
+ workqueue_sysfs_unregister(wq);
- for_each_node(node) {
- if (wq_calc_node_cpumask(attrs, node, -1, tmp_attrs->cpumask)) {
- pwq_tbl[node] = alloc_unbound_pwq(wq, tmp_attrs);
- if (!pwq_tbl[node])
- goto enomem_pwq;
- } else {
- dfl_pwq->refcnt++;
- pwq_tbl[node] = dfl_pwq;
+ if (wq->rescuer)
+ kthread_stop(wq->rescuer->task);
+
+ if (!(wq->flags & WQ_UNBOUND)) {
+ /*
+ * The base ref is never dropped on per-cpu pwqs. Directly
+ * schedule RCU free.
+ */
+ call_rcu_sched(&wq->rcu, rcu_free_wq);
+ } else {
+ /*
+ * We're the sole accessor of @wq at this point. Directly
+ * access numa_pwq_tbl[] and dfl_pwq to put the base refs.
+ * @wq will be freed when the last pwq is released.
+ */
+ for_each_node(node) {
+ pwq = rcu_access_pointer(wq->numa_pwq_tbl[node]);
+ RCU_INIT_POINTER(wq->numa_pwq_tbl[node], NULL);
+ put_pwq_unlocked(pwq);
}
+
+ /*
+ * Put dfl_pwq. @wq may be freed any time after dfl_pwq is
+ * put. Don't access it afterwards.
+ */
+ pwq = wq->dfl_pwq;
+ wq->dfl_pwq = NULL;
+ put_pwq_unlocked(pwq);
}
+}
+EXPORT_SYMBOL_GPL(destroy_workqueue);
- mutex_unlock(&wq_pool_mutex);
+/**
+ * workqueue_set_max_active - adjust max_active of a workqueue
+ * @wq: target workqueue
+ * @max_active: new max_active value.
+ *
+ * Set max_active of @wq to @max_active.
+ *
+ * CONTEXT:
+ * Don't call from IRQ context.
+ */
+void workqueue_set_max_active(struct workqueue_struct *wq, int max_active)
+{
+ struct pool_workqueue *pwq;
- /* all pwqs have been created successfully, let's install'em */
- mutex_lock(&wq->mutex);
+ /* disallow meddling with max_active for ordered workqueues */
+ if (WARN_ON(wq->flags & __WQ_ORDERED))
+ return;
- copy_workqueue_attrs(wq->unbound_attrs, new_attrs);
+ max_active = wq_clamp_max_active(max_active, wq->flags, wq->name);
- /* save the previous pwq and install the new one */
- for_each_node(node)
- pwq_tbl[node] = numa_pwq_tbl_install(wq, node, pwq_tbl[node]);
+ mutex_lock(&wq->mutex);
- /* @dfl_pwq might not have been used, ensure it's linked */
- link_pwq(dfl_pwq);
- swap(wq->dfl_pwq, dfl_pwq);
+ wq->saved_max_active = max_active;
- mutex_unlock(&wq->mutex);
+ for_each_pwq(pwq, wq)
+ pwq_adjust_max_active(pwq);
- /* put the old pwqs */
- for_each_node(node)
- put_pwq_unlocked(pwq_tbl[node]);
- put_pwq_unlocked(dfl_pwq);
+ mutex_unlock(&wq->mutex);
+}
+EXPORT_SYMBOL_GPL(workqueue_set_max_active);
- put_online_cpus();
- ret = 0;
- /* fall through */
-out_free:
- free_workqueue_attrs(tmp_attrs);
- free_workqueue_attrs(new_attrs);
- kfree(pwq_tbl);
- return ret;
+/**
+ * current_is_workqueue_rescuer - is %current workqueue rescuer?
+ *
+ * Determine whether %current is a workqueue rescuer. Can be used from
+ * work functions to determine whether it's being run off the rescuer task.
+ *
+ * Return: %true if %current is a workqueue rescuer. %false otherwise.
+ */
+bool current_is_workqueue_rescuer(void)
+{
+ struct worker *worker = current_wq_worker();
-enomem_pwq:
- free_unbound_pwq(dfl_pwq);
- for_each_node(node)
- if (pwq_tbl && pwq_tbl[node] != dfl_pwq)
- free_unbound_pwq(pwq_tbl[node]);
- mutex_unlock(&wq_pool_mutex);
- put_online_cpus();
-enomem:
- ret = -ENOMEM;
- goto out_free;
+ return worker && worker->rescue_wq;
}
/**
- * wq_update_unbound_numa - update NUMA affinity of a wq for CPU hot[un]plug
- * @wq: the target workqueue
- * @cpu: the CPU coming up or going down
- * @online: whether @cpu is coming up or going down
+ * workqueue_congested - test whether a workqueue is congested
+ * @cpu: CPU in question
+ * @wq: target workqueue
*
- * This function is to be called from %CPU_DOWN_PREPARE, %CPU_ONLINE and
- * %CPU_DOWN_FAILED. @cpu is being hot[un]plugged, update NUMA affinity of
- * @wq accordingly.
+ * Test whether @wq's cpu workqueue for @cpu is congested. There is
+ * no synchronization around this function and the test result is
+ * unreliable and only useful as advisory hints or for debugging.
*
- * If NUMA affinity can't be adjusted due to memory allocation failure, it
- * falls back to @wq->dfl_pwq which may not be optimal but is always
- * correct.
+ * If @cpu is WORK_CPU_UNBOUND, the test is performed on the local CPU.
+ * Note that both per-cpu and unbound workqueues may be associated with
+ * multiple pool_workqueues which have separate congested states. A
+ * workqueue being congested on one CPU doesn't mean the workqueue is also
+ * contested on other CPUs / NUMA nodes.
*
- * Note that when the last allowed CPU of a NUMA node goes offline for a
- * workqueue with a cpumask spanning multiple nodes, the workers which were
- * already executing the work items for the workqueue will lose their CPU
- * affinity and may execute on any CPU. This is similar to how per-cpu
- * workqueues behave on CPU_DOWN. If a workqueue user wants strict
- * affinity, it's the user's responsibility to flush the work item from
- * CPU_DOWN_PREPARE.
+ * Return:
+ * %true if congested, %false otherwise.
*/
-static void wq_update_unbound_numa(struct workqueue_struct *wq, int cpu,
- bool online)
+bool workqueue_congested(int cpu, struct workqueue_struct *wq)
{
- int node = cpu_to_node(cpu);
- int cpu_off = online ? -1 : cpu;
- struct pool_workqueue *old_pwq = NULL, *pwq;
- struct workqueue_attrs *target_attrs;
- cpumask_t *cpumask;
+ struct pool_workqueue *pwq;
+ bool ret;
- lockdep_assert_held(&wq_pool_mutex);
+ rcu_read_lock_sched();
- if (!wq_numa_enabled || !(wq->flags & WQ_UNBOUND))
- return;
+ if (cpu == WORK_CPU_UNBOUND)
+ cpu = smp_processor_id();
- /*
- * We don't wanna alloc/free wq_attrs for each wq for each CPU.
- * Let's use a preallocated one. The following buf is protected by
- * CPU hotplug exclusion.
- */
- target_attrs = wq_update_unbound_numa_attrs_buf;
- cpumask = target_attrs->cpumask;
+ if (!(wq->flags & WQ_UNBOUND))
+ pwq = per_cpu_ptr(wq->cpu_pwqs, cpu);
+ else
+ pwq = unbound_pwq_by_node(wq, cpu_to_node(cpu));
- mutex_lock(&wq->mutex);
- if (wq->unbound_attrs->no_numa)
- goto out_unlock;
+ ret = !list_empty(&pwq->delayed_works);
+ rcu_read_unlock_sched();
- copy_workqueue_attrs(target_attrs, wq->unbound_attrs);
- pwq = unbound_pwq_by_node(wq, node);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(workqueue_congested);
- /*
- * Let's determine what needs to be done. If the target cpumask is
- * different from wq's, we need to compare it to @pwq's and create
- * a new one if they don't match. If the target cpumask equals
- * wq's, the default pwq should be used.
- */
- if (wq_calc_node_cpumask(wq->unbound_attrs, node, cpu_off, cpumask)) {
- if (cpumask_equal(cpumask, pwq->pool->attrs->cpumask))
- goto out_unlock;
- } else {
- goto use_dfl_pwq;
- }
+/**
+ * work_busy - test whether a work is currently pending or running
+ * @work: the work to be tested
+ *
+ * Test whether @work is currently pending or running. There is no
+ * synchronization around this function and the test result is
+ * unreliable and only useful as advisory hints or for debugging.
+ *
+ * Return:
+ * OR'd bitmask of WORK_BUSY_* bits.
+ */
+unsigned int work_busy(struct work_struct *work)
+{
+ struct worker_pool *pool;
+ unsigned long flags;
+ unsigned int ret = 0;
- mutex_unlock(&wq->mutex);
+ if (work_pending(work))
+ ret |= WORK_BUSY_PENDING;
- /* create a new pwq */
- pwq = alloc_unbound_pwq(wq, target_attrs);
- if (!pwq) {
- pr_warn("workqueue: allocation failed while updating NUMA affinity of \"%s\"\n",
- wq->name);
- mutex_lock(&wq->mutex);
- goto use_dfl_pwq;
+ local_irq_save(flags);
+ pool = get_work_pool(work);
+ if (pool) {
+ spin_lock(&pool->lock);
+ if (find_worker_executing_work(pool, work))
+ ret |= WORK_BUSY_RUNNING;
+ spin_unlock(&pool->lock);
}
+ local_irq_restore(flags);
- /*
- * Install the new pwq. As this function is called only from CPU
- * hotplug callbacks and applying a new attrs is wrapped with
- * get/put_online_cpus(), @wq->unbound_attrs couldn't have changed
- * inbetween.
- */
- mutex_lock(&wq->mutex);
- old_pwq = numa_pwq_tbl_install(wq, node, pwq);
- goto out_unlock;
+ return ret;
+}
+EXPORT_SYMBOL_GPL(work_busy);
-use_dfl_pwq:
- spin_lock_irq(&wq->dfl_pwq->pool->lock);
- get_pwq(wq->dfl_pwq);
- spin_unlock_irq(&wq->dfl_pwq->pool->lock);
- old_pwq = numa_pwq_tbl_install(wq, node, wq->dfl_pwq);
-out_unlock:
- mutex_unlock(&wq->mutex);
- put_pwq_unlocked(old_pwq);
+/**
+ * set_worker_desc - set description for the current work item
+ * @fmt: printf-style format string
+ * @...: arguments for the format string
+ *
+ * This function can be called by a running work function to describe what
+ * the work item is about. If the worker task gets dumped, this
+ * information will be printed out together to help debugging. The
+ * description can be at most WORKER_DESC_LEN including the trailing '\0'.
+ */
+void set_worker_desc(const char *fmt, ...)
+{
+ struct worker *worker = current_wq_worker();
+ va_list args;
+
+ if (worker) {
+ va_start(args, fmt);
+ vsnprintf(worker->desc, sizeof(worker->desc), fmt, args);
+ va_end(args);
+ worker->desc_valid = true;
+ }
}
-static int alloc_and_link_pwqs(struct workqueue_struct *wq)
+/**
+ * print_worker_info - print out worker information and description
+ * @log_lvl: the log level to use when printing
+ * @task: target task
+ *
+ * If @task is a worker and currently executing a work item, print out the
+ * name of the workqueue being serviced and worker description set with
+ * set_worker_desc() by the currently executing work item.
+ *
+ * This function can be safely called on any task as long as the
+ * task_struct itself is accessible. While safe, this function isn't
+ * synchronized and may print out mixups or garbages of limited length.
+ */
+void print_worker_info(const char *log_lvl, struct task_struct *task)
{
- bool highpri = wq->flags & WQ_HIGHPRI;
- int cpu, ret;
+ work_func_t *fn = NULL;
+ char name[WQ_NAME_LEN] = { };
+ char desc[WORKER_DESC_LEN] = { };
+ struct pool_workqueue *pwq = NULL;
+ struct workqueue_struct *wq = NULL;
+ bool desc_valid = false;
+ struct worker *worker;
- if (!(wq->flags & WQ_UNBOUND)) {
- wq->cpu_pwqs = alloc_percpu(struct pool_workqueue);
- if (!wq->cpu_pwqs)
- return -ENOMEM;
+ if (!(task->flags & PF_WQ_WORKER))
+ return;
- for_each_possible_cpu(cpu) {
- struct pool_workqueue *pwq =
- per_cpu_ptr(wq->cpu_pwqs, cpu);
- struct worker_pool *cpu_pools =
- per_cpu(cpu_worker_pools, cpu);
+ /*
+ * This function is called without any synchronization and @task
+ * could be in any state. Be careful with dereferences.
+ */
+ worker = probe_kthread_data(task);
- init_pwq(pwq, wq, &cpu_pools[highpri]);
+ /*
+ * Carefully copy the associated workqueue's workfn and name. Keep
+ * the original last '\0' in case the original contains garbage.
+ */
+ probe_kernel_read(&fn, &worker->current_func, sizeof(fn));
+ probe_kernel_read(&pwq, &worker->current_pwq, sizeof(pwq));
+ probe_kernel_read(&wq, &pwq->wq, sizeof(wq));
+ probe_kernel_read(name, wq->name, sizeof(name) - 1);
- mutex_lock(&wq->mutex);
- link_pwq(pwq);
- mutex_unlock(&wq->mutex);
- }
- return 0;
- } else if (wq->flags & __WQ_ORDERED) {
- ret = apply_workqueue_attrs(wq, ordered_wq_attrs[highpri]);
- /* there should only be single pwq for ordering guarantee */
- WARN(!ret && (wq->pwqs.next != &wq->dfl_pwq->pwqs_node ||
- wq->pwqs.prev != &wq->dfl_pwq->pwqs_node),
- "ordering guarantee broken for workqueue %s\n", wq->name);
- return ret;
- } else {
- return apply_workqueue_attrs(wq, unbound_std_wq_attrs[highpri]);
+ /* copy worker description */
+ probe_kernel_read(&desc_valid, &worker->desc_valid, sizeof(desc_valid));
+ if (desc_valid)
+ probe_kernel_read(desc, worker->desc, sizeof(desc) - 1);
+
+ if (fn || name[0] || desc[0]) {
+ printk("%sWorkqueue: %s %pf", log_lvl, name, fn);
+ if (desc[0])
+ pr_cont(" (%s)", desc);
+ pr_cont("\n");
}
}
-static int wq_clamp_max_active(int max_active, unsigned int flags,
- const char *name)
+static void pr_cont_pool_info(struct worker_pool *pool)
{
- int lim = flags & WQ_UNBOUND ? WQ_UNBOUND_MAX_ACTIVE : WQ_MAX_ACTIVE;
+ pr_cont(" cpus=%*pbl", nr_cpumask_bits, pool->attrs->cpumask);
+ if (pool->node != NUMA_NO_NODE)
+ pr_cont(" node=%d", pool->node);
+ pr_cont(" flags=0x%x nice=%d", pool->flags, pool->attrs->nice);
+}
- if (max_active < 1 || max_active > lim)
- pr_warn("workqueue: max_active %d requested for %s is out of range, clamping between %d and %d\n",
- max_active, name, 1, lim);
+static void pr_cont_work(bool comma, struct work_struct *work)
+{
+ if (work->func == wq_barrier_func) {
+ struct wq_barrier *barr;
- return clamp_val(max_active, 1, lim);
+ barr = container_of(work, struct wq_barrier, work);
+
+ pr_cont("%s BAR(%d)", comma ? "," : "",
+ task_pid_nr(barr->task));
+ } else {
+ pr_cont("%s %pf", comma ? "," : "", work->func);
+ }
}
-struct workqueue_struct *__alloc_workqueue_key(const char *fmt,
- unsigned int flags,
- int max_active,
- struct lock_class_key *key,
- const char *lock_name, ...)
+static void show_pwq(struct pool_workqueue *pwq)
{
- size_t tbl_size = 0;
- va_list args;
- struct workqueue_struct *wq;
- struct pool_workqueue *pwq;
-
- /* see the comment above the definition of WQ_POWER_EFFICIENT */
- if ((flags & WQ_POWER_EFFICIENT) && wq_power_efficient)
- flags |= WQ_UNBOUND;
+ struct worker_pool *pool = pwq->pool;
+ struct work_struct *work;
+ struct worker *worker;
+ bool has_in_flight = false, has_pending = false;
+ int bkt;
- /* allocate wq and format name */
- if (flags & WQ_UNBOUND)
- tbl_size = nr_node_ids * sizeof(wq->numa_pwq_tbl[0]);
+ pr_info(" pwq %d:", pool->id);
+ pr_cont_pool_info(pool);
- wq = kzalloc(sizeof(*wq) + tbl_size, GFP_KERNEL);
- if (!wq)
- return NULL;
+ pr_cont(" active=%d/%d%s\n", pwq->nr_active, pwq->max_active,
+ !list_empty(&pwq->mayday_node) ? " MAYDAY" : "");
- if (flags & WQ_UNBOUND) {
- wq->unbound_attrs = alloc_workqueue_attrs(GFP_KERNEL);
- if (!wq->unbound_attrs)
- goto err_free_wq;
+ hash_for_each(pool->busy_hash, bkt, worker, hentry) {
+ if (worker->current_pwq == pwq) {
+ has_in_flight = true;
+ break;
+ }
}
+ if (has_in_flight) {
+ bool comma = false;
- va_start(args, lock_name);
- vsnprintf(wq->name, sizeof(wq->name), fmt, args);
- va_end(args);
-
- max_active = max_active ?: WQ_DFL_ACTIVE;
- max_active = wq_clamp_max_active(max_active, flags, wq->name);
+ pr_info(" in-flight:");
+ hash_for_each(pool->busy_hash, bkt, worker, hentry) {
+ if (worker->current_pwq != pwq)
+ continue;
- /* init wq */
- wq->flags = flags;
- wq->saved_max_active = max_active;
- mutex_init(&wq->mutex);
- atomic_set(&wq->nr_pwqs_to_flush, 0);
- INIT_LIST_HEAD(&wq->pwqs);
- INIT_LIST_HEAD(&wq->flusher_queue);
- INIT_LIST_HEAD(&wq->flusher_overflow);
- INIT_LIST_HEAD(&wq->maydays);
+ pr_cont("%s %d%s:%pf", comma ? "," : "",
+ task_pid_nr(worker->task),
+ worker == pwq->wq->rescuer ? "(RESCUER)" : "",
+ worker->current_func);
+ list_for_each_entry(work, &worker->scheduled, entry)
+ pr_cont_work(false, work);
+ comma = true;
+ }
+ pr_cont("\n");
+ }
- lockdep_init_map(&wq->lockdep_map, lock_name, key, 0);
- INIT_LIST_HEAD(&wq->list);
+ list_for_each_entry(work, &pool->worklist, entry) {
+ if (get_work_pwq(work) == pwq) {
+ has_pending = true;
+ break;
+ }
+ }
+ if (has_pending) {
+ bool comma = false;
- if (alloc_and_link_pwqs(wq) < 0)
- goto err_free_wq;
+ pr_info(" pending:");
+ list_for_each_entry(work, &pool->worklist, entry) {
+ if (get_work_pwq(work) != pwq)
+ continue;
- /*
- * Workqueues which may be used during memory reclaim should
- * have a rescuer to guarantee forward progress.
- */
- if (flags & WQ_MEM_RECLAIM) {
- struct worker *rescuer;
+ pr_cont_work(comma, work);
+ comma = !(*work_data_bits(work) & WORK_STRUCT_LINKED);
+ }
+ pr_cont("\n");
+ }
- rescuer = alloc_worker(NUMA_NO_NODE);
- if (!rescuer)
- goto err_destroy;
+ if (!list_empty(&pwq->delayed_works)) {
+ bool comma = false;
- rescuer->rescue_wq = wq;
- rescuer->task = kthread_create(rescuer_thread, rescuer, "%s",
- wq->name);
- if (IS_ERR(rescuer->task)) {
- kfree(rescuer);
- goto err_destroy;
+ pr_info(" delayed:");
+ list_for_each_entry(work, &pwq->delayed_works, entry) {
+ pr_cont_work(comma, work);
+ comma = !(*work_data_bits(work) & WORK_STRUCT_LINKED);
}
-
- wq->rescuer = rescuer;
- rescuer->task->flags |= PF_NO_SETAFFINITY;
- wake_up_process(rescuer->task);
+ pr_cont("\n");
}
+}
- if ((wq->flags & WQ_SYSFS) && workqueue_sysfs_register(wq))
- goto err_destroy;
+/**
+ * show_workqueue_state - dump workqueue state
+ *
+ * Called from a sysrq handler and prints out all busy workqueues and
+ * pools.
+ */
+void show_workqueue_state(void)
+{
+ struct workqueue_struct *wq;
+ struct worker_pool *pool;
+ unsigned long flags;
+ int pi;
- /*
- * wq_pool_mutex protects global freeze state and workqueues list.
- * Grab it, adjust max_active and add the new @wq to workqueues
- * list.
- */
- mutex_lock(&wq_pool_mutex);
+ rcu_read_lock_sched();
- mutex_lock(&wq->mutex);
- for_each_pwq(pwq, wq)
- pwq_adjust_max_active(pwq);
- mutex_unlock(&wq->mutex);
+ pr_info("Showing busy workqueues and worker pools:\n");
- list_add(&wq->list, &workqueues);
+ list_for_each_entry_rcu(wq, &workqueues, list) {
+ struct pool_workqueue *pwq;
+ bool idle = true;
- mutex_unlock(&wq_pool_mutex);
+ for_each_pwq(pwq, wq) {
+ if (pwq->nr_active || !list_empty(&pwq->delayed_works)) {
+ idle = false;
+ break;
+ }
+ }
+ if (idle)
+ continue;
- return wq;
+ pr_info("workqueue %s: flags=0x%x\n", wq->name, wq->flags);
-err_free_wq:
- free_workqueue_attrs(wq->unbound_attrs);
- kfree(wq);
- return NULL;
-err_destroy:
- destroy_workqueue(wq);
- return NULL;
+ for_each_pwq(pwq, wq) {
+ spin_lock_irqsave(&pwq->pool->lock, flags);
+ if (pwq->nr_active || !list_empty(&pwq->delayed_works))
+ show_pwq(pwq);
+ spin_unlock_irqrestore(&pwq->pool->lock, flags);
+ }
+ }
+
+ for_each_pool(pool, pi) {
+ struct worker *worker;
+ bool first = true;
+
+ spin_lock_irqsave(&pool->lock, flags);
+ if (pool->nr_workers == pool->nr_idle)
+ goto next_pool;
+
+ pr_info("pool %d:", pool->id);
+ pr_cont_pool_info(pool);
+ pr_cont(" workers=%d", pool->nr_workers);
+ if (pool->manager)
+ pr_cont(" manager: %d",
+ task_pid_nr(pool->manager->task));
+ list_for_each_entry(worker, &pool->idle_list, entry) {
+ pr_cont(" %s%d", first ? "idle: " : "",
+ task_pid_nr(worker->task));
+ first = false;
+ }
+ pr_cont("\n");
+ next_pool:
+ spin_unlock_irqrestore(&pool->lock, flags);
+ }
+
+ rcu_read_unlock_sched();
}
-EXPORT_SYMBOL_GPL(__alloc_workqueue_key);
-/**
- * destroy_workqueue - safely terminate a workqueue
- * @wq: target workqueue
+/*
+ * CPU hotplug.
*
- * Safely destroy a workqueue. All work currently pending will be done first.
+ * There are two challenges in supporting CPU hotplug. Firstly, there
+ * are a lot of assumptions on strong associations among work, pwq and
+ * pool which make migrating pending and scheduled works very
+ * difficult to implement without impacting hot paths. Secondly,
+ * worker pools serve mix of short, long and very long running works making
+ * blocked draining impractical.
+ *
+ * This is solved by allowing the pools to be disassociated from the CPU
+ * running as an unbound one and allowing it to be reattached later if the
+ * cpu comes back online.
*/
-void destroy_workqueue(struct workqueue_struct *wq)
-{
- struct pool_workqueue *pwq;
- int node;
-
- /* drain it before proceeding with destruction */
- drain_workqueue(wq);
-
- /* sanity checks */
- mutex_lock(&wq->mutex);
- for_each_pwq(pwq, wq) {
- int i;
- for (i = 0; i < WORK_NR_COLORS; i++) {
- if (WARN_ON(pwq->nr_in_flight[i])) {
- mutex_unlock(&wq->mutex);
- return;
- }
- }
+static void wq_unbind_fn(struct work_struct *work)
+{
+ int cpu = smp_processor_id();
+ struct worker_pool *pool;
+ struct worker *worker;
- if (WARN_ON((pwq != wq->dfl_pwq) && (pwq->refcnt > 1)) ||
- WARN_ON(pwq->nr_active) ||
- WARN_ON(!list_empty(&pwq->delayed_works))) {
- mutex_unlock(&wq->mutex);
- return;
- }
- }
- mutex_unlock(&wq->mutex);
+ for_each_cpu_worker_pool(pool, cpu) {
+ mutex_lock(&pool->attach_mutex);
+ spin_lock_irq(&pool->lock);
- /*
- * wq list is used to freeze wq, remove from list after
- * flushing is complete in case freeze races us.
- */
- mutex_lock(&wq_pool_mutex);
- list_del_init(&wq->list);
- mutex_unlock(&wq_pool_mutex);
+ /*
+ * We've blocked all attach/detach operations. Make all workers
+ * unbound and set DISASSOCIATED. Before this, all workers
+ * except for the ones which are still executing works from
+ * before the last CPU down must be on the cpu. After
+ * this, they may become diasporas.
+ */
+ for_each_pool_worker(worker, pool)
+ worker->flags |= WORKER_UNBOUND;
- workqueue_sysfs_unregister(wq);
+ pool->flags |= POOL_DISASSOCIATED;
- if (wq->rescuer) {
- kthread_stop(wq->rescuer->task);
- kfree(wq->rescuer);
- wq->rescuer = NULL;
- }
+ spin_unlock_irq(&pool->lock);
+ mutex_unlock(&pool->attach_mutex);
- if (!(wq->flags & WQ_UNBOUND)) {
/*
- * The base ref is never dropped on per-cpu pwqs. Directly
- * free the pwqs and wq.
+ * Call schedule() so that we cross rq->lock and thus can
+ * guarantee sched callbacks see the %WORKER_UNBOUND flag.
+ * This is necessary as scheduler callbacks may be invoked
+ * from other cpus.
*/
- free_percpu(wq->cpu_pwqs);
- kfree(wq);
- } else {
+ schedule();
+
/*
- * We're the sole accessor of @wq at this point. Directly
- * access numa_pwq_tbl[] and dfl_pwq to put the base refs.
- * @wq will be freed when the last pwq is released.
+ * Sched callbacks are disabled now. Zap nr_running.
+ * After this, nr_running stays zero and need_more_worker()
+ * and keep_working() are always true as long as the
+ * worklist is not empty. This pool now behaves as an
+ * unbound (in terms of concurrency management) pool which
+ * are served by workers tied to the pool.
*/
- for_each_node(node) {
- pwq = rcu_access_pointer(wq->numa_pwq_tbl[node]);
- RCU_INIT_POINTER(wq->numa_pwq_tbl[node], NULL);
- put_pwq_unlocked(pwq);
- }
+ atomic_set(&pool->nr_running, 0);
/*
- * Put dfl_pwq. @wq may be freed any time after dfl_pwq is
- * put. Don't access it afterwards.
+ * With concurrency management just turned off, a busy
+ * worker blocking could lead to lengthy stalls. Kick off
+ * unbound chain execution of currently pending work items.
*/
- pwq = wq->dfl_pwq;
- wq->dfl_pwq = NULL;
- put_pwq_unlocked(pwq);
+ spin_lock_irq(&pool->lock);
+ wake_up_worker(pool);
+ spin_unlock_irq(&pool->lock);
}
}
-EXPORT_SYMBOL_GPL(destroy_workqueue);
/**
- * workqueue_set_max_active - adjust max_active of a workqueue
- * @wq: target workqueue
- * @max_active: new max_active value.
- *
- * Set max_active of @wq to @max_active.
+ * rebind_workers - rebind all workers of a pool to the associated CPU
+ * @pool: pool of interest
*
- * CONTEXT:
- * Don't call from IRQ context.
+ * @pool->cpu is coming online. Rebind all workers to the CPU.
*/
-void workqueue_set_max_active(struct workqueue_struct *wq, int max_active)
+static void rebind_workers(struct worker_pool *pool)
{
- struct pool_workqueue *pwq;
+ struct worker *worker;
- /* disallow meddling with max_active for ordered workqueues */
- if (WARN_ON(wq->flags & __WQ_ORDERED))
- return;
+ lockdep_assert_held(&pool->attach_mutex);
- max_active = wq_clamp_max_active(max_active, wq->flags, wq->name);
+ /*
+ * Restore CPU affinity of all workers. As all idle workers should
+ * be on the run-queue of the associated CPU before any local
+ * wake-ups for concurrency management happen, restore CPU affinty
+ * of all workers first and then clear UNBOUND. As we're called
+ * from CPU_ONLINE, the following shouldn't fail.
+ */
+ for_each_pool_worker(worker, pool)
+ WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
+ pool->attrs->cpumask) < 0);
- mutex_lock(&wq->mutex);
+ spin_lock_irq(&pool->lock);
+ pool->flags &= ~POOL_DISASSOCIATED;
- wq->saved_max_active = max_active;
+ for_each_pool_worker(worker, pool) {
+ unsigned int worker_flags = worker->flags;
- for_each_pwq(pwq, wq)
- pwq_adjust_max_active(pwq);
+ /*
+ * A bound idle worker should actually be on the runqueue
+ * of the associated CPU for local wake-ups targeting it to
+ * work. Kick all idle workers so that they migrate to the
+ * associated CPU. Doing this in the same loop as
+ * replacing UNBOUND with REBOUND is safe as no worker will
+ * be bound before @pool->lock is released.
+ */
+ if (worker_flags & WORKER_IDLE)
+ wake_up_process(worker->task);
- mutex_unlock(&wq->mutex);
+ /*
+ * We want to clear UNBOUND but can't directly call
+ * worker_clr_flags() or adjust nr_running. Atomically
+ * replace UNBOUND with another NOT_RUNNING flag REBOUND.
+ * @worker will clear REBOUND using worker_clr_flags() when
+ * it initiates the next execution cycle thus restoring
+ * concurrency management. Note that when or whether
+ * @worker clears REBOUND doesn't affect correctness.
+ *
+ * ACCESS_ONCE() is necessary because @worker->flags may be
+ * tested without holding any lock in
+ * wq_worker_waking_up(). Without it, NOT_RUNNING test may
+ * fail incorrectly leading to premature concurrency
+ * management operations.
+ */
+ WARN_ON_ONCE(!(worker_flags & WORKER_UNBOUND));
+ worker_flags |= WORKER_REBOUND;
+ worker_flags &= ~WORKER_UNBOUND;
+ ACCESS_ONCE(worker->flags) = worker_flags;
+ }
+
+ spin_unlock_irq(&pool->lock);
}
-EXPORT_SYMBOL_GPL(workqueue_set_max_active);
/**
- * current_is_workqueue_rescuer - is %current workqueue rescuer?
- *
- * Determine whether %current is a workqueue rescuer. Can be used from
- * work functions to determine whether it's being run off the rescuer task.
+ * restore_unbound_workers_cpumask - restore cpumask of unbound workers
+ * @pool: unbound pool of interest
+ * @cpu: the CPU which is coming up
*
- * Return: %true if %current is a workqueue rescuer. %false otherwise.
+ * An unbound pool may end up with a cpumask which doesn't have any online
+ * CPUs. When a worker of such pool get scheduled, the scheduler resets
+ * its cpus_allowed. If @cpu is in @pool's cpumask which didn't have any
+ * online CPU before, cpus_allowed of all its workers should be restored.
*/
-bool current_is_workqueue_rescuer(void)
+static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
{
- struct worker *worker = current_wq_worker();
+ static cpumask_t cpumask;
+ struct worker *worker;
- return worker && worker->rescue_wq;
+ lockdep_assert_held(&pool->attach_mutex);
+
+ /* is @cpu allowed for @pool? */
+ if (!cpumask_test_cpu(cpu, pool->attrs->cpumask))
+ return;
+
+ /* is @cpu the only online CPU? */
+ cpumask_and(&cpumask, pool->attrs->cpumask, cpu_online_mask);
+ if (cpumask_weight(&cpumask) != 1)
+ return;
+
+ /* as we're called from CPU_ONLINE, the following shouldn't fail */
+ for_each_pool_worker(worker, pool)
+ WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
+ pool->attrs->cpumask) < 0);
}
-/**
- * workqueue_congested - test whether a workqueue is congested
- * @cpu: CPU in question
- * @wq: target workqueue
- *
- * Test whether @wq's cpu workqueue for @cpu is congested. There is
- * no synchronization around this function and the test result is
- * unreliable and only useful as advisory hints or for debugging.
- *
- * If @cpu is WORK_CPU_UNBOUND, the test is performed on the local CPU.
- * Note that both per-cpu and unbound workqueues may be associated with
- * multiple pool_workqueues which have separate congested states. A
- * workqueue being congested on one CPU doesn't mean the workqueue is also
- * contested on other CPUs / NUMA nodes.
- *
- * Return:
- * %true if congested, %false otherwise.
+/*
+ * Workqueues should be brought up before normal priority CPU notifiers.
+ * This will be registered high priority CPU notifier.
*/
-bool workqueue_congested(int cpu, struct workqueue_struct *wq)
+static int workqueue_cpu_up_callback(struct notifier_block *nfb,
+ unsigned long action,
+ void *hcpu)
{
- struct pool_workqueue *pwq;
- bool ret;
+ int cpu = (unsigned long)hcpu;
+ struct worker_pool *pool;
+ struct workqueue_struct *wq;
+ int pi;
- rcu_read_lock_sched();
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_UP_PREPARE:
+ for_each_cpu_worker_pool(pool, cpu) {
+ if (pool->nr_workers)
+ continue;
+ if (!create_worker(pool))
+ return NOTIFY_BAD;
+ }
+ break;
- if (cpu == WORK_CPU_UNBOUND)
- cpu = smp_processor_id();
+ case CPU_DOWN_FAILED:
+ case CPU_ONLINE:
+ mutex_lock(&wq_pool_mutex);
- if (!(wq->flags & WQ_UNBOUND))
- pwq = per_cpu_ptr(wq->cpu_pwqs, cpu);
- else
- pwq = unbound_pwq_by_node(wq, cpu_to_node(cpu));
+ for_each_pool(pool, pi) {
+ mutex_lock(&pool->attach_mutex);
- ret = !list_empty(&pwq->delayed_works);
- rcu_read_unlock_sched();
+ if (pool->cpu == cpu)
+ rebind_workers(pool);
+ else if (pool->cpu < 0)
+ restore_unbound_workers_cpumask(pool, cpu);
- return ret;
+ mutex_unlock(&pool->attach_mutex);
+ }
+
+ /* update NUMA affinity of unbound workqueues */
+ list_for_each_entry(wq, &workqueues, list)
+ wq_update_unbound_numa(wq, cpu, true);
+
+ mutex_unlock(&wq_pool_mutex);
+ break;
+ }
+ return NOTIFY_OK;
}
-EXPORT_SYMBOL_GPL(workqueue_congested);
-/**
- * work_busy - test whether a work is currently pending or running
- * @work: the work to be tested
- *
- * Test whether @work is currently pending or running. There is no
- * synchronization around this function and the test result is
- * unreliable and only useful as advisory hints or for debugging.
- *
- * Return:
- * OR'd bitmask of WORK_BUSY_* bits.
+/*
+ * Workqueues should be brought down after normal priority CPU notifiers.
+ * This will be registered as low priority CPU notifier.
*/
-unsigned int work_busy(struct work_struct *work)
+static int workqueue_cpu_down_callback(struct notifier_block *nfb,
+ unsigned long action,
+ void *hcpu)
{
- struct worker_pool *pool;
- unsigned long flags;
- unsigned int ret = 0;
+ int cpu = (unsigned long)hcpu;
+ struct work_struct unbind_work;
+ struct workqueue_struct *wq;
- if (work_pending(work))
- ret |= WORK_BUSY_PENDING;
+ switch (action & ~CPU_TASKS_FROZEN) {
+ case CPU_DOWN_PREPARE:
+ /* unbinding per-cpu workers should happen on the local CPU */
+ INIT_WORK_ONSTACK(&unbind_work, wq_unbind_fn);
+ queue_work_on(cpu, system_highpri_wq, &unbind_work);
- local_irq_save(flags);
- pool = get_work_pool(work);
- if (pool) {
- spin_lock(&pool->lock);
- if (find_worker_executing_work(pool, work))
- ret |= WORK_BUSY_RUNNING;
- spin_unlock(&pool->lock);
+ /* update NUMA affinity of unbound workqueues */
+ mutex_lock(&wq_pool_mutex);
+ list_for_each_entry(wq, &workqueues, list)
+ wq_update_unbound_numa(wq, cpu, false);
+ mutex_unlock(&wq_pool_mutex);
+
+ /* wait for per-cpu unbinding to finish */
+ flush_work(&unbind_work);
+ destroy_work_on_stack(&unbind_work);
+ break;
}
- local_irq_restore(flags);
+ return NOTIFY_OK;
+}
- return ret;
+#ifdef CONFIG_SMP
+
+struct work_for_cpu {
+ struct work_struct work;
+ long (*fn)(void *);
+ void *arg;
+ long ret;
+};
+
+static void work_for_cpu_fn(struct work_struct *work)
+{
+ struct work_for_cpu *wfc = container_of(work, struct work_for_cpu, work);
+
+ wfc->ret = wfc->fn(wfc->arg);
}
-EXPORT_SYMBOL_GPL(work_busy);
/**
- * set_worker_desc - set description for the current work item
- * @fmt: printf-style format string
- * @...: arguments for the format string
+ * work_on_cpu - run a function in user context on a particular cpu
+ * @cpu: the cpu to run on
+ * @fn: the function to run
+ * @arg: the function arg
*
- * This function can be called by a running work function to describe what
- * the work item is about. If the worker task gets dumped, this
- * information will be printed out together to help debugging. The
- * description can be at most WORKER_DESC_LEN including the trailing '\0'.
+ * It is up to the caller to ensure that the cpu doesn't go offline.
+ * The caller must not hold any locks which would prevent @fn from completing.
+ *
+ * Return: The value @fn returns.
*/
-void set_worker_desc(const char *fmt, ...)
+long work_on_cpu(int cpu, long (*fn)(void *), void *arg)
{
- struct worker *worker = current_wq_worker();
- va_list args;
+ struct work_for_cpu wfc = { .fn = fn, .arg = arg };
- if (worker) {
- va_start(args, fmt);
- vsnprintf(worker->desc, sizeof(worker->desc), fmt, args);
- va_end(args);
- worker->desc_valid = true;
- }
+ INIT_WORK_ONSTACK(&wfc.work, work_for_cpu_fn);
+ schedule_work_on(cpu, &wfc.work);
+ flush_work(&wfc.work);
+ destroy_work_on_stack(&wfc.work);
+ return wfc.ret;
}
+EXPORT_SYMBOL_GPL(work_on_cpu);
+#endif /* CONFIG_SMP */
+
+#ifdef CONFIG_FREEZER
/**
- * print_worker_info - print out worker information and description
- * @log_lvl: the log level to use when printing
- * @task: target task
+ * freeze_workqueues_begin - begin freezing workqueues
*
- * If @task is a worker and currently executing a work item, print out the
- * name of the workqueue being serviced and worker description set with
- * set_worker_desc() by the currently executing work item.
+ * Start freezing workqueues. After this function returns, all freezable
+ * workqueues will queue new works to their delayed_works list instead of
+ * pool->worklist.
*
- * This function can be safely called on any task as long as the
- * task_struct itself is accessible. While safe, this function isn't
- * synchronized and may print out mixups or garbages of limited length.
+ * CONTEXT:
+ * Grabs and releases wq_pool_mutex, wq->mutex and pool->lock's.
*/
-void print_worker_info(const char *log_lvl, struct task_struct *task)
+void freeze_workqueues_begin(void)
{
- work_func_t *fn = NULL;
- char name[WQ_NAME_LEN] = { };
- char desc[WORKER_DESC_LEN] = { };
- struct pool_workqueue *pwq = NULL;
- struct workqueue_struct *wq = NULL;
- bool desc_valid = false;
- struct worker *worker;
-
- if (!(task->flags & PF_WQ_WORKER))
- return;
-
- /*
- * This function is called without any synchronization and @task
- * could be in any state. Be careful with dereferences.
- */
- worker = probe_kthread_data(task);
+ struct workqueue_struct *wq;
+ struct pool_workqueue *pwq;
- /*
- * Carefully copy the associated workqueue's workfn and name. Keep
- * the original last '\0' in case the original contains garbage.
- */
- probe_kernel_read(&fn, &worker->current_func, sizeof(fn));
- probe_kernel_read(&pwq, &worker->current_pwq, sizeof(pwq));
- probe_kernel_read(&wq, &pwq->wq, sizeof(wq));
- probe_kernel_read(name, wq->name, sizeof(name) - 1);
+ mutex_lock(&wq_pool_mutex);
- /* copy worker description */
- probe_kernel_read(&desc_valid, &worker->desc_valid, sizeof(desc_valid));
- if (desc_valid)
- probe_kernel_read(desc, worker->desc, sizeof(desc) - 1);
+ WARN_ON_ONCE(workqueue_freezing);
+ workqueue_freezing = true;
- if (fn || name[0] || desc[0]) {
- printk("%sWorkqueue: %s %pf", log_lvl, name, fn);
- if (desc[0])
- pr_cont(" (%s)", desc);
- pr_cont("\n");
+ list_for_each_entry(wq, &workqueues, list) {
+ mutex_lock(&wq->mutex);
+ for_each_pwq(pwq, wq)
+ pwq_adjust_max_active(pwq);
+ mutex_unlock(&wq->mutex);
}
+
+ mutex_unlock(&wq_pool_mutex);
}
-/*
- * CPU hotplug.
+/**
+ * freeze_workqueues_busy - are freezable workqueues still busy?
*
- * There are two challenges in supporting CPU hotplug. Firstly, there
- * are a lot of assumptions on strong associations among work, pwq and
- * pool which make migrating pending and scheduled works very
- * difficult to implement without impacting hot paths. Secondly,
- * worker pools serve mix of short, long and very long running works making
- * blocked draining impractical.
+ * Check whether freezing is complete. This function must be called
+ * between freeze_workqueues_begin() and thaw_workqueues().
*
- * This is solved by allowing the pools to be disassociated from the CPU
- * running as an unbound one and allowing it to be reattached later if the
- * cpu comes back online.
+ * CONTEXT:
+ * Grabs and releases wq_pool_mutex.
+ *
+ * Return:
+ * %true if some freezable workqueues are still busy. %false if freezing
+ * is complete.
*/
-
-static void wq_unbind_fn(struct work_struct *work)
+bool freeze_workqueues_busy(void)
{
- int cpu = smp_processor_id();
- struct worker_pool *pool;
- struct worker *worker;
+ bool busy = false;
+ struct workqueue_struct *wq;
+ struct pool_workqueue *pwq;
- for_each_cpu_worker_pool(pool, cpu) {
- mutex_lock(&pool->attach_mutex);
- spin_lock_irq(&pool->lock);
+ mutex_lock(&wq_pool_mutex);
+
+ WARN_ON_ONCE(!workqueue_freezing);
+ list_for_each_entry(wq, &workqueues, list) {
+ if (!(wq->flags & WQ_FREEZABLE))
+ continue;
/*
- * We've blocked all attach/detach operations. Make all workers
- * unbound and set DISASSOCIATED. Before this, all workers
- * except for the ones which are still executing works from
- * before the last CPU down must be on the cpu. After
- * this, they may become diasporas.
+ * nr_active is monotonically decreasing. It's safe
+ * to peek without lock.
*/
- for_each_pool_worker(worker, pool)
- worker->flags |= WORKER_UNBOUND;
+ rcu_read_lock_sched();
+ for_each_pwq(pwq, wq) {
+ WARN_ON_ONCE(pwq->nr_active < 0);
+ if (pwq->nr_active) {
+ busy = true;
+ rcu_read_unlock_sched();
+ goto out_unlock;
+ }
+ }
+ rcu_read_unlock_sched();
+ }
+out_unlock:
+ mutex_unlock(&wq_pool_mutex);
+ return busy;
+}
- pool->flags |= POOL_DISASSOCIATED;
+/**
+ * thaw_workqueues - thaw workqueues
+ *
+ * Thaw workqueues. Normal queueing is restored and all collected
+ * frozen works are transferred to their respective pool worklists.
+ *
+ * CONTEXT:
+ * Grabs and releases wq_pool_mutex, wq->mutex and pool->lock's.
+ */
+void thaw_workqueues(void)
+{
+ struct workqueue_struct *wq;
+ struct pool_workqueue *pwq;
- spin_unlock_irq(&pool->lock);
- mutex_unlock(&pool->attach_mutex);
+ mutex_lock(&wq_pool_mutex);
- /*
- * Call schedule() so that we cross rq->lock and thus can
- * guarantee sched callbacks see the %WORKER_UNBOUND flag.
- * This is necessary as scheduler callbacks may be invoked
- * from other cpus.
- */
- schedule();
+ if (!workqueue_freezing)
+ goto out_unlock;
- /*
- * Sched callbacks are disabled now. Zap nr_running.
- * After this, nr_running stays zero and need_more_worker()
- * and keep_working() are always true as long as the
- * worklist is not empty. This pool now behaves as an
- * unbound (in terms of concurrency management) pool which
- * are served by workers tied to the pool.
- */
- atomic_set(&pool->nr_running, 0);
+ workqueue_freezing = false;
- /*
- * With concurrency management just turned off, a busy
- * worker blocking could lead to lengthy stalls. Kick off
- * unbound chain execution of currently pending work items.
- */
- spin_lock_irq(&pool->lock);
- wake_up_worker(pool);
- spin_unlock_irq(&pool->lock);
+ /* restore max_active and repopulate worklist */
+ list_for_each_entry(wq, &workqueues, list) {
+ mutex_lock(&wq->mutex);
+ for_each_pwq(pwq, wq)
+ pwq_adjust_max_active(pwq);
+ mutex_unlock(&wq->mutex);
}
+
+out_unlock:
+ mutex_unlock(&wq_pool_mutex);
}
+#endif /* CONFIG_FREEZER */
-/**
- * rebind_workers - rebind all workers of a pool to the associated CPU
- * @pool: pool of interest
+#ifdef CONFIG_SYSFS
+/*
+ * Workqueues with WQ_SYSFS flag set is visible to userland via
+ * /sys/bus/workqueue/devices/WQ_NAME. All visible workqueues have the
+ * following attributes.
*
- * @pool->cpu is coming online. Rebind all workers to the CPU.
+ * per_cpu RO bool : whether the workqueue is per-cpu or unbound
+ * max_active RW int : maximum number of in-flight work items
+ *
+ * Unbound workqueues have the following extra attributes.
+ *
+ * id RO int : the associated pool ID
+ * nice RW int : nice value of the workers
+ * cpumask RW mask : bitmask of allowed CPUs for the workers
*/
-static void rebind_workers(struct worker_pool *pool)
+struct wq_device {
+ struct workqueue_struct *wq;
+ struct device dev;
+};
+
+static struct workqueue_struct *dev_to_wq(struct device *dev)
{
- struct worker *worker;
+ struct wq_device *wq_dev = container_of(dev, struct wq_device, dev);
- lockdep_assert_held(&pool->attach_mutex);
+ return wq_dev->wq;
+}
- /*
- * Restore CPU affinity of all workers. As all idle workers should
- * be on the run-queue of the associated CPU before any local
- * wake-ups for concurrency management happen, restore CPU affinty
- * of all workers first and then clear UNBOUND. As we're called
- * from CPU_ONLINE, the following shouldn't fail.
- */
- for_each_pool_worker(worker, pool)
- WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
- pool->attrs->cpumask) < 0);
+static ssize_t per_cpu_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
- spin_lock_irq(&pool->lock);
- pool->flags &= ~POOL_DISASSOCIATED;
+ return scnprintf(buf, PAGE_SIZE, "%d\n", (bool)!(wq->flags & WQ_UNBOUND));
+}
+static DEVICE_ATTR_RO(per_cpu);
- for_each_pool_worker(worker, pool) {
- unsigned int worker_flags = worker->flags;
+static ssize_t max_active_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
- /*
- * A bound idle worker should actually be on the runqueue
- * of the associated CPU for local wake-ups targeting it to
- * work. Kick all idle workers so that they migrate to the
- * associated CPU. Doing this in the same loop as
- * replacing UNBOUND with REBOUND is safe as no worker will
- * be bound before @pool->lock is released.
- */
- if (worker_flags & WORKER_IDLE)
- wake_up_process(worker->task);
+ return scnprintf(buf, PAGE_SIZE, "%d\n", wq->saved_max_active);
+}
- /*
- * We want to clear UNBOUND but can't directly call
- * worker_clr_flags() or adjust nr_running. Atomically
- * replace UNBOUND with another NOT_RUNNING flag REBOUND.
- * @worker will clear REBOUND using worker_clr_flags() when
- * it initiates the next execution cycle thus restoring
- * concurrency management. Note that when or whether
- * @worker clears REBOUND doesn't affect correctness.
- *
- * ACCESS_ONCE() is necessary because @worker->flags may be
- * tested without holding any lock in
- * wq_worker_waking_up(). Without it, NOT_RUNNING test may
- * fail incorrectly leading to premature concurrency
- * management operations.
- */
- WARN_ON_ONCE(!(worker_flags & WORKER_UNBOUND));
- worker_flags |= WORKER_REBOUND;
- worker_flags &= ~WORKER_UNBOUND;
- ACCESS_ONCE(worker->flags) = worker_flags;
+static ssize_t max_active_store(struct device *dev,
+ struct device_attribute *attr, const char *buf,
+ size_t count)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ int val;
+
+ if (sscanf(buf, "%d", &val) != 1 || val <= 0)
+ return -EINVAL;
+
+ workqueue_set_max_active(wq, val);
+ return count;
+}
+static DEVICE_ATTR_RW(max_active);
+
+static struct attribute *wq_sysfs_attrs[] = {
+ &dev_attr_per_cpu.attr,
+ &dev_attr_max_active.attr,
+ NULL,
+};
+ATTRIBUTE_GROUPS(wq_sysfs);
+
+static ssize_t wq_pool_ids_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ const char *delim = "";
+ int node, written = 0;
+
+ rcu_read_lock_sched();
+ for_each_node(node) {
+ written += scnprintf(buf + written, PAGE_SIZE - written,
+ "%s%d:%d", delim, node,
+ unbound_pwq_by_node(wq, node)->pool->id);
+ delim = " ";
}
+ written += scnprintf(buf + written, PAGE_SIZE - written, "\n");
+ rcu_read_unlock_sched();
- spin_unlock_irq(&pool->lock);
+ return written;
}
-/**
- * restore_unbound_workers_cpumask - restore cpumask of unbound workers
- * @pool: unbound pool of interest
- * @cpu: the CPU which is coming up
- *
- * An unbound pool may end up with a cpumask which doesn't have any online
- * CPUs. When a worker of such pool get scheduled, the scheduler resets
- * its cpus_allowed. If @cpu is in @pool's cpumask which didn't have any
- * online CPU before, cpus_allowed of all its workers should be restored.
- */
-static void restore_unbound_workers_cpumask(struct worker_pool *pool, int cpu)
+static ssize_t wq_nice_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
{
- static cpumask_t cpumask;
- struct worker *worker;
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ int written;
- lockdep_assert_held(&pool->attach_mutex);
+ mutex_lock(&wq->mutex);
+ written = scnprintf(buf, PAGE_SIZE, "%d\n", wq->unbound_attrs->nice);
+ mutex_unlock(&wq->mutex);
- /* is @cpu allowed for @pool? */
- if (!cpumask_test_cpu(cpu, pool->attrs->cpumask))
- return;
+ return written;
+}
- /* is @cpu the only online CPU? */
- cpumask_and(&cpumask, pool->attrs->cpumask, cpu_online_mask);
- if (cpumask_weight(&cpumask) != 1)
- return;
+/* prepare workqueue_attrs for sysfs store operations */
+static struct workqueue_attrs *wq_sysfs_prep_attrs(struct workqueue_struct *wq)
+{
+ struct workqueue_attrs *attrs;
- /* as we're called from CPU_ONLINE, the following shouldn't fail */
- for_each_pool_worker(worker, pool)
- WARN_ON_ONCE(set_cpus_allowed_ptr(worker->task,
- pool->attrs->cpumask) < 0);
+ attrs = alloc_workqueue_attrs(GFP_KERNEL);
+ if (!attrs)
+ return NULL;
+
+ mutex_lock(&wq->mutex);
+ copy_workqueue_attrs(attrs, wq->unbound_attrs);
+ mutex_unlock(&wq->mutex);
+ return attrs;
}
-/*
- * Workqueues should be brought up before normal priority CPU notifiers.
- * This will be registered high priority CPU notifier.
- */
-static int workqueue_cpu_up_callback(struct notifier_block *nfb,
- unsigned long action,
- void *hcpu)
+static ssize_t wq_nice_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
{
- int cpu = (unsigned long)hcpu;
- struct worker_pool *pool;
- struct workqueue_struct *wq;
- int pi;
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ struct workqueue_attrs *attrs;
+ int ret;
- switch (action & ~CPU_TASKS_FROZEN) {
- case CPU_UP_PREPARE:
- for_each_cpu_worker_pool(pool, cpu) {
- if (pool->nr_workers)
- continue;
- if (!create_worker(pool))
- return NOTIFY_BAD;
- }
- break;
+ attrs = wq_sysfs_prep_attrs(wq);
+ if (!attrs)
+ return -ENOMEM;
- case CPU_DOWN_FAILED:
- case CPU_ONLINE:
- mutex_lock(&wq_pool_mutex);
+ if (sscanf(buf, "%d", &attrs->nice) == 1 &&
+ attrs->nice >= MIN_NICE && attrs->nice <= MAX_NICE)
+ ret = apply_workqueue_attrs(wq, attrs);
+ else
+ ret = -EINVAL;
- for_each_pool(pool, pi) {
- mutex_lock(&pool->attach_mutex);
+ free_workqueue_attrs(attrs);
+ return ret ?: count;
+}
- if (pool->cpu == cpu)
- rebind_workers(pool);
- else if (pool->cpu < 0)
- restore_unbound_workers_cpumask(pool, cpu);
+static ssize_t wq_cpumask_show(struct device *dev,
+ struct device_attribute *attr, char *buf)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ int written;
+
+ mutex_lock(&wq->mutex);
+ written = scnprintf(buf, PAGE_SIZE, "%*pb\n",
+ cpumask_pr_args(wq->unbound_attrs->cpumask));
+ mutex_unlock(&wq->mutex);
+ return written;
+}
+
+static ssize_t wq_cpumask_store(struct device *dev,
+ struct device_attribute *attr,
+ const char *buf, size_t count)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ struct workqueue_attrs *attrs;
+ int ret;
+
+ attrs = wq_sysfs_prep_attrs(wq);
+ if (!attrs)
+ return -ENOMEM;
+
+ ret = cpumask_parse(buf, attrs->cpumask);
+ if (!ret)
+ ret = apply_workqueue_attrs(wq, attrs);
+
+ free_workqueue_attrs(attrs);
+ return ret ?: count;
+}
- mutex_unlock(&pool->attach_mutex);
- }
+static ssize_t wq_numa_show(struct device *dev, struct device_attribute *attr,
+ char *buf)
+{
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ int written;
- /* update NUMA affinity of unbound workqueues */
- list_for_each_entry(wq, &workqueues, list)
- wq_update_unbound_numa(wq, cpu, true);
+ mutex_lock(&wq->mutex);
+ written = scnprintf(buf, PAGE_SIZE, "%d\n",
+ !wq->unbound_attrs->no_numa);
+ mutex_unlock(&wq->mutex);
- mutex_unlock(&wq_pool_mutex);
- break;
- }
- return NOTIFY_OK;
+ return written;
}
-/*
- * Workqueues should be brought down after normal priority CPU notifiers.
- * This will be registered as low priority CPU notifier.
- */
-static int workqueue_cpu_down_callback(struct notifier_block *nfb,
- unsigned long action,
- void *hcpu)
+static ssize_t wq_numa_store(struct device *dev, struct device_attribute *attr,
+ const char *buf, size_t count)
{
- int cpu = (unsigned long)hcpu;
- struct work_struct unbind_work;
- struct workqueue_struct *wq;
-
- switch (action & ~CPU_TASKS_FROZEN) {
- case CPU_DOWN_PREPARE:
- /* unbinding per-cpu workers should happen on the local CPU */
- INIT_WORK_ONSTACK(&unbind_work, wq_unbind_fn);
- queue_work_on(cpu, system_highpri_wq, &unbind_work);
+ struct workqueue_struct *wq = dev_to_wq(dev);
+ struct workqueue_attrs *attrs;
+ int v, ret;
- /* update NUMA affinity of unbound workqueues */
- mutex_lock(&wq_pool_mutex);
- list_for_each_entry(wq, &workqueues, list)
- wq_update_unbound_numa(wq, cpu, false);
- mutex_unlock(&wq_pool_mutex);
+ attrs = wq_sysfs_prep_attrs(wq);
+ if (!attrs)
+ return -ENOMEM;
- /* wait for per-cpu unbinding to finish */
- flush_work(&unbind_work);
- destroy_work_on_stack(&unbind_work);
- break;
+ ret = -EINVAL;
+ if (sscanf(buf, "%d", &v) == 1) {
+ attrs->no_numa = !v;
+ ret = apply_workqueue_attrs(wq, attrs);
}
- return NOTIFY_OK;
+
+ free_workqueue_attrs(attrs);
+ return ret ?: count;
}
-#ifdef CONFIG_SMP
+static struct device_attribute wq_sysfs_unbound_attrs[] = {
+ __ATTR(pool_ids, 0444, wq_pool_ids_show, NULL),
+ __ATTR(nice, 0644, wq_nice_show, wq_nice_store),
+ __ATTR(cpumask, 0644, wq_cpumask_show, wq_cpumask_store),
+ __ATTR(numa, 0644, wq_numa_show, wq_numa_store),
+ __ATTR_NULL,
+};
-struct work_for_cpu {
- struct work_struct work;
- long (*fn)(void *);
- void *arg;
- long ret;
+static struct bus_type wq_subsys = {
+ .name = "workqueue",
+ .dev_groups = wq_sysfs_groups,
};
-static void work_for_cpu_fn(struct work_struct *work)
+static int __init wq_sysfs_init(void)
{
- struct work_for_cpu *wfc = container_of(work, struct work_for_cpu, work);
-
- wfc->ret = wfc->fn(wfc->arg);
+ return subsys_virtual_register(&wq_subsys, NULL);
}
+core_initcall(wq_sysfs_init);
-/**
- * work_on_cpu - run a function in user context on a particular cpu
- * @cpu: the cpu to run on
- * @fn: the function to run
- * @arg: the function arg
- *
- * It is up to the caller to ensure that the cpu doesn't go offline.
- * The caller must not hold any locks which would prevent @fn from completing.
- *
- * Return: The value @fn returns.
- */
-long work_on_cpu(int cpu, long (*fn)(void *), void *arg)
+static void wq_device_release(struct device *dev)
{
- struct work_for_cpu wfc = { .fn = fn, .arg = arg };
+ struct wq_device *wq_dev = container_of(dev, struct wq_device, dev);
- INIT_WORK_ONSTACK(&wfc.work, work_for_cpu_fn);
- schedule_work_on(cpu, &wfc.work);
- flush_work(&wfc.work);
- destroy_work_on_stack(&wfc.work);
- return wfc.ret;
+ kfree(wq_dev);
}
-EXPORT_SYMBOL_GPL(work_on_cpu);
-#endif /* CONFIG_SMP */
-
-#ifdef CONFIG_FREEZER
/**
- * freeze_workqueues_begin - begin freezing workqueues
+ * workqueue_sysfs_register - make a workqueue visible in sysfs
+ * @wq: the workqueue to register
*
- * Start freezing workqueues. After this function returns, all freezable
- * workqueues will queue new works to their delayed_works list instead of
- * pool->worklist.
+ * Expose @wq in sysfs under /sys/bus/workqueue/devices.
+ * alloc_workqueue*() automatically calls this function if WQ_SYSFS is set
+ * which is the preferred method.
*
- * CONTEXT:
- * Grabs and releases wq_pool_mutex, wq->mutex and pool->lock's.
+ * Workqueue user should use this function directly iff it wants to apply
+ * workqueue_attrs before making the workqueue visible in sysfs; otherwise,
+ * apply_workqueue_attrs() may race against userland updating the
+ * attributes.
+ *
+ * Return: 0 on success, -errno on failure.
*/
-void freeze_workqueues_begin(void)
+int workqueue_sysfs_register(struct workqueue_struct *wq)
{
- struct workqueue_struct *wq;
- struct pool_workqueue *pwq;
-
- mutex_lock(&wq_pool_mutex);
+ struct wq_device *wq_dev;
+ int ret;
- WARN_ON_ONCE(workqueue_freezing);
- workqueue_freezing = true;
+ /*
+ * Adjusting max_active or creating new pwqs by applyting
+ * attributes breaks ordering guarantee. Disallow exposing ordered
+ * workqueues.
+ */
+ if (WARN_ON(wq->flags & __WQ_ORDERED))
+ return -EINVAL;
- list_for_each_entry(wq, &workqueues, list) {
- mutex_lock(&wq->mutex);
- for_each_pwq(pwq, wq)
- pwq_adjust_max_active(pwq);
- mutex_unlock(&wq->mutex);
- }
+ wq->wq_dev = wq_dev = kzalloc(sizeof(*wq_dev), GFP_KERNEL);
+ if (!wq_dev)
+ return -ENOMEM;
- mutex_unlock(&wq_pool_mutex);
-}
+ wq_dev->wq = wq;
+ wq_dev->dev.bus = &wq_subsys;
+ wq_dev->dev.init_name = wq->name;
+ wq_dev->dev.release = wq_device_release;
-/**
- * freeze_workqueues_busy - are freezable workqueues still busy?
- *
- * Check whether freezing is complete. This function must be called
- * between freeze_workqueues_begin() and thaw_workqueues().
- *
- * CONTEXT:
- * Grabs and releases wq_pool_mutex.
- *
- * Return:
- * %true if some freezable workqueues are still busy. %false if freezing
- * is complete.
- */
-bool freeze_workqueues_busy(void)
-{
- bool busy = false;
- struct workqueue_struct *wq;
- struct pool_workqueue *pwq;
+ /*
+ * unbound_attrs are created separately. Suppress uevent until
+ * everything is ready.
+ */
+ dev_set_uevent_suppress(&wq_dev->dev, true);
- mutex_lock(&wq_pool_mutex);
+ ret = device_register(&wq_dev->dev);
+ if (ret) {
+ kfree(wq_dev);
+ wq->wq_dev = NULL;
+ return ret;
+ }
- WARN_ON_ONCE(!workqueue_freezing);
+ if (wq->flags & WQ_UNBOUND) {
+ struct device_attribute *attr;
- list_for_each_entry(wq, &workqueues, list) {
- if (!(wq->flags & WQ_FREEZABLE))
- continue;
- /*
- * nr_active is monotonically decreasing. It's safe
- * to peek without lock.
- */
- rcu_read_lock_sched();
- for_each_pwq(pwq, wq) {
- WARN_ON_ONCE(pwq->nr_active < 0);
- if (pwq->nr_active) {
- busy = true;
- rcu_read_unlock_sched();
- goto out_unlock;
+ for (attr = wq_sysfs_unbound_attrs; attr->attr.name; attr++) {
+ ret = device_create_file(&wq_dev->dev, attr);
+ if (ret) {
+ device_unregister(&wq_dev->dev);
+ wq->wq_dev = NULL;
+ return ret;
}
}
- rcu_read_unlock_sched();
}
-out_unlock:
- mutex_unlock(&wq_pool_mutex);
- return busy;
+
+ dev_set_uevent_suppress(&wq_dev->dev, false);
+ kobject_uevent(&wq_dev->dev.kobj, KOBJ_ADD);
+ return 0;
}
/**
- * thaw_workqueues - thaw workqueues
- *
- * Thaw workqueues. Normal queueing is restored and all collected
- * frozen works are transferred to their respective pool worklists.
+ * workqueue_sysfs_unregister - undo workqueue_sysfs_register()
+ * @wq: the workqueue to unregister
*
- * CONTEXT:
- * Grabs and releases wq_pool_mutex, wq->mutex and pool->lock's.
+ * If @wq is registered to sysfs by workqueue_sysfs_register(), unregister.
*/
-void thaw_workqueues(void)
+static void workqueue_sysfs_unregister(struct workqueue_struct *wq)
{
- struct workqueue_struct *wq;
- struct pool_workqueue *pwq;
-
- mutex_lock(&wq_pool_mutex);
-
- if (!workqueue_freezing)
- goto out_unlock;
-
- workqueue_freezing = false;
+ struct wq_device *wq_dev = wq->wq_dev;
- /* restore max_active and repopulate worklist */
- list_for_each_entry(wq, &workqueues, list) {
- mutex_lock(&wq->mutex);
- for_each_pwq(pwq, wq)
- pwq_adjust_max_active(pwq);
- mutex_unlock(&wq->mutex);
- }
+ if (!wq->wq_dev)
+ return;
-out_unlock:
- mutex_unlock(&wq_pool_mutex);
+ wq->wq_dev = NULL;
+ device_unregister(&wq_dev->dev);
}
-#endif /* CONFIG_FREEZER */
+#else /* CONFIG_SYSFS */
+static void workqueue_sysfs_unregister(struct workqueue_struct *wq) { }
+#endif /* CONFIG_SYSFS */
static void __init wq_numa_init(void)
{
data corruption or a sporadic crash at a later stage once the region
is examined. The runtime overhead introduced is minimal.
+config DEBUG_TIMEKEEPING
+ bool "Enable extra timekeeping sanity checking"
+ help
+ This option will enable additional timekeeping sanity checks
+ which may be helpful when diagnosing issues where timekeeping
+ problems are suspected.
+
+ This may include checks in the timekeeping hotpaths, so this
+ option may have a (very small) performance impact to some
+ workloads.
+
+ If unsure, say N.
+
config TIMER_STATS
bool "Collect kernel timers statistics"
depends on DEBUG_KERNEL && PROC_FS
obj-y += bcd.o div64.o sort.o parser.o halfmd4.o debug_locks.o random32.o \
bust_spinlocks.o kasprintf.o bitmap.o scatterlist.o \
- gcd.o lcm.o list_sort.o uuid.o flex_array.o clz_ctz.o \
+ gcd.o lcm.o list_sort.o uuid.o flex_array.o iov_iter.o clz_ctz.o \
bsearch.o find_last_bit.o find_next_bit.o llist.o memweight.o kfifo.o \
percpu-refcount.o percpu_ida.o rhashtable.o reciprocal_div.o
obj-y += string_helpers.o
return npages;
}
EXPORT_SYMBOL(iov_iter_npages);
+
+const void *dup_iter(struct iov_iter *new, struct iov_iter *old, gfp_t flags)
+{
+ *new = *old;
+ if (new->type & ITER_BVEC)
+ return new->bvec = kmemdup(new->bvec,
+ new->nr_segs * sizeof(struct bio_vec),
+ flags);
+ else
+ /* iovec and kvec have identical layout */
+ return new->iov = kmemdup(new->iov,
+ new->nr_segs * sizeof(struct iovec),
+ flags);
+}
+EXPORT_SYMBOL(dup_iter);
return 0;
}
EXPORT_SYMBOL_GPL(lcm);
+
+unsigned long lcm_not_zero(unsigned long a, unsigned long b)
+{
+ unsigned long l = lcm(a, b);
+
+ if (l)
+ return l;
+
+ return (b ? : a);
+}
+EXPORT_SYMBOL_GPL(lcm_not_zero);
#define CMPXCHG_LOOP(CODE, SUCCESS) do { \
struct lockref old; \
BUILD_BUG_ON(sizeof(old) != 8); \
- old.lock_count = ACCESS_ONCE(lockref->lock_count); \
+ old.lock_count = READ_ONCE(lockref->lock_count); \
while (likely(arch_spin_value_unlocked(old.lock.rlock.raw_lock))) { \
struct lockref new = old, prev = old; \
CODE \
/* Error: request to write beyond destination buffer */
if (cpy > oend)
goto _output_error;
+ if ((ref + COPYLENGTH) > oend ||
+ (op + COPYLENGTH) > oend)
+ goto _output_error;
LZ4_SECURECOPY(ref, op, (oend - COPYLENGTH));
while (op < cpy)
*op++ = *ref++;
int minlen = min_t(int, count, nla_len(src));
memcpy(dest, nla_data(src), minlen);
+ if (count > minlen)
+ memset(dest + minlen, 0, count - minlen);
return minlen;
}
#include <linux/kernel.h>
#include <linux/init.h>
#include <linux/log2.h>
+#include <linux/sched.h>
#include <linux/slab.h>
#include <linux/vmalloc.h>
#include <linux/mm.h>
static struct bucket_table *bucket_table_alloc(struct rhashtable *ht,
size_t nbuckets)
{
- struct bucket_table *tbl;
+ struct bucket_table *tbl = NULL;
size_t size;
int i;
size = sizeof(*tbl) + nbuckets * sizeof(tbl->buckets[0]);
- tbl = kzalloc(size, GFP_KERNEL | __GFP_NOWARN);
+ if (size <= (PAGE_SIZE << PAGE_ALLOC_COSTLY_ORDER))
+ tbl = kzalloc(size, GFP_KERNEL | __GFP_NOWARN | __GFP_NORETRY);
if (tbl == NULL)
tbl = vzalloc(size);
-
if (tbl == NULL)
return NULL;
* @ht: hash table
* @new_size: new table size
*/
-bool rht_grow_above_75(const struct rhashtable *ht, size_t new_size)
+static bool rht_grow_above_75(const struct rhashtable *ht, size_t new_size)
{
/* Expand table when exceeding 75% load */
return atomic_read(&ht->nelems) > (new_size / 4 * 3) &&
- (ht->p.max_shift && atomic_read(&ht->shift) < ht->p.max_shift);
+ (!ht->p.max_shift || atomic_read(&ht->shift) < ht->p.max_shift);
}
-EXPORT_SYMBOL_GPL(rht_grow_above_75);
/**
* rht_shrink_below_30 - returns true if nelems < 0.3 * table-size
* @ht: hash table
* @new_size: new table size
*/
-bool rht_shrink_below_30(const struct rhashtable *ht, size_t new_size)
+static bool rht_shrink_below_30(const struct rhashtable *ht, size_t new_size)
{
/* Shrink table beneath 30% load */
return atomic_read(&ht->nelems) < (new_size * 3 / 10) &&
(atomic_read(&ht->shift) > ht->p.min_shift);
}
-EXPORT_SYMBOL_GPL(rht_shrink_below_30);
static void lock_buckets(struct bucket_table *new_tbl,
struct bucket_table *old_tbl, unsigned int hash)
}
}
unlock_buckets(new_tbl, old_tbl, new_hash);
+ cond_resched();
}
/* Unzip interleaved hash chains */
complete = false;
unlock_buckets(new_tbl, old_tbl, old_hash);
+ cond_resched();
}
}
tbl->buckets[new_hash + new_tbl->size]);
unlock_buckets(new_tbl, tbl, new_hash);
+ cond_resched();
}
/* Publish the new, valid hash table */
list_for_each_entry(walker, &ht->walkers, list)
walker->resize = true;
- if (ht->p.grow_decision && ht->p.grow_decision(ht, tbl->size))
+ if (rht_grow_above_75(ht, tbl->size))
rhashtable_expand(ht);
- else if (ht->p.shrink_decision && ht->p.shrink_decision(ht, tbl->size))
+ else if (rht_shrink_below_30(ht, tbl->size))
rhashtable_shrink(ht);
-
unlock:
mutex_unlock(&ht->mutex);
}
-static void rhashtable_wakeup_worker(struct rhashtable *ht)
-{
- struct bucket_table *tbl = rht_dereference_rcu(ht->tbl, ht);
- struct bucket_table *new_tbl = rht_dereference_rcu(ht->future_tbl, ht);
- size_t size = tbl->size;
-
- /* Only adjust the table if no resizing is currently in progress. */
- if (tbl == new_tbl &&
- ((ht->p.grow_decision && ht->p.grow_decision(ht, size)) ||
- (ht->p.shrink_decision && ht->p.shrink_decision(ht, size))))
- schedule_work(&ht->run_work);
-}
-
static void __rhashtable_insert(struct rhashtable *ht, struct rhash_head *obj,
- struct bucket_table *tbl, u32 hash)
+ struct bucket_table *tbl,
+ const struct bucket_table *old_tbl, u32 hash)
{
+ bool no_resize_running = tbl == old_tbl;
struct rhash_head *head;
hash = rht_bucket_index(tbl, hash);
rcu_assign_pointer(tbl->buckets[hash], obj);
atomic_inc(&ht->nelems);
-
- rhashtable_wakeup_worker(ht);
+ if (no_resize_running && rht_grow_above_75(ht, tbl->size))
+ schedule_work(&ht->run_work);
}
/**
hash = obj_raw_hashfn(ht, rht_obj(ht, obj));
lock_buckets(tbl, old_tbl, hash);
- __rhashtable_insert(ht, obj, tbl, hash);
+ __rhashtable_insert(ht, obj, tbl, old_tbl, hash);
unlock_buckets(tbl, old_tbl, hash);
rcu_read_unlock();
unlock_buckets(new_tbl, old_tbl, new_hash);
if (ret) {
+ bool no_resize_running = new_tbl == old_tbl;
+
atomic_dec(&ht->nelems);
- rhashtable_wakeup_worker(ht);
+ if (no_resize_running && rht_shrink_below_30(ht, new_tbl->size))
+ schedule_work(&ht->run_work);
}
rcu_read_unlock();
goto exit;
}
- __rhashtable_insert(ht, obj, new_tbl, new_hash);
+ __rhashtable_insert(ht, obj, new_tbl, old_tbl, new_hash);
exit:
unlock_buckets(new_tbl, old_tbl, new_hash);
if (!iter->walker)
return -ENOMEM;
+ INIT_LIST_HEAD(&iter->walker->list);
+ iter->walker->resize = false;
+
mutex_lock(&ht->mutex);
list_add(&iter->walker->list, &ht->walkers);
mutex_unlock(&ht->mutex);
if (!ht->p.hash_rnd)
get_random_bytes(&ht->p.hash_rnd, sizeof(ht->p.hash_rnd));
- if (ht->p.grow_decision || ht->p.shrink_decision)
- INIT_WORK(&ht->run_work, rht_deferred_worker);
+ INIT_WORK(&ht->run_work, rht_deferred_worker);
return 0;
}
{
ht->being_destroyed = true;
- if (ht->p.grow_decision || ht->p.shrink_decision)
- cancel_work_sync(&ht->run_work);
+ cancel_work_sync(&ht->run_work);
mutex_lock(&ht->mutex);
bucket_table_free(rht_dereference(ht->tbl, ht));
if (s->len < s->size) {
len = vsnprintf(s->buffer + s->len, s->size - s->len, fmt, args);
- if (seq_buf_can_fit(s, len)) {
+ if (s->len + len < s->size) {
s->len += len;
return 0;
}
if (s->len < s->size) {
ret = bstr_printf(s->buffer + s->len, len, fmt, binary);
- if (seq_buf_can_fit(s, ret)) {
+ if (s->len + ret < s->size) {
s->len += ret;
return 0;
}
return err;
}
+static struct rhashtable ht;
+
static int __init test_rht_init(void)
{
- struct rhashtable ht;
struct rhashtable_params params = {
.nelem_hint = TEST_HT_SIZE,
.head_offset = offsetof(struct test_obj, node),
.key_offset = offsetof(struct test_obj, value),
.key_len = sizeof(int),
.hashfn = jhash,
+ .max_shift = 1, /* we expand/shrink manually here */
.nulls_base = (3U << RHT_BASE_SHIFT),
- .grow_decision = rht_grow_above_75,
- .shrink_decision = rht_shrink_below_30,
};
int err;
return err;
}
+static void __exit test_rht_exit(void)
+{
+}
+
module_init(test_rht_init);
+module_exit(test_rht_exit);
MODULE_LICENSE("GPL v2");
mm_init.o mmu_context.o percpu.o slab_common.o \
compaction.o vmacache.o \
interval_tree.o list_lru.o workingset.o \
- iov_iter.o debug.o $(mmu-y)
+ debug.o $(mmu-y)
obj-y += init-mm.o
return (1UL << (align_order - cma->order_per_bit)) - 1;
}
+/*
+ * Find a PFN aligned to the specified order and return an offset represented in
+ * order_per_bits.
+ */
static unsigned long cma_bitmap_aligned_offset(struct cma *cma, int align_order)
{
- unsigned int alignment;
-
if (align_order <= cma->order_per_bit)
return 0;
- alignment = 1UL << (align_order - cma->order_per_bit);
- return ALIGN(cma->base_pfn, alignment) -
- (cma->base_pfn >> cma->order_per_bit);
+
+ return (ALIGN(cma->base_pfn, (1UL << align_order))
+ - cma->base_pfn) >> cma->order_per_bit;
}
static unsigned long cma_bitmap_maxno(struct cma *cma)
int target_nid, last_cpupid = -1;
bool page_locked;
bool migrated = false;
+ bool was_writable;
int flags = 0;
/* A PROT_NONE fault should not end up here */
flags |= TNF_FAULT_LOCAL;
}
- /*
- * Avoid grouping on DSO/COW pages in specific and RO pages
- * in general, RO pages shouldn't hurt as much anyway since
- * they can be in shared cache state.
- */
- if (!pmd_write(pmd))
+ /* See similar comment in do_numa_page for explanation */
+ if (!(vma->vm_flags & VM_WRITE))
flags |= TNF_NO_GROUP;
/*
if (migrated) {
flags |= TNF_MIGRATED;
page_nid = target_nid;
- }
+ } else
+ flags |= TNF_MIGRATE_FAIL;
goto out;
clear_pmdnuma:
BUG_ON(!PageLocked(page));
+ was_writable = pmd_write(pmd);
pmd = pmd_modify(pmd, vma->vm_page_prot);
+ pmd = pmd_mkyoung(pmd);
+ if (was_writable)
+ pmd = pmd_mkwrite(pmd);
set_pmd_at(mm, haddr, pmdp, pmd);
update_mmu_cache_pmd(vma, addr, pmdp);
unlock_page(page);
if (__pmd_trans_huge_lock(pmd, vma, &ptl) == 1) {
pmd_t entry;
+ bool preserve_write = prot_numa && pmd_write(*pmd);
+ ret = 1;
/*
* Avoid trapping faults against the zero page. The read-only
*/
if (prot_numa && is_huge_zero_pmd(*pmd)) {
spin_unlock(ptl);
- return 0;
+ return ret;
}
if (!prot_numa || !pmd_protnone(*pmd)) {
- ret = 1;
entry = pmdp_get_and_clear_notify(mm, addr, pmd);
entry = pmd_modify(entry, newprot);
+ if (preserve_write)
+ entry = pmd_mkwrite(entry);
ret = HPAGE_PMD_NR;
set_pmd_at(mm, addr, pmd, entry);
- BUG_ON(pmd_write(entry));
+ BUG_ON(!preserve_write && pmd_write(entry));
}
spin_unlock(ptl);
}
__SetPageHead(page);
__ClearPageReserved(page);
for (i = 1; i < nr_pages; i++, p = mem_map_next(p, page, i)) {
- __SetPageTail(p);
/*
* For gigantic hugepages allocated through bootmem at
* boot, it's safer to be consistent with the not-gigantic
__ClearPageReserved(p);
set_page_count(p, 0);
p->first_page = page;
+ /* Make sure p->first_page is always valid for PageTail() */
+ smp_wmb();
+ __SetPageTail(p);
}
}
#include <linux/stacktrace.h>
#include <linux/string.h>
#include <linux/types.h>
+#include <linux/vmalloc.h>
#include <linux/kasan.h>
#include "kasan.h"
GFP_KERNEL | __GFP_HIGHMEM | __GFP_ZERO,
PAGE_KERNEL, VM_NO_GUARD, NUMA_NO_NODE,
__builtin_return_address(0));
- return ret ? 0 : -ENOMEM;
+
+ if (ret) {
+ find_vm_area(addr)->flags |= VM_KASAN;
+ return 0;
+ }
+
+ return -ENOMEM;
}
-void kasan_module_free(void *addr)
+void kasan_free_shadow(const struct vm_struct *vm)
{
- vfree(kasan_mem_to_shadow(addr));
+ if (vm->flags & VM_KASAN)
+ vfree(kasan_mem_to_shadow(vm->addr));
}
static void register_global(struct kasan_global *global)
* on for the root memcg is enough.
*/
if (cgroup_on_dfl(root_css->cgroup))
- mem_cgroup_from_css(root_css)->use_hierarchy = true;
+ root_mem_cgroup->use_hierarchy = true;
+ else
+ root_mem_cgroup->use_hierarchy = false;
}
static u64 memory_current_read(struct cgroup_subsys_state *css,
unsigned long low = ACCESS_ONCE(memcg->low);
if (low == PAGE_COUNTER_MAX)
- seq_puts(m, "infinity\n");
+ seq_puts(m, "max\n");
else
seq_printf(m, "%llu\n", (u64)low * PAGE_SIZE);
int err;
buf = strstrip(buf);
- err = page_counter_memparse(buf, "infinity", &low);
+ err = page_counter_memparse(buf, "max", &low);
if (err)
return err;
unsigned long high = ACCESS_ONCE(memcg->high);
if (high == PAGE_COUNTER_MAX)
- seq_puts(m, "infinity\n");
+ seq_puts(m, "max\n");
else
seq_printf(m, "%llu\n", (u64)high * PAGE_SIZE);
int err;
buf = strstrip(buf);
- err = page_counter_memparse(buf, "infinity", &high);
+ err = page_counter_memparse(buf, "max", &high);
if (err)
return err;
unsigned long max = ACCESS_ONCE(memcg->memory.limit);
if (max == PAGE_COUNTER_MAX)
- seq_puts(m, "infinity\n");
+ seq_puts(m, "max\n");
else
seq_printf(m, "%llu\n", (u64)max * PAGE_SIZE);
int err;
buf = strstrip(buf);
- err = page_counter_memparse(buf, "infinity", &max);
+ err = page_counter_memparse(buf, "max", &max);
if (err)
return err;
if (memcg == root_mem_cgroup)
return false;
- if (page_counter_read(&memcg->memory) > memcg->low)
+ if (page_counter_read(&memcg->memory) >= memcg->low)
return false;
while (memcg != root) {
if (memcg == root_mem_cgroup)
break;
- if (page_counter_read(&memcg->memory) > memcg->low)
+ if (page_counter_read(&memcg->memory) >= memcg->low)
return false;
}
return true;
int last_cpupid;
int target_nid;
bool migrated = false;
+ bool was_writable = pte_write(pte);
int flags = 0;
/* A PROT_NONE fault should not end up here */
/* Make it present again */
pte = pte_modify(pte, vma->vm_page_prot);
pte = pte_mkyoung(pte);
+ if (was_writable)
+ pte = pte_mkwrite(pte);
set_pte_at(mm, addr, ptep, pte);
update_mmu_cache(vma, addr, ptep);
}
/*
- * Avoid grouping on DSO/COW pages in specific and RO pages
- * in general, RO pages shouldn't hurt as much anyway since
- * they can be in shared cache state.
+ * Avoid grouping on RO pages in general. RO pages shouldn't hurt as
+ * much anyway since they can be in shared cache state. This misses
+ * the case where a mapping is writable but the process never writes
+ * to it but pte_write gets cleared during protection updates and
+ * pte_dirty has unpredictable behaviour between PTE scan updates,
+ * background writeback, dirty balancing and application behaviour.
*/
- if (!pte_write(pte))
+ if (!(vma->vm_flags & VM_WRITE))
flags |= TNF_NO_GROUP;
/*
if (migrated) {
page_nid = target_nid;
flags |= TNF_MIGRATED;
- }
+ } else
+ flags |= TNF_MIGRATE_FAIL;
out:
if (page_nid != -1)
return NULL;
arch_refresh_nodedata(nid, pgdat);
+ } else {
+ /* Reset the nr_zones and classzone_idx to 0 before reuse */
+ pgdat->nr_zones = 0;
+ pgdat->classzone_idx = 0;
}
/* we can use NODE_DATA(nid) from here */
if (is_vmalloc_addr(zone->wait_table))
vfree(zone->wait_table);
}
-
- /*
- * Since there is no way to guarentee the address of pgdat/zone is not
- * on stack of any kernel threads or used by other kernel objects
- * without reference counting or other symchronizing method, do not
- * reset node_data and free pgdat here. Just reset it to 0 and reuse
- * the memory when the node is online again.
- */
- memset(pgdat, 0, sizeof(*pgdat));
}
EXPORT_SYMBOL(try_offline_node);
int can_do_mlock(void)
{
- if (capable(CAP_IPC_LOCK))
- return 1;
if (rlimit(RLIMIT_MEMLOCK) != 0)
return 1;
+ if (capable(CAP_IPC_LOCK))
+ return 1;
return 0;
}
EXPORT_SYMBOL(can_do_mlock);
importer->anon_vma = exporter->anon_vma;
error = anon_vma_clone(importer, exporter);
- if (error) {
- importer->anon_vma = NULL;
+ if (error)
return error;
- }
}
}
oldpte = *pte;
if (pte_present(oldpte)) {
pte_t ptent;
+ bool preserve_write = prot_numa && pte_write(oldpte);
/*
* Avoid trapping faults against the zero or KSM
ptent = ptep_modify_prot_start(mm, addr, pte);
ptent = pte_modify(ptent, newprot);
+ if (preserve_write)
+ ptent = pte_mkwrite(ptent);
/* Avoid taking write faults for known dirty pages */
if (dirty_accountable && pte_dirty(ptent) &&
old_len = new_len;
old_addr = new_addr;
new_addr = -ENOMEM;
- } else if (vma->vm_file && vma->vm_file->f_op->mremap)
- vma->vm_file->f_op->mremap(vma->vm_file, new_vma);
+ } else if (vma->vm_file && vma->vm_file->f_op->mremap) {
+ err = vma->vm_file->f_op->mremap(vma->vm_file, new_vma);
+ if (err < 0) {
+ move_page_tables(new_vma, new_addr, vma, old_addr,
+ moved_len, true);
+ return err;
+ }
+ }
/* Conceal VM_ACCOUNT so old reservation is not undone */
if (vm_flags & VM_ACCOUNT) {
EXPORT_SYMBOL(high_memory);
struct page *mem_map;
unsigned long max_mapnr;
+EXPORT_SYMBOL(max_mapnr);
unsigned long highest_memmap_pfn;
struct percpu_counter vm_committed_as;
int sysctl_overcommit_memory = OVERCOMMIT_GUESS; /* heuristic overcommit */
if (sysctl_nr_trim_pages && total - point >= sysctl_nr_trim_pages) {
total = point;
kdebug("try to alloc exact %lu pages", total);
- base = alloc_pages_exact(len, GFP_KERNEL);
- } else {
- base = (void *)__get_free_pages(GFP_KERNEL, order);
}
+ base = alloc_pages_exact(total << PAGE_SHIFT, GFP_KERNEL);
if (!base)
goto enomem;
* bw * elapsed + write_bandwidth * (period - elapsed)
* write_bandwidth = ---------------------------------------------------
* period
+ *
+ * @written may have decreased due to account_page_redirty().
+ * Avoid underflowing @bw calculation.
*/
- bw = written - bdi->written_stamp;
+ bw = written - min(written, bdi->written_stamp);
bw *= HZ;
if (unlikely(elapsed > period)) {
do_div(bw, elapsed);
unsigned long now)
{
static DEFINE_SPINLOCK(dirty_lock);
- static unsigned long update_time;
+ static unsigned long update_time = INITIAL_JIFFIES;
/*
* check locklessly first to optimize away locking for the most time
if (ac->high_zoneidx < ZONE_NORMAL)
goto out;
/* The OOM killer does not compensate for light reclaim */
- if (!(gfp_mask & __GFP_FS))
+ if (!(gfp_mask & __GFP_FS)) {
+ /*
+ * XXX: Page reclaim didn't yield anything,
+ * and the OOM killer can't be invoked, but
+ * keep looping as per should_alloc_retry().
+ */
+ *did_some_progress = 1;
goto out;
+ }
/*
* GFP_THISNODE contains __GFP_NORETRY and we never hit this.
* Sanity check for bare calls of __GFP_THISNODE, not real OOM.
goto out;
}
/* Exhausted what can be done so it's blamo time */
- if (out_of_memory(ac->zonelist, gfp_mask, order, ac->nodemask, false))
+ if (out_of_memory(ac->zonelist, gfp_mask, order, ac->nodemask, false)
+ || WARN_ON_ONCE(gfp_mask & __GFP_NOFAIL))
*did_some_progress = 1;
out:
oom_zonelist_unlock(ac->zonelist, gfp_mask);
if (!is_migrate_isolate_page(buddy)) {
__isolate_free_page(page, order);
+ kernel_map_pages(page, (1 << order), 1);
set_page_refcounted(page);
isolated_page = page;
}
vma = vma->vm_next;
err = walk_page_test(start, next, walk);
- if (err > 0)
+ if (err > 0) {
+ /*
+ * positive return values are purely for
+ * controlling the pagewalk, so should never
+ * be passed to the callers.
+ */
+ err = 0;
continue;
+ }
if (err < 0)
break;
}
* and, from the second one, the backing allocator (currently either vm or
* km) provides translation.
*
- * The addr can be tranlated simply without checking if it falls into the
+ * The addr can be translated simply without checking if it falls into the
* first chunk. But the current code reflects better how percpu allocator
* actually works, and the verification can discover both bugs in percpu
* allocator itself and per_cpu_ptr_to_phys() callers. So we keep current
* and other parameters considering needed percpu size, allocation
* atom size and distances between CPUs.
*
- * Groups are always mutliples of atom size and CPUs which are of
+ * Groups are always multiples of atom size and CPUs which are of
* LOCAL_DISTANCE both ways are grouped together and share space for
* units in the same group. The returned configuration is guaranteed
* to have CPUs on different nodes on different groups and >=75% usage
return 0;
enomem_failure:
+ /*
+ * dst->anon_vma is dropped here otherwise its degree can be incorrectly
+ * decremented in unlink_anon_vmas().
+ * We can safely do this because callers of anon_vma_clone() don't care
+ * about dst->anon_vma if anon_vma_clone() failed.
+ */
+ dst->anon_vma = NULL;
unlink_anon_vmas(dst);
return -ENOMEM;
}
bool shmem_mapping(struct address_space *mapping)
{
+ if (!mapping->host)
+ return false;
+
return mapping->host->i_sb->s_op == &shmem_ops;
}
do {
tid = this_cpu_read(s->cpu_slab->tid);
c = raw_cpu_ptr(s->cpu_slab);
- } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
+ } while (IS_ENABLED(CONFIG_PREEMPT) &&
+ unlikely(tid != READ_ONCE(c->tid)));
/*
* Irqless object alloc/free algorithm used here depends on sequence
do {
tid = this_cpu_read(s->cpu_slab->tid);
c = raw_cpu_ptr(s->cpu_slab);
- } while (IS_ENABLED(CONFIG_PREEMPT) && unlikely(tid != c->tid));
+ } while (IS_ENABLED(CONFIG_PREEMPT) &&
+ unlikely(tid != READ_ONCE(c->tid)));
/* Same with comment on barrier() in slab_alloc_node() */
barrier();
spin_unlock(&vmap_area_lock);
vmap_debug_free_range(va->va_start, va->va_end);
+ kasan_free_shadow(vm);
free_unmap_vmap_area(va);
vm->size -= PAGE_SIZE;
static void p9_virtio_remove(struct virtio_device *vdev)
{
struct virtio_chan *chan = vdev->priv;
-
- if (chan->inuse)
- p9_virtio_close(chan->client);
- vdev->config->del_vqs(vdev);
+ unsigned long warning_time;
mutex_lock(&virtio_9p_lock);
+
+ /* Remove self from list so we don't get new users. */
list_del(&chan->chan_list);
+ warning_time = jiffies;
+
+ /* Wait for existing users to close. */
+ while (chan->inuse) {
+ mutex_unlock(&virtio_9p_lock);
+ msleep(250);
+ if (time_after(jiffies, warning_time + 10 * HZ)) {
+ dev_emerg(&vdev->dev,
+ "p9_virtio_remove: waiting for device in use.\n");
+ warning_time = jiffies;
+ }
+ mutex_lock(&virtio_9p_lock);
+ }
+
mutex_unlock(&virtio_9p_lock);
+
+ vdev->config->del_vqs(vdev);
+
sysfs_remove_file(&(vdev->dev.kobj), &dev_attr_mount_tag.attr);
kobject_uevent(&(vdev->dev.kobj), KOBJ_CHANGE);
kfree(chan->tag);
{
int err;
+ BUILD_BUG_ON(sizeof(struct br_input_skb_cb) > FIELD_SIZEOF(struct sk_buff, cb));
+
err = stp_proto_register(&br_stp_proto);
if (err < 0) {
pr_err("bridge: can't register sap for STP\n");
*/
del_nbp(p);
+ dev_set_mtu(br->dev, br_min_mtu(br));
+
spin_lock_bh(&br->lock);
changed_addr = br_stp_recalculate_bridge_id(br);
spin_unlock_bh(&br->lock);
int copylen;
ret = -EOPNOTSUPP;
- if (m->msg_flags&MSG_OOB)
+ if (flags & MSG_OOB)
goto read_error;
skb = skb_recv_datagram(sk, flags, 0 , &ret);
u16 tmp;
u16 len;
u16 hdrchks;
- u16 pktchks;
+ int pktchks;
struct cffrml *this;
this = container_obj(layr);
return skb->len;
}
-inline u16 cfpkt_iterate(struct cfpkt *pkt,
- u16 (*iter_func)(u16, void *, u16),
- u16 data)
+int cfpkt_iterate(struct cfpkt *pkt,
+ u16 (*iter_func)(u16, void *, u16),
+ u16 data)
{
/*
* Don't care about the performance hit of linearizing,
goto inval_skb;
}
+ skb->ip_summed = CHECKSUM_UNNECESSARY;
+
+ skb_reset_mac_header(skb);
skb_reset_network_header(skb);
skb_reset_transport_header(skb);
IPPROTO_TCP, &sock);
if (ret)
return ret;
- sock->sk->sk_allocation = GFP_NOFS | __GFP_MEMALLOC;
+ sock->sk->sk_allocation = GFP_NOFS;
#ifdef CONFIG_LOCKDEP
lockdep_set_class(&sock->sk->sk_lock, &socket_class);
ret);
}
- sk_set_memalloc(sock->sk);
-
con->sock = sock;
return 0;
}
{
struct ceph_connection *con = container_of(work, struct ceph_connection,
work.work);
- unsigned long pflags = current->flags;
bool fault;
- current->flags |= PF_MEMALLOC;
-
mutex_lock(&con->mutex);
while (true) {
int ret;
con_fault_finish(con);
con->ops->put(con);
-
- tsk_restore_flags(current, pflags, PF_MEMALLOC);
}
/*
__get_user(kmsg->msg_controllen, &umsg->msg_controllen) ||
__get_user(kmsg->msg_flags, &umsg->msg_flags))
return -EFAULT;
+
+ if (!uaddr)
+ kmsg->msg_namelen = 0;
+
+ if (kmsg->msg_namelen < 0)
+ return -EINVAL;
+
if (kmsg->msg_namelen > sizeof(struct sockaddr_storage))
kmsg->msg_namelen = sizeof(struct sockaddr_storage);
kmsg->msg_control = compat_ptr(tmp3);
COMPAT_SYSCALL_DEFINE3(sendmsg, int, fd, struct compat_msghdr __user *, msg, unsigned int, flags)
{
- if (flags & MSG_CMSG_COMPAT)
- return -EINVAL;
return __sys_sendmsg(fd, (struct user_msghdr __user *)msg, flags | MSG_CMSG_COMPAT);
}
COMPAT_SYSCALL_DEFINE4(sendmmsg, int, fd, struct compat_mmsghdr __user *, mmsg,
unsigned int, vlen, unsigned int, flags)
{
- if (flags & MSG_CMSG_COMPAT)
- return -EINVAL;
return __sys_sendmmsg(fd, (struct mmsghdr __user *)mmsg, vlen,
flags | MSG_CMSG_COMPAT);
}
COMPAT_SYSCALL_DEFINE3(recvmsg, int, fd, struct compat_msghdr __user *, msg, unsigned int, flags)
{
- if (flags & MSG_CMSG_COMPAT)
- return -EINVAL;
return __sys_recvmsg(fd, (struct user_msghdr __user *)msg, flags | MSG_CMSG_COMPAT);
}
int datagrams;
struct timespec ktspec;
- if (flags & MSG_CMSG_COMPAT)
- return -EINVAL;
-
if (timeout == NULL)
return __sys_recvmmsg(fd, (struct mmsghdr __user *)mmsg, vlen,
flags | MSG_CMSG_COMPAT, NULL);
return false;
while (*name) {
- if (*name == '/' || isspace(*name))
+ if (*name == '/' || *name == ':' || isspace(*name))
return false;
name++;
}
#define skb_update_prio(skb)
#endif
-static DEFINE_PER_CPU(int, xmit_recursion);
+DEFINE_PER_CPU(int, xmit_recursion);
+EXPORT_SYMBOL(xmit_recursion);
+
#define RECURSION_LIMIT 10
/**
[NETIF_F_RXALL_BIT] = "rx-all",
[NETIF_F_HW_L2FW_DOFFLOAD_BIT] = "l2-fwd-offload",
[NETIF_F_BUSY_POLL_BIT] = "busy-poll",
+ [NETIF_F_HW_SWITCH_OFFLOAD_BIT] = "hw-switch-offload",
};
static const char
spin_lock(&net->rules_mod_lock);
list_del_rcu(&ops->list);
- fib_rules_cleanup_ops(ops);
spin_unlock(&net->rules_mod_lock);
+ fib_rules_cleanup_ops(ops);
call_rcu(&ops->rcu, fib_rules_put_rcu);
}
EXPORT_SYMBOL_GPL(fib_rules_unregister);
return 0;
nla_put_failure:
+ kfree(d->xstats);
+ d->xstats = NULL;
+ d->xstats_len = 0;
spin_unlock_bh(d->lock);
return -1;
}
gnet_stats_copy_app(struct gnet_dump *d, void *st, int len)
{
if (d->compat_xstats) {
- d->xstats = st;
+ d->xstats = kmemdup(st, len, GFP_ATOMIC);
+ if (!d->xstats)
+ goto err_out;
d->xstats_len = len;
}
return gnet_stats_copy(d, TCA_STATS_APP, st, len);
return 0;
+
+err_out:
+ d->xstats_len = 0;
+ spin_unlock_bh(d->lock);
+ return -1;
}
EXPORT_SYMBOL(gnet_stats_copy_app);
return -1;
}
+ kfree(d->xstats);
+ d->xstats = NULL;
+ d->xstats_len = 0;
spin_unlock_bh(d->lock);
return 0;
}
*/
int peernet2id(struct net *net, struct net *peer)
{
- int id = __peernet2id(net, peer, true);
+ bool alloc = atomic_read(&peer->count) == 0 ? false : true;
+ int id;
+ id = __peernet2id(net, peer, alloc);
return id >= 0 ? id : NETNSA_NSID_NOT_ASSIGNED;
}
EXPORT_SYMBOL(peernet2id);
return len;
i += len;
+ if ((value > 1) &&
+ (!(pkt_dev->odev->priv_flags & IFF_TX_SKB_SHARING)))
+ return -ENOTSUPP;
pkt_dev->burst = value < 1 ? 1 : value;
sprintf(pg_result, "OK: burst=%d", pkt_dev->burst);
return count;
s_h = cb->args[0];
s_idx = cb->args[1];
- rcu_read_lock();
cb->seq = net->dev_base_seq;
/* A hack to preserve kernel<->userspace interface.
for (h = s_h; h < NETDEV_HASHENTRIES; h++, s_idx = 0) {
idx = 0;
head = &net->dev_index_head[h];
- hlist_for_each_entry_rcu(dev, head, index_hlist) {
+ hlist_for_each_entry(dev, head, index_hlist) {
if (idx < s_idx)
goto cont;
err = rtnl_fill_ifinfo(skb, dev, RTM_NEWLINK,
}
}
out:
- rcu_read_unlock();
cb->args[1] = idx;
cb->args[0] = h;
struct ifinfomsg *ifm,
struct nlattr **tb)
{
- struct net_device *dev;
+ struct net_device *dev, *aux;
int err;
- for_each_netdev(net, dev) {
+ for_each_netdev_safe(net, dev, aux) {
if (dev->group == group) {
err = do_setlink(skb, dev, ifm, tb, NULL, 0);
if (err < 0)
}
if (1) {
- struct nlattr *attr[ops ? ops->maxtype + 1 : 0];
- struct nlattr *slave_attr[m_ops ? m_ops->slave_maxtype + 1 : 0];
+ struct nlattr *attr[ops ? ops->maxtype + 1 : 1];
+ struct nlattr *slave_attr[m_ops ? m_ops->slave_maxtype + 1 : 1];
struct nlattr **data = NULL;
struct nlattr **slave_data = NULL;
struct net *dest_net, *link_net = NULL;
if (IS_ERR(dest_net))
return PTR_ERR(dest_net);
+ err = -EPERM;
+ if (!netlink_ns_capable(skb, dest_net->user_ns, CAP_NET_ADMIN))
+ goto out;
+
if (tb[IFLA_LINK_NETNSID]) {
int id = nla_get_s32(tb[IFLA_LINK_NETNSID]);
err = -EINVAL;
goto out;
}
+ err = -EPERM;
+ if (!netlink_ns_capable(skb, link_net->user_ns, CAP_NET_ADMIN))
+ goto out;
}
dev = rtnl_create_link(link_net ? : dest_net, ifname,
}
}
err = rtnl_configure_link(dev, ifm);
- if (err < 0) {
- if (ops->newlink) {
- LIST_HEAD(list_kill);
-
- ops->dellink(dev, &list_kill);
- unregister_netdevice_many(&list_kill);
- } else {
- unregister_netdevice(dev);
- }
- goto out;
- }
-
+ if (err < 0)
+ goto out_unregister;
if (link_net) {
err = dev_change_net_namespace(dev, dest_net, ifname);
if (err < 0)
- unregister_netdevice(dev);
+ goto out_unregister;
}
out:
if (link_net)
put_net(link_net);
put_net(dest_net);
return err;
+out_unregister:
+ if (ops->newlink) {
+ LIST_HEAD(list_kill);
+
+ ops->dellink(dev, &list_kill);
+ unregister_netdevice_many(&list_kill);
+ } else {
+ unregister_netdevice(dev);
+ }
+ goto out;
}
}
{
struct sk_buff_head *q = &sk->sk_error_queue;
struct sk_buff *skb, *skb_next;
+ unsigned long flags;
int err = 0;
- spin_lock_bh(&q->lock);
+ spin_lock_irqsave(&q->lock, flags);
skb = __skb_dequeue(q);
if (skb && (skb_next = skb_peek(q)))
err = SKB_EXT_ERR(skb_next)->ee.ee_errno;
- spin_unlock_bh(&q->lock);
+ spin_unlock_irqrestore(&q->lock, flags);
sk->sk_err = err;
if (err)
struct sock *sk, int tstype)
{
struct sk_buff *skb;
- bool tsonly = sk->sk_tsflags & SOF_TIMESTAMPING_OPT_TSONLY;
+ bool tsonly;
+
+ if (!sk)
+ return;
- if (!sk || !skb_may_tx_timestamp(sk, tsonly))
+ tsonly = sk->sk_tsflags & SOF_TIMESTAMPING_OPT_TSONLY;
+ if (!skb_may_tx_timestamp(sk, tsonly))
return;
if (tsonly)
skb->ignore_df = 0;
skb_dst_drop(skb);
skb->mark = 0;
- skb->sender_cpu = 0;
+ skb_sender_cpu_clear(skb);
skb_init_secmark(skb);
secpath_reset(skb);
nf_reset(skb);
sock_reset_flag(sk, bit);
}
+bool sk_mc_loop(struct sock *sk)
+{
+ if (dev_recursion_level())
+ return false;
+ if (!sk)
+ return true;
+ switch (sk->sk_family) {
+ case AF_INET:
+ return inet_sk(sk)->mc_loop;
+#if IS_ENABLED(CONFIG_IPV6)
+ case AF_INET6:
+ return inet6_sk(sk)->mc_loop;
+#endif
+ }
+ WARN_ON(1);
+ return true;
+}
+EXPORT_SYMBOL(sk_mc_loop);
+
/*
* This is meant for all protocols to use and covers goings on
* at the socket level. Everything here is generic.
}
EXPORT_SYMBOL(sock_rfree);
+/*
+ * Buffer destructor for skbs that are not used directly in read or write
+ * path, e.g. for error handler skbs. Automatically called from kfree_skb.
+ */
void sock_efree(struct sk_buff *skb)
{
sock_put(skb->sk);
static int zero = 0;
static int one = 1;
static int ushort_max = USHRT_MAX;
+static int min_sndbuf = SOCK_MIN_SNDBUF;
+static int min_rcvbuf = SOCK_MIN_RCVBUF;
static int net_msg_warn; /* Unused, but still a sysctl */
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = &min_sndbuf,
},
{
.procname = "rmem_max",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = &min_rcvbuf,
},
{
.procname = "wmem_default",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = &min_sndbuf,
},
{
.procname = "rmem_default",
.maxlen = sizeof(int),
.mode = 0644,
.proc_handler = proc_dointvec_minmax,
- .extra1 = &one,
+ .extra1 = &min_rcvbuf,
},
{
.procname = "dev_weight",
if (decnet_debug_level & 16)
printk(KERN_DEBUG
"dn_route_output_slow: initial checks complete."
- " dst=%o4x src=%04x oif=%d try_hard=%d\n",
+ " dst=%04x src=%04x oif=%d try_hard=%d\n",
le16_to_cpu(fld.daddr), le16_to_cpu(fld.saddr),
fld.flowidn_oif, try_hard);
void __exit dn_fib_rules_cleanup(void)
{
+ rtnl_lock();
fib_rules_unregister(dn_fib_rules_ops);
+ rtnl_unlock();
rcu_barrier();
}
#ifdef CONFIG_OF
static int dsa_of_setup_routing_table(struct dsa_platform_data *pd,
struct dsa_chip_data *cd,
- int chip_index,
+ int chip_index, int port_index,
struct device_node *link)
{
- int ret;
const __be32 *reg;
- int link_port_addr;
int link_sw_addr;
struct device_node *parent_sw;
int len;
if (!reg || (len != sizeof(*reg) * 2))
return -EINVAL;
+ /*
+ * Get the destination switch number from the second field of its 'reg'
+ * property, i.e. for "reg = <0x19 1>" sw_addr is '1'.
+ */
link_sw_addr = be32_to_cpup(reg + 1);
if (link_sw_addr >= pd->nr_chips)
memset(cd->rtable, -1, pd->nr_chips * sizeof(s8));
}
- reg = of_get_property(link, "reg", NULL);
- if (!reg) {
- ret = -EINVAL;
- goto out;
- }
-
- link_port_addr = be32_to_cpup(reg);
-
- cd->rtable[link_sw_addr] = link_port_addr;
+ cd->rtable[link_sw_addr] = port_index;
return 0;
-out:
- kfree(cd->rtable);
- return ret;
}
static void dsa_of_free_platform_data(struct dsa_platform_data *pd)
if (!strcmp(port_name, "dsa") && link &&
pd->nr_chips > 1) {
ret = dsa_of_setup_routing_table(pd, cd,
- chip_index, link);
+ chip_index, port_index, link);
if (ret)
goto out_free_chip;
}
struct hsr_port *port;
hsr = netdev_priv(hsr_dev);
+
+ rtnl_lock();
hsr_for_each_port(hsr, port)
hsr_del_port(port);
+ rtnl_unlock();
del_timer_sync(&hsr->prune_timer);
del_timer_sync(&hsr->announce_timer);
return NOTIFY_DONE; /* Not an HSR device */
hsr = netdev_priv(dev);
port = hsr_port_get_hsr(hsr, HSR_PT_MASTER);
+ if (port == NULL) {
+ /* Resend of notification concerning removed device? */
+ return NOTIFY_DONE;
+ }
} else {
hsr = port->hsr;
}
list_del_rcu(&port->port_list);
if (port != master) {
- netdev_update_features(master->dev);
- dev_set_mtu(master->dev, hsr_get_max_mtu(hsr));
+ if (master != NULL) {
+ netdev_update_features(master->dev);
+ dev_set_mtu(master->dev, hsr_get_max_mtu(hsr));
+ }
netdev_rx_handler_unregister(port->dev);
dev_set_promiscuity(port->dev, -1);
}
*/
synchronize_rcu();
- dev_put(port->dev);
+
+ if (port != master)
+ dev_put(port->dev);
}
{
unsigned int i;
+ rtnl_lock();
#ifdef CONFIG_IP_MULTIPLE_TABLES
fib4_rules_exit(net);
#endif
-
- rtnl_lock();
for (i = 0; i < FIB_TABLE_HASHSZ; i++) {
struct fib_table *tb;
struct hlist_head *head;
release_sock(sk);
if (reqsk_queue_empty(&icsk->icsk_accept_queue))
timeo = schedule_timeout(timeo);
+ sched_annotate_sleep();
lock_sock(sk);
err = 0;
if (!reqsk_queue_empty(&icsk->icsk_accept_queue))
mutex_unlock(&inet_diag_table_mutex);
}
+static size_t inet_sk_attr_size(void)
+{
+ return nla_total_size(sizeof(struct tcp_info))
+ + nla_total_size(1) /* INET_DIAG_SHUTDOWN */
+ + nla_total_size(1) /* INET_DIAG_TOS */
+ + nla_total_size(1) /* INET_DIAG_TCLASS */
+ + nla_total_size(sizeof(struct inet_diag_meminfo))
+ + nla_total_size(sizeof(struct inet_diag_msg))
+ + nla_total_size(SK_MEMINFO_VARS * sizeof(u32))
+ + nla_total_size(TCP_CA_NAME_MAX)
+ + nla_total_size(sizeof(struct tcpvegas_info))
+ + 64;
+}
+
int inet_sk_diag_fill(struct sock *sk, struct inet_connection_sock *icsk,
struct sk_buff *skb, struct inet_diag_req_v2 *req,
struct user_namespace *user_ns,
if (err)
goto out;
- rep = nlmsg_new(sizeof(struct inet_diag_msg) +
- sizeof(struct inet_diag_meminfo) +
- sizeof(struct tcp_info) + 64, GFP_KERNEL);
+ rep = nlmsg_new(inet_sk_attr_size(), GFP_KERNEL);
if (!rep) {
err = -ENOMEM;
goto out;
if (unlikely(opt->optlen))
ip_forward_options(skb);
+ skb_sender_cpu_clear(skb);
return dst_output(skb);
}
struct sk_buff *ip_check_defrag(struct sk_buff *skb, u32 user)
{
struct iphdr iph;
+ int netoff;
u32 len;
if (skb->protocol != htons(ETH_P_IP))
return skb;
- if (!skb_copy_bits(skb, 0, &iph, sizeof(iph)))
+ netoff = skb_network_offset(skb);
+
+ if (skb_copy_bits(skb, netoff, &iph, sizeof(iph)) < 0)
return skb;
if (iph.ihl < 5 || iph.version != 4)
return skb;
len = ntohs(iph.tot_len);
- if (skb->len < len || len < (iph.ihl * 4))
+ if (skb->len < netoff + len || len < (iph.ihl * 4))
return skb;
if (ip_is_fragment(&iph)) {
skb = skb_share_check(skb, GFP_ATOMIC);
if (skb) {
- if (!pskb_may_pull(skb, iph.ihl*4))
+ if (!pskb_may_pull(skb, netoff + iph.ihl * 4))
return skb;
- if (pskb_trim_rcsum(skb, len))
+ if (pskb_trim_rcsum(skb, netoff + len))
return skb;
memset(IPCB(skb), 0, sizeof(struct inet_skb_parm));
if (ip_defrag(skb, user))
cork->length += length;
if (((length > mtu) || (skb && skb_is_gso(skb))) &&
(sk->sk_protocol == IPPROTO_UDP) &&
- (rt->dst.dev->features & NETIF_F_UFO) && !rt->dst.header_len) {
+ (rt->dst.dev->features & NETIF_F_UFO) && !rt->dst.header_len &&
+ (sk->sk_type == SOCK_DGRAM)) {
err = ip_ufo_append_data(sk, queue, getfrag, from, length,
hh_len, fragheaderlen, transhdrlen,
maxfraglen, flags);
kfree_skb(skb);
}
-static bool ipv4_pktinfo_prepare_errqueue(const struct sock *sk,
- const struct sk_buff *skb,
- int ee_origin)
+/* IPv4 supports cmsg on all imcp errors and some timestamps
+ *
+ * Timestamp code paths do not initialize the fields expected by cmsg:
+ * the PKTINFO fields in skb->cb[]. Fill those in here.
+ */
+static bool ipv4_datagram_support_cmsg(const struct sock *sk,
+ struct sk_buff *skb,
+ int ee_origin)
{
- struct in_pktinfo *info = PKTINFO_SKB_CB(skb);
+ struct in_pktinfo *info;
+
+ if (ee_origin == SO_EE_ORIGIN_ICMP)
+ return true;
- if ((ee_origin != SO_EE_ORIGIN_TIMESTAMPING) ||
- (!(sk->sk_tsflags & SOF_TIMESTAMPING_OPT_CMSG)) ||
+ if (ee_origin == SO_EE_ORIGIN_LOCAL)
+ return false;
+
+ /* Support IP_PKTINFO on tstamp packets if requested, to correlate
+ * timestamp with egress dev. Not possible for packets without dev
+ * or without payload (SOF_TIMESTAMPING_OPT_TSONLY).
+ */
+ if ((!(sk->sk_tsflags & SOF_TIMESTAMPING_OPT_CMSG)) ||
(!skb->dev))
return false;
+ info = PKTINFO_SKB_CB(skb);
info->ipi_spec_dst.s_addr = ip_hdr(skb)->saddr;
info->ipi_ifindex = skb->dev->ifindex;
return true;
serr = SKB_EXT_ERR(skb);
- if (sin && skb->len) {
+ if (sin && serr->port) {
sin->sin_family = AF_INET;
sin->sin_addr.s_addr = *(__be32 *)(skb_network_header(skb) +
serr->addr_offset);
sin = &errhdr.offender;
memset(sin, 0, sizeof(*sin));
- if (skb->len &&
- (serr->ee.ee_origin == SO_EE_ORIGIN_ICMP ||
- ipv4_pktinfo_prepare_errqueue(sk, skb, serr->ee.ee_origin))) {
+ if (ipv4_datagram_support_cmsg(sk, skb, serr->ee.ee_origin)) {
sin->sin_family = AF_INET;
sin->sin_addr.s_addr = ip_hdr(skb)->saddr;
if (inet_sk(sk)->cmsg_flags)
return 0;
err2:
- kfree(mrt);
+ ipmr_free_table(mrt);
err1:
fib_rules_unregister(ops);
return err;
{
struct mr_table *mrt, *next;
+ rtnl_lock();
list_for_each_entry_safe(mrt, next, &net->ipv4.mr_tables, list) {
list_del(&mrt->list);
ipmr_free_table(mrt);
}
fib_rules_unregister(net->ipv4.mr_rules_ops);
+ rtnl_unlock();
}
#else
#define ipmr_for_each_table(mrt, net) \
static void __net_exit ipmr_rules_exit(struct net *net)
{
+ rtnl_lock();
ipmr_free_table(net->ipv4.mrt);
+ net->ipv4.mrt = NULL;
+ rtnl_unlock();
}
#endif
&chainname, &comment, &rulenum) != 0)
break;
- nf_log_packet(net, AF_INET, hook, skb, in, out, &trace_loginfo,
- "TRACE: %s:%s:%s:%u ",
- tablename, chainname, comment, rulenum);
+ nf_log_trace(net, AF_INET, hook, skb, in, out, &trace_loginfo,
+ "TRACE: %s:%s:%s:%u ",
+ tablename, chainname, comment, rulenum);
}
#endif
kgid_t low, high;
int ret = 0;
+ if (sk->sk_family == AF_INET6)
+ sk->sk_ipv6only = 1;
+
inet_get_ping_group_range_net(net, &low, &high);
if (gid_lte(low, group) && gid_lte(group, high))
return 0;
if (addr_len < sizeof(*addr))
return -EINVAL;
+ if (addr->sin_family != AF_INET &&
+ !(addr->sin_family == AF_UNSPEC &&
+ addr->sin_addr.s_addr == htonl(INADDR_ANY)))
+ return -EAFNOSUPPORT;
+
pr_debug("ping_check_bind_addr(sk=%p,addr=%pI4,port=%d)\n",
sk, &addr->sin_addr.s_addr, ntohs(addr->sin_port));
return -EINVAL;
if (addr->sin6_family != AF_INET6)
- return -EINVAL;
+ return -EAFNOSUPPORT;
pr_debug("ping_check_bind_addr(sk=%p,addr=%pI6c,port=%d)\n",
sk, addr->sin6_addr.s6_addr, ntohs(addr->sin6_port));
if (msg->msg_namelen < sizeof(*usin))
return -EINVAL;
if (usin->sin_family != AF_INET)
- return -EINVAL;
+ return -EAFNOSUPPORT;
daddr = usin->sin_addr.s_addr;
/* no remote port */
} else {
int large_allowed)
{
struct tcp_sock *tp = tcp_sk(sk);
- u32 new_size_goal, size_goal, hlen;
+ u32 new_size_goal, size_goal;
if (!large_allowed || !sk_can_gso(sk))
return mss_now;
- /* Maybe we should/could use sk->sk_prot->max_header here ? */
- hlen = inet_csk(sk)->icsk_af_ops->net_header_len +
- inet_csk(sk)->icsk_ext_hdr_len +
- tp->tcp_header_len;
-
- new_size_goal = sk->sk_gso_max_size - 1 - hlen;
+ /* Note : tcp_tso_autosize() will eventually split this later */
+ new_size_goal = sk->sk_gso_max_size - 1 - MAX_TCP_HEADER;
new_size_goal = tcp_bound_to_half_wnd(tp, new_size_goal);
/* We try hard to avoid divides here */
*/
void tcp_cong_avoid_ai(struct tcp_sock *tp, u32 w, u32 acked)
{
+ /* If credits accumulated at a higher w, apply them gently now. */
+ if (tp->snd_cwnd_cnt >= w) {
+ tp->snd_cwnd_cnt = 0;
+ tp->snd_cwnd++;
+ }
+
tp->snd_cwnd_cnt += acked;
if (tp->snd_cwnd_cnt >= w) {
u32 delta = tp->snd_cwnd_cnt / w;
}
}
- if (ca->cnt == 0) /* cannot be zero */
- ca->cnt = 1;
+ /* The maximum rate of cwnd increase CUBIC allows is 1 packet per
+ * 2 packets ACKed, meaning cwnd grows at 1.5x per RTT.
+ */
+ ca->cnt = max(ca->cnt, 2U);
}
static void bictcp_cong_avoid(struct sock *sk, u32 ack, u32 acked)
if (!first_ackt.v64)
first_ackt = last_ackt;
- if (!(sacked & TCPCB_SACKED_ACKED))
+ if (!(sacked & TCPCB_SACKED_ACKED)) {
reord = min(pkts_acked, reord);
- if (!after(scb->end_seq, tp->high_seq))
- flag |= FLAG_ORIG_SACK_ACKED;
+ if (!after(scb->end_seq, tp->high_seq))
+ flag |= FLAG_ORIG_SACK_ACKED;
+ }
}
if (sacked & TCPCB_SACKED_ACKED)
return false;
/* If we filled the congestion window, do not expand. */
- if (tp->packets_out >= tp->snd_cwnd)
+ if (tcp_packets_in_flight(tp) >= tp->snd_cwnd)
return false;
return true;
skb->sk = sk;
skb->destructor = sock_edemux;
if (sk->sk_state != TCP_TIME_WAIT) {
- struct dst_entry *dst = sk->sk_rx_dst;
+ struct dst_entry *dst = READ_ONCE(sk->sk_rx_dst);
if (dst)
dst = dst_check(dst, 0);
} else {
/* Socket is locked, keep trying until memory is available. */
for (;;) {
- skb = alloc_skb_fclone(MAX_TCP_HEADER,
- sk->sk_allocation);
+ skb = sk_stream_alloc_skb(sk, 0, sk->sk_allocation);
if (skb)
break;
yield();
}
-
- /* Reserve space for headers and prepare control bits. */
- skb_reserve(skb, MAX_TCP_HEADER);
/* FIN eats a sequence byte, write_seq advanced by tcp_queue_skb(). */
tcp_init_nondata_skb(skb, tp->write_seq,
TCPHDR_ACK | TCPHDR_FIN);
return err;
IPCB(skb)->flags |= IPSKB_XFRM_TUNNEL_SIZE;
+ skb->protocol = htons(ETH_P_IP);
return x->outer_mode->output2(x, skb);
}
int xfrm4_output_finish(struct sk_buff *skb)
{
memset(IPCB(skb), 0, sizeof(*IPCB(skb)));
- skb->protocol = htons(ETH_P_IP);
#ifdef CONFIG_NETFILTER
IPCB(skb)->flags |= IPSKB_XFRM_TRANSFORMED;
return ret;
}
+static
+int addrconf_sysctl_mtu(struct ctl_table *ctl, int write,
+ void __user *buffer, size_t *lenp, loff_t *ppos)
+{
+ struct inet6_dev *idev = ctl->extra1;
+ int min_mtu = IPV6_MIN_MTU;
+ struct ctl_table lctl;
+
+ lctl = *ctl;
+ lctl.extra1 = &min_mtu;
+ lctl.extra2 = idev ? &idev->dev->mtu : NULL;
+
+ return proc_dointvec_minmax(&lctl, write, buffer, lenp, ppos);
+}
+
static void dev_disable_change(struct inet6_dev *idev)
{
struct netdev_notifier_info info;
.data = &ipv6_devconf.mtu6,
.maxlen = sizeof(int),
.mode = 0644,
- .proc_handler = proc_dointvec,
+ .proc_handler = addrconf_sysctl_mtu,
},
{
.procname = "accept_ra",
kfree_skb(skb);
}
-static void ip6_datagram_prepare_pktinfo_errqueue(struct sk_buff *skb)
+/* IPv6 supports cmsg on all origins aside from SO_EE_ORIGIN_LOCAL.
+ *
+ * At one point, excluding local errors was a quick test to identify icmp/icmp6
+ * errors. This is no longer true, but the test remained, so the v6 stack,
+ * unlike v4, also honors cmsg requests on all wifi and timestamp errors.
+ *
+ * Timestamp code paths do not initialize the fields expected by cmsg:
+ * the PKTINFO fields in skb->cb[]. Fill those in here.
+ */
+static bool ip6_datagram_support_cmsg(struct sk_buff *skb,
+ struct sock_exterr_skb *serr)
{
- int ifindex = skb->dev ? skb->dev->ifindex : -1;
+ if (serr->ee.ee_origin == SO_EE_ORIGIN_ICMP ||
+ serr->ee.ee_origin == SO_EE_ORIGIN_ICMP6)
+ return true;
+
+ if (serr->ee.ee_origin == SO_EE_ORIGIN_LOCAL)
+ return false;
+
+ if (!skb->dev)
+ return false;
if (skb->protocol == htons(ETH_P_IPV6))
- IP6CB(skb)->iif = ifindex;
+ IP6CB(skb)->iif = skb->dev->ifindex;
else
- PKTINFO_SKB_CB(skb)->ipi_ifindex = ifindex;
+ PKTINFO_SKB_CB(skb)->ipi_ifindex = skb->dev->ifindex;
+
+ return true;
}
/*
serr = SKB_EXT_ERR(skb);
- if (sin && skb->len) {
+ if (sin && serr->port) {
const unsigned char *nh = skb_network_header(skb);
sin->sin6_family = AF_INET6;
sin->sin6_flowinfo = 0;
memcpy(&errhdr.ee, &serr->ee, sizeof(struct sock_extended_err));
sin = &errhdr.offender;
memset(sin, 0, sizeof(*sin));
- if (serr->ee.ee_origin != SO_EE_ORIGIN_LOCAL && skb->len) {
+
+ if (ip6_datagram_support_cmsg(skb, serr)) {
sin->sin6_family = AF_INET6;
- if (np->rxopt.all) {
- if (serr->ee.ee_origin != SO_EE_ORIGIN_ICMP &&
- serr->ee.ee_origin != SO_EE_ORIGIN_ICMP6)
- ip6_datagram_prepare_pktinfo_errqueue(skb);
+ if (np->rxopt.all)
ip6_datagram_recv_common_ctl(sk, msg, skb);
- }
if (skb->protocol == htons(ETH_P_IPV6)) {
sin->sin6_addr = ipv6_hdr(skb)->saddr;
if (np->rxopt.all)
goto again;
flp6->saddr = saddr;
}
+ err = rt->dst.error;
goto out;
}
again:
static void __net_exit fib6_rules_net_exit(struct net *net)
{
+ rtnl_lock();
fib_rules_unregister(net->ipv6.fib6_rules_ops);
+ rtnl_unlock();
}
static struct pernet_operations fib6_rules_net_ops = {
static inline int ip6_forward_finish(struct sk_buff *skb)
{
+ skb_sender_cpu_clear(skb);
return dst_output(skb);
}
{
struct sk_buff *frag;
struct rt6_info *rt = (struct rt6_info *)skb_dst(skb);
- struct ipv6_pinfo *np = skb->sk ? inet6_sk(skb->sk) : NULL;
+ struct ipv6_pinfo *np = skb->sk && !dev_recursion_level() ?
+ inet6_sk(skb->sk) : NULL;
struct ipv6hdr *tmp_hdr;
struct frag_hdr *fh;
unsigned int mtu, hlen, left, len;
if (((length > mtu) ||
(skb && skb_is_gso(skb))) &&
(sk->sk_protocol == IPPROTO_UDP) &&
- (rt->dst.dev->features & NETIF_F_UFO)) {
+ (rt->dst.dev->features & NETIF_F_UFO) &&
+ (sk->sk_type == SOCK_DGRAM)) {
err = ip6_ufo_append_data(sk, queue, getfrag, from, length,
hh_len, fragheaderlen,
transhdrlen, mtu, flags, rt);
* Create tunnel matching given parameters.
*
* Return:
- * created tunnel or NULL
+ * created tunnel or error pointer
**/
static struct ip6_tnl *ip6_tnl_create(struct net *net, struct __ip6_tnl_parm *p)
struct net_device *dev;
struct ip6_tnl *t;
char name[IFNAMSIZ];
- int err;
+ int err = -ENOMEM;
if (p->name[0])
strlcpy(name, p->name, IFNAMSIZ);
failed_free:
ip6_dev_free(dev);
failed:
- return NULL;
+ return ERR_PTR(err);
}
/**
* tunnel device is created and registered for use.
*
* Return:
- * matching tunnel or NULL
+ * matching tunnel or error pointer
**/
static struct ip6_tnl *ip6_tnl_locate(struct net *net,
if (ipv6_addr_equal(local, &t->parms.laddr) &&
ipv6_addr_equal(remote, &t->parms.raddr)) {
if (create)
- return NULL;
+ return ERR_PTR(-EEXIST);
return t;
}
}
if (!create)
- return NULL;
+ return ERR_PTR(-ENODEV);
return ip6_tnl_create(net, p);
}
}
ip6_tnl_parm_from_user(&p1, &p);
t = ip6_tnl_locate(net, &p1, 0);
- if (t == NULL)
+ if (IS_ERR(t))
t = netdev_priv(dev);
} else {
memset(&p, 0, sizeof(p));
ip6_tnl_parm_from_user(&p1, &p);
t = ip6_tnl_locate(net, &p1, cmd == SIOCADDTUNNEL);
if (cmd == SIOCCHGTUNNEL) {
- if (t != NULL) {
+ if (!IS_ERR(t)) {
if (t->dev != dev) {
err = -EEXIST;
break;
else
err = ip6_tnl_update(t, &p1);
}
- if (t) {
+ if (!IS_ERR(t)) {
err = 0;
ip6_tnl_parm_to_user(&p, &t->parms);
if (copy_to_user(ifr->ifr_ifru.ifru_data, &p, sizeof(p)))
err = -EFAULT;
- } else
- err = (cmd == SIOCADDTUNNEL ? -ENOBUFS : -ENOENT);
+ } else {
+ err = PTR_ERR(t);
+ }
break;
case SIOCDELTUNNEL:
err = -EPERM;
err = -ENOENT;
ip6_tnl_parm_from_user(&p1, &p);
t = ip6_tnl_locate(net, &p1, 0);
- if (t == NULL)
+ if (IS_ERR(t))
break;
err = -EPERM;
if (t->dev == ip6n->fb_tnl_dev)
struct nlattr *tb[], struct nlattr *data[])
{
struct net *net = dev_net(dev);
- struct ip6_tnl *nt;
+ struct ip6_tnl *nt, *t;
nt = netdev_priv(dev);
ip6_tnl_netlink_parms(data, &nt->parms);
- if (ip6_tnl_locate(net, &nt->parms, 0))
+ t = ip6_tnl_locate(net, &nt->parms, 0);
+ if (!IS_ERR(t))
return -EEXIST;
return ip6_tnl_create2(dev);
ip6_tnl_netlink_parms(data, &p);
t = ip6_tnl_locate(net, &p, 0);
-
- if (t) {
+ if (!IS_ERR(t)) {
if (t->dev != dev)
return -EEXIST;
} else
return 0;
err2:
- kfree(mrt);
+ ip6mr_free_table(mrt);
err1:
fib_rules_unregister(ops);
return err;
list_del(&mrt->list);
ip6mr_free_table(mrt);
}
- rtnl_unlock();
fib_rules_unregister(net->ipv6.mr6_rules_ops);
+ rtnl_unlock();
}
#else
#define ip6mr_for_each_table(mrt, net) \
static void ip6mr_free_table(struct mr6_table *mrt)
{
- del_timer(&mrt->ipmr_expire_timer);
+ del_timer_sync(&mrt->ipmr_expire_timer);
mroute_clean_tables(mrt);
kfree(mrt);
}
if (rt)
rt6_set_expires(rt, jiffies + (HZ * lifetime));
if (ra_msg->icmph.icmp6_hop_limit) {
- in6_dev->cnf.hop_limit = ra_msg->icmph.icmp6_hop_limit;
+ /* Only set hop_limit on the interface if it is higher than
+ * the current hop_limit.
+ */
+ if (in6_dev->cnf.hop_limit < ra_msg->icmph.icmp6_hop_limit) {
+ in6_dev->cnf.hop_limit = ra_msg->icmph.icmp6_hop_limit;
+ } else {
+ ND_PRINTK(2, warn, "RA: Got route advertisement with lower hop_limit than current\n");
+ }
if (rt)
dst_metric_set(&rt->dst, RTAX_HOPLIMIT,
ra_msg->icmph.icmp6_hop_limit);
&chainname, &comment, &rulenum) != 0)
break;
- nf_log_packet(net, AF_INET6, hook, skb, in, out, &trace_loginfo,
- "TRACE: %s:%s:%s:%u ",
- tablename, chainname, comment, rulenum);
+ nf_log_trace(net, AF_INET6, hook, skb, in, out, &trace_loginfo,
+ "TRACE: %s:%s:%s:%u ",
+ tablename, chainname, comment, rulenum);
}
#endif
if (msg->msg_name) {
DECLARE_SOCKADDR(struct sockaddr_in6 *, u, msg->msg_name);
- if (msg->msg_namelen < sizeof(struct sockaddr_in6) ||
- u->sin6_family != AF_INET6) {
+ if (msg->msg_namelen < sizeof(*u))
return -EINVAL;
+ if (u->sin6_family != AF_INET6) {
+ return -EAFNOSUPPORT;
}
if (sk->sk_bound_dev_if &&
sk->sk_bound_dev_if != u->sin6_scope_id) {
TCP_SKB_CB(skb)->sacked = 0;
}
+static void tcp_v6_restore_cb(struct sk_buff *skb)
+{
+ /* We need to move header back to the beginning if xfrm6_policy_check()
+ * and tcp_v6_fill_cb() are going to be called again.
+ */
+ memmove(IP6CB(skb), &TCP_SKB_CB(skb)->header.h6,
+ sizeof(struct inet6_skb_parm));
+}
+
static int tcp_v6_rcv(struct sk_buff *skb)
{
const struct tcphdr *th;
inet_twsk_deschedule(tw, &tcp_death_row);
inet_twsk_put(tw);
sk = sk2;
+ tcp_v6_restore_cb(skb);
goto process;
}
/* Fall through to ACK */
tcp_v6_timewait_ack(sk, skb);
break;
case TCP_TW_RST:
+ tcp_v6_restore_cb(skb);
goto no_tcp_socket;
case TCP_TW_SUCCESS:
;
skb->sk = sk;
skb->destructor = sock_edemux;
if (sk->sk_state != TCP_TIME_WAIT) {
- struct dst_entry *dst = sk->sk_rx_dst;
+ struct dst_entry *dst = READ_ONCE(sk->sk_rx_dst);
if (dst)
dst = dst_check(dst, inet6_sk(sk)->rx_dst_cookie);
fptr = (struct frag_hdr *)(skb_network_header(skb) + unfrag_ip6hlen);
fptr->nexthdr = nexthdr;
fptr->reserved = 0;
- if (skb_shinfo(skb)->ip6_frag_id)
- fptr->identification = skb_shinfo(skb)->ip6_frag_id;
- else
- ipv6_select_ident(fptr,
- (struct rt6_info *)skb_dst(skb));
+ if (!skb_shinfo(skb)->ip6_frag_id)
+ ipv6_proxy_select_ident(skb);
+ fptr->identification = skb_shinfo(skb)->ip6_frag_id;
/* Fragment the skb. ipv6 header and the remaining fields of the
* fragment header are updated in ipv6_gso_segment()
return err;
skb->ignore_df = 1;
+ skb->protocol = htons(ETH_P_IPV6);
return x->outer_mode->output2(x, skb);
}
int xfrm6_output_finish(struct sk_buff *skb)
{
memset(IP6CB(skb), 0, sizeof(*IP6CB(skb)));
- skb->protocol = htons(ETH_P_IPV6);
#ifdef CONFIG_NETFILTER
IP6CB(skb)->flags |= IP6SKB_XFRM_TRANSFORMED;
#if IS_ENABLED(CONFIG_IPV6_MIP6)
case IPPROTO_MH:
+ offset += ipv6_optlen(exthdr);
if (!onlyproto && pskb_may_pull(skb, nh + offset + 3 - skb->data)) {
struct ip6_mh *mh;
orig_jiffies = jiffies;
/* Set poll time to 200 ms */
- poll_time = IRDA_MIN(timeout, msecs_to_jiffies(200));
+ poll_time = msecs_to_jiffies(200);
+ if (timeout)
+ poll_time = min_t(unsigned long, timeout, poll_time);
spin_lock_irqsave(&self->spinlock, flags);
while (self->tx_skb && self->tx_skb->len) {
break;
}
spin_unlock_irqrestore(&self->spinlock, flags);
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
}
/*
/* Put ourselves on the wait queue to be woken up */
add_wait_queue(&irnet_events.rwait, &wait);
- current->state = TASK_INTERRUPTIBLE;
+ set_current_state(TASK_INTERRUPTIBLE);
for(;;)
{
/* If there is unread events */
/* Yield and wait to be woken up */
schedule();
}
- current->state = TASK_RUNNING;
+ __set_current_state(TASK_RUNNING);
remove_wait_queue(&irnet_events.rwait, &wait);
/* Did we got it ? */
noblock, &err);
else
skb = sock_alloc_send_skb(sk, len, noblock, &err);
- if (!skb) {
- err = -ENOMEM;
+ if (!skb)
goto out;
- }
if (iucv->transport == AF_IUCV_TRANS_HIPER)
skb_reserve(skb, sizeof(struct af_iucv_trans_hdr) + ETH_HLEN);
if (memcpy_from_msg(skb_put(skb, len), msg, len)) {
l2tp_wq = alloc_workqueue("l2tp", WQ_UNBOUND, 0);
if (!l2tp_wq) {
pr_err("alloc_workqueue failed\n");
+ unregister_pernet_device(&l2tp_net_ops);
rc = -ENOMEM;
goto out;
}
container_of(h, struct tid_ampdu_rx, rcu_head);
int i;
- del_timer_sync(&tid_rx->reorder_timer);
-
for (i = 0; i < tid_rx->buf_size; i++)
__skb_queue_purge(&tid_rx->reorder_buf[i]);
kfree(tid_rx->reorder_buf);
del_timer_sync(&tid_rx->session_timer);
+ /* make sure ieee80211_sta_reorder_release() doesn't re-arm the timer */
+ spin_lock_bh(&tid_rx->reorder_lock);
+ tid_rx->removed = true;
+ spin_unlock_bh(&tid_rx->reorder_lock);
+ del_timer_sync(&tid_rx->reorder_timer);
+
call_rcu(&tid_rx->rcu_head, ieee80211_free_tid_rx);
}
if (ieee80211_chanctx_refcount(local, ctx) == 0)
ieee80211_free_chanctx(local, ctx);
+ sdata->radar_required = false;
+
/* Unreserving may ready an in-place reservation. */
if (use_reserved_switch)
ieee80211_vif_use_reserved_switch(local);
ieee80211_recalc_smps_chanctx(local, ctx);
ieee80211_recalc_radar_chanctx(local, ctx);
out:
+ if (ret)
+ sdata->radar_required = false;
+
mutex_unlock(&local->chanctx_mtx);
return ret;
}
#define IEEE80211_UNSET_POWER_LEVEL INT_MIN
/*
- * Some APs experience problems when working with U-APSD. Decrease the
- * probability of that happening by using legacy mode for all ACs but VO.
- * The AP that caused us trouble was a Cisco 4410N. It ignores our
- * setting, and always treats non-VO ACs as legacy.
+ * Some APs experience problems when working with U-APSD. Decreasing the
+ * probability of that happening by using legacy mode for all ACs but VO isn't
+ * enough.
+ *
+ * Cisco 4410N originally forced us to enable VO by default only because it
+ * treated non-VO ACs as legacy.
+ *
+ * However some APs (notably Netgear R7000) silently reclassify packets to
+ * different ACs. Since u-APSD ACs require trigger frames for frame retrieval
+ * clients would never see some frames (e.g. ARP responses) or would fetch them
+ * accidentally after a long time.
+ *
+ * It makes little sense to enable u-APSD queues by default because it needs
+ * userspace applications to be aware of it to actually take advantage of the
+ * possible additional powersavings. Implicitly depending on driver autotrigger
+ * frame support doesn't make much sense.
*/
-#define IEEE80211_DEFAULT_UAPSD_QUEUES \
- IEEE80211_WMM_IE_STA_QOSINFO_AC_VO
+#define IEEE80211_DEFAULT_UAPSD_QUEUES 0
#define IEEE80211_DEFAULT_MAX_SP_LEN \
IEEE80211_WMM_IE_STA_QOSINFO_SP_ALL
unsigned int flags;
bool csa_waiting_bcn;
+ bool csa_ignored_same_chan;
bool beacon_crc_valid;
u32 beacon_crc;
return;
}
+ if (cfg80211_chandef_identical(&csa_ie.chandef,
+ &sdata->vif.bss_conf.chandef)) {
+ if (ifmgd->csa_ignored_same_chan)
+ return;
+ sdata_info(sdata,
+ "AP %pM tries to chanswitch to same channel, ignore\n",
+ ifmgd->associated->bssid);
+ ifmgd->csa_ignored_same_chan = true;
+ return;
+ }
+
mutex_lock(&local->mtx);
mutex_lock(&local->chanctx_mtx);
conf = rcu_dereference_protected(sdata->vif.chanctx_conf,
sdata->vif.csa_active = true;
sdata->csa_chandef = csa_ie.chandef;
sdata->csa_block_tx = csa_ie.mode;
+ ifmgd->csa_ignored_same_chan = false;
if (sdata->csa_block_tx)
ieee80211_stop_vif_queues(local, sdata,
sdata->vif.csa_active = false;
ifmgd->csa_waiting_bcn = false;
+ ifmgd->csa_ignored_same_chan = false;
if (sdata->csa_block_tx) {
ieee80211_wake_vif_queues(local, sdata,
IEEE80211_QUEUE_STOP_REASON_CSA);
(1ULL << WLAN_EID_CHANNEL_SWITCH) |
(1ULL << WLAN_EID_PWR_CONSTRAINT) |
(1ULL << WLAN_EID_HT_CAPABILITY) |
- (1ULL << WLAN_EID_HT_OPERATION);
+ (1ULL << WLAN_EID_HT_OPERATION) |
+ (1ULL << WLAN_EID_EXT_CHANSWITCH_ANN);
static void ieee80211_rx_mgmt_beacon(struct ieee80211_sub_if_data *sdata,
struct ieee80211_mgmt *mgmt, size_t len,
rate++;
mi->sample_deferred++;
} else {
- if (!msr->sample_limit != 0)
+ if (!msr->sample_limit)
return;
mi->sample_packets++;
set_release_timer:
- mod_timer(&tid_agg_rx->reorder_timer,
- tid_agg_rx->reorder_time[j] + 1 +
- HT_RX_REORDER_BUF_TIMEOUT);
+ if (!tid_agg_rx->removed)
+ mod_timer(&tid_agg_rx->reorder_timer,
+ tid_agg_rx->reorder_time[j] + 1 +
+ HT_RX_REORDER_BUF_TIMEOUT);
} else {
del_timer(&tid_agg_rx->reorder_timer);
}
hdr = (struct ieee80211_hdr *) skb->data;
mesh_hdr = (struct ieee80211s_hdr *) (skb->data + hdrlen);
+ if (ieee80211_drop_unencrypted(rx, hdr->frame_control))
+ return RX_DROP_MONITOR;
+
/* frame is in RMC, don't forward */
if (ieee80211_is_data(hdr->frame_control) &&
is_multicast_ether_addr(hdr->addr1) &&
* @reorder_lock: serializes access to reorder buffer, see below.
* @auto_seq: used for offloaded BA sessions to automatically pick head_seq_and
* and ssn.
+ * @removed: this session is removed (but might have been found due to RCU)
*
* This structure's lifetime is managed by RCU, assignments to
* the array holding it must hold the aggregation mutex.
u16 timeout;
u8 dialog_token;
bool auto_seq;
+ bool removed;
};
/**
if (tx->sdata->control_port_no_encrypt)
info->flags |= IEEE80211_TX_INTFL_DONT_ENCRYPT;
info->control.flags |= IEEE80211_TX_CTRL_PORT_CTRL_PROTO;
+ info->flags |= IEEE80211_TX_CTL_USE_MINRATE;
}
return TX_CONTINUE;
wdev_iter = &sdata_iter->wdev;
if (sdata_iter == sdata ||
- rcu_access_pointer(sdata_iter->vif.chanctx_conf) == NULL ||
+ !ieee80211_sdata_running(sdata_iter) ||
local->hw.wiphy->software_iftypes & BIT(wdev_iter->iftype))
continue;
if (udest.af == 0)
udest.af = svc->af;
- if (udest.af != svc->af) {
+ if (udest.af != svc->af && cmd != IPVS_CMD_DEL_DEST) {
/* The synchronization protocol is incompatible
* with mixed family services
*/
IP_VS_DBG(2, "BACKUP, add new conn. failed\n");
return;
}
+ if (!(flags & IP_VS_CONN_F_TEMPLATE))
+ kfree(param->pe_data);
}
if (opt)
(opt_flags & IPVS_OPT_F_SEQ_DATA ? &opt : NULL)
);
#endif
+ ip_vs_pe_put(param.pe);
return 0;
/* Error exit */
out:
}
EXPORT_SYMBOL(nf_log_packet);
+void nf_log_trace(struct net *net,
+ u_int8_t pf,
+ unsigned int hooknum,
+ const struct sk_buff *skb,
+ const struct net_device *in,
+ const struct net_device *out,
+ const struct nf_loginfo *loginfo, const char *fmt, ...)
+{
+ va_list args;
+ char prefix[NF_LOG_PREFIXLEN];
+ const struct nf_logger *logger;
+
+ rcu_read_lock();
+ logger = rcu_dereference(net->nf.nf_loggers[pf]);
+ if (logger) {
+ va_start(args, fmt);
+ vsnprintf(prefix, sizeof(prefix), fmt, args);
+ va_end(args);
+ logger->logfn(net, pf, hooknum, skb, in, out, loginfo, prefix);
+ }
+ rcu_read_unlock();
+}
+EXPORT_SYMBOL(nf_log_trace);
+
#define S_SIZE (1024 - (sizeof(unsigned int) + 1))
struct nf_log_buf {
static inline void nft_rule_clear(struct net *net, struct nft_rule *rule)
{
- rule->genmask = 0;
+ rule->genmask &= ~(1 << gencursor_next(net));
}
static int
if (nla[NFTA_CHAIN_POLICY]) {
if ((chain != NULL &&
- !(chain->flags & NFT_BASE_CHAIN)) ||
+ !(chain->flags & NFT_BASE_CHAIN)))
+ return -EOPNOTSUPP;
+
+ if (chain == NULL &&
nla[NFTA_CHAIN_HOOK] == NULL)
return -EOPNOTSUPP;
}
nla_nest_end(skb, list);
- if (rule->ulen &&
- nla_put(skb, NFTA_RULE_USERDATA, rule->ulen, nft_userdata(rule)))
- goto nla_put_failure;
+ if (rule->udata) {
+ struct nft_userdata *udata = nft_userdata(rule);
+ if (nla_put(skb, NFTA_RULE_USERDATA, udata->len + 1,
+ udata->data) < 0)
+ goto nla_put_failure;
+ }
nlmsg_end(skb, nlh);
return 0;
struct nft_table *table;
struct nft_chain *chain;
struct nft_rule *rule, *old_rule = NULL;
+ struct nft_userdata *udata;
struct nft_trans *trans = NULL;
struct nft_expr *expr;
struct nft_ctx ctx;
struct nlattr *tmp;
- unsigned int size, i, n, ulen = 0;
+ unsigned int size, i, n, ulen = 0, usize = 0;
int err, rem;
bool create;
u64 handle, pos_handle;
n++;
}
}
+ /* Check for overflow of dlen field */
+ err = -EFBIG;
+ if (size >= 1 << 12)
+ goto err1;
- if (nla[NFTA_RULE_USERDATA])
+ if (nla[NFTA_RULE_USERDATA]) {
ulen = nla_len(nla[NFTA_RULE_USERDATA]);
+ if (ulen > 0)
+ usize = sizeof(struct nft_userdata) + ulen;
+ }
err = -ENOMEM;
- rule = kzalloc(sizeof(*rule) + size + ulen, GFP_KERNEL);
+ rule = kzalloc(sizeof(*rule) + size + usize, GFP_KERNEL);
if (rule == NULL)
goto err1;
rule->handle = handle;
rule->dlen = size;
- rule->ulen = ulen;
+ rule->udata = ulen ? 1 : 0;
- if (ulen)
- nla_memcpy(nft_userdata(rule), nla[NFTA_RULE_USERDATA], ulen);
+ if (ulen) {
+ udata = nft_userdata(rule);
+ udata->len = ulen - 1;
+ nla_memcpy(udata->data, nla[NFTA_RULE_USERDATA], ulen);
+ }
expr = nft_expr_first(rule);
for (i = 0; i < n; i++) {
err3:
list_del_rcu(&rule->list);
- if (trans) {
- list_del_rcu(&nft_trans_rule(trans)->list);
- nft_rule_clear(net, nft_trans_rule(trans));
- nft_trans_destroy(trans);
- chain->use++;
- }
err2:
nf_tables_rule_destroy(&ctx, rule);
err1:
&te->elem,
NFT_MSG_DELSETELEM, 0);
te->set->ops->get(te->set, &te->elem);
- te->set->ops->remove(te->set, &te->elem);
nft_data_uninit(&te->elem.key, NFT_DATA_VALUE);
- if (te->elem.flags & NFT_SET_MAP) {
- nft_data_uninit(&te->elem.data,
- te->set->dtype);
- }
+ if (te->set->flags & NFT_SET_MAP &&
+ !(te->elem.flags & NFT_SET_ELEM_INTERVAL_END))
+ nft_data_uninit(&te->elem.data, te->set->dtype);
+ te->set->ops->remove(te->set, &te->elem);
nft_trans_destroy(trans);
break;
}
{
struct net *net = sock_net(skb->sk);
struct nft_trans *trans, *next;
- struct nft_set *set;
+ struct nft_trans_elem *te;
list_for_each_entry_safe(trans, next, &net->nft.commit_list, list) {
switch (trans->msg_type) {
break;
case NFT_MSG_NEWSETELEM:
nft_trans_elem_set(trans)->nelems--;
- set = nft_trans_elem_set(trans);
- set->ops->get(set, &nft_trans_elem(trans));
- set->ops->remove(set, &nft_trans_elem(trans));
+ te = (struct nft_trans_elem *)trans->data;
+ te->set->ops->get(te->set, &te->elem);
+ nft_data_uninit(&te->elem.key, NFT_DATA_VALUE);
+ if (te->set->flags & NFT_SET_MAP &&
+ !(te->elem.flags & NFT_SET_ELEM_INTERVAL_END))
+ nft_data_uninit(&te->elem.data, te->set->dtype);
+ te->set->ops->remove(te->set, &te->elem);
nft_trans_destroy(trans);
break;
case NFT_MSG_DELSETELEM:
{
struct net *net = dev_net(pkt->in ? pkt->in : pkt->out);
- nf_log_packet(net, pkt->xt.family, pkt->ops->hooknum, pkt->skb, pkt->in,
- pkt->out, &trace_loginfo, "TRACE: %s:%s:%s:%u ",
- chain->table->name, chain->name, comments[type],
- rulenum);
+ nf_log_trace(net, pkt->xt.family, pkt->ops->hooknum, pkt->skb, pkt->in,
+ pkt->out, &trace_loginfo, "TRACE: %s:%s:%s:%u ",
+ chain->table->name, chain->name, comments[type],
+ rulenum);
}
unsigned int
if (!tb[NFCTH_TUPLE_L3PROTONUM] || !tb[NFCTH_TUPLE_L4PROTONUM])
return -EINVAL;
+ /* Not all fields are initialized so first zero the tuple */
+ memset(tuple, 0, sizeof(struct nf_conntrack_tuple));
+
tuple->src.l3num = ntohs(nla_get_be16(tb[NFCTH_TUPLE_L3PROTONUM]));
tuple->dst.protonum = nla_get_u8(tb[NFCTH_TUPLE_L4PROTONUM]);
nft_target_set_tgchk_param(struct xt_tgchk_param *par,
const struct nft_ctx *ctx,
struct xt_target *target, void *info,
- union nft_entry *entry, u8 proto, bool inv)
+ union nft_entry *entry, u16 proto, bool inv)
{
par->net = ctx->net;
par->table = ctx->table->name;
entry->e4.ip.invflags = inv ? IPT_INV_PROTO : 0;
break;
case AF_INET6:
+ if (proto)
+ entry->e6.ipv6.flags |= IP6T_F_PROTO;
+
entry->e6.ipv6.proto = proto;
entry->e6.ipv6.invflags = inv ? IP6T_INV_PROTO : 0;
break;
case NFPROTO_BRIDGE:
- entry->ebt.ethproto = proto;
+ entry->ebt.ethproto = (__force __be16)proto;
entry->ebt.invflags = inv ? EBT_IPROTO : 0;
break;
}
[NFTA_RULE_COMPAT_FLAGS] = { .type = NLA_U32 },
};
-static int nft_parse_compat(const struct nlattr *attr, u8 *proto, bool *inv)
+static int nft_parse_compat(const struct nlattr *attr, u16 *proto, bool *inv)
{
struct nlattr *tb[NFTA_RULE_COMPAT_MAX+1];
u32 flags;
struct xt_target *target = expr->ops->data;
struct xt_tgchk_param par;
size_t size = XT_ALIGN(nla_len(tb[NFTA_TARGET_INFO]));
- u8 proto = 0;
+ u16 proto = 0;
bool inv = false;
union nft_entry e = {};
int ret;
static void
nft_match_set_mtchk_param(struct xt_mtchk_param *par, const struct nft_ctx *ctx,
struct xt_match *match, void *info,
- union nft_entry *entry, u8 proto, bool inv)
+ union nft_entry *entry, u16 proto, bool inv)
{
par->net = ctx->net;
par->table = ctx->table->name;
entry->e4.ip.invflags = inv ? IPT_INV_PROTO : 0;
break;
case AF_INET6:
+ if (proto)
+ entry->e6.ipv6.flags |= IP6T_F_PROTO;
+
entry->e6.ipv6.proto = proto;
entry->e6.ipv6.invflags = inv ? IP6T_INV_PROTO : 0;
break;
case NFPROTO_BRIDGE:
- entry->ebt.ethproto = proto;
+ entry->ebt.ethproto = (__force __be16)proto;
entry->ebt.invflags = inv ? EBT_IPROTO : 0;
break;
}
struct xt_match *match = expr->ops->data;
struct xt_mtchk_param par;
size_t size = XT_ALIGN(nla_len(tb[NFTA_MATCH_INFO]));
- u8 proto = 0;
+ u16 proto = 0;
bool inv = false;
union nft_entry e = {};
int ret;
struct xt_match *match = nft_match->ops.data;
if (strcmp(match->name, mt_name) == 0 &&
- match->revision == rev && match->family == family)
+ match->revision == rev && match->family == family) {
+ if (!try_module_get(match->me))
+ return ERR_PTR(-ENOENT);
+
return &nft_match->ops;
+ }
}
match = xt_request_find_match(family, mt_name, rev);
struct xt_target *target = nft_target->ops.data;
if (strcmp(target->name, tg_name) == 0 &&
- target->revision == rev && target->family == family)
+ target->revision == rev && target->family == family) {
+ if (!try_module_get(target->me))
+ return ERR_PTR(-ENOENT);
+
return &nft_target->ops;
+ }
}
target = xt_request_find_target(family, tg_name, rev);
iter->err = err;
goto out;
}
+
+ continue;
}
if (iter->count < iter->skip)
.key_offset = offsetof(struct nft_hash_elem, key),
.key_len = set->klen,
.hashfn = jhash,
- .grow_decision = rht_grow_above_75,
- .shrink_decision = rht_shrink_below_30,
};
return rhashtable_init(priv, ¶ms);
{
const struct ip6t_ip6 *i = par->entryinfo;
- if ((i->proto == IPPROTO_TCP || i->proto == IPPROTO_UDP)
- && !(i->flags & IP6T_INV_PROTO))
+ if ((i->proto == IPPROTO_TCP || i->proto == IPPROTO_UDP) &&
+ !(i->invflags & IP6T_INV_PROTO))
return 0;
pr_info("Can be used only in combination with "
mutex_lock(&recent_mutex);
t = recent_table_lookup(recent_net, info->name);
if (t != NULL) {
- if (info->hit_count > t->nstamps_max_mask) {
- pr_info("hitcount (%u) is larger than packets to be remembered (%u) for table %s\n",
- info->hit_count, t->nstamps_max_mask + 1,
- info->name);
- ret = -EINVAL;
- goto out;
+ if (nstamp_mask > t->nstamps_max_mask) {
+ spin_lock_bh(&recent_lock);
+ recent_table_flush(t);
+ t->nstamps_max_mask = nstamp_mask;
+ spin_unlock_bh(&recent_lock);
}
t->refcnt++;
extract_icmp6_fields(const struct sk_buff *skb,
unsigned int outside_hdrlen,
int *protocol,
- struct in6_addr **raddr,
- struct in6_addr **laddr,
+ const struct in6_addr **raddr,
+ const struct in6_addr **laddr,
__be16 *rport,
- __be16 *lport)
+ __be16 *lport,
+ struct ipv6hdr *ipv6_var)
{
- struct ipv6hdr *inside_iph, _inside_iph;
+ const struct ipv6hdr *inside_iph;
struct icmp6hdr *icmph, _icmph;
__be16 *ports, _ports[2];
u8 inside_nexthdr;
if (icmph->icmp6_type & ICMPV6_INFOMSG_MASK)
return 1;
- inside_iph = skb_header_pointer(skb, outside_hdrlen + sizeof(_icmph), sizeof(_inside_iph), &_inside_iph);
+ inside_iph = skb_header_pointer(skb, outside_hdrlen + sizeof(_icmph),
+ sizeof(*ipv6_var), ipv6_var);
if (inside_iph == NULL)
return 1;
inside_nexthdr = inside_iph->nexthdr;
- inside_hdrlen = ipv6_skip_exthdr(skb, outside_hdrlen + sizeof(_icmph) + sizeof(_inside_iph),
+ inside_hdrlen = ipv6_skip_exthdr(skb, outside_hdrlen + sizeof(_icmph) +
+ sizeof(*ipv6_var),
&inside_nexthdr, &inside_fragoff);
if (inside_hdrlen < 0)
return 1; /* hjm: Packet has no/incomplete transport layer headers. */
static bool
socket_mt6_v1_v2(const struct sk_buff *skb, struct xt_action_param *par)
{
- struct ipv6hdr *iph = ipv6_hdr(skb);
+ struct ipv6hdr ipv6_var, *iph = ipv6_hdr(skb);
struct udphdr _hdr, *hp = NULL;
struct sock *sk = skb->sk;
- struct in6_addr *daddr = NULL, *saddr = NULL;
+ const struct in6_addr *daddr = NULL, *saddr = NULL;
__be16 uninitialized_var(dport), uninitialized_var(sport);
int thoff = 0, uninitialized_var(tproto);
const struct xt_socket_mtinfo1 *info = (struct xt_socket_mtinfo1 *) par->matchinfo;
} else if (tproto == IPPROTO_ICMPV6) {
if (extract_icmp6_fields(skb, thoff, &tproto, &saddr, &daddr,
- &sport, &dport))
+ &sport, &dport, &ipv6_var))
return false;
} else {
return false;
.key_len = sizeof(u32), /* portid */
.hashfn = jhash,
.max_shift = 16, /* 64K */
- .grow_decision = rht_grow_above_75,
- .shrink_decision = rht_shrink_below_30,
};
if (err != 0)
return 0;
}
-static void __net_exit ovs_exit_net(struct net *net)
+static void __net_exit list_vports_from_net(struct net *net, struct net *dnet,
+ struct list_head *head)
{
- struct datapath *dp, *dp_next;
struct ovs_net *ovs_net = net_generic(net, ovs_net_id);
+ struct datapath *dp;
+
+ list_for_each_entry(dp, &ovs_net->dps, list_node) {
+ int i;
+
+ for (i = 0; i < DP_VPORT_HASH_BUCKETS; i++) {
+ struct vport *vport;
+
+ hlist_for_each_entry(vport, &dp->ports[i], dp_hash_node) {
+ struct netdev_vport *netdev_vport;
+
+ if (vport->ops->type != OVS_VPORT_TYPE_INTERNAL)
+ continue;
+
+ netdev_vport = netdev_vport_priv(vport);
+ if (dev_net(netdev_vport->dev) == dnet)
+ list_add(&vport->detach_list, head);
+ }
+ }
+ }
+}
+
+static void __net_exit ovs_exit_net(struct net *dnet)
+{
+ struct datapath *dp, *dp_next;
+ struct ovs_net *ovs_net = net_generic(dnet, ovs_net_id);
+ struct vport *vport, *vport_next;
+ struct net *net;
+ LIST_HEAD(head);
ovs_lock();
list_for_each_entry_safe(dp, dp_next, &ovs_net->dps, list_node)
__dp_destroy(dp);
+
+ rtnl_lock();
+ for_each_net(net)
+ list_vports_from_net(net, dnet, &head);
+ rtnl_unlock();
+
+ /* Detach all vports from given namespace. */
+ list_for_each_entry_safe(vport, vport_next, &head, detach_list) {
+ list_del(&vport->detach_list);
+ ovs_dp_detach_port(vport);
+ }
+
ovs_unlock();
cancel_work_sync(&ovs_net->dp_notify_work);
struct sk_buff *skb)
{
const struct nlattr *ovs_key = nla_data(a);
+ struct nlattr *nla;
size_t key_len = nla_len(ovs_key) / 2;
/* Revert the conversion we did from a non-masked set action to
* masked set action.
*/
- if (nla_put(skb, OVS_ACTION_ATTR_SET, nla_len(a) - key_len, ovs_key))
+ nla = nla_nest_start(skb, OVS_ACTION_ATTR_SET);
+ if (!nla)
return -EMSGSIZE;
+ if (nla_put(skb, nla_type(ovs_key), key_len, nla_data(ovs_key)))
+ return -EMSGSIZE;
+
+ nla_nest_end(skb, nla);
return 0;
}
ASSERT_OVSL();
hlist_del_rcu(&vport->hash_node);
-
- vport->ops->destroy(vport);
-
module_put(vport->ops->owner);
+ vport->ops->destroy(vport);
}
/**
* @ops: Class structure.
* @percpu_stats: Points to per-CPU statistics used and maintained by vport
* @err_stats: Points to error statistics used and maintained by vport
+ * @detach_list: list used for detaching vport in net-exit call.
*/
struct vport {
struct rcu_head rcu;
struct pcpu_sw_netstats __percpu *percpu_stats;
struct vport_err_stats err_stats;
+ struct list_head detach_list;
};
/**
if (pkc->last_kactive_blk_num == pkc->kactive_blk_num) {
if (!frozen) {
+ if (!BLOCK_NUM_PKTS(pbd)) {
+ /* An empty block. Just refresh the timer. */
+ goto refresh_timer;
+ }
prb_retire_current_block(pkc, po, TP_STATUS_BLK_TMO);
if (!prb_dispatch_next_block(pkc, po))
goto refresh_timer;
h1->ts_last_pkt.ts_sec = last_pkt->tp_sec;
h1->ts_last_pkt.ts_nsec = last_pkt->tp_nsec;
} else {
- /* Ok, we tmo'd - so get the current time */
+ /* Ok, we tmo'd - so get the current time.
+ *
+ * It shouldn't really happen as we don't close empty
+ * blocks. See prb_retire_rx_blk_timer_expired().
+ */
struct timespec ts;
getnstimeofday(&ts);
h1->ts_last_pkt.ts_sec = ts.tv_sec;
return 0;
}
+ if (fanout_has_flag(f, PACKET_FANOUT_FLAG_DEFRAG)) {
+ skb = ip_check_defrag(skb, IP_DEFRAG_AF_PACKET);
+ if (!skb)
+ return 0;
+ }
switch (f->type) {
case PACKET_FANOUT_HASH:
default:
- if (fanout_has_flag(f, PACKET_FANOUT_FLAG_DEFRAG)) {
- skb = ip_check_defrag(skb, IP_DEFRAG_AF_PACKET);
- if (!skb)
- return 0;
- }
idx = fanout_demux_hash(f, skb, num);
break;
case PACKET_FANOUT_LB:
return 0;
}
-static void packet_dev_mclist(struct net_device *dev, struct packet_mclist *i, int what)
+static void packet_dev_mclist_delete(struct net_device *dev,
+ struct packet_mclist **mlp)
{
- for ( ; i; i = i->next) {
- if (i->ifindex == dev->ifindex)
- packet_dev_mc(dev, i, what);
+ struct packet_mclist *ml;
+
+ while ((ml = *mlp) != NULL) {
+ if (ml->ifindex == dev->ifindex) {
+ packet_dev_mc(dev, ml, -1);
+ *mlp = ml->next;
+ kfree(ml);
+ } else
+ mlp = &ml->next;
}
}
packet_dev_mc(dev, ml, -1);
kfree(ml);
}
- rtnl_unlock();
- return 0;
+ break;
}
}
rtnl_unlock();
- return -EADDRNOTAVAIL;
+ return 0;
}
static void packet_flush_mclist(struct sock *sk)
switch (msg) {
case NETDEV_UNREGISTER:
if (po->mclist)
- packet_dev_mclist(dev, po->mclist, -1);
+ packet_dev_mclist_delete(dev, &po->mclist);
/* fallthrough */
case NETDEV_DOWN:
int *unpinned);
static void rds_iw_destroy_fastreg(struct rds_iw_mr_pool *pool, struct rds_iw_mr *ibmr);
-static int rds_iw_get_device(struct rds_sock *rs, struct rds_iw_device **rds_iwdev, struct rdma_cm_id **cm_id)
+static int rds_iw_get_device(struct sockaddr_in *src, struct sockaddr_in *dst,
+ struct rds_iw_device **rds_iwdev,
+ struct rdma_cm_id **cm_id)
{
struct rds_iw_device *iwdev;
struct rds_iw_cm_id *i_cm_id;
src_addr->sin_port,
dst_addr->sin_addr.s_addr,
dst_addr->sin_port,
- rs->rs_bound_addr,
- rs->rs_bound_port,
- rs->rs_conn_addr,
- rs->rs_conn_port);
+ src->sin_addr.s_addr,
+ src->sin_port,
+ dst->sin_addr.s_addr,
+ dst->sin_port);
#ifdef WORKING_TUPLE_DETECTION
- if (src_addr->sin_addr.s_addr == rs->rs_bound_addr &&
- src_addr->sin_port == rs->rs_bound_port &&
- dst_addr->sin_addr.s_addr == rs->rs_conn_addr &&
- dst_addr->sin_port == rs->rs_conn_port) {
+ if (src_addr->sin_addr.s_addr == src->sin_addr.s_addr &&
+ src_addr->sin_port == src->sin_port &&
+ dst_addr->sin_addr.s_addr == dst->sin_addr.s_addr &&
+ dst_addr->sin_port == dst->sin_port) {
#else
/* FIXME - needs to compare the local and remote
* ipaddr/port tuple, but the ipaddr is the only
* zero'ed. It doesn't appear to be properly populated
* during connection setup...
*/
- if (src_addr->sin_addr.s_addr == rs->rs_bound_addr) {
+ if (src_addr->sin_addr.s_addr == src->sin_addr.s_addr) {
#endif
spin_unlock_irq(&iwdev->spinlock);
*rds_iwdev = iwdev;
{
struct sockaddr_in *src_addr, *dst_addr;
struct rds_iw_device *rds_iwdev_old;
- struct rds_sock rs;
struct rdma_cm_id *pcm_id;
int rc;
src_addr = (struct sockaddr_in *)&cm_id->route.addr.src_addr;
dst_addr = (struct sockaddr_in *)&cm_id->route.addr.dst_addr;
- rs.rs_bound_addr = src_addr->sin_addr.s_addr;
- rs.rs_bound_port = src_addr->sin_port;
- rs.rs_conn_addr = dst_addr->sin_addr.s_addr;
- rs.rs_conn_port = dst_addr->sin_port;
-
- rc = rds_iw_get_device(&rs, &rds_iwdev_old, &pcm_id);
+ rc = rds_iw_get_device(src_addr, dst_addr, &rds_iwdev_old, &pcm_id);
if (rc)
rds_iw_remove_cm_id(rds_iwdev, cm_id);
struct rds_iw_device *rds_iwdev;
struct rds_iw_mr *ibmr = NULL;
struct rdma_cm_id *cm_id;
+ struct sockaddr_in src = {
+ .sin_addr.s_addr = rs->rs_bound_addr,
+ .sin_port = rs->rs_bound_port,
+ };
+ struct sockaddr_in dst = {
+ .sin_addr.s_addr = rs->rs_conn_addr,
+ .sin_port = rs->rs_conn_port,
+ };
int ret;
- ret = rds_iw_get_device(rs, &rds_iwdev, &cm_id);
+ ret = rds_iw_get_device(&src, &dst, &rds_iwdev, &cm_id);
if (ret || !cm_id) {
ret = -ENODEV;
goto out;
struct rxrpc_header *hdr;
struct sk_buff *txb;
unsigned long *p_txb, resend_at;
- int loop, stop;
+ bool stop;
+ int loop;
u8 resend;
_enter("{%d,%d,%d,%d},",
atomic_read(&call->sequence),
CIRC_CNT(call->acks_head, call->acks_tail, call->acks_winsz));
- stop = 0;
+ stop = false;
resend = 0;
resend_at = 0;
_proto("Tx DATA %%%u { #%d }",
ntohl(sp->hdr.serial), ntohl(sp->hdr.seq));
if (rxrpc_send_packet(call->conn->trans, txb) < 0) {
- stop = 0;
+ stop = true;
sp->resend_at = jiffies + 3;
} else {
sp->resend_at =
- jiffies + rxrpc_resend_timeout * HZ;
+ jiffies + rxrpc_resend_timeout;
}
}
_leave("UDP socket errqueue empty");
return;
}
- if (!skb->len) {
+ serr = SKB_EXT_ERR(skb);
+ if (!skb->len && serr->ee.ee_origin == SO_EE_ORIGIN_TIMESTAMPING) {
_leave("UDP empty message");
kfree_skb(skb);
return;
rxrpc_new_skb(skb);
- serr = SKB_EXT_ERR(skb);
addr = *(__be32 *)(skb_network_header(skb) + serr->addr_offset);
port = serr->port;
if (!skb) {
/* nothing remains on the queue */
if (copied &&
- (msg->msg_flags & MSG_PEEK || timeo == 0))
+ (flags & MSG_PEEK || timeo == 0))
goto out;
/* wait for a message to turn up */
struct tcf_result *res)
{
struct tcf_bpf *b = a->priv;
- int action;
- int filter_res;
+ int action, filter_res;
spin_lock(&b->tcf_lock);
+
b->tcf_tm.lastuse = jiffies;
bstats_update(&b->tcf_bstats, skb);
- action = b->tcf_action;
filter_res = BPF_PROG_RUN(b->filter, skb);
- if (filter_res == 0) {
- /* Return code 0 from the BPF program
- * is being interpreted as a drop here.
- */
- action = TC_ACT_SHOT;
+
+ /* A BPF program may overwrite the default action opcode.
+ * Similarly as in cls_bpf, if filter_res == -1 we use the
+ * default action specified from tc.
+ *
+ * In case a different well-known TC_ACT opcode has been
+ * returned, it will overwrite the default one.
+ *
+ * For everything else that is unkown, TC_ACT_UNSPEC is
+ * returned.
+ */
+ switch (filter_res) {
+ case TC_ACT_PIPE:
+ case TC_ACT_RECLASSIFY:
+ case TC_ACT_OK:
+ action = filter_res;
+ break;
+ case TC_ACT_SHOT:
+ action = filter_res;
b->tcf_qstats.drops++;
+ break;
+ case TC_ACT_UNSPEC:
+ action = b->tcf_action;
+ break;
+ default:
+ action = TC_ACT_UNSPEC;
+ break;
}
spin_unlock(&b->tcf_lock);
struct tc_u_common *tp_c;
int refcnt;
unsigned int divisor;
- struct tc_u_knode __rcu *ht[1];
struct rcu_head rcu;
+ /* The 'ht' field MUST be the last field in structure to allow for
+ * more entries allocated at end of structure.
+ */
+ struct tc_u_knode __rcu *ht[1];
};
struct tc_u_common {
* to replay the request.
*/
module_put(em->ops->owner);
+ em->ops = NULL;
err = -EAGAIN;
}
#endif
if (len > INT_MAX)
len = INT_MAX;
+ if (unlikely(!access_ok(VERIFY_READ, buff, len)))
+ return -EFAULT;
sock = sockfd_lookup_light(fd, &err, &fput_needed);
if (!sock)
goto out;
if (size > INT_MAX)
size = INT_MAX;
+ if (unlikely(!access_ok(VERIFY_WRITE, ubuf, size)))
+ return -EFAULT;
sock = sockfd_lookup_light(fd, &err, &fput_needed);
if (!sock)
goto out;
for (i = 0; i < arg->npages && arg->pages[i]; i++)
__free_page(arg->pages[i]);
+
+ kfree(arg->pages);
}
static int gssp_alloc_receive_pages(struct gssx_arg_accept_sec_context *arg)
/* number of additional gid's */
if (get_int(&mesg, &N))
goto out;
+ if (N < 0 || N > NGROUPS_MAX)
+ goto out;
status = -ENOMEM;
rsci.cred.cr_group_info = groups_alloc(N);
if (rsci.cred.cr_group_info == NULL)
poll_wait(filp, &queue_wait, wait);
/* alway allow write */
- mask = POLL_OUT | POLLWRNORM;
+ mask = POLLOUT | POLLWRNORM;
if (!rp)
return mask;
struct super_block *pipefs_sb;
int err;
- err = rpc_clnt_debugfs_register(clnt);
- if (err)
- return err;
+ rpc_clnt_debugfs_register(clnt);
pipefs_sb = rpc_get_sb_net(net);
if (pipefs_sb) {
.release = tasks_release,
};
-int
+void
rpc_clnt_debugfs_register(struct rpc_clnt *clnt)
{
- int len, err;
+ int len;
char name[24]; /* enough for "../../rpc_xprt/ + 8 hex digits + NULL */
+ struct rpc_xprt *xprt;
/* Already registered? */
- if (clnt->cl_debugfs)
- return 0;
+ if (clnt->cl_debugfs || !rpc_clnt_dir)
+ return;
len = snprintf(name, sizeof(name), "%x", clnt->cl_clid);
if (len >= sizeof(name))
- return -EINVAL;
+ return;
/* make the per-client dir */
clnt->cl_debugfs = debugfs_create_dir(name, rpc_clnt_dir);
if (!clnt->cl_debugfs)
- return -ENOMEM;
+ return;
/* make tasks file */
- err = -ENOMEM;
if (!debugfs_create_file("tasks", S_IFREG | S_IRUSR, clnt->cl_debugfs,
clnt, &tasks_fops))
goto out_err;
- err = -EINVAL;
rcu_read_lock();
+ xprt = rcu_dereference(clnt->cl_xprt);
+ /* no "debugfs" dentry? Don't bother with the symlink. */
+ if (!xprt->debugfs) {
+ rcu_read_unlock();
+ return;
+ }
len = snprintf(name, sizeof(name), "../../rpc_xprt/%s",
- rcu_dereference(clnt->cl_xprt)->debugfs->d_name.name);
+ xprt->debugfs->d_name.name);
rcu_read_unlock();
+
if (len >= sizeof(name))
goto out_err;
- err = -ENOMEM;
if (!debugfs_create_symlink("xprt", clnt->cl_debugfs, name))
goto out_err;
- return 0;
+ return;
out_err:
debugfs_remove_recursive(clnt->cl_debugfs);
clnt->cl_debugfs = NULL;
- return err;
}
void
.release = xprt_info_release,
};
-int
+void
rpc_xprt_debugfs_register(struct rpc_xprt *xprt)
{
int len, id;
static atomic_t cur_id;
char name[9]; /* 8 hex digits + NULL term */
+ if (!rpc_xprt_dir)
+ return;
+
id = (unsigned int)atomic_inc_return(&cur_id);
len = snprintf(name, sizeof(name), "%x", id);
if (len >= sizeof(name))
- return -EINVAL;
+ return;
/* make the per-client dir */
xprt->debugfs = debugfs_create_dir(name, rpc_xprt_dir);
if (!xprt->debugfs)
- return -ENOMEM;
+ return;
/* make tasks file */
if (!debugfs_create_file("info", S_IFREG | S_IRUSR, xprt->debugfs,
xprt, &xprt_info_fops)) {
debugfs_remove_recursive(xprt->debugfs);
xprt->debugfs = NULL;
- return -ENOMEM;
}
-
- return 0;
}
void
sunrpc_debugfs_exit(void)
{
debugfs_remove_recursive(topdir);
+ topdir = NULL;
+ rpc_clnt_dir = NULL;
+ rpc_xprt_dir = NULL;
}
-int __init
+void __init
sunrpc_debugfs_init(void)
{
topdir = debugfs_create_dir("sunrpc", NULL);
if (!topdir)
- goto out;
+ return;
rpc_clnt_dir = debugfs_create_dir("rpc_clnt", topdir);
if (!rpc_clnt_dir)
if (!rpc_xprt_dir)
goto out_remove;
- return 0;
+ return;
out_remove:
debugfs_remove_recursive(topdir);
topdir = NULL;
-out:
- return -ENOMEM;
+ rpc_clnt_dir = NULL;
}
if (err)
goto out4;
- err = sunrpc_debugfs_init();
- if (err)
- goto out5;
-
+ sunrpc_debugfs_init();
#if IS_ENABLED(CONFIG_SUNRPC_DEBUG)
rpc_register_sysctl();
#endif
init_socket_xprt(); /* clnt sock transport */
return 0;
-out5:
- unregister_rpc_pipefs();
out4:
unregister_pernet_subsys(&sunrpc_net_ops);
out3:
*/
struct rpc_xprt *xprt_create_transport(struct xprt_create *args)
{
- int err;
struct rpc_xprt *xprt;
struct xprt_class *t;
return ERR_PTR(-ENOMEM);
}
- err = rpc_xprt_debugfs_register(xprt);
- if (err) {
- xprt_destroy(xprt);
- return ERR_PTR(err);
- }
+ rpc_xprt_debugfs_register(xprt);
dprintk("RPC: created transport %p with %u slots\n", xprt,
xprt->max_reqs);
struct rpc_xprt *xprt = rep->rr_xprt;
struct rpcrdma_xprt *r_xprt = rpcx_to_rdmax(xprt);
__be32 *iptr;
- int credits, rdmalen, status;
+ int rdmalen, status;
unsigned long cwnd;
+ u32 credits;
/* Check status. If bad, signal disconnect and return rep to pool */
if (rep->rr_len == ~0U) {
*/
struct rpcrdma_buffer {
spinlock_t rb_lock; /* protects indexes */
- int rb_max_requests;/* client max requests */
+ u32 rb_max_requests;/* client max requests */
struct list_head rb_mws; /* optional memory windows/fmrs/frmrs */
struct list_head rb_all;
int rb_send_index;
static void __exit tipc_exit(void)
{
tipc_bearer_cleanup();
+ unregister_pernet_subsys(&tipc_net_ops);
tipc_netlink_stop();
tipc_netlink_compat_stop();
tipc_socket_stop();
tipc_unregister_sysctl();
- unregister_pernet_subsys(&tipc_net_ops);
pr_info("Deactivated\n");
}
/* Clean up all queues, except inputq: */
__skb_queue_purge(&l_ptr->outqueue);
__skb_queue_purge(&l_ptr->deferred_queue);
- skb_queue_splice_init(&l_ptr->wakeupq, &l_ptr->inputq);
- if (!skb_queue_empty(&l_ptr->inputq))
+ if (!owner->inputq)
+ owner->inputq = &l_ptr->inputq;
+ skb_queue_splice_init(&l_ptr->wakeupq, owner->inputq);
+ if (!skb_queue_empty(owner->inputq))
owner->action_flags |= TIPC_MSG_EVT;
- owner->inputq = &l_ptr->inputq;
l_ptr->next_out = NULL;
l_ptr->unacked_window = 0;
l_ptr->checkpoint = 1;
.hashfn = jhash,
.max_shift = 20, /* 1M */
.min_shift = 8, /* 256 */
- .grow_decision = rht_grow_above_75,
- .shrink_decision = rht_shrink_below_30,
};
return rhashtable_init(&tn->sk_rht, &rht_params);
regulatory_exit();
out_fail_reg:
debugfs_remove(ieee80211_debugfs_dir);
+ nl80211_exit();
out_fail_nl80211:
unregister_netdevice_notifier(&cfg80211_netdev_notifier);
out_fail_notifier:
return err;
}
- msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
- if (!msg)
- return -ENOMEM;
-
err = parse_monitor_flags(type == NL80211_IFTYPE_MONITOR ?
info->attrs[NL80211_ATTR_MNTR_FLAGS] : NULL,
&flags);
!(rdev->wiphy.features & NL80211_FEATURE_ACTIVE_MONITOR))
return -EOPNOTSUPP;
+ msg = nlmsg_new(NLMSG_DEFAULT_SIZE, GFP_KERNEL);
+ if (!msg)
+ return -ENOMEM;
+
wdev = rdev_add_virtual_intf(rdev,
nla_data(info->attrs[NL80211_ATTR_IFNAME]),
type, err ? NULL : &flags, ¶ms);
if (parse_station_flags(info, dev->ieee80211_ptr->iftype, ¶ms))
return -EINVAL;
+ /* HT/VHT requires QoS, but if we don't have that just ignore HT/VHT
+ * as userspace might just pass through the capabilities from the IEs
+ * directly, rather than enforcing this restriction and returning an
+ * error in this case.
+ */
+ if (!(params.sta_flags_set & BIT(NL80211_STA_FLAG_WME))) {
+ params.ht_capa = NULL;
+ params.vht_capa = NULL;
+ }
+
/* When you run into this, adjust the code below for the new flag */
BUILD_BUG_ON(NL80211_STA_FLAG_MAX != 7);
}
for (j = 0; j < match->n_channels; j++) {
- if (nla_put_u32(msg,
- NL80211_ATTR_WIPHY_FREQ,
- match->channels[j])) {
+ if (nla_put_u32(msg, j, match->channels[j])) {
nla_nest_cancel(msg, nl_freqs);
nla_nest_cancel(msg, nl_match);
goto out;
/* We keep a static world regulatory domain in case of the absence of CRDA */
static const struct ieee80211_regdomain world_regdom = {
- .n_reg_rules = 6,
+ .n_reg_rules = 8,
.alpha2 = "00",
.reg_rules = {
/* IEEE 802.11b/g, channels 1..11 */
* have the xfrm_state's. We need to wait for KM to
* negotiate new SA's or bail out with error.*/
if (net->xfrm.sysctl_larval_drop) {
- dst_release(dst);
- xfrm_pols_put(pols, drop_pols);
XFRM_INC_STATS(net, LINUX_MIB_XFRMOUTNOSTATES);
-
- return ERR_PTR(-EREMOTE);
+ err = -EREMOTE;
+ goto error;
}
err = -EAGAIN;
error:
dst_release(dst);
dropdst:
- dst_release(dst_orig);
+ if (!(flags & XFRM_LOOKUP_KEEP_DST_REF))
+ dst_release(dst_orig);
xfrm_pols_put(pols, drop_pols);
return ERR_PTR(err);
}
struct sock *sk, int flags)
{
struct dst_entry *dst = xfrm_lookup(net, dst_orig, fl, sk,
- flags | XFRM_LOOKUP_QUEUE);
+ flags | XFRM_LOOKUP_QUEUE |
+ XFRM_LOOKUP_KEEP_DST_REF);
if (IS_ERR(dst) && PTR_ERR(dst) == -EREMOTE)
return make_blackhole(net, dst_orig->ops->family, dst_orig);
--- /dev/null
+# nothing to do for the initialization of this package
goto out;
/* No partial writes. */
- length = EINVAL;
+ length = -EINVAL;
if (*ppos != 0)
goto out;
if (info->count < 1)
return -EINVAL;
+ if (!*info->id.name)
+ return -EINVAL;
+ if (strnlen(info->id.name, sizeof(info->id.name)) >= sizeof(info->id.name))
+ return -EINVAL;
access = info->access == 0 ? SNDRV_CTL_ELEM_ACCESS_READWRITE :
(info->access & (SNDRV_CTL_ELEM_ACCESS_READWRITE|
SNDRV_CTL_ELEM_ACCESS_INACTIVE|
if (! snd_pcm_playback_empty(substream)) {
snd_pcm_do_start(substream, SNDRV_PCM_STATE_DRAINING);
snd_pcm_post_start(substream, SNDRV_PCM_STATE_DRAINING);
+ } else {
+ runtime->status->state = SNDRV_PCM_STATE_SETUP;
}
break;
case SNDRV_PCM_STATE_RUNNING:
int pitchbend = chan->midi_pitchbend;
int segment;
+ if (pitchbend < -0x2000)
+ pitchbend = -0x2000;
if (pitchbend > 0x1FFF)
pitchbend = 0x1FFF;
*/
#define MAX_MIDI_RX_BLOCKS 8
-#define TRANSFER_DELAY_TICKS 0x2e00 /* 479.17 µs */
+#define TRANSFER_DELAY_TICKS 0x2e00 /* 479.17 microseconds */
/* isochronous header parameters */
#define ISO_DATA_LENGTH_SHIFT 16
int amdtp_stream_init(struct amdtp_stream *s, struct fw_unit *unit,
enum amdtp_stream_direction dir, enum cip_flags flags)
{
- s->unit = fw_unit_get(unit);
+ s->unit = unit;
s->direction = dir;
s->flags = flags;
s->context = ERR_PTR(-1);
{
WARN_ON(amdtp_stream_running(s));
mutex_destroy(&s->mutex);
- fw_unit_put(s->unit);
}
EXPORT_SYMBOL(amdtp_stream_destroy);
return err;
}
+/*
+ * This module releases the FireWire unit data after all ALSA character devices
+ * are released by applications. This is for releasing stream data or finishing
+ * transactions safely. Thus at returning from .remove(), this module still keep
+ * references for the unit.
+ */
static void
bebob_card_free(struct snd_card *card)
{
struct snd_bebob *bebob = card->private_data;
+ snd_bebob_stream_destroy_duplex(bebob);
+ fw_unit_put(bebob->unit);
+
+ kfree(bebob->maudio_special_quirk);
+
if (bebob->card_index >= 0) {
mutex_lock(&devices_mutex);
clear_bit(bebob->card_index, devices_used);
card->private_free = bebob_card_free;
bebob->card = card;
- bebob->unit = unit;
+ bebob->unit = fw_unit_get(unit);
bebob->spec = spec;
mutex_init(&bebob->mutex);
spin_lock_init(&bebob->lock);
if (bebob == NULL)
return;
- kfree(bebob->maudio_special_quirk);
+ /* Awake bus-reset waiters. */
+ if (!completion_done(&bebob->bus_reset))
+ complete_all(&bebob->bus_reset);
- snd_bebob_stream_destroy_duplex(bebob);
- snd_card_disconnect(bebob->card);
+ /* No need to wait for releasing card object in this context. */
snd_card_free_when_closed(bebob->card);
}
struct fw_device *device = fw_parent_device(unit);
int err, rcode;
u64 date;
- __be32 cues[3] = {
- MAUDIO_BOOTLOADER_CUE1,
- MAUDIO_BOOTLOADER_CUE2,
- MAUDIO_BOOTLOADER_CUE3
+ __le32 cues[3] = {
+ cpu_to_le32(MAUDIO_BOOTLOADER_CUE1),
+ cpu_to_le32(MAUDIO_BOOTLOADER_CUE2),
+ cpu_to_le32(MAUDIO_BOOTLOADER_CUE3)
};
/* check date of software used to build */
static void
destroy_both_connections(struct snd_bebob *bebob)
{
- break_both_connections(bebob);
-
cmp_connection_destroy(&bebob->in_conn);
cmp_connection_destroy(&bebob->out_conn);
}
mutex_unlock(&bebob->mutex);
}
+/*
+ * This function should be called before starting streams or after stopping
+ * streams.
+ */
void snd_bebob_stream_destroy_duplex(struct snd_bebob *bebob)
{
- mutex_lock(&bebob->mutex);
-
- amdtp_stream_pcm_abort(&bebob->rx_stream);
- amdtp_stream_pcm_abort(&bebob->tx_stream);
-
- amdtp_stream_stop(&bebob->rx_stream);
- amdtp_stream_stop(&bebob->tx_stream);
-
amdtp_stream_destroy(&bebob->rx_stream);
amdtp_stream_destroy(&bebob->tx_stream);
destroy_both_connections(bebob);
-
- mutex_unlock(&bebob->mutex);
}
/*
return err;
}
+/*
+ * This function should be called before starting streams or after stopping
+ * streams.
+ */
static void destroy_stream(struct snd_dice *dice, struct amdtp_stream *stream)
{
- amdtp_stream_destroy(stream);
+ struct fw_iso_resources *resources;
if (stream == &dice->tx_stream)
- fw_iso_resources_destroy(&dice->tx_resources);
+ resources = &dice->tx_resources;
else
- fw_iso_resources_destroy(&dice->rx_resources);
+ resources = &dice->rx_resources;
+
+ amdtp_stream_destroy(stream);
+ fw_iso_resources_destroy(resources);
}
int snd_dice_stream_init_duplex(struct snd_dice *dice)
goto end;
err = init_stream(dice, &dice->rx_stream);
+ if (err < 0)
+ destroy_stream(dice, &dice->tx_stream);
end:
return err;
}
{
snd_dice_transaction_clear_enable(dice);
- stop_stream(dice, &dice->tx_stream);
destroy_stream(dice, &dice->tx_stream);
-
- stop_stream(dice, &dice->rx_stream);
destroy_stream(dice, &dice->rx_stream);
dice->substreams_counter = 0;
strcpy(card->mixername, "DICE");
}
+/*
+ * This module releases the FireWire unit data after all ALSA character devices
+ * are released by applications. This is for releasing stream data or finishing
+ * transactions safely. Thus at returning from .remove(), this module still keep
+ * references for the unit.
+ */
static void dice_card_free(struct snd_card *card)
{
struct snd_dice *dice = card->private_data;
+ snd_dice_stream_destroy_duplex(dice);
snd_dice_transaction_destroy(dice);
+ fw_unit_put(dice->unit);
+
mutex_destroy(&dice->mutex);
}
dice = card->private_data;
dice->card = card;
- dice->unit = unit;
+ dice->unit = fw_unit_get(unit);
card->private_free = dice_card_free;
spin_lock_init(&dice->lock);
{
struct snd_dice *dice = dev_get_drvdata(&unit->device);
- snd_card_disconnect(dice->card);
-
- snd_dice_stream_destroy_duplex(dice);
-
+ /* No need to wait for releasing card object in this context. */
snd_card_free_when_closed(dice->card);
}
return err;
}
+/*
+ * This module releases the FireWire unit data after all ALSA character devices
+ * are released by applications. This is for releasing stream data or finishing
+ * transactions safely. Thus at returning from .remove(), this module still keep
+ * references for the unit.
+ */
static void
efw_card_free(struct snd_card *card)
{
struct snd_efw *efw = card->private_data;
+ snd_efw_stream_destroy_duplex(efw);
+ snd_efw_transaction_remove_instance(efw);
+ fw_unit_put(efw->unit);
+
+ kfree(efw->resp_buf);
+
if (efw->card_index >= 0) {
mutex_lock(&devices_mutex);
clear_bit(efw->card_index, devices_used);
}
mutex_destroy(&efw->mutex);
- kfree(efw->resp_buf);
}
static int
card->private_free = efw_card_free;
efw->card = card;
- efw->unit = unit;
+ efw->unit = fw_unit_get(unit);
mutex_init(&efw->mutex);
spin_lock_init(&efw->lock);
init_waitqueue_head(&efw->hwdep_wait);
{
struct snd_efw *efw = dev_get_drvdata(&unit->device);
- snd_efw_stream_destroy_duplex(efw);
- snd_efw_transaction_remove_instance(efw);
-
- snd_card_disconnect(efw->card);
+ /* No need to wait for releasing card object in this context. */
snd_card_free_when_closed(efw->card);
}
return err;
}
+/*
+ * This function should be called before starting the stream or after stopping
+ * the streams.
+ */
static void
destroy_stream(struct snd_efw *efw, struct amdtp_stream *stream)
{
- stop_stream(efw, stream);
-
- amdtp_stream_destroy(stream);
+ struct cmp_connection *conn;
if (stream == &efw->tx_stream)
- cmp_connection_destroy(&efw->out_conn);
+ conn = &efw->out_conn;
else
- cmp_connection_destroy(&efw->in_conn);
+ conn = &efw->in_conn;
+
+ amdtp_stream_destroy(stream);
+ cmp_connection_destroy(&efw->out_conn);
}
static int
void snd_efw_stream_destroy_duplex(struct snd_efw *efw)
{
- mutex_lock(&efw->mutex);
-
destroy_stream(efw, &efw->rx_stream);
destroy_stream(efw, &efw->tx_stream);
-
- mutex_unlock(&efw->mutex);
}
void snd_efw_stream_lock_changed(struct snd_efw *efw)
int fw_iso_resources_init(struct fw_iso_resources *r, struct fw_unit *unit)
{
r->channels_mask = ~0uLL;
- r->unit = fw_unit_get(unit);
+ r->unit = unit;
mutex_init(&r->mutex);
r->allocated = false;
{
WARN_ON(r->allocated);
mutex_destroy(&r->mutex);
- fw_unit_put(r->unit);
}
EXPORT_SYMBOL(fw_iso_resources_destroy);
}
/* Wait first packet */
- err = amdtp_stream_wait_callback(stream, CALLBACK_TIMEOUT);
- if (err < 0)
+ if (!amdtp_stream_wait_callback(stream, CALLBACK_TIMEOUT)) {
stop_stream(oxfw, stream);
+ err = -ETIMEDOUT;
+ }
end:
return err;
}
stop_stream(oxfw, stream);
}
+/*
+ * This function should be called before starting the stream or after stopping
+ * the streams.
+ */
void snd_oxfw_stream_destroy_simplex(struct snd_oxfw *oxfw,
struct amdtp_stream *stream)
{
else
conn = &oxfw->in_conn;
- stop_stream(oxfw, stream);
-
amdtp_stream_destroy(stream);
cmp_connection_destroy(conn);
}
return err;
}
+/*
+ * This module releases the FireWire unit data after all ALSA character devices
+ * are released by applications. This is for releasing stream data or finishing
+ * transactions safely. Thus at returning from .remove(), this module still keep
+ * references for the unit.
+ */
static void oxfw_card_free(struct snd_card *card)
{
struct snd_oxfw *oxfw = card->private_data;
unsigned int i;
+ snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream);
+ if (oxfw->has_output)
+ snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->tx_stream);
+
+ fw_unit_put(oxfw->unit);
+
for (i = 0; i < SND_OXFW_STREAM_FORMAT_ENTRIES; i++) {
kfree(oxfw->tx_stream_formats[i]);
kfree(oxfw->rx_stream_formats[i]);
oxfw = card->private_data;
oxfw->card = card;
mutex_init(&oxfw->mutex);
- oxfw->unit = unit;
+ oxfw->unit = fw_unit_get(unit);
oxfw->device_info = (const struct device_info *)id->driver_data;
spin_lock_init(&oxfw->lock);
init_waitqueue_head(&oxfw->hwdep_wait);
{
struct snd_oxfw *oxfw = dev_get_drvdata(&unit->device);
- snd_card_disconnect(oxfw->card);
-
- snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->rx_stream);
- if (oxfw->has_output)
- snd_oxfw_stream_destroy_simplex(oxfw, &oxfw->tx_stream);
-
+ /* No need to wait for releasing card object in this context. */
snd_card_free_when_closed(oxfw->card);
}
spin_lock_init(&chip->mixer_lock);
strcpy(card->mixername, "MSND Pinnacle Mixer");
- for (idx = 0; idx < ARRAY_SIZE(snd_msnd_controls); idx++)
+ for (idx = 0; idx < ARRAY_SIZE(snd_msnd_controls); idx++) {
err = snd_ctl_add(card,
snd_ctl_new1(snd_msnd_controls + idx, chip));
if (err < 0)
return err;
+ }
return 0;
}
dev_err(chip->card->dev, "cannot allocate CORB/RIRB\n");
return err;
}
-EXPORT_SYMBOL_GPL(azx_alloc_cmd_io);
static void azx_init_cmd_io(struct azx *chip)
{
azx_writeb(chip, RIRBCTL, AZX_RBCTL_DMA_EN | AZX_RBCTL_IRQ_EN);
spin_unlock_irq(&chip->reg_lock);
}
-EXPORT_SYMBOL_GPL(azx_init_cmd_io);
static void azx_free_cmd_io(struct azx *chip)
{
azx_writeb(chip, CORBCTL, 0);
spin_unlock_irq(&chip->reg_lock);
}
-EXPORT_SYMBOL_GPL(azx_free_cmd_io);
static unsigned int azx_command_addr(u32 cmd)
{
}
}
- if (!bus->no_response_fallback)
+ if (bus->no_response_fallback)
return -1;
if (!chip->polling_mode && chip->poll_count < 2) {
else
return azx_corb_send_cmd(bus, val);
}
-EXPORT_SYMBOL_GPL(azx_send_cmd);
/* get a response */
static unsigned int azx_get_response(struct hda_bus *bus,
else
return azx_rirb_get_response(bus, addr);
}
-EXPORT_SYMBOL_GPL(azx_get_response);
#ifdef CONFIG_SND_HDA_DSP_LOADER
/*
return val;
}
+/* is this a stereo widget or a stereo-to-mono mix? */
+static bool is_stereo_amps(struct hda_codec *codec, hda_nid_t nid, int dir)
+{
+ unsigned int wcaps = get_wcaps(codec, nid);
+ hda_nid_t conn;
+
+ if (wcaps & AC_WCAP_STEREO)
+ return true;
+ if (dir != HDA_INPUT || get_wcaps_type(wcaps) != AC_WID_AUD_MIX)
+ return false;
+ if (snd_hda_get_num_conns(codec, nid) != 1)
+ return false;
+ if (snd_hda_get_connections(codec, nid, &conn, 1) < 0)
+ return false;
+ return !!(get_wcaps(codec, conn) & AC_WCAP_STEREO);
+}
+
/* initialize the amp value (only at the first time) */
static void init_amp(struct hda_codec *codec, hda_nid_t nid, int dir, int idx)
{
unsigned int caps = query_amp_caps(codec, nid, dir);
int val = get_amp_val_to_activate(codec, nid, dir, caps, false);
- snd_hda_codec_amp_init_stereo(codec, nid, dir, idx, 0xff, val);
+
+ if (is_stereo_amps(codec, nid, dir))
+ snd_hda_codec_amp_init_stereo(codec, nid, dir, idx, 0xff, val);
+ else
+ snd_hda_codec_amp_init(codec, nid, 0, dir, idx, 0xff, val);
+}
+
+/* update the amp, doing in stereo or mono depending on NID */
+static int update_amp(struct hda_codec *codec, hda_nid_t nid, int dir, int idx,
+ unsigned int mask, unsigned int val)
+{
+ if (is_stereo_amps(codec, nid, dir))
+ return snd_hda_codec_amp_stereo(codec, nid, dir, idx,
+ mask, val);
+ else
+ return snd_hda_codec_amp_update(codec, nid, 0, dir, idx,
+ mask, val);
}
/* calculate amp value mask we can modify;
return;
val &= mask;
- snd_hda_codec_amp_stereo(codec, nid, dir, idx, mask, val);
+ update_amp(codec, nid, dir, idx, mask, val);
}
static void activate_amp_out(struct hda_codec *codec, struct nid_path *path,
has_amp = nid_has_mute(codec, mix, HDA_INPUT);
for (i = 0; i < nums; i++) {
if (has_amp)
- snd_hda_codec_amp_stereo(codec, mix,
- HDA_INPUT, i,
- 0xff, HDA_AMP_MUTE);
+ update_amp(codec, mix, HDA_INPUT, i,
+ 0xff, HDA_AMP_MUTE);
else if (nid_has_volume(codec, conn[i], HDA_OUTPUT))
- snd_hda_codec_amp_stereo(codec, conn[i],
- HDA_OUTPUT, 0,
- 0xff, HDA_AMP_MUTE);
+ update_amp(codec, conn[i], HDA_OUTPUT, 0,
+ 0xff, HDA_AMP_MUTE);
}
}
.driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH_NOPM },
/* Panther Point */
{ PCI_DEVICE(0x8086, 0x1e20),
- .driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH },
+ .driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH_NOPM },
/* Lynx Point */
{ PCI_DEVICE(0x8086, 0x8c20),
.driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH },
.driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH },
/* Sunrise Point */
{ PCI_DEVICE(0x8086, 0xa170),
- .driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_PCH },
+ .driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_SKYLAKE },
/* Sunrise Point-LP */
{ PCI_DEVICE(0x8086, 0x9d70),
.driver_data = AZX_DRIVER_PCH | AZX_DCAPS_INTEL_SKYLAKE },
(caps & AC_AMPCAP_MUTE) >> AC_AMPCAP_MUTE_SHIFT);
}
+/* is this a stereo widget or a stereo-to-mono mix? */
+static bool is_stereo_amps(struct hda_codec *codec, hda_nid_t nid,
+ int dir, unsigned int wcaps, int indices)
+{
+ hda_nid_t conn;
+
+ if (wcaps & AC_WCAP_STEREO)
+ return true;
+ /* check for a stereo-to-mono mix; it must be:
+ * only a single connection, only for input, and only a mixer widget
+ */
+ if (indices != 1 || dir != HDA_INPUT ||
+ get_wcaps_type(wcaps) != AC_WID_AUD_MIX)
+ return false;
+
+ if (snd_hda_get_raw_connections(codec, nid, &conn, 1) < 0)
+ return false;
+ /* the connection source is a stereo? */
+ wcaps = snd_hda_param_read(codec, conn, AC_PAR_AUDIO_WIDGET_CAP);
+ return !!(wcaps & AC_WCAP_STEREO);
+}
+
static void print_amp_vals(struct snd_info_buffer *buffer,
struct hda_codec *codec, hda_nid_t nid,
- int dir, int stereo, int indices)
+ int dir, unsigned int wcaps, int indices)
{
unsigned int val;
+ bool stereo;
int i;
+ stereo = is_stereo_amps(codec, nid, dir, wcaps, indices);
+
dir = dir == HDA_OUTPUT ? AC_AMP_GET_OUTPUT : AC_AMP_GET_INPUT;
for (i = 0; i < indices; i++) {
snd_iprintf(buffer, " [");
(codec->single_adc_amp &&
wid_type == AC_WID_AUD_IN))
print_amp_vals(buffer, codec, nid, HDA_INPUT,
- wid_caps & AC_WCAP_STEREO,
- 1);
+ wid_caps, 1);
else
print_amp_vals(buffer, codec, nid, HDA_INPUT,
- wid_caps & AC_WCAP_STEREO,
- conn_len);
+ wid_caps, conn_len);
}
if (wid_caps & AC_WCAP_OUT_AMP) {
snd_iprintf(buffer, " Amp-Out caps: ");
if (wid_type == AC_WID_PIN &&
codec->pin_amp_workaround)
print_amp_vals(buffer, codec, nid, HDA_OUTPUT,
- wid_caps & AC_WCAP_STEREO,
- conn_len);
+ wid_caps, conn_len);
else
print_amp_vals(buffer, codec, nid, HDA_OUTPUT,
- wid_caps & AC_WCAP_STEREO, 1);
+ wid_caps, 1);
}
switch (wid_type) {
SND_PCI_QUIRK(0x106b, 0x1c00, "MacBookPro 8,1", CS420X_MBP81),
SND_PCI_QUIRK(0x106b, 0x2000, "iMac 12,2", CS420X_IMAC27_122),
SND_PCI_QUIRK(0x106b, 0x2800, "MacBookPro 10,1", CS420X_MBP101),
+ SND_PCI_QUIRK(0x106b, 0x5600, "MacBookAir 5,2", CS420X_MBP81),
SND_PCI_QUIRK(0x106b, 0x5b00, "MacBookAir 4,2", CS420X_MBA42),
SND_PCI_QUIRK_VENDOR(0x106b, "Apple", CS420X_APPLE),
{} /* terminator */
return -ENOMEM;
spec->gen.automute_hook = cs_automute;
+ codec->single_adc_amp = 1;
snd_hda_pick_fixup(codec, cs420x_models, cs420x_fixup_tbl,
cs420x_fixups);
CXT_PINCFG_LENOVO_TP410,
CXT_PINCFG_LEMOTE_A1004,
CXT_PINCFG_LEMOTE_A1205,
+ CXT_PINCFG_COMPAQ_CQ60,
CXT_FIXUP_STEREO_DMIC,
CXT_FIXUP_INC_MIC_BOOST,
CXT_FIXUP_HEADPHONE_MIC_PIN,
.type = HDA_FIXUP_PINS,
.v.pins = cxt_pincfg_lemote,
},
+ [CXT_PINCFG_COMPAQ_CQ60] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ /* 0x17 was falsely set up as a mic, it should 0x1d */
+ { 0x17, 0x400001f0 },
+ { 0x1d, 0x97a70120 },
+ { }
+ }
+ },
[CXT_FIXUP_STEREO_DMIC] = {
.type = HDA_FIXUP_FUNC,
.v.func = cxt_fixup_stereo_dmic,
};
static const struct snd_pci_quirk cxt5051_fixups[] = {
+ SND_PCI_QUIRK(0x103c, 0x360b, "Compaq CQ60", CXT_PINCFG_COMPAQ_CQ60),
SND_PCI_QUIRK(0x17aa, 0x20f2, "Lenovo X200", CXT_PINCFG_LENOVO_X200),
{}
};
{
/* We currently only handle front, HP */
static hda_nid_t pins[] = {
- 0x0f, 0x10, 0x14, 0x15, 0
+ 0x0f, 0x10, 0x14, 0x15, 0x17, 0
};
hda_nid_t *p;
for (p = pins; *p; p++)
if (!hp_pin)
return;
+
+ msleep(30);
hp_pin_sense = snd_hda_jack_detect(codec, hp_pin);
/* Index 0x43 Direct Drive HP AMP LPM Control 1 */
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_process_coef_fw(codec, coef0255);
break;
case 0x10ec0233:
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_write_coef_idx(codec, 0x45, 0xc489);
snd_hda_set_pin_ctl_cache(codec, hp_pin, 0);
alc_process_coef_fw(codec, coef0255);
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_process_coef_fw(codec, coef0255);
break;
case 0x10ec0233:
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_process_coef_fw(codec, coef0255);
break;
case 0x10ec0233:
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_process_coef_fw(codec, coef0255);
break;
case 0x10ec0233:
switch (codec->vendor_id) {
case 0x10ec0255:
+ case 0x10ec0256:
alc_process_coef_fw(codec, coef0255);
msleep(300);
val = alc_read_coef_idx(codec, 0x46);
ALC269_FIXUP_QUANTA_MUTE,
ALC269_FIXUP_LIFEBOOK,
ALC269_FIXUP_LIFEBOOK_EXTMIC,
+ ALC269_FIXUP_LIFEBOOK_HP_PIN,
ALC269_FIXUP_AMIC,
ALC269_FIXUP_DMIC,
ALC269VB_FIXUP_AMIC,
{ }
},
},
+ [ALC269_FIXUP_LIFEBOOK_HP_PIN] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ { 0x21, 0x0221102f }, /* HP out */
+ { }
+ },
+ },
[ALC269_FIXUP_AMIC] = {
.type = HDA_FIXUP_PINS,
.v.pins = (const struct hda_pintbl[]) {
SND_PCI_QUIRK(0x104d, 0x9084, "Sony VAIO", ALC275_FIXUP_SONY_HWEQ),
SND_PCI_QUIRK(0x104d, 0x9099, "Sony VAIO S13", ALC275_FIXUP_SONY_DISABLE_AAMIX),
SND_PCI_QUIRK(0x10cf, 0x1475, "Lifebook", ALC269_FIXUP_LIFEBOOK),
+ SND_PCI_QUIRK(0x10cf, 0x15dc, "Lifebook T731", ALC269_FIXUP_LIFEBOOK_HP_PIN),
SND_PCI_QUIRK(0x10cf, 0x1845, "Lifebook U904", ALC269_FIXUP_LIFEBOOK_EXTMIC),
SND_PCI_QUIRK(0x144d, 0xc109, "Samsung Ativ book 9 (NP900X3G)", ALC269_FIXUP_INV_DMIC),
SND_PCI_QUIRK(0x1458, 0xfa53, "Gigabyte BXBT-2807", ALC283_FIXUP_BXBT2807_MIC),
SND_PCI_QUIRK(0x17aa, 0x501a, "Thinkpad", ALC283_FIXUP_INT_MIC),
SND_PCI_QUIRK(0x17aa, 0x501e, "Thinkpad L440", ALC292_FIXUP_TPT440_DOCK),
SND_PCI_QUIRK(0x17aa, 0x5026, "Thinkpad", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
+ SND_PCI_QUIRK(0x17aa, 0x5036, "Thinkpad T450s", ALC292_FIXUP_TPT440_DOCK),
SND_PCI_QUIRK(0x17aa, 0x5109, "Thinkpad", ALC269_FIXUP_LIMIT_INT_MIC_BOOST),
SND_PCI_QUIRK(0x17aa, 0x3bf8, "Quanta FL1", ALC269_FIXUP_PCM_44K),
SND_PCI_QUIRK(0x17aa, 0x9e54, "LENOVO NB", ALC269_FIXUP_LENOVO_EAPD),
{0x17, 0x40000000},
{0x1d, 0x40700001},
{0x21, 0x02211040}),
+ SND_HDA_PIN_QUIRK(0x10ec0255, 0x1028, "Dell", ALC255_FIXUP_DELL1_MIC_NO_PRESENCE,
+ ALC255_STANDARD_PINS,
+ {0x12, 0x90a60170},
+ {0x14, 0x90170140},
+ {0x17, 0x40000000},
+ {0x1d, 0x40700001},
+ {0x21, 0x02211050}),
+ SND_HDA_PIN_QUIRK(0x10ec0256, 0x1028, "Dell", ALC255_FIXUP_DELL1_MIC_NO_PRESENCE,
+ {0x12, 0x90a60140},
+ {0x13, 0x40000000},
+ {0x14, 0x90170110},
+ {0x19, 0x411111f0},
+ {0x1a, 0x411111f0},
+ {0x1b, 0x411111f0},
+ {0x1d, 0x40700001},
+ {0x1e, 0x411111f0},
+ {0x21, 0x02211020}),
SND_HDA_PIN_QUIRK(0x10ec0280, 0x103c, "HP", ALC280_FIXUP_HP_GPIO4,
{0x12, 0x90a60130},
{0x13, 0x40000000},
STAC_ALIENWARE_M17X,
STAC_92HD89XX_HP_FRONT_JACK,
STAC_92HD89XX_HP_Z1_G2_RIGHT_MIC_JACK,
+ STAC_92HD73XX_ASUS_MOBO,
STAC_92HD73XX_MODELS
};
[STAC_92HD89XX_HP_Z1_G2_RIGHT_MIC_JACK] = {
.type = HDA_FIXUP_PINS,
.v.pins = stac92hd89xx_hp_z1_g2_right_mic_jack_pin_configs,
- }
+ },
+ [STAC_92HD73XX_ASUS_MOBO] = {
+ .type = HDA_FIXUP_PINS,
+ .v.pins = (const struct hda_pintbl[]) {
+ /* enable 5.1 and SPDIF out */
+ { 0x0c, 0x01014411 },
+ { 0x0d, 0x01014410 },
+ { 0x0e, 0x01014412 },
+ { 0x22, 0x014b1180 },
+ { }
+ }
+ },
};
static const struct hda_model_fixup stac92hd73xx_models[] = {
{ .id = STAC_DELL_M6_BOTH, .name = "dell-m6" },
{ .id = STAC_DELL_EQ, .name = "dell-eq" },
{ .id = STAC_ALIENWARE_M17X, .name = "alienware" },
+ { .id = STAC_92HD73XX_ASUS_MOBO, .name = "asus-mobo" },
{}
};
"HP Z1 G2", STAC_92HD89XX_HP_Z1_G2_RIGHT_MIC_JACK),
SND_PCI_QUIRK(PCI_VENDOR_ID_HP, 0x2b17,
"unknown HP", STAC_92HD89XX_HP_FRONT_JACK),
+ SND_PCI_QUIRK(PCI_VENDOR_ID_ASUSTEK, 0x83f8, "ASUS AT4NM10",
+ STAC_92HD73XX_ASUS_MOBO),
{} /* terminator */
};
#include <sound/pcm_params.h>
#include <sound/soc.h>
-#include <asm/mach-types.h>
-
#include "../codecs/wm8731.h"
#include "atmel-pcm.h"
#include "atmel_ssc_dai.h"
int ret;
if (!np) {
- if (!(machine_is_at91sam9g20ek() ||
- machine_is_at91sam9g20ek_2mmc()))
- return -ENODEV;
+ return -ENODEV;
}
ret = atmel_ssc_set_audio(0);
card->dev = &pdev->dev;
/* Parse device node info */
- if (np) {
- ret = snd_soc_of_parse_card_name(card, "atmel,model");
- if (ret)
- goto err;
-
- ret = snd_soc_of_parse_audio_routing(card,
- "atmel,audio-routing");
- if (ret)
- goto err;
-
- /* Parse codec info */
- at91sam9g20ek_dai.codec_name = NULL;
- codec_np = of_parse_phandle(np, "atmel,audio-codec", 0);
- if (!codec_np) {
- dev_err(&pdev->dev, "codec info missing\n");
- return -EINVAL;
- }
- at91sam9g20ek_dai.codec_of_node = codec_np;
-
- /* Parse dai and platform info */
- at91sam9g20ek_dai.cpu_dai_name = NULL;
- at91sam9g20ek_dai.platform_name = NULL;
- cpu_np = of_parse_phandle(np, "atmel,ssc-controller", 0);
- if (!cpu_np) {
- dev_err(&pdev->dev, "dai and pcm info missing\n");
- return -EINVAL;
- }
- at91sam9g20ek_dai.cpu_of_node = cpu_np;
- at91sam9g20ek_dai.platform_of_node = cpu_np;
-
- of_node_put(codec_np);
- of_node_put(cpu_np);
+ ret = snd_soc_of_parse_card_name(card, "atmel,model");
+ if (ret)
+ goto err;
+
+ ret = snd_soc_of_parse_audio_routing(card,
+ "atmel,audio-routing");
+ if (ret)
+ goto err;
+
+ /* Parse codec info */
+ at91sam9g20ek_dai.codec_name = NULL;
+ codec_np = of_parse_phandle(np, "atmel,audio-codec", 0);
+ if (!codec_np) {
+ dev_err(&pdev->dev, "codec info missing\n");
+ return -EINVAL;
+ }
+ at91sam9g20ek_dai.codec_of_node = codec_np;
+
+ /* Parse dai and platform info */
+ at91sam9g20ek_dai.cpu_dai_name = NULL;
+ at91sam9g20ek_dai.platform_name = NULL;
+ cpu_np = of_parse_phandle(np, "atmel,ssc-controller", 0);
+ if (!cpu_np) {
+ dev_err(&pdev->dev, "dai and pcm info missing\n");
+ return -EINVAL;
}
+ at91sam9g20ek_dai.cpu_of_node = cpu_np;
+ at91sam9g20ek_dai.platform_of_node = cpu_np;
+
+ of_node_put(codec_np);
+ of_node_put(cpu_np);
ret = snd_soc_register_card(card);
if (ret) {
config SND_EP93XX_SOC_SNAPPERCL15
tristate "SoC Audio support for Bluewater Systems Snapper CL15 module"
- depends on SND_EP93XX_SOC && MACH_SNAPPER_CL15
+ depends on SND_EP93XX_SOC && MACH_SNAPPER_CL15 && I2C
select SND_EP93XX_SOC_I2S
select SND_SOC_TLV320AIC23_I2C
help
select SND_SOC_MAX98088 if I2C
select SND_SOC_MAX98090 if I2C
select SND_SOC_MAX98095 if I2C
- select SND_SOC_MAX98357A
+ select SND_SOC_MAX98357A if GPIOLIB
select SND_SOC_MAX9850 if I2C
select SND_SOC_MAX9768 if I2C
select SND_SOC_MAX9877 if I2C
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct adav80x *adav80x = snd_soc_codec_get_drvdata(codec);
- unsigned int deemph = ucontrol->value.enumerated.item[0];
+ unsigned int deemph = ucontrol->value.integer.value[0];
if (deemph > 1)
return -EINVAL;
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct adav80x *adav80x = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = adav80x->deemph;
+ ucontrol->value.integer.value[0] = adav80x->deemph;
return 0;
};
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct ak4641_priv *ak4641 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
if (deemph > 1)
return -EINVAL;
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct ak4641_priv *ak4641 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = ak4641->deemph;
+ ucontrol->value.integer.value[0] = ak4641->deemph;
return 0;
};
};
static const struct snd_soc_dapm_route ak4671_intercon[] = {
- {"DAC Left", "NULL", "PMPLL"},
- {"DAC Right", "NULL", "PMPLL"},
- {"ADC Left", "NULL", "PMPLL"},
- {"ADC Right", "NULL", "PMPLL"},
+ {"DAC Left", NULL, "PMPLL"},
+ {"DAC Right", NULL, "PMPLL"},
+ {"ADC Left", NULL, "PMPLL"},
+ {"ADC Right", NULL, "PMPLL"},
/* Outputs */
- {"LOUT1", "NULL", "LOUT1 Mixer"},
- {"ROUT1", "NULL", "ROUT1 Mixer"},
- {"LOUT2", "NULL", "LOUT2 Mix Amp"},
- {"ROUT2", "NULL", "ROUT2 Mix Amp"},
- {"LOUT3", "NULL", "LOUT3 Mixer"},
- {"ROUT3", "NULL", "ROUT3 Mixer"},
+ {"LOUT1", NULL, "LOUT1 Mixer"},
+ {"ROUT1", NULL, "ROUT1 Mixer"},
+ {"LOUT2", NULL, "LOUT2 Mix Amp"},
+ {"ROUT2", NULL, "ROUT2 Mix Amp"},
+ {"LOUT3", NULL, "LOUT3 Mixer"},
+ {"ROUT3", NULL, "ROUT3 Mixer"},
{"LOUT1 Mixer", "DACL", "DAC Left"},
{"ROUT1 Mixer", "DACR", "DAC Right"},
{"LOUT2 Mixer", "DACHL", "DAC Left"},
{"ROUT2 Mixer", "DACHR", "DAC Right"},
- {"LOUT2 Mix Amp", "NULL", "LOUT2 Mixer"},
- {"ROUT2 Mix Amp", "NULL", "ROUT2 Mixer"},
+ {"LOUT2 Mix Amp", NULL, "LOUT2 Mixer"},
+ {"ROUT2 Mix Amp", NULL, "ROUT2 Mixer"},
{"LOUT3 Mixer", "DACSL", "DAC Left"},
{"ROUT3 Mixer", "DACSR", "DAC Right"},
{"LIN2", NULL, "Mic Bias"},
{"RIN2", NULL, "Mic Bias"},
- {"ADC Left", "NULL", "LIN MUX"},
- {"ADC Right", "NULL", "RIN MUX"},
+ {"ADC Left", NULL, "LIN MUX"},
+ {"ADC Right", NULL, "RIN MUX"},
/* Analog Loops */
- {"LIN1 Mixing Circuit", "NULL", "LIN1"},
- {"RIN1 Mixing Circuit", "NULL", "RIN1"},
- {"LIN2 Mixing Circuit", "NULL", "LIN2"},
- {"RIN2 Mixing Circuit", "NULL", "RIN2"},
- {"LIN3 Mixing Circuit", "NULL", "LIN3"},
- {"RIN3 Mixing Circuit", "NULL", "RIN3"},
- {"LIN4 Mixing Circuit", "NULL", "LIN4"},
- {"RIN4 Mixing Circuit", "NULL", "RIN4"},
+ {"LIN1 Mixing Circuit", NULL, "LIN1"},
+ {"RIN1 Mixing Circuit", NULL, "RIN1"},
+ {"LIN2 Mixing Circuit", NULL, "LIN2"},
+ {"RIN2 Mixing Circuit", NULL, "RIN2"},
+ {"LIN3 Mixing Circuit", NULL, "LIN3"},
+ {"RIN3 Mixing Circuit", NULL, "RIN3"},
+ {"LIN4 Mixing Circuit", NULL, "LIN4"},
+ {"RIN4 Mixing Circuit", NULL, "RIN4"},
{"LOUT1 Mixer", "LINL1", "LIN1 Mixing Circuit"},
{"ROUT1 Mixer", "RINR1", "RIN1 Mixing Circuit"},
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct cs4271_private *cs4271 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = cs4271->deemph;
+ ucontrol->value.integer.value[0] = cs4271->deemph;
return 0;
}
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct cs4271_private *cs4271 = snd_soc_codec_get_drvdata(codec);
- cs4271->deemph = ucontrol->value.enumerated.item[0];
+ cs4271->deemph = ucontrol->value.integer.value[0];
return cs4271_set_deemph(codec);
}
static const struct snd_soc_dapm_route da732x_dapm_routes[] = {
/* Inputs */
- {"AUX1L PGA", "NULL", "AUX1L"},
- {"AUX1R PGA", "NULL", "AUX1R"},
+ {"AUX1L PGA", NULL, "AUX1L"},
+ {"AUX1R PGA", NULL, "AUX1R"},
{"MIC1 PGA", NULL, "MIC1"},
- {"MIC2 PGA", "NULL", "MIC2"},
- {"MIC3 PGA", "NULL", "MIC3"},
+ {"MIC2 PGA", NULL, "MIC2"},
+ {"MIC3 PGA", NULL, "MIC3"},
/* Capture Path */
{"ADC1 Left MUX", "MIC1", "MIC1 PGA"},
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = es8328->deemph;
+ ucontrol->value.integer.value[0] = es8328->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct es8328_priv *es8328 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
int ret;
if (deemph > 1)
* max98357a.c -- MAX98357A ALSA SoC Codec driver
*/
-#include <linux/module.h>
+#include <linux/device.h>
+#include <linux/err.h>
#include <linux/gpio.h>
+#include <linux/gpio/consumer.h>
+#include <linux/kernel.h>
+#include <linux/mod_devicetable.h>
+#include <linux/module.h>
+#include <linux/of.h>
+#include <linux/platform_device.h>
+#include <sound/pcm.h>
#include <sound/soc.h>
+#include <sound/soc-dai.h>
+#include <sound/soc-dapm.h>
#define DRV_NAME "max98357a"
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct pcm1681_private *priv = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = priv->deemph;
+ ucontrol->value.integer.value[0] = priv->deemph;
return 0;
}
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct pcm1681_private *priv = snd_soc_codec_get_drvdata(codec);
- priv->deemph = ucontrol->value.enumerated.item[0];
+ priv->deemph = ucontrol->value.integer.value[0];
return pcm1681_set_deemph(codec);
}
ret, pcm512x->pll_out);
return ret;
}
-
- gpio = PCM512x_G1OE << (4 - 1);
- ret = regmap_update_bits(pcm512x->regmap, PCM512x_GPIO_EN,
- gpio, gpio);
- if (ret != 0) {
- dev_err(codec->dev, "Failed to enable gpio %d: %d\n",
- 4, ret);
- return ret;
- }
-
- gpio = PCM512x_GPIO_OUTPUT_1 + 4 - 1;
- ret = regmap_update_bits(pcm512x->regmap, gpio,
- PCM512x_GxSL, PCM512x_GxSL_PLLLK);
- if (ret != 0) {
- dev_err(codec->dev,
- "Failed to output pll lock on %d: %d\n",
- ret, 4);
- return ret;
- }
}
ret = regmap_update_bits(pcm512x->regmap, PCM512x_SYNCHRONIZE,
.ident = "Dell Dino",
.matches = {
DMI_MATCH(DMI_SYS_VENDOR, "Dell Inc."),
- DMI_MATCH(DMI_BOARD_NAME, "0144P8")
+ DMI_MATCH(DMI_PRODUCT_NAME, "XPS 13 9343")
}
},
{ }
case RT5670_ADC_EQ_CTRL1:
case RT5670_EQ_CTRL1:
case RT5670_ALC_CTRL_1:
- case RT5670_IRQ_CTRL1:
case RT5670_IRQ_CTRL2:
case RT5670_INT_IRQ_ST:
case RT5670_IL_CMD:
regmap_write(rt5670->regmap, RT5670_RESET, 0);
+ regmap_read(rt5670->regmap, RT5670_VENDOR_ID, &val);
+ if (val >= 4)
+ regmap_write(rt5670->regmap, RT5670_GPIO_CTRL3, 0x0980);
+ else
+ regmap_write(rt5670->regmap, RT5670_GPIO_CTRL3, 0x0d00);
+
ret = regmap_register_patch(rt5670->regmap, init_list,
ARRAY_SIZE(init_list));
if (ret != 0)
{ "IB45 Bypass Mux", "Bypass", "IB45 Mux" },
{ "IB45 Bypass Mux", "Pass SRC", "IB45 Mux" },
- { "IB6 Mux", "IF1 DAC 6", "IF1 DAC6" },
- { "IB6 Mux", "IF2 DAC 6", "IF2 DAC6" },
+ { "IB6 Mux", "IF1 DAC 6", "IF1 DAC6 Mux" },
+ { "IB6 Mux", "IF2 DAC 6", "IF2 DAC6 Mux" },
{ "IB6 Mux", "SLB DAC 6", "SLB DAC6" },
{ "IB6 Mux", "STO4 ADC MIX L", "Stereo4 ADC MIXL" },
{ "IB6 Mux", "IF4 DAC L", "IF4 DAC L" },
{ "IB6 Mux", "STO2 ADC MIX L", "Stereo2 ADC MIXL" },
{ "IB6 Mux", "STO3 ADC MIX L", "Stereo3 ADC MIXL" },
- { "IB7 Mux", "IF1 DAC 7", "IF1 DAC7" },
- { "IB7 Mux", "IF2 DAC 7", "IF2 DAC7" },
+ { "IB7 Mux", "IF1 DAC 7", "IF1 DAC7 Mux" },
+ { "IB7 Mux", "IF2 DAC 7", "IF2 DAC7 Mux" },
{ "IB7 Mux", "SLB DAC 7", "SLB DAC7" },
{ "IB7 Mux", "STO4 ADC MIX R", "Stereo4 ADC MIXR" },
{ "IB7 Mux", "IF4 DAC R", "IF4 DAC R" },
{ "DAC1 FS", NULL, "DAC1 MIXL" },
{ "DAC1 FS", NULL, "DAC1 MIXR" },
- { "DAC2 L Mux", "IF1 DAC 2", "IF1 DAC2" },
- { "DAC2 L Mux", "IF2 DAC 2", "IF2 DAC2" },
+ { "DAC2 L Mux", "IF1 DAC 2", "IF1 DAC2 Mux" },
+ { "DAC2 L Mux", "IF2 DAC 2", "IF2 DAC2 Mux" },
{ "DAC2 L Mux", "IF3 DAC L", "IF3 DAC L" },
{ "DAC2 L Mux", "IF4 DAC L", "IF4 DAC L" },
{ "DAC2 L Mux", "SLB DAC 2", "SLB DAC2" },
{ "DAC2 L Mux", "OB 2", "OutBound2" },
- { "DAC2 R Mux", "IF1 DAC 3", "IF1 DAC3" },
- { "DAC2 R Mux", "IF2 DAC 3", "IF2 DAC3" },
+ { "DAC2 R Mux", "IF1 DAC 3", "IF1 DAC3 Mux" },
+ { "DAC2 R Mux", "IF2 DAC 3", "IF2 DAC3 Mux" },
{ "DAC2 R Mux", "IF3 DAC R", "IF3 DAC R" },
{ "DAC2 R Mux", "IF4 DAC R", "IF4 DAC R" },
{ "DAC2 R Mux", "SLB DAC 3", "SLB DAC3" },
{ "DAC2 R Mux", "Haptic Generator", "Haptic Generator" },
{ "DAC2 R Mux", "VAD ADC", "VAD ADC Mux" },
- { "DAC3 L Mux", "IF1 DAC 4", "IF1 DAC4" },
- { "DAC3 L Mux", "IF2 DAC 4", "IF2 DAC4" },
+ { "DAC3 L Mux", "IF1 DAC 4", "IF1 DAC4 Mux" },
+ { "DAC3 L Mux", "IF2 DAC 4", "IF2 DAC4 Mux" },
{ "DAC3 L Mux", "IF3 DAC L", "IF3 DAC L" },
{ "DAC3 L Mux", "IF4 DAC L", "IF4 DAC L" },
{ "DAC3 L Mux", "SLB DAC 4", "SLB DAC4" },
{ "DAC3 L Mux", "OB 4", "OutBound4" },
- { "DAC3 R Mux", "IF1 DAC 5", "IF1 DAC4" },
- { "DAC3 R Mux", "IF2 DAC 5", "IF2 DAC4" },
+ { "DAC3 R Mux", "IF1 DAC 5", "IF1 DAC5 Mux" },
+ { "DAC3 R Mux", "IF2 DAC 5", "IF2 DAC5 Mux" },
{ "DAC3 R Mux", "IF3 DAC R", "IF3 DAC R" },
{ "DAC3 R Mux", "IF4 DAC R", "IF4 DAC R" },
{ "DAC3 R Mux", "SLB DAC 5", "SLB DAC5" },
{ "DAC3 R Mux", "OB 5", "OutBound5" },
- { "DAC4 L Mux", "IF1 DAC 6", "IF1 DAC6" },
- { "DAC4 L Mux", "IF2 DAC 6", "IF2 DAC6" },
+ { "DAC4 L Mux", "IF1 DAC 6", "IF1 DAC6 Mux" },
+ { "DAC4 L Mux", "IF2 DAC 6", "IF2 DAC6 Mux" },
{ "DAC4 L Mux", "IF3 DAC L", "IF3 DAC L" },
{ "DAC4 L Mux", "IF4 DAC L", "IF4 DAC L" },
{ "DAC4 L Mux", "SLB DAC 6", "SLB DAC6" },
{ "DAC4 L Mux", "OB 6", "OutBound6" },
- { "DAC4 R Mux", "IF1 DAC 7", "IF1 DAC7" },
- { "DAC4 R Mux", "IF2 DAC 7", "IF2 DAC7" },
+ { "DAC4 R Mux", "IF1 DAC 7", "IF1 DAC7 Mux" },
+ { "DAC4 R Mux", "IF2 DAC 7", "IF2 DAC7 Mux" },
{ "DAC4 R Mux", "IF3 DAC R", "IF3 DAC R" },
{ "DAC4 R Mux", "IF4 DAC R", "IF4 DAC R" },
{ "DAC4 R Mux", "SLB DAC 7", "SLB DAC7" },
/* Enable VDDC charge pump */
ana_pwr |= SGTL5000_VDDC_CHRGPMP_POWERUP;
} else if (vddio >= 3100 && vdda >= 3100) {
- /*
- * if vddio and vddd > 3.1v,
- * charge pump should be clean before set ana_pwr
- */
- snd_soc_update_bits(codec, SGTL5000_CHIP_ANA_POWER,
- SGTL5000_VDDC_CHRGPMP_POWERUP, 0);
-
+ ana_pwr &= ~SGTL5000_VDDC_CHRGPMP_POWERUP;
/* VDDC use VDDIO rail */
lreg_ctrl |= SGTL5000_VDDC_ASSN_OVRD;
lreg_ctrl |= SGTL5000_VDDC_MAN_ASSN_VDDIO <<
/* speaker map */
{ "IHFOUTL", NULL, "Speaker Rail"},
{ "IHFOUTR", NULL, "Speaker Rail"},
- { "IHFOUTL", "NULL", "Speaker Left Playback"},
- { "IHFOUTR", "NULL", "Speaker Right Playback"},
+ { "IHFOUTL", NULL, "Speaker Left Playback"},
+ { "IHFOUTR", NULL, "Speaker Right Playback"},
{ "Speaker Left Playback", NULL, "Speaker Left Filter"},
{ "Speaker Right Playback", NULL, "Speaker Right Filter"},
{ "Speaker Left Filter", NULL, "IHFDAC Left"},
};
static const struct regmap_range sta32x_write_regs_range[] = {
- regmap_reg_range(STA32X_CONFA, STA32X_AUTO2),
- regmap_reg_range(STA32X_C1CFG, STA32X_FDRC2),
+ regmap_reg_range(STA32X_CONFA, STA32X_FDRC2),
};
static const struct regmap_range sta32x_read_regs_range[] = {
- regmap_reg_range(STA32X_CONFA, STA32X_AUTO2),
- regmap_reg_range(STA32X_C1CFG, STA32X_FDRC2),
+ regmap_reg_range(STA32X_CONFA, STA32X_FDRC2),
};
static const struct regmap_range sta32x_volatile_regs_range[] = {
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct tas5086_private *priv = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = priv->deemph;
+ ucontrol->value.integer.value[0] = priv->deemph;
return 0;
}
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct tas5086_private *priv = snd_soc_codec_get_drvdata(codec);
- priv->deemph = ucontrol->value.enumerated.item[0];
+ priv->deemph = ucontrol->value.integer.value[0];
return tas5086_set_deemph(codec);
}
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm2000_priv *wm2000 = dev_get_drvdata(codec->dev);
- ucontrol->value.enumerated.item[0] = wm2000->anc_active;
+ ucontrol->value.integer.value[0] = wm2000->anc_active;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm2000_priv *wm2000 = dev_get_drvdata(codec->dev);
- int anc_active = ucontrol->value.enumerated.item[0];
+ int anc_active = ucontrol->value.integer.value[0];
int ret;
if (anc_active > 1)
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm2000_priv *wm2000 = dev_get_drvdata(codec->dev);
- ucontrol->value.enumerated.item[0] = wm2000->spk_ena;
+ ucontrol->value.integer.value[0] = wm2000->spk_ena;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm2000_priv *wm2000 = dev_get_drvdata(codec->dev);
- int val = ucontrol->value.enumerated.item[0];
+ int val = ucontrol->value.integer.value[0];
int ret;
if (val > 1)
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8731_priv *wm8731 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = wm8731->deemph;
+ ucontrol->value.integer.value[0] = wm8731->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8731_priv *wm8731 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
int ret = 0;
if (deemph > 1)
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8903_priv *wm8903 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = wm8903->deemph;
+ ucontrol->value.integer.value[0] = wm8903->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8903_priv *wm8903 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
int ret = 0;
if (deemph > 1)
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8904_priv *wm8904 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = wm8904->deemph;
+ ucontrol->value.integer.value[0] = wm8904->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8904_priv *wm8904 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
if (deemph > 1)
return -EINVAL;
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8955_priv *wm8955 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = wm8955->deemph;
+ ucontrol->value.integer.value[0] = wm8955->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8955_priv *wm8955 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
if (deemph > 1)
return -EINVAL;
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8960_priv *wm8960 = snd_soc_codec_get_drvdata(codec);
- ucontrol->value.enumerated.item[0] = wm8960->deemph;
+ ucontrol->value.integer.value[0] = wm8960->deemph;
return 0;
}
{
struct snd_soc_codec *codec = snd_soc_kcontrol_codec(kcontrol);
struct wm8960_priv *wm8960 = snd_soc_codec_get_drvdata(codec);
- int deemph = ucontrol->value.enumerated.item[0];
+ int deemph = ucontrol->value.integer.value[0];
if (deemph > 1)
return -EINVAL;
struct snd_soc_dapm_context *dapm = snd_soc_dapm_kcontrol_dapm(kcontrol);
struct snd_soc_codec *codec = snd_soc_dapm_to_codec(dapm);
struct wm9712_priv *wm9712 = snd_soc_codec_get_drvdata(codec);
- unsigned int val = ucontrol->value.enumerated.item[0];
+ unsigned int val = ucontrol->value.integer.value[0];
struct soc_mixer_control *mc =
(struct soc_mixer_control *)kcontrol->private_value;
unsigned int mixer, mask, shift, old;
mutex_lock(&wm9712->lock);
old = wm9712->hp_mixer[mixer];
- if (ucontrol->value.enumerated.item[0])
+ if (ucontrol->value.integer.value[0])
wm9712->hp_mixer[mixer] |= mask;
else
wm9712->hp_mixer[mixer] &= ~mask;
mixer = mc->shift >> 8;
shift = mc->shift & 0xff;
- ucontrol->value.enumerated.item[0] =
+ ucontrol->value.integer.value[0] =
(wm9712->hp_mixer[mixer] >> shift) & 1;
return 0;
struct snd_soc_dapm_context *dapm = snd_soc_dapm_kcontrol_dapm(kcontrol);
struct snd_soc_codec *codec = snd_soc_dapm_to_codec(dapm);
struct wm9713_priv *wm9713 = snd_soc_codec_get_drvdata(codec);
- unsigned int val = ucontrol->value.enumerated.item[0];
+ unsigned int val = ucontrol->value.integer.value[0];
struct soc_mixer_control *mc =
(struct soc_mixer_control *)kcontrol->private_value;
unsigned int mixer, mask, shift, old;
mutex_lock(&wm9713->lock);
old = wm9713->hp_mixer[mixer];
- if (ucontrol->value.enumerated.item[0])
+ if (ucontrol->value.integer.value[0])
wm9713->hp_mixer[mixer] |= mask;
else
wm9713->hp_mixer[mixer] &= ~mask;
mixer = mc->shift >> 8;
shift = mc->shift & 0xff;
- ucontrol->value.enumerated.item[0] =
+ ucontrol->value.integer.value[0] =
(wm9713->hp_mixer[mixer] >> shift) & 1;
return 0;
enum spdif_txrate index, bool round)
{
const u32 rate[] = { 32000, 44100, 48000, 96000, 192000 };
- bool is_sysclk = clk == spdif_priv->sysclk;
+ bool is_sysclk = clk_is_match(clk, spdif_priv->sysclk);
u64 rate_ideal, rate_actual, sub;
u32 sysclk_dfmin, sysclk_dfmax;
u32 txclk_df, sysclk_df, arate;
spdif_priv->txclk_src[index], rate[index]);
dev_dbg(&pdev->dev, "use txclk df %d for %dHz sample rate\n",
spdif_priv->txclk_df[index], rate[index]);
- if (spdif_priv->txclk[index] == spdif_priv->sysclk)
+ if (clk_is_match(spdif_priv->txclk[index], spdif_priv->sysclk))
dev_dbg(&pdev->dev, "use sysclk df %d for %dHz sample rate\n",
spdif_priv->sysclk_df[index], rate[index]);
dev_dbg(&pdev->dev, "the best rate for %dHz sample rate is %dHz\n",
factor = (div2 + 1) * (7 * psr + 1) * 2;
for (i = 0; i < 255; i++) {
- /* The bclk rate must be smaller than 1/5 sysclk rate */
- if (factor * (i + 1) < 5)
- continue;
-
- tmprate = freq * factor * (i + 2);
+ tmprate = freq * factor * (i + 1);
if (baudclk_is_used)
clkrate = clk_get_rate(ssi_private->baudclk);
else
clkrate = clk_round_rate(ssi_private->baudclk, tmprate);
+ /*
+ * Hardware limitation: The bclk rate must be
+ * never greater than 1/5 IPG clock rate
+ */
+ if (clkrate * 5 > clk_get_rate(ssi_private->clk))
+ continue;
+
clkrate /= factor;
afreq = clkrate / (i + 1);
ssi_private->dma_params_tx.addr = ssi_private->ssi_phys + CCSR_SSI_STX0;
ssi_private->dma_params_rx.addr = ssi_private->ssi_phys + CCSR_SSI_SRX0;
- ret = !of_property_read_u32_array(np, "dmas", dmas, 4);
+ ret = of_property_read_u32_array(np, "dmas", dmas, 4);
if (ssi_private->use_dma && !ret && dmas[2] == IMX_DMATYPE_SSI_DUAL) {
ssi_private->use_dual_fifo = true;
/* When using dual fifo mode, we need to keep watermark
strlen(dai_link->cpu_dai_name) +
strlen(dai_link->codec_dai_name) + 2,
GFP_KERNEL);
+ if (!name) {
+ ret = -ENOMEM;
+ goto dai_link_of_err;
+ }
+
sprintf(name, "%s-%s", dai_link->cpu_dai_name,
dai_link->codec_dai_name);
dai_link->name = dai_link->stream_name = name;
enum sst_task {
SST_TASK_SBA = 1,
- SST_TASK_MMX,
+ SST_TASK_MMX = 3,
};
enum sst_type {
module = (void *)module + sizeof(*module) + module->mod_size;
}
- /* allocate scratch mem regions */
- sst_block_alloc_scratch(dsp);
-
return 0;
}
int sst_hsw_dsp_load(struct sst_hsw *hsw)
{
struct sst_dsp *dsp = hsw->dsp;
+ struct sst_fw *sst_fw, *t;
int ret;
dev_dbg(hsw->dev, "loading audio DSP....");
return ret;
}
- ret = sst_fw_reload(hsw->sst_fw);
- if (ret < 0) {
- dev_err(hsw->dev, "error: SST FW reload failed\n");
- sst_dsp_dma_put_channel(dsp);
- return -ENOMEM;
+ list_for_each_entry_safe_reverse(sst_fw, t, &dsp->fw_list, list) {
+ ret = sst_fw_reload(sst_fw);
+ if (ret < 0) {
+ dev_err(hsw->dev, "error: SST FW reload failed\n");
+ sst_dsp_dma_put_channel(dsp);
+ return -ENOMEM;
+ }
}
+ ret = sst_block_alloc_scratch(hsw->dsp);
+ if (ret < 0)
+ return -EINVAL;
sst_dsp_dma_put_channel(dsp);
return 0;
int sst_hsw_dsp_runtime_sleep(struct sst_hsw *hsw)
{
- sst_fw_unload(hsw->sst_fw);
- sst_block_free_scratch(hsw->dsp);
+ struct sst_fw *sst_fw, *t;
+ struct sst_dsp *dsp = hsw->dsp;
+
+ list_for_each_entry_safe(sst_fw, t, &dsp->fw_list, list) {
+ sst_fw_unload(sst_fw);
+ }
+ sst_block_free_scratch(dsp);
hsw->boot_complete = false;
- sst_dsp_sleep(hsw->dsp);
+ sst_dsp_sleep(dsp);
return 0;
}
goto fw_err;
}
+ /* allocate scratch mem regions */
+ ret = sst_block_alloc_scratch(hsw->dsp);
+ if (ret < 0)
+ goto boot_err;
+
/* wait for DSP boot completion */
sst_dsp_boot(hsw->dsp);
ret = wait_event_timeout(hsw->boot_wait, hsw->boot_complete,
spin_lock_irqsave(&ctx->ipc_spin_lock, irq_flags);
- shim_regs->imrx = sst_shim_read64(shim, SST_IMRX),
+ shim_regs->imrx = sst_shim_read64(shim, SST_IMRX);
+ shim_regs->csr = sst_shim_read64(shim, SST_CSR);
+
spin_unlock_irqrestore(&ctx->ipc_spin_lock, irq_flags);
}
*/
spin_lock_irqsave(&ctx->ipc_spin_lock, irq_flags);
sst_shim_write64(shim, SST_IMRX, shim_regs->imrx),
+ sst_shim_write64(shim, SST_CSR, shim_regs->csr),
spin_unlock_irqrestore(&ctx->ipc_spin_lock, irq_flags);
}
* initially active. So change the state to active before
* enabling the pm
*/
+
+ if (!acpi_disabled)
+ pm_runtime_set_active(ctx->dev);
+
pm_runtime_enable(ctx->dev);
if (acpi_disabled)
synchronize_irq(ctx->irq_num);
flush_workqueue(ctx->post_msg_wq);
+ ctx->ops->reset(ctx);
/* save the shim registers because PMC doesn't save state */
sst_save_shim64(ctx, ctx->shim, ctx->shim_regs64);
if (PTR_ERR(priv->extclk) == -EPROBE_DEFER)
return -EPROBE_DEFER;
} else {
- if (priv->extclk == priv->clk) {
+ if (clk_is_match(priv->extclk, priv->clk)) {
devm_clk_put(&pdev->dev, priv->extclk);
priv->extclk = ERR_PTR(-EINVAL);
} else {
return ret;
card = devm_kzalloc(dev, sizeof(*card), GFP_KERNEL);
+ if (!card)
+ return -ENOMEM;
+
card->name = devm_kasprintf(dev, GFP_KERNEL,
"HDMI %s", dev_name(ad->dssdev));
card->owner = THIS_MODULE;
case OMAP_MCBSP_SYSCLK_CLKX_EXT:
regs->srgr2 |= CLKSM;
+ regs->pcr0 |= SCLKME;
+ /*
+ * If McBSP is master but yet the CLKX/CLKR pin drives the SRG,
+ * disable output on those pins. This enables to inject the
+ * reference clock through CLKX/CLKR. For this to work
+ * set_dai_sysclk() _needs_ to be called after set_dai_fmt().
+ */
+ regs->pcr0 &= ~CLKXM;
+ break;
case OMAP_MCBSP_SYSCLK_CLKR_EXT:
regs->pcr0 |= SCLKME;
+ /* Disable ouput on CLKR pin in master mode */
+ regs->pcr0 &= ~CLKRM;
break;
default:
err = -ENODEV;
struct snd_pcm *pcm = rtd->pcm;
int ret;
- ret = dma_coerce_mask_and_coherent(card->dev, DMA_BIT_MASK(64));
+ ret = dma_coerce_mask_and_coherent(card->dev, DMA_BIT_MASK(32));
if (ret)
return ret;
config SND_SOC_SPEYSIDE
tristate "Audio support for Wolfson Speyside"
- depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410
+ depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410 && I2C && SPI_MASTER
select SND_SAMSUNG_I2S
select SND_SOC_WM8996
select SND_SOC_WM9081
config SND_SOC_BELLS
tristate "Audio support for Wolfson Bells"
- depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410 && MFD_ARIZONA
+ depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410 && MFD_ARIZONA && I2C && SPI_MASTER
select SND_SAMSUNG_I2S
select SND_SOC_WM5102
select SND_SOC_WM5110
config SND_SOC_LITTLEMILL
tristate "Audio support for Wolfson Littlemill"
- depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410
+ depends on SND_SOC_SAMSUNG && MACH_WLF_CRAGG_6410 && I2C
select SND_SAMSUNG_I2S
select MFD_WM8994
select SND_SOC_WM8994
config SND_SOC_ODROIDX2
tristate "Audio support for Odroid-X2 and Odroid-U3"
- depends on SND_SOC_SAMSUNG
+ depends on SND_SOC_SAMSUNG && I2C
select SND_SOC_MAX98090
select SND_SAMSUNG_I2S
help
config SND_SOC_ARNDALE_RT5631_ALC5631
tristate "Audio support for RT5631(ALC5631) on Arndale Board"
- depends on SND_SOC_SAMSUNG
+ depends on SND_SOC_SAMSUNG && I2C
select SND_SAMSUNG_I2S
select SND_SOC_RT5631
goto exit_snd_probe;
}
+ dev_set_drvdata(dev, priv);
+
/*
* asoc register
*/
goto exit_snd_soc;
}
- dev_set_drvdata(dev, priv);
-
pm_runtime_enable(dev);
dev_info(dev, "probed\n");
if (!buf)
return -ENOMEM;
+ mutex_lock(&client_mutex);
+
list_for_each_entry(codec, &codec_list, list) {
len = snprintf(buf + ret, PAGE_SIZE - ret, "%s\n",
codec->component.name);
}
}
+ mutex_unlock(&client_mutex);
+
if (ret >= 0)
ret = simple_read_from_buffer(user_buf, count, ppos, buf, ret);
if (!buf)
return -ENOMEM;
+ mutex_lock(&client_mutex);
+
list_for_each_entry(component, &component_list, list) {
list_for_each_entry(dai, &component->dai_list, list) {
len = snprintf(buf + ret, PAGE_SIZE - ret, "%s\n",
}
}
+ mutex_unlock(&client_mutex);
+
ret = simple_read_from_buffer(user_buf, count, ppos, buf, ret);
kfree(buf);
if (!buf)
return -ENOMEM;
+ mutex_lock(&client_mutex);
+
list_for_each_entry(platform, &platform_list, list) {
len = snprintf(buf + ret, PAGE_SIZE - ret, "%s\n",
platform->component.name);
}
}
+ mutex_unlock(&client_mutex);
+
ret = simple_read_from_buffer(user_buf, count, ppos, buf, ret);
kfree(buf);
{
struct snd_soc_component *component;
+ lockdep_assert_held(&client_mutex);
+
list_for_each_entry(component, &component_list, list) {
if (of_node) {
if (component->dev->of_node == of_node)
struct snd_soc_component *component;
struct snd_soc_dai *dai;
+ lockdep_assert_held(&client_mutex);
+
/* Find CPU DAI from registered DAIs*/
list_for_each_entry(component, &component_list, list) {
if (dlc->of_node && component->dev->of_node != dlc->of_node)
struct snd_soc_codec *codec;
int ret, i, order;
+ mutex_lock(&client_mutex);
mutex_lock_nested(&card->mutex, SND_SOC_CARD_CLASS_INIT);
/* bind DAIs */
card->instantiated = 1;
snd_soc_dapm_sync(&card->dapm);
mutex_unlock(&card->mutex);
+ mutex_unlock(&client_mutex);
return 0;
base_error:
mutex_unlock(&card->mutex);
+ mutex_unlock(&client_mutex);
return ret;
}
list_del(&component->list);
}
-static void snd_soc_component_del(struct snd_soc_component *component)
-{
- mutex_lock(&client_mutex);
- snd_soc_component_del_unlocked(component);
- mutex_unlock(&client_mutex);
-}
-
int snd_soc_register_component(struct device *dev,
const struct snd_soc_component_driver *cmpnt_drv,
struct snd_soc_dai_driver *dai_drv,
{
struct snd_soc_component *cmpnt;
+ mutex_lock(&client_mutex);
list_for_each_entry(cmpnt, &component_list, list) {
if (dev == cmpnt->dev && cmpnt->registered_as_component)
goto found;
}
+ mutex_unlock(&client_mutex);
return;
found:
- snd_soc_component_del(cmpnt);
+ snd_soc_component_del_unlocked(cmpnt);
+ mutex_unlock(&client_mutex);
snd_soc_component_cleanup(cmpnt);
kfree(cmpnt);
}
{
struct snd_soc_platform *platform;
+ mutex_lock(&client_mutex);
list_for_each_entry(platform, &platform_list, list) {
- if (dev == platform->dev)
+ if (dev == platform->dev) {
+ mutex_unlock(&client_mutex);
return platform;
+ }
}
+ mutex_unlock(&client_mutex);
return NULL;
}
{
struct snd_soc_codec *codec;
+ mutex_lock(&client_mutex);
list_for_each_entry(codec, &codec_list, list) {
if (dev == codec->dev)
goto found;
}
+ mutex_unlock(&client_mutex);
return;
found:
-
- mutex_lock(&client_mutex);
list_del(&codec->list);
snd_soc_component_del_unlocked(&codec->component);
mutex_unlock(&client_mutex);
for (; p < buf_end; ++p) {
short pv = le16_to_cpu(*p);
int val = (pv * volume[chn & 1]) >> 8;
- pv = clamp(val, 0x7fff, -0x8000);
+ pv = clamp(val, -0x8000, 0x7fff);
*p = cpu_to_le16(pv);
++chn;
}
val = p[0] + (p[1] << 8) + ((signed char)p[2] << 16);
val = (val * volume[chn & 1]) >> 8;
- val = clamp(val, 0x7fffff, -0x800000);
+ val = clamp(val, -0x800000, 0x7fffff);
p[0] = val;
p[1] = val >> 8;
p[2] = val >> 16;
short pov = le16_to_cpu(*po);
short piv = le16_to_cpu(*pi);
int val = pov + ((piv * volume) >> 8);
- pov = clamp(val, 0x7fff, -0x8000);
+ pov = clamp(val, -0x8000, 0x7fff);
*po = cpu_to_le16(pov);
}
}
{ USB_ID(0x041e, 0x3040), 2, 2, 6, 6, 2, 0x6e91 }, /* Live! 24-bit */
{ USB_ID(0x041e, 0x3042), 0, 1, 1, 1, 1, 0x000d }, /* Usb X-Fi S51 */
{ USB_ID(0x041e, 0x30df), 0, 1, 1, 1, 1, 0x000d }, /* Usb X-Fi S51 Pro */
+ { USB_ID(0x041e, 0x3237), 0, 1, 1, 1, 1, 0x000d }, /* Usb X-Fi S51 Pro */
{ USB_ID(0x041e, 0x3048), 2, 2, 6, 6, 2, 0x6e91 }, /* Toshiba SB0500 */
};
}
}
},
+{
+ USB_DEVICE(0x0582, 0x0159),
+ .driver_info = (unsigned long) & (const struct snd_usb_audio_quirk) {
+ /* .vendor_name = "Roland", */
+ /* .product_name = "UA-22", */
+ .ifnum = QUIRK_ANY_INTERFACE,
+ .type = QUIRK_COMPOSITE,
+ .data = (const struct snd_usb_audio_quirk[]) {
+ {
+ .ifnum = 0,
+ .type = QUIRK_AUDIO_STANDARD_INTERFACE
+ },
+ {
+ .ifnum = 1,
+ .type = QUIRK_AUDIO_STANDARD_INTERFACE
+ },
+ {
+ .ifnum = 2,
+ .type = QUIRK_MIDI_FIXED_ENDPOINT,
+ .data = & (const struct snd_usb_midi_endpoint_info) {
+ .out_cables = 0x0001,
+ .in_cables = 0x0001
+ }
+ },
+ {
+ .ifnum = -1
+ }
+ }
+ }
+},
/* this catches most recent vendor-specific Roland devices */
{
.match_flags = USB_DEVICE_ID_MATCH_VENDOR |
bool snd_usb_get_sample_rate_quirk(struct snd_usb_audio *chip)
{
- /* MS Lifecam HD-5000 doesn't support reading the sample rate. */
- return chip->usb_id == USB_ID(0x045E, 0x076D);
+ /* devices which do not support reading the sample rate. */
+ switch (chip->usb_id) {
+ case USB_ID(0x045E, 0x076D): /* MS Lifecam HD-5000 */
+ case USB_ID(0x04D8, 0xFEEA): /* Benchmark DAC1 Pre */
+ return true;
+ }
+ return false;
}
/* Marantz/Denon USB DACs need a vendor cmd to switch
-MEMCPY_FN(__memcpy,
+MEMCPY_FN(memcpy_orig,
"x86-64-unrolled",
"unrolled memcpy() in arch/x86/lib/memcpy_64.S")
-MEMCPY_FN(memcpy_c,
+MEMCPY_FN(__memcpy,
"x86-64-movsq",
"movsq-based memcpy() in arch/x86/lib/memcpy_64.S")
-MEMCPY_FN(memcpy_c_e,
+MEMCPY_FN(memcpy_erms,
"x86-64-movsb",
"movsb-based memcpy() in arch/x86/lib/memcpy_64.S")
#define memcpy MEMCPY /* don't hide glibc's memcpy() */
#define altinstr_replacement text
#define globl p2align 4; .globl
-#define Lmemcpy_c globl memcpy_c; memcpy_c
-#define Lmemcpy_c_e globl memcpy_c_e; memcpy_c_e
#include "../../../arch/x86/lib/memcpy_64.S"
/*
* We need to provide note.GNU-stack section, saying that we want
"Specify length of memory to copy. "
"Available units: B, KB, MB, GB and TB (upper and lower)"),
OPT_STRING('r', "routine", &routine, "default",
- "Specify routine to copy"),
+ "Specify routine to copy, \"all\" runs all available routines"),
OPT_INTEGER('i', "iterations", &iterations,
"repeat memcpy() invocation this number of times"),
OPT_BOOLEAN('c', "cycle", &use_cycle,
const char *const *usage;
};
-static int bench_mem_common(int argc, const char **argv,
- const char *prefix __maybe_unused,
- struct bench_mem_info *info)
+static void __bench_mem_routine(struct bench_mem_info *info, int r_idx, size_t len, double totallen)
{
- int i;
- size_t len;
- double totallen;
+ const struct routine *r = &info->routines[r_idx];
double result_bps[2];
u64 result_cycle[2];
- argc = parse_options(argc, argv, options,
- info->usage, 0);
-
- if (no_prefault && only_prefault) {
- fprintf(stderr, "Invalid options: -o and -n are mutually exclusive\n");
- return 1;
- }
-
- if (use_cycle)
- init_cycle();
-
- len = (size_t)perf_atoll((char *)length_str);
- totallen = (double)len * iterations;
-
result_cycle[0] = result_cycle[1] = 0ULL;
result_bps[0] = result_bps[1] = 0.0;
- if ((s64)len <= 0) {
- fprintf(stderr, "Invalid length:%s\n", length_str);
- return 1;
- }
-
- /* same to without specifying either of prefault and no-prefault */
- if (only_prefault && no_prefault)
- only_prefault = no_prefault = false;
-
- for (i = 0; info->routines[i].name; i++) {
- if (!strcmp(info->routines[i].name, routine))
- break;
- }
- if (!info->routines[i].name) {
- printf("Unknown routine:%s\n", routine);
- printf("Available routines...\n");
- for (i = 0; info->routines[i].name; i++) {
- printf("\t%s ... %s\n",
- info->routines[i].name, info->routines[i].desc);
- }
- return 1;
- }
+ printf("Routine %s (%s)\n", r->name, r->desc);
if (bench_format == BENCH_FORMAT_DEFAULT)
printf("# Copying %s Bytes ...\n\n", length_str);
if (!only_prefault && !no_prefault) {
/* show both of results */
if (use_cycle) {
- result_cycle[0] =
- info->do_cycle(&info->routines[i], len, false);
- result_cycle[1] =
- info->do_cycle(&info->routines[i], len, true);
+ result_cycle[0] = info->do_cycle(r, len, false);
+ result_cycle[1] = info->do_cycle(r, len, true);
} else {
- result_bps[0] =
- info->do_gettimeofday(&info->routines[i],
- len, false);
- result_bps[1] =
- info->do_gettimeofday(&info->routines[i],
- len, true);
+ result_bps[0] = info->do_gettimeofday(r, len, false);
+ result_bps[1] = info->do_gettimeofday(r, len, true);
}
} else {
- if (use_cycle) {
- result_cycle[pf] =
- info->do_cycle(&info->routines[i],
- len, only_prefault);
- } else {
- result_bps[pf] =
- info->do_gettimeofday(&info->routines[i],
- len, only_prefault);
- }
+ if (use_cycle)
+ result_cycle[pf] = info->do_cycle(r, len, only_prefault);
+ else
+ result_bps[pf] = info->do_gettimeofday(r, len, only_prefault);
}
switch (bench_format) {
die("unknown format: %d\n", bench_format);
break;
}
+}
+
+static int bench_mem_common(int argc, const char **argv,
+ const char *prefix __maybe_unused,
+ struct bench_mem_info *info)
+{
+ int i;
+ size_t len;
+ double totallen;
+
+ argc = parse_options(argc, argv, options,
+ info->usage, 0);
+
+ if (no_prefault && only_prefault) {
+ fprintf(stderr, "Invalid options: -o and -n are mutually exclusive\n");
+ return 1;
+ }
+
+ if (use_cycle)
+ init_cycle();
+
+ len = (size_t)perf_atoll((char *)length_str);
+ totallen = (double)len * iterations;
+
+ if ((s64)len <= 0) {
+ fprintf(stderr, "Invalid length:%s\n", length_str);
+ return 1;
+ }
+
+ /* same to without specifying either of prefault and no-prefault */
+ if (only_prefault && no_prefault)
+ only_prefault = no_prefault = false;
+
+ if (!strncmp(routine, "all", 3)) {
+ for (i = 0; info->routines[i].name; i++)
+ __bench_mem_routine(info, i, len, totallen);
+ return 0;
+ }
+
+ for (i = 0; info->routines[i].name; i++) {
+ if (!strcmp(info->routines[i].name, routine))
+ break;
+ }
+ if (!info->routines[i].name) {
+ printf("Unknown routine:%s\n", routine);
+ printf("Available routines...\n");
+ for (i = 0; info->routines[i].name; i++) {
+ printf("\t%s ... %s\n",
+ info->routines[i].name, info->routines[i].desc);
+ }
+ return 1;
+ }
+
+ __bench_mem_routine(info, i, len, totallen);
return 0;
}
memcpy_t fn = r->fn.memcpy;
int i;
- memcpy_alloc_mem(&src, &dst, len);
+ memcpy_alloc_mem(&dst, &src, len);
if (prefault)
fn(dst, src, len);
void *src = NULL, *dst = NULL;
int i;
- memcpy_alloc_mem(&src, &dst, len);
+ memcpy_alloc_mem(&dst, &src, len);
if (prefault)
fn(dst, src, len);
-MEMSET_FN(__memset,
+MEMSET_FN(memset_orig,
"x86-64-unrolled",
"unrolled memset() in arch/x86/lib/memset_64.S")
-MEMSET_FN(memset_c,
+MEMSET_FN(__memset,
"x86-64-stosq",
"movsq-based memset() in arch/x86/lib/memset_64.S")
-MEMSET_FN(memset_c_e,
+MEMSET_FN(memset_erms,
"x86-64-stosb",
"movsb-based memset() in arch/x86/lib/memset_64.S")
#define memset MEMSET /* don't hide glibc's memset() */
#define altinstr_replacement text
#define globl p2align 4; .globl
-#define Lmemset_c globl memset_c; memset_c
-#define Lmemset_c_e globl memset_c_e; memset_c_e
#include "../../../arch/x86/lib/memset_64.S"
/*
endif
endif
+ifeq ($(RAW_ARCH),sparc64)
+ ARCH ?= sparc
+endif
+
ARCH ?= $(RAW_ARCH)
LP64 := $(shell echo __LP64__ | ${CC} ${CFLAGS} -E -x c - | tail -n 1)
$(BUILD)
test-pthread-attr-setaffinity-np.bin:
- $(BUILD) -Werror -lpthread
+ $(BUILD) -D_GNU_SOURCE -Werror -lpthread
test-stackprotector-all.bin:
$(BUILD) -Werror -fstack-protector-all
{
int ret = 0;
pthread_attr_t thread_attr;
+ cpu_set_t cs;
pthread_attr_init(&thread_attr);
/* don't care abt exact args, just the API itself in libpthread */
- ret = pthread_attr_setaffinity_np(&thread_attr, 0, NULL);
+ ret = pthread_attr_setaffinity_np(&thread_attr, sizeof(cs), &cs);
return ret;
}
static void ins__delete(struct ins_operands *ops)
{
+ if (ops == NULL)
+ return;
zfree(&ops->source.raw);
zfree(&ops->source.name);
zfree(&ops->target.raw);
if (cpu < 0)
cpu = 0;
+ /*
+ * Using -1 for the pid is a workaround to avoid gratuitous jump label
+ * changes.
+ */
while (1) {
/* check cloexec flag */
fd = sys_perf_event_open(&attr, pid, cpu, -1,
err, strerror_r(err, sbuf, sizeof(sbuf)));
/* not supported, confirm error related to PERF_FLAG_FD_CLOEXEC */
- fd = sys_perf_event_open(&attr, pid, cpu, -1, 0);
+ while (1) {
+ fd = sys_perf_event_open(&attr, pid, cpu, -1, 0);
+ if (fd < 0 && pid == -1 && errno == EACCES) {
+ pid = 0;
+ continue;
+ }
+ break;
+ }
err = errno;
+ if (fd >= 0)
+ close(fd);
+
if (WARN_ONCE(fd < 0 && err != EBUSY,
"perf_event_open(..., 0) failed unexpectedly with error %d (%s)\n",
err, strerror_r(err, sbuf, sizeof(sbuf))))
return -1;
- close(fd);
-
return 0;
}
int mask;
int refcnt;
unsigned int prev;
- char event_copy[PERF_SAMPLE_MAX_SIZE];
+ char event_copy[PERF_SAMPLE_MAX_SIZE] __attribute__((aligned(8)));
};
struct perf_evlist {
/* Just disable it so we can build arch/x86/lib/memcpy_64.S for perf bench: */
#define altinstruction_entry #
+#define ALTERNATIVE_2 #
#endif
#include <symbol/kallsyms.h>
#include "debug.h"
+#ifndef EM_AARCH64
+#define EM_AARCH64 183 /* ARM 64 bit */
+#endif
+
+
#ifdef HAVE_CPLUS_DEMANGLE_SUPPORT
extern char *cplus_demangle(const char *, int);
$(OUTPUT)cpupower: $(UTIL_OBJS) $(OUTPUT)libcpupower.so.$(LIB_MAJ)
$(ECHO) " CC " $@
- $(QUIET) $(CC) $(CFLAGS) $(LDFLAGS) $(UTIL_OBJS) -lcpupower -Wl,-rpath=./ -lrt -lpci -L$(OUTPUT) -o $@
+ $(QUIET) $(CC) $(CFLAGS) $(LDFLAGS) $(UTIL_OBJS) -lcpupower -lrt -lpci -L$(OUTPUT) -o $@
$(QUIET) $(STRIPCMD) $@
$(OUTPUT)po/$(PACKAGE).pot: $(UTIL_SRC)
TARGETS += timers
TARGETS += user
TARGETS += vm
+TARGETS += x86
#Please keep the TARGETS list alphabetically sorted
TARGETS_HOTPLUG = cpu-hotplug
TARGETS_HOTPLUG += memory-hotplug
+# Clear LDFLAGS and MAKEFLAGS if called from main
+# Makefile to avoid test build failures when test
+# Makefile doesn't have explicit build rules.
+ifeq (1,$(MAKELEVEL))
+undefine LDFLAGS
+override MAKEFLAGS =
+endif
+
all:
for TARGET in $(TARGETS); do \
make -C $$TARGET; \
make -C $$TARGET clean; \
done;
+INSTALL_PATH ?= install
+INSTALL_PATH := $(abspath $(INSTALL_PATH))
+ALL_SCRIPT := $(INSTALL_PATH)/run_kselftest.sh
+
+install:
+ifdef INSTALL_PATH
+ @# Ask all targets to install their files
+ mkdir -p $(INSTALL_PATH)
+ for TARGET in $(TARGETS); do \
+ mkdir -p $(INSTALL_PATH)/$$TARGET ; \
+ make -C $$TARGET INSTALL_PATH=$(INSTALL_PATH)/$$TARGET install; \
+ done;
+
+ @# Ask all targets to emit their test scripts
+ echo "#!/bin/bash" > $(ALL_SCRIPT)
+ echo "cd \$$(dirname \$$0)" >> $(ALL_SCRIPT)
+ echo "ROOT=\$$PWD" >> $(ALL_SCRIPT)
+
+ for TARGET in $(TARGETS); do \
+ echo "echo ; echo Running tests in $$TARGET" >> $(ALL_SCRIPT); \
+ echo "echo ========================================" >> $(ALL_SCRIPT); \
+ echo "cd $$TARGET" >> $(ALL_SCRIPT); \
+ make -s --no-print-directory -C $$TARGET emit_tests >> $(ALL_SCRIPT); \
+ echo "cd \$$ROOT" >> $(ALL_SCRIPT); \
+ done;
+
+ chmod u+x $(ALL_SCRIPT)
+else
+ $(error Error: set INSTALL_PATH to use install)
+endif
+
clean:
for TARGET in $(TARGETS); do \
make -C $$TARGET clean; \
done;
+
+.PHONY: install
echo "Not an x86 target, can't build breakpoints selftests"
endif
-run_tests:
- @./breakpoint_test || echo "breakpoints selftests: [FAIL]"
+TEST_PROGS := breakpoint_test
+
+include ../lib.mk
clean:
rm -fr breakpoint_test
all:
-run_tests:
- @/bin/bash ./on-off-test.sh || echo "cpu-hotplug selftests: [FAIL]"
+TEST_PROGS := cpu-on-off-test.sh
+
+include ../lib.mk
run_full_test:
- @/bin/bash ./on-off-test.sh -a || echo "cpu-hotplug selftests: [FAIL]"
+ @/bin/bash ./cpu-on-off-test.sh -a || echo "cpu-hotplug selftests: [FAIL]"
clean:
-CC = $(CROSS_COMPILE)gcc
CFLAGS = -Wall
test_objs = open-unlink create-read
all: $(test_objs)
-run_tests: all
- @/bin/bash ./efivarfs.sh || echo "efivarfs selftests: [FAIL]"
+TEST_PROGS := efivarfs.sh
+TEST_FILES := $(test_objs)
+
+include ../lib.mk
clean:
rm -f $(test_objs)
-CC = $(CROSS_COMPILE)gcc
CFLAGS = -Wall
BINARIES = execveat
DEPS = execveat.symlink execveat.denatured script subdir
%: %.c
$(CC) $(CFLAGS) -o $@ $^
-run_tests: all
- ./execveat
+TEST_PROGS := execveat
+TEST_FILES := $(DEPS)
+
+include ../lib.mk
+
+override EMIT_TESTS := echo "mkdir -p subdir; (./execveat && echo \"selftests: execveat [PASS]\") || echo \"selftests: execveat [FAIL]\""
clean:
rm -rf $(BINARIES) $(DEPS) subdir.moved execveat.moved xxxxx*
#ifdef __NR_execveat
return syscall(__NR_execveat, fd, path, argv, envp, flags);
#else
- errno = -ENOSYS;
+ errno = ENOSYS;
return -1;
#endif
}
int fd_cloexec = open_or_die("execveat", O_RDONLY|O_CLOEXEC);
int fd_script_cloexec = open_or_die("script", O_RDONLY|O_CLOEXEC);
+ /* Check if we have execveat at all, and bail early if not */
+ errno = 0;
+ execveat_(-1, NULL, NULL, NULL, 0);
+ if (errno == ENOSYS) {
+ printf("[FAIL] ENOSYS calling execveat - no kernel support?\n");
+ return 1;
+ }
+
/* Change file position to confirm it doesn't affect anything */
lseek(fd, 10, SEEK_SET);
# No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
all:
-fw_filesystem:
- @if /bin/sh ./fw_filesystem.sh ; then \
- echo "fw_filesystem: ok"; \
- else \
- echo "fw_filesystem: [FAIL]"; \
- exit 1; \
- fi
+TEST_PROGS := fw_filesystem.sh fw_userhelper.sh
-fw_userhelper:
- @if /bin/sh ./fw_userhelper.sh ; then \
- echo "fw_userhelper: ok"; \
- else \
- echo "fw_userhelper: [FAIL]"; \
- exit 1; \
- fi
-
-run_tests: all fw_filesystem fw_userhelper
+include ../lib.mk
# Nothing to clean up.
clean:
-
-.PHONY: all clean run_tests fw_filesystem fw_userhelper
all:
-run_tests:
- @/bin/sh ./ftracetest || echo "ftrace selftests: [FAIL]"
+TEST_PROGS := ftracetest
+
+include ../lib.mk
clean:
rm -rf logs/*
# description: Basic event tracing check
test -f available_events -a -f set_event -a -d events
# check scheduler events are available
-grep -q sched available_events && exit 0 || exit -1
\ No newline at end of file
+grep -q sched available_events && exit 0 || exit $FAIL
fail() { #msg
do_reset
echo $1
- exit -1
+ exit $FAIL
+}
+
+yield() {
+ ping localhost -c 1 || sleep .001 || usleep 1 || sleep 1
}
if [ ! -f set_event -o ! -d events/sched ]; then
do_reset
echo 'sched:sched_switch' > set_event
-usleep 1
+
+yield
count=`cat trace | grep sched_switch | wc -l`
if [ $count -eq 0 ]; then
do_reset
echo 1 > events/sched/sched_switch/enable
-usleep 1
+
+yield
count=`cat trace | grep sched_switch | wc -l`
if [ $count -eq 0 ]; then
do_reset
echo 0 > events/sched/sched_switch/enable
-usleep 1
+
+yield
count=`cat trace | grep sched_switch | wc -l`
if [ $count -ne 0 ]; then
fail() { #msg
do_reset
echo $1
- exit -1
+ exit $FAIL
+}
+
+yield() {
+ ping localhost -c 1 || sleep .001 || usleep 1 || sleep 1
}
if [ ! -f set_event -o ! -d events/sched ]; then
do_reset
echo 'sched:*' > set_event
-usleep 1
+
+yield
count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l`
if [ $count -lt 3 ]; then
do_reset
echo 1 > events/sched/enable
-usleep 1
+
+yield
count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l`
if [ $count -lt 3 ]; then
do_reset
echo 0 > events/sched/enable
-usleep 1
+
+yield
count=`cat trace | grep -v ^# | awk '{ print $5 }' | sort -u | wc -l`
if [ $count -ne 0 ]; then
fail() { #msg
do_reset
echo $1
- exit -1
+ exit $FAIL
+}
+
+yield() {
+ ping localhost -c 1 || sleep .001 || usleep 1 || sleep 1
}
if [ ! -f available_events -o ! -f set_event -o ! -d events ]; then
do_reset
echo '*:*' > set_event
+
+yield
+
count=`cat trace | grep -v ^# | wc -l`
if [ $count -eq 0 ]; then
fail "none of events are recorded"
do_reset
echo 1 > events/enable
+
+yield
+
count=`cat trace | grep -v ^# | wc -l`
if [ $count -eq 0 ]; then
fail "none of events are recorded"
do_reset
echo 0 > events/enable
+
+yield
+
count=`cat trace | grep -v ^# | wc -l`
if [ $count -ne 0 ]; then
fail "any of events should not be recorded"
do_reset() {
reset_tracer
- echo 0 > /proc/sys/kernel/stack_tracer_enabled
+ if [ -e /proc/sys/kernel/stack_tracer_enabled ]; then
+ echo 0 > /proc/sys/kernel/stack_tracer_enabled
+ fi
enable_tracing
clear_trace
echo > set_ftrace_filter
fail() { # msg
do_reset
echo $1
- exit -1
+ exit $FAIL
}
disable_tracing
fail() { # msg
do_reset
echo $1
- exit -1
+ exit $FAIL
}
disable_tracing
reset_tracer
echo > set_ftrace_filter
echo $1
- exit -1
+ exit $FAIL
}
echo "Testing function tracer with profiler:"
--- /dev/null
+#!/bin/bash
+#
+# gen_kselftest_tar
+# Generate kselftest tarball
+# Author: Shuah Khan <shuahkh@osg.samsung.com>
+# Copyright (C) 2015 Samsung Electronics Co., Ltd.
+
+# This software may be freely redistributed under the terms of the GNU
+# General Public License (GPLv2).
+
+# main
+main()
+{
+ if [ "$#" -eq 0 ]; then
+ echo "$0: Generating default compression gzip"
+ copts="cvzf"
+ ext=".tar.gz"
+ else
+ case "$1" in
+ tar)
+ copts="cvf"
+ ext=".tar"
+ ;;
+ targz)
+ copts="cvzf"
+ ext=".tar.gz"
+ ;;
+ tarbz2)
+ copts="cvjf"
+ ext=".tar.bz2"
+ ;;
+ tarxz)
+ copts="cvJf"
+ ext=".tar.xz"
+ ;;
+ *)
+ echo "Unknown tarball format $1"
+ exit 1
+ ;;
+ esac
+ fi
+
+ install_dir=./kselftest
+
+# Run install using INSTALL_KSFT_PATH override to generate install
+# directory
+./kselftest_install.sh
+tar $copts kselftest${ext} $install_dir
+echo "Kselftest archive kselftest${ext} created!"
+
+# clean up install directory
+rm -rf kselftest
+}
+
+main "$@"
CFLAGS += -I../../../../usr/include/
all:
-ifeq ($(ARCH),x86)
- gcc $(CFLAGS) msgque.c -o msgque_test
-else
- echo "Not an x86 target, can't build msgque selftest"
-endif
+ $(CC) $(CFLAGS) msgque.c -o msgque_test
+
+TEST_PROGS := msgque_test
-run_tests: all
- ./msgque_test
+include ../lib.mk
clean:
rm -fr ./msgque_test
-CC := $(CROSS_COMPILE)$(CC)
CFLAGS += -I../../../../usr/include/
all: kcmp_test
-run_tests: all
- @./kcmp_test || echo "kcmp_test: [FAIL]"
+TEST_PROGS := kcmp_test
+
+include ../lib.mk
clean:
$(RM) kcmp_test kcmp-test-file
--- /dev/null
+#!/bin/bash
+#
+# Kselftest Install
+# Install kselftest tests
+# Author: Shuah Khan <shuahkh@osg.samsung.com>
+# Copyright (C) 2015 Samsung Electronics Co., Ltd.
+
+# This software may be freely redistributed under the terms of the GNU
+# General Public License (GPLv2).
+
+install_loc=`pwd`
+
+main()
+{
+ if [ $(basename $install_loc) != "selftests" ]; then
+ echo "$0: Please run it in selftests directory ..."
+ exit 1;
+ fi
+ if [ "$#" -eq 0 ]; then
+ echo "$0: Installing in default location - $install_loc ..."
+ elif [ ! -d "$1" ]; then
+ echo "$0: $1 doesn't exist!!"
+ exit 1;
+ else
+ install_loc=$1
+ echo "$0: Installing in specified location - $install_loc ..."
+ fi
+
+ install_dir=$install_loc/kselftest
+
+# Create install directory
+ mkdir -p $install_dir
+# Build tests
+ INSTALL_PATH=$install_dir make install
+}
+
+main "$@"
--- /dev/null
+# This mimics the top-level Makefile. We do it explicitly here so that this
+# Makefile can operate with or without the kbuild infrastructure.
+CC := $(CROSS_COMPILE)gcc
+
+define RUN_TESTS
+ @for TEST in $(TEST_PROGS); do \
+ (./$$TEST && echo "selftests: $$TEST [PASS]") || echo "selftests: $$TEST [FAIL]"; \
+ done;
+endef
+
+run_tests: all
+ $(RUN_TESTS)
+
+define INSTALL_RULE
+ mkdir -p $(INSTALL_PATH)
+ install -t $(INSTALL_PATH) $(TEST_PROGS) $(TEST_PROGS_EXTENDED) $(TEST_FILES)
+endef
+
+install: all
+ifdef INSTALL_PATH
+ $(INSTALL_RULE)
+else
+ $(error Error: set INSTALL_PATH to use install)
+endif
+
+define EMIT_TESTS
+ @for TEST in $(TEST_PROGS); do \
+ echo "(./$$TEST && echo \"selftests: $$TEST [PASS]\") || echo \"selftests: $$TEST [FAIL]\""; \
+ done;
+endef
+
+emit_tests:
+ $(EMIT_TESTS)
+
+.PHONY: run_tests all clean install emit_tests
+CC = $(CROSS_COMPILE)gcc
CFLAGS += -D_FILE_OFFSET_BITS=64
CFLAGS += -I../../../../include/uapi/
CFLAGS += -I../../../../include/
+CFLAGS += -I../../../../usr/include/
all:
- gcc $(CFLAGS) memfd_test.c -o memfd_test
+ $(CC) $(CFLAGS) memfd_test.c -o memfd_test
-run_tests: all
- gcc $(CFLAGS) memfd_test.c -o memfd_test
- @./memfd_test || echo "memfd_test: [FAIL]"
+TEST_PROGS := memfd_test
+
+include ../lib.mk
build_fuse:
- gcc $(CFLAGS) fuse_mnt.c `pkg-config fuse --cflags --libs` -o fuse_mnt
- gcc $(CFLAGS) fuse_test.c -o fuse_test
+ $(CC) $(CFLAGS) fuse_mnt.c `pkg-config fuse --cflags --libs` -o fuse_mnt
+ $(CC) $(CFLAGS) fuse_test.c -o fuse_test
run_fuse: build_fuse
@./run_fuse_test.sh || echo "fuse_test: [FAIL]"
all:
-run_tests:
- @/bin/bash ./on-off-test.sh -r 2 || echo "memory-hotplug selftests: [FAIL]"
+include ../lib.mk
+
+TEST_PROGS := mem-on-off-test.sh
+override RUN_TESTS := ./mem-on-off-test.sh -r 2 || echo "selftests: memory-hotplug [FAIL]"
+override EMIT_TESTS := echo "$(RUN_TESTS)"
run_full_test:
- @/bin/bash ./on-off-test.sh || echo "memory-hotplug selftests: [FAIL]"
+ @/bin/bash ./mem-on-off-test.sh || echo "memory-hotplug selftests: [FAIL]"
clean:
--- /dev/null
+unprivileged-remount-test
# Makefile for mount selftests.
-
+CFLAGS = -Wall \
+ -O2
all: unprivileged-remount-test
unprivileged-remount-test: unprivileged-remount-test.c
- gcc -Wall -O2 unprivileged-remount-test.c -o unprivileged-remount-test
+ $(CC) $(CFLAGS) unprivileged-remount-test.c -o unprivileged-remount-test
-# Allow specific tests to be selected.
-test_unprivileged_remount: unprivileged-remount-test
- @if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi
+include ../lib.mk
-run_tests: all test_unprivileged_remount
+TEST_PROGS := unprivileged-remount-test
+override RUN_TESTS := if [ -f /proc/self/uid_map ] ; then ./unprivileged-remount-test ; fi
+override EMIT_TESTS := echo "$(RUN_TESTS)"
clean:
rm -f unprivileged-remount-test
-
-.PHONY: all test_unprivileged_remount
+CFLAGS = -O2
+
all:
- gcc -O2 mq_open_tests.c -o mq_open_tests -lrt
- gcc -O2 -o mq_perf_tests mq_perf_tests.c -lrt -lpthread -lpopt
+ $(CC) $(CFLAGS) mq_open_tests.c -o mq_open_tests -lrt
+ $(CC) $(CFLAGS) -o mq_perf_tests mq_perf_tests.c -lrt -lpthread -lpopt
+
+include ../lib.mk
+
+override define RUN_TESTS
+ @./mq_open_tests /test1 || echo "selftests: mq_open_tests [FAIL]"
+ @./mq_perf_tests || echo "selftests: mq_perf_tests [FAIL]"
+endef
+
+TEST_PROGS := mq_open_tests mq_perf_tests
-run_tests:
- @./mq_open_tests /test1 || echo "mq_open_tests: [FAIL]"
- @./mq_perf_tests || echo "mq_perf_tests: [FAIL]"
+override define EMIT_TESTS
+ echo "./mq_open_tests /test1 || echo \"selftests: mq_open_tests [FAIL]\""
+ echo "./mq_perf_tests || echo \"selftests: mq_perf_tests [FAIL]\""
+endef
clean:
rm -f mq_open_tests mq_perf_tests
# Makefile for net selftests
-CC = $(CROSS_COMPILE)gcc
CFLAGS = -Wall -O2 -g
CFLAGS += -I../../../../usr/include/
%: %.c
$(CC) $(CFLAGS) -o $@ $^
-run_tests: all
- @/bin/sh ./run_netsocktests || echo "sockettests: [FAIL]"
- @/bin/sh ./run_afpackettests || echo "afpackettests: [FAIL]"
- ./test_bpf.sh
+TEST_PROGS := run_netsocktests run_afpackettests test_bpf.sh
+TEST_FILES := $(NET_PROGS)
+
+include ../lib.mk
+
clean:
$(RM) $(NET_PROGS)
GIT_VERSION = $(shell git describe --always --long --dirty || echo "unknown")
-CC := $(CROSS_COMPILE)$(CC)
CFLAGS := -Wall -O2 -flto -Wall -Werror -DGIT_VERSION='"$(GIT_VERSION)"' -I$(CURDIR) $(CFLAGS)
-export CC CFLAGS
+export CFLAGS
TARGETS = pmu copyloops mm tm primitives stringloops
$(TARGETS):
$(MAKE) -k -C $@ all
-run_tests: all
+include ../lib.mk
+
+override define RUN_TESTS
@for TARGET in $(TARGETS); do \
$(MAKE) -C $$TARGET run_tests; \
done;
+endef
+
+override define INSTALL_RULE
+ @for TARGET in $(TARGETS); do \
+ $(MAKE) -C $$TARGET install; \
+ done;
+endef
+
+override define EMIT_TESTS
+ @for TARGET in $(TARGETS); do \
+ $(MAKE) -s -C $$TARGET emit_tests; \
+ done;
+endef
clean:
@for TARGET in $(TARGETS); do \
tags:
find . -name '*.c' -o -name '*.h' | xargs ctags
-.PHONY: all run_tests clean tags $(TARGETS)
+.PHONY: tags $(TARGETS)
# Use our CFLAGS for the implicit .S rule
ASFLAGS = $(CFLAGS)
-PROGS := copyuser_64 copyuser_power7 memcpy_64 memcpy_power7
+TEST_PROGS := copyuser_64 copyuser_power7 memcpy_64 memcpy_power7
EXTRA_SOURCES := validate.c ../harness.c
-all: $(PROGS)
+all: $(TEST_PROGS)
copyuser_64: CPPFLAGS += -D COPY_LOOP=test___copy_tofrom_user_base
copyuser_power7: CPPFLAGS += -D COPY_LOOP=test___copy_tofrom_user_power7
memcpy_64: CPPFLAGS += -D COPY_LOOP=test_memcpy
memcpy_power7: CPPFLAGS += -D COPY_LOOP=test_memcpy_power7
-$(PROGS): $(EXTRA_SOURCES)
+$(TEST_PROGS): $(EXTRA_SOURCES)
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
clean:
- rm -f $(PROGS) *.o
-
-.PHONY: all run_tests clean
+ rm -f $(TEST_PROGS) *.o
noarg:
$(MAKE) -C ../
-PROGS := hugetlb_vs_thp_test subpage_prot
+TEST_PROGS := hugetlb_vs_thp_test subpage_prot
-all: $(PROGS) tempfile
+all: $(TEST_PROGS) tempfile
-$(PROGS): ../harness.c
+$(TEST_PROGS): ../harness.c
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
tempfile:
dd if=/dev/zero of=tempfile bs=64k count=1
clean:
- rm -f $(PROGS) tempfile
-
-.PHONY: all run_tests clean
+ rm -f $(TEST_PROGS) tempfile
noarg:
$(MAKE) -C ../
-PROGS := count_instructions l3_bank_test per_event_excludes
+TEST_PROGS := count_instructions l3_bank_test per_event_excludes
EXTRA_SOURCES := ../harness.c event.c lib.c
-SUB_TARGETS = ebb
+all: $(TEST_PROGS) ebb
-all: $(PROGS) $(SUB_TARGETS)
-
-$(PROGS): $(EXTRA_SOURCES)
+$(TEST_PROGS): $(EXTRA_SOURCES)
# loop.S can only be built 64-bit
count_instructions: loop.S count_instructions.c $(EXTRA_SOURCES)
$(CC) $(CFLAGS) -m64 -o $@ $^
-run_tests: all sub_run_tests
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
-clean: sub_clean
- rm -f $(PROGS) loop.o
+DEFAULT_RUN_TESTS := $(RUN_TESTS)
+override define RUN_TESTS
+ $(DEFAULT_RUN_TESTS)
+ $(MAKE) -C ebb run_tests
+endef
-$(SUB_TARGETS):
- $(MAKE) -k -C $@ all
+DEFAULT_EMIT_TESTS := $(EMIT_TESTS)
+override define EMIT_TESTS
+ $(DEFAULT_EMIT_TESTS)
+ $(MAKE) -s -C ebb emit_tests
+endef
-sub_run_tests: all
- @for TARGET in $(SUB_TARGETS); do \
- $(MAKE) -C $$TARGET run_tests; \
- done;
+DEFAULT_INSTALL := $(INSTALL_RULE)
+override define INSTALL_RULE
+ $(DEFAULT_INSTALL_RULE)
+ $(MAKE) -C ebb install
+endef
-sub_clean:
- @for TARGET in $(SUB_TARGETS); do \
- $(MAKE) -C $$TARGET clean; \
- done;
+clean:
+ rm -f $(TEST_PROGS) loop.o
+ $(MAKE) -C ebb clean
+
+ebb:
+ $(MAKE) -k -C $@ all
-.PHONY: all run_tests clean sub_run_tests sub_clean $(SUB_TARGETS)
+.PHONY: all run_tests clean ebb
# The EBB handler is 64-bit code and everything links against it
CFLAGS += -m64
-PROGS := reg_access_test event_attributes_test cycles_test \
+TEST_PROGS := reg_access_test event_attributes_test cycles_test \
cycles_with_freeze_test pmc56_overflow_test \
ebb_vs_cpu_event_test cpu_event_vs_ebb_test \
cpu_event_pinned_vs_ebb_test task_event_vs_ebb_test \
lost_exception_test no_handler_test \
cycles_with_mmcr2_test
-all: $(PROGS)
+all: $(TEST_PROGS)
-$(PROGS): ../../harness.c ../event.c ../lib.c ebb.c ebb_handler.S trace.c busy_loop.S
+$(TEST_PROGS): ../../harness.c ../event.c ../lib.c ebb.c ebb_handler.S trace.c busy_loop.S
instruction_count_test: ../loop.S
lost_exception_test: ../lib.c
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../../lib.mk
clean:
- rm -f $(PROGS)
+ rm -f $(TEST_PROGS)
CFLAGS += -I$(CURDIR)
-PROGS := load_unaligned_zeropad
+TEST_PROGS := load_unaligned_zeropad
-all: $(PROGS)
+all: $(TEST_PROGS)
-$(PROGS): ../harness.c
+$(TEST_PROGS): ../harness.c
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
clean:
- rm -f $(PROGS) *.o
-
-.PHONY: all run_tests clean
+ rm -f $(TEST_PROGS) *.o
CFLAGS += -m64
CFLAGS += -I$(CURDIR)
-PROGS := memcmp
+TEST_PROGS := memcmp
EXTRA_SOURCES := memcmp_64.S ../harness.c
-all: $(PROGS)
+all: $(TEST_PROGS)
-$(PROGS): $(EXTRA_SOURCES)
+$(TEST_PROGS): $(EXTRA_SOURCES)
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
clean:
- rm -f $(PROGS) *.o
-
-.PHONY: all run_tests clean
+ rm -f $(TEST_PROGS) *.o
-PROGS := tm-resched-dscr
+TEST_PROGS := tm-resched-dscr
-all: $(PROGS)
+all: $(TEST_PROGS)
-$(PROGS): ../harness.c
+$(TEST_PROGS): ../harness.c
-run_tests: all
- @-for PROG in $(PROGS); do \
- ./$$PROG; \
- done;
+include ../../lib.mk
clean:
- rm -f $(PROGS) *.o
-
-.PHONY: all run_tests clean
+ rm -f $(TEST_PROGS) *.o
clean:
rm -f peeksiginfo
-run_tests: all
- @./peeksiginfo || echo "peeksiginfo selftests: [FAIL]"
+TEST_PROGS := peeksiginfo
+
+include ../lib.mk
-CC = $(CROSS_COMPILE)gcc
-
all: get_size
get_size: get_size.c
$(CC) -static -ffreestanding -nostartfiles -s $< -o $@
-run_tests: all
- ./get_size
+TEST_PROGS := get_size
+
+include ../lib.mk
clean:
$(RM) get_size
# No binaries, but make sure arg-less "make" doesn't trigger "run_tests".
all:
-# Allow specific tests to be selected.
-test_num:
- @/bin/sh ./run_numerictests
+TEST_PROGS := run_numerictests run_stringtests
+TEST_FILES := common_tests
-test_string:
- @/bin/sh ./run_stringtests
-
-run_tests: all test_num test_string
+include ../lib.mk
# Nothing to clean up.
clean:
-
-.PHONY: all run_tests clean test_num test_string
-all:
- gcc posix_timers.c -o posix_timers -lrt
+CC = $(CROSS_COMPILE)gcc
+BUILD_FLAGS = -DKTEST
+CFLAGS += -O3 -Wl,-no-as-needed -Wall $(BUILD_FLAGS)
+LDFLAGS += -lrt -lpthread
-run_tests: all
- ./posix_timers
+# these are all "safe" tests that don't modify
+# system time or require escalated privledges
+TEST_PROGS = posix_timers nanosleep nsleep-lat set-timer-lat mqueue-lat \
+ inconsistency-check raw_skew threadtest rtctest
+
+TEST_PROGS_EXTENDED = alarmtimer-suspend valid-adjtimex change_skew \
+ skew_consistency clocksource-switch leap-a-day \
+ leapcrash set-tai set-2038
+
+bins = $(TEST_PROGS) $(TEST_PROGS_EXTENDED)
+
+all: ${bins}
+
+include ../lib.mk
+
+# these tests require escalated privledges
+# and may modify the system time or trigger
+# other behavior like suspend
+run_destructive_tests: run_tests
+ ./alarmtimer-suspend
+ ./valid-adjtimex
+ ./change_skew
+ ./skew_consistency
+ ./clocksource-switch
+ ./leap-a-day -s -i 10
+ ./leapcrash
+ ./set-tai
+ ./set-2038
clean:
- rm -f ./posix_timers
+ rm -f ${bins}
--- /dev/null
+/* alarmtimer suspend test
+ * John Stultz (john.stultz@linaro.org)
+ * (C) Copyright Linaro 2013
+ * Licensed under the GPLv2
+ *
+ * This test makes sure the alarmtimer & RTC wakeup code is
+ * functioning.
+ *
+ * To build:
+ * $ gcc alarmtimer-suspend.c -o alarmtimer-suspend -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <unistd.h>
+#include <time.h>
+#include <string.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <pthread.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define CLOCK_REALTIME 0
+#define CLOCK_MONOTONIC 1
+#define CLOCK_PROCESS_CPUTIME_ID 2
+#define CLOCK_THREAD_CPUTIME_ID 3
+#define CLOCK_MONOTONIC_RAW 4
+#define CLOCK_REALTIME_COARSE 5
+#define CLOCK_MONOTONIC_COARSE 6
+#define CLOCK_BOOTTIME 7
+#define CLOCK_REALTIME_ALARM 8
+#define CLOCK_BOOTTIME_ALARM 9
+#define CLOCK_HWSPECIFIC 10
+#define CLOCK_TAI 11
+#define NR_CLOCKIDS 12
+
+
+#define NSEC_PER_SEC 1000000000ULL
+#define UNREASONABLE_LAT (NSEC_PER_SEC * 4) /* hopefully we resume in 4secs */
+
+#define SUSPEND_SECS 15
+int alarmcount;
+int alarm_clock_id;
+struct timespec start_time;
+
+
+char *clockstring(int clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+
+long long timespec_sub(struct timespec a, struct timespec b)
+{
+ long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
+
+ ret -= NSEC_PER_SEC * a.tv_sec + a.tv_nsec;
+ return ret;
+}
+
+int final_ret = 0;
+
+void sigalarm(int signo)
+{
+ long long delta_ns;
+ struct timespec ts;
+
+ clock_gettime(alarm_clock_id, &ts);
+ alarmcount++;
+
+ delta_ns = timespec_sub(start_time, ts);
+ delta_ns -= NSEC_PER_SEC * SUSPEND_SECS * alarmcount;
+
+ printf("ALARM(%i): %ld:%ld latency: %lld ns ", alarmcount, ts.tv_sec,
+ ts.tv_nsec, delta_ns);
+
+ if (delta_ns > UNREASONABLE_LAT) {
+ printf("[FAIL]\n");
+ final_ret = -1;
+ } else
+ printf("[OK]\n");
+
+}
+
+int main(void)
+{
+ timer_t tm1;
+ struct itimerspec its1, its2;
+ struct sigevent se;
+ struct sigaction act;
+ int signum = SIGRTMAX;
+
+ /* Set up signal handler: */
+ sigfillset(&act.sa_mask);
+ act.sa_flags = 0;
+ act.sa_handler = sigalarm;
+ sigaction(signum, &act, NULL);
+
+ /* Set up timer: */
+ memset(&se, 0, sizeof(se));
+ se.sigev_notify = SIGEV_SIGNAL;
+ se.sigev_signo = signum;
+ se.sigev_value.sival_int = 0;
+
+ for (alarm_clock_id = CLOCK_REALTIME_ALARM;
+ alarm_clock_id <= CLOCK_BOOTTIME_ALARM;
+ alarm_clock_id++) {
+
+ alarmcount = 0;
+ timer_create(alarm_clock_id, &se, &tm1);
+
+ clock_gettime(alarm_clock_id, &start_time);
+ printf("Start time (%s): %ld:%ld\n", clockstring(alarm_clock_id),
+ start_time.tv_sec, start_time.tv_nsec);
+ printf("Setting alarm for every %i seconds\n", SUSPEND_SECS);
+ its1.it_value = start_time;
+ its1.it_value.tv_sec += SUSPEND_SECS;
+ its1.it_interval.tv_sec = SUSPEND_SECS;
+ its1.it_interval.tv_nsec = 0;
+
+ timer_settime(tm1, TIMER_ABSTIME, &its1, &its2);
+
+ while (alarmcount < 5)
+ sleep(1); /* First 5 alarms, do nothing */
+
+ printf("Starting suspend loops\n");
+ while (alarmcount < 10) {
+ int ret;
+
+ sleep(1);
+ ret = system("echo mem > /sys/power/state");
+ if (ret)
+ break;
+ }
+ timer_delete(tm1);
+ }
+ if (final_ret)
+ return ksft_exit_fail();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* ADJ_FREQ Skew change test
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2012
+ * Licensed under the GPLv2
+ *
+ * NOTE: This is a meta-test which cranks the ADJ_FREQ knob and
+ * then uses other tests to detect problems. Thus this test requires
+ * that the raw_skew, inconsistency-check and nanosleep tests be
+ * present in the same directory it is run from.
+ *
+ * To build:
+ * $ gcc change_skew.c -o change_skew -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <time.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000LL
+
+
+int change_skew_test(int ppm)
+{
+ struct timex tx;
+ int ret;
+
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = ppm << 16;
+
+ ret = adjtimex(&tx);
+ if (ret < 0) {
+ printf("Error adjusting freq\n");
+ return ret;
+ }
+
+ ret = system("./raw_skew");
+ ret |= system("./inconsistency-check");
+ ret |= system("./nanosleep");
+
+ return ret;
+}
+
+
+int main(int argv, char **argc)
+{
+ struct timex tx;
+ int i, ret;
+
+ int ppm[5] = {0, 250, 500, -250, -500};
+
+ /* Kill ntpd */
+ ret = system("killall -9 ntpd");
+
+ /* Make sure there's no offset adjustment going on */
+ tx.modes = ADJ_OFFSET;
+ tx.offset = 0;
+ ret = adjtimex(&tx);
+
+ if (ret < 0) {
+ printf("Maybe you're not running as root?\n");
+ return -1;
+ }
+
+ for (i = 0; i < 5; i++) {
+ printf("Using %i ppm adjustment\n", ppm[i]);
+ ret = change_skew_test(ppm[i]);
+ if (ret)
+ break;
+ }
+
+ /* Set things back */
+ tx.modes = ADJ_FREQUENCY;
+ tx.offset = 0;
+ adjtimex(&tx);
+
+ if (ret) {
+ printf("[FAIL]");
+ return ksft_exit_fail();
+ }
+ printf("[OK]");
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Clocksource change test
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2012
+ * Licensed under the GPLv2
+ *
+ * NOTE: This is a meta-test which quickly changes the clocksourc and
+ * then uses other tests to detect problems. Thus this test requires
+ * that the inconsistency-check and nanosleep tests be present in the
+ * same directory it is run from.
+ *
+ * To build:
+ * $ gcc clocksource-switch.c -o clocksource-switch -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <string.h>
+#include <sys/wait.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+
+int get_clocksources(char list[][30])
+{
+ int fd, i;
+ size_t size;
+ char buf[512];
+ char *head, *tmp;
+
+ fd = open("/sys/devices/system/clocksource/clocksource0/available_clocksource", O_RDONLY);
+
+ size = read(fd, buf, 512);
+
+ close(fd);
+
+ for (i = 0; i < 30; i++)
+ list[i][0] = '\0';
+
+ head = buf;
+ i = 0;
+ while (head - buf < size) {
+ /* Find the next space */
+ for (tmp = head; *tmp != ' '; tmp++) {
+ if (*tmp == '\n')
+ break;
+ if (*tmp == '\0')
+ break;
+ }
+ *tmp = '\0';
+ strcpy(list[i], head);
+ head = tmp + 1;
+ i++;
+ }
+
+ return i-1;
+}
+
+int get_cur_clocksource(char *buf, size_t size)
+{
+ int fd;
+
+ fd = open("/sys/devices/system/clocksource/clocksource0/current_clocksource", O_RDONLY);
+
+ size = read(fd, buf, size);
+
+ return 0;
+}
+
+int change_clocksource(char *clocksource)
+{
+ int fd;
+ size_t size;
+
+ fd = open("/sys/devices/system/clocksource/clocksource0/current_clocksource", O_WRONLY);
+
+ if (fd < 0)
+ return -1;
+
+ size = write(fd, clocksource, strlen(clocksource));
+
+ if (size < 0)
+ return -1;
+
+ close(fd);
+ return 0;
+}
+
+
+int run_tests(int secs)
+{
+ int ret;
+ char buf[255];
+
+ sprintf(buf, "./inconsistency-check -t %i", secs);
+ ret = system(buf);
+ if (ret)
+ return ret;
+ ret = system("./nanosleep");
+ return ret;
+}
+
+
+char clocksource_list[10][30];
+
+int main(int argv, char **argc)
+{
+ char orig_clk[512];
+ int count, i, status;
+ pid_t pid;
+
+ get_cur_clocksource(orig_clk, 512);
+
+ count = get_clocksources(clocksource_list);
+
+ if (change_clocksource(clocksource_list[0])) {
+ printf("Error: You probably need to run this as root\n");
+ return -1;
+ }
+
+ /* Check everything is sane before we start switching asyncrhonously */
+ for (i = 0; i < count; i++) {
+ printf("Validating clocksource %s\n", clocksource_list[i]);
+ if (change_clocksource(clocksource_list[i])) {
+ status = -1;
+ goto out;
+ }
+ if (run_tests(5)) {
+ status = -1;
+ goto out;
+ }
+ }
+
+
+ printf("Running Asyncrhonous Switching Tests...\n");
+ pid = fork();
+ if (!pid)
+ return run_tests(60);
+
+ while (pid != waitpid(pid, &status, WNOHANG))
+ for (i = 0; i < count; i++)
+ if (change_clocksource(clocksource_list[i])) {
+ status = -1;
+ goto out;
+ }
+out:
+ change_clocksource(orig_clk);
+
+ if (status)
+ return ksft_exit_fail();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Time inconsistency check test
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2003, 2004, 2005, 2012
+ * (C) Copyright Linaro Limited 2015
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc inconsistency-check.c -o inconsistency-check -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define CALLS_PER_LOOP 64
+#define NSEC_PER_SEC 1000000000ULL
+
+#define CLOCK_REALTIME 0
+#define CLOCK_MONOTONIC 1
+#define CLOCK_PROCESS_CPUTIME_ID 2
+#define CLOCK_THREAD_CPUTIME_ID 3
+#define CLOCK_MONOTONIC_RAW 4
+#define CLOCK_REALTIME_COARSE 5
+#define CLOCK_MONOTONIC_COARSE 6
+#define CLOCK_BOOTTIME 7
+#define CLOCK_REALTIME_ALARM 8
+#define CLOCK_BOOTTIME_ALARM 9
+#define CLOCK_HWSPECIFIC 10
+#define CLOCK_TAI 11
+#define NR_CLOCKIDS 12
+
+char *clockstring(int clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+/* returns 1 if a <= b, 0 otherwise */
+static inline int in_order(struct timespec a, struct timespec b)
+{
+ /* use unsigned to avoid false positives on 2038 rollover */
+ if ((unsigned long)a.tv_sec < (unsigned long)b.tv_sec)
+ return 1;
+ if ((unsigned long)a.tv_sec > (unsigned long)b.tv_sec)
+ return 0;
+ if (a.tv_nsec > b.tv_nsec)
+ return 0;
+ return 1;
+}
+
+
+
+int consistency_test(int clock_type, unsigned long seconds)
+{
+ struct timespec list[CALLS_PER_LOOP];
+ int i, inconsistent;
+ long now, then;
+ time_t t;
+ char *start_str;
+
+ clock_gettime(clock_type, &list[0]);
+ now = then = list[0].tv_sec;
+
+ /* timestamp start of test */
+ t = time(0);
+ start_str = ctime(&t);
+
+ while (seconds == -1 || now - then < seconds) {
+ inconsistent = 0;
+
+ /* Fill list */
+ for (i = 0; i < CALLS_PER_LOOP; i++)
+ clock_gettime(clock_type, &list[i]);
+
+ /* Check for inconsistencies */
+ for (i = 0; i < CALLS_PER_LOOP - 1; i++)
+ if (!in_order(list[i], list[i+1]))
+ inconsistent = i;
+
+ /* display inconsistency */
+ if (inconsistent) {
+ unsigned long long delta;
+
+ printf("\%s\n", start_str);
+ for (i = 0; i < CALLS_PER_LOOP; i++) {
+ if (i == inconsistent)
+ printf("--------------------\n");
+ printf("%lu:%lu\n", list[i].tv_sec,
+ list[i].tv_nsec);
+ if (i == inconsistent + 1)
+ printf("--------------------\n");
+ }
+ delta = list[inconsistent].tv_sec * NSEC_PER_SEC;
+ delta += list[inconsistent].tv_nsec;
+ delta -= list[inconsistent+1].tv_sec * NSEC_PER_SEC;
+ delta -= list[inconsistent+1].tv_nsec;
+ printf("Delta: %llu ns\n", delta);
+ fflush(0);
+ /* timestamp inconsistency*/
+ t = time(0);
+ printf("%s\n", ctime(&t));
+ printf("[FAILED]\n");
+ return -1;
+ }
+ now = list[0].tv_sec;
+ }
+ printf("[OK]\n");
+ return 0;
+}
+
+
+int main(int argc, char *argv[])
+{
+ int clockid, opt;
+ int userclock = CLOCK_REALTIME;
+ int maxclocks = NR_CLOCKIDS;
+ int runtime = 10;
+ struct timespec ts;
+
+ /* Process arguments */
+ while ((opt = getopt(argc, argv, "t:c:")) != -1) {
+ switch (opt) {
+ case 't':
+ runtime = atoi(optarg);
+ break;
+ case 'c':
+ userclock = atoi(optarg);
+ maxclocks = userclock + 1;
+ break;
+ default:
+ printf("Usage: %s [-t <secs>] [-c <clockid>]\n", argv[0]);
+ printf(" -t: Number of seconds to run\n");
+ printf(" -c: clockid to use (default, all clockids)\n");
+ exit(-1);
+ }
+ }
+
+ setbuf(stdout, NULL);
+
+ for (clockid = userclock; clockid < maxclocks; clockid++) {
+
+ if (clockid == CLOCK_HWSPECIFIC)
+ continue;
+
+ if (!clock_gettime(clockid, &ts)) {
+ printf("Consistent %-30s ", clockstring(clockid));
+ if (consistency_test(clockid, runtime))
+ return ksft_exit_fail();
+ }
+ }
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Leap second stress test
+ * by: John Stultz (john.stultz@linaro.org)
+ * (C) Copyright IBM 2012
+ * (C) Copyright 2013, 2015 Linaro Limited
+ * Licensed under the GPLv2
+ *
+ * This test signals the kernel to insert a leap second
+ * every day at midnight GMT. This allows for stessing the
+ * kernel's leap-second behavior, as well as how well applications
+ * handle the leap-second discontinuity.
+ *
+ * Usage: leap-a-day [-s] [-i <num>]
+ *
+ * Options:
+ * -s: Each iteration, set the date to 10 seconds before midnight GMT.
+ * This speeds up the number of leapsecond transitions tested,
+ * but because it calls settimeofday frequently, advancing the
+ * time by 24 hours every ~16 seconds, it may cause application
+ * disruption.
+ *
+ * -i: Number of iterations to run (default: infinite)
+ *
+ * Other notes: Disabling NTP prior to running this is advised, as the two
+ * may conflict in their commands to the kernel.
+ *
+ * To build:
+ * $ gcc leap-a-day.c -o leap-a-day -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#include <unistd.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000ULL
+#define CLOCK_TAI 11
+
+/* returns 1 if a <= b, 0 otherwise */
+static inline int in_order(struct timespec a, struct timespec b)
+{
+ if (a.tv_sec < b.tv_sec)
+ return 1;
+ if (a.tv_sec > b.tv_sec)
+ return 0;
+ if (a.tv_nsec > b.tv_nsec)
+ return 0;
+ return 1;
+}
+
+struct timespec timespec_add(struct timespec ts, unsigned long long ns)
+{
+ ts.tv_nsec += ns;
+ while (ts.tv_nsec >= NSEC_PER_SEC) {
+ ts.tv_nsec -= NSEC_PER_SEC;
+ ts.tv_sec++;
+ }
+ return ts;
+}
+
+char *time_state_str(int state)
+{
+ switch (state) {
+ case TIME_OK: return "TIME_OK";
+ case TIME_INS: return "TIME_INS";
+ case TIME_DEL: return "TIME_DEL";
+ case TIME_OOP: return "TIME_OOP";
+ case TIME_WAIT: return "TIME_WAIT";
+ case TIME_BAD: return "TIME_BAD";
+ }
+ return "ERROR";
+}
+
+/* clear NTP time_status & time_state */
+int clear_time_state(void)
+{
+ struct timex tx;
+ int ret;
+
+ /*
+ * We have to call adjtime twice here, as kernels
+ * prior to 6b1859dba01c7 (included in 3.5 and
+ * -stable), had an issue with the state machine
+ * and wouldn't clear the STA_INS/DEL flag directly.
+ */
+ tx.modes = ADJ_STATUS;
+ tx.status = STA_PLL;
+ ret = adjtimex(&tx);
+
+ /* Clear maxerror, as it can cause UNSYNC to be set */
+ tx.modes = ADJ_MAXERROR;
+ tx.maxerror = 0;
+ ret = adjtimex(&tx);
+
+ /* Clear the status */
+ tx.modes = ADJ_STATUS;
+ tx.status = 0;
+ ret = adjtimex(&tx);
+
+ return ret;
+}
+
+/* Make sure we cleanup on ctrl-c */
+void handler(int unused)
+{
+ clear_time_state();
+ exit(0);
+}
+
+/* Test for known hrtimer failure */
+void test_hrtimer_failure(void)
+{
+ struct timespec now, target;
+
+ clock_gettime(CLOCK_REALTIME, &now);
+ target = timespec_add(now, NSEC_PER_SEC/2);
+ clock_nanosleep(CLOCK_REALTIME, TIMER_ABSTIME, &target, NULL);
+ clock_gettime(CLOCK_REALTIME, &now);
+
+ if (!in_order(target, now))
+ printf("ERROR: hrtimer early expiration failure observed.\n");
+}
+
+int main(int argc, char **argv)
+{
+ int settime = 0;
+ int tai_time = 0;
+ int insert = 1;
+ int iterations = -1;
+ int opt;
+
+ /* Process arguments */
+ while ((opt = getopt(argc, argv, "sti:")) != -1) {
+ switch (opt) {
+ case 's':
+ printf("Setting time to speed up testing\n");
+ settime = 1;
+ break;
+ case 'i':
+ iterations = atoi(optarg);
+ break;
+ case 't':
+ tai_time = 1;
+ break;
+ default:
+ printf("Usage: %s [-s] [-i <iterations>]\n", argv[0]);
+ printf(" -s: Set time to right before leap second each iteration\n");
+ printf(" -i: Number of iterations\n");
+ printf(" -t: Print TAI time\n");
+ exit(-1);
+ }
+ }
+
+ /* Make sure TAI support is present if -t was used */
+ if (tai_time) {
+ struct timespec ts;
+
+ if (clock_gettime(CLOCK_TAI, &ts)) {
+ printf("System doesn't support CLOCK_TAI\n");
+ ksft_exit_fail();
+ }
+ }
+
+ signal(SIGINT, handler);
+ signal(SIGKILL, handler);
+
+ if (iterations < 0)
+ printf("This runs continuously. Press ctrl-c to stop\n");
+ else
+ printf("Running for %i iterations. Press ctrl-c to stop\n", iterations);
+
+ printf("\n");
+ while (1) {
+ int ret;
+ struct timespec ts;
+ struct timex tx;
+ time_t now, next_leap;
+
+ /* Get the current time */
+ clock_gettime(CLOCK_REALTIME, &ts);
+
+ /* Calculate the next possible leap second 23:59:60 GMT */
+ next_leap = ts.tv_sec;
+ next_leap += 86400 - (next_leap % 86400);
+
+ if (settime) {
+ struct timeval tv;
+
+ tv.tv_sec = next_leap - 10;
+ tv.tv_usec = 0;
+ settimeofday(&tv, NULL);
+ printf("Setting time to %s", ctime(&tv.tv_sec));
+ }
+
+ /* Reset NTP time state */
+ clear_time_state();
+
+ /* Set the leap second insert flag */
+ tx.modes = ADJ_STATUS;
+ if (insert)
+ tx.status = STA_INS;
+ else
+ tx.status = STA_DEL;
+ ret = adjtimex(&tx);
+ if (ret < 0) {
+ printf("Error: Problem setting STA_INS/STA_DEL!: %s\n",
+ time_state_str(ret));
+ return ksft_exit_fail();
+ }
+
+ /* Validate STA_INS was set */
+ tx.modes = 0;
+ ret = adjtimex(&tx);
+ if (tx.status != STA_INS && tx.status != STA_DEL) {
+ printf("Error: STA_INS/STA_DEL not set!: %s\n",
+ time_state_str(ret));
+ return ksft_exit_fail();
+ }
+
+ if (tai_time) {
+ printf("Using TAI time,"
+ " no inconsistencies should be seen!\n");
+ }
+
+ printf("Scheduling leap second for %s", ctime(&next_leap));
+
+ /* Wake up 3 seconds before leap */
+ ts.tv_sec = next_leap - 3;
+ ts.tv_nsec = 0;
+
+ while (clock_nanosleep(CLOCK_REALTIME, TIMER_ABSTIME, &ts, NULL))
+ printf("Something woke us up, returning to sleep\n");
+
+ /* Validate STA_INS is still set */
+ tx.modes = 0;
+ ret = adjtimex(&tx);
+ if (tx.status != STA_INS && tx.status != STA_DEL) {
+ printf("Something cleared STA_INS/STA_DEL, setting it again.\n");
+ tx.modes = ADJ_STATUS;
+ if (insert)
+ tx.status = STA_INS;
+ else
+ tx.status = STA_DEL;
+ ret = adjtimex(&tx);
+ }
+
+ /* Check adjtimex output every half second */
+ now = tx.time.tv_sec;
+ while (now < next_leap + 2) {
+ char buf[26];
+ struct timespec tai;
+
+ tx.modes = 0;
+ ret = adjtimex(&tx);
+
+ if (tai_time) {
+ clock_gettime(CLOCK_TAI, &tai);
+ printf("%ld sec, %9ld ns\t%s\n",
+ tai.tv_sec,
+ tai.tv_nsec,
+ time_state_str(ret));
+ } else {
+ ctime_r(&tx.time.tv_sec, buf);
+ buf[strlen(buf)-1] = 0; /*remove trailing\n */
+
+ printf("%s + %6ld us (%i)\t%s\n",
+ buf,
+ tx.time.tv_usec,
+ tx.tai,
+ time_state_str(ret));
+ }
+ now = tx.time.tv_sec;
+ /* Sleep for another half second */
+ ts.tv_sec = 0;
+ ts.tv_nsec = NSEC_PER_SEC / 2;
+ clock_nanosleep(CLOCK_MONOTONIC, 0, &ts, NULL);
+ }
+ /* Switch to using other mode */
+ insert = !insert;
+
+ /* Note if kernel has known hrtimer failure */
+ test_hrtimer_failure();
+
+ printf("Leap complete\n\n");
+
+ if ((iterations != -1) && !(--iterations))
+ break;
+ }
+
+ clear_time_state();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Demo leapsecond deadlock
+ * by: John Stultz (john.stultz@linaro.org)
+ * (C) Copyright IBM 2012
+ * (C) Copyright 2013, 2015 Linaro Limited
+ * Licensed under the GPL
+ *
+ * This test demonstrates leapsecond deadlock that is possibe
+ * on kernels from 2.6.26 to 3.3.
+ *
+ * WARNING: THIS WILL LIKELY HARDHANG SYSTEMS AND MAY LOSE DATA
+ * RUN AT YOUR OWN RISK!
+ * To build:
+ * $ gcc leapcrash.c -o leapcrash -lrt
+ */
+
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+
+
+/* clear NTP time_status & time_state */
+int clear_time_state(void)
+{
+ struct timex tx;
+ int ret;
+
+ /*
+ * We have to call adjtime twice here, as kernels
+ * prior to 6b1859dba01c7 (included in 3.5 and
+ * -stable), had an issue with the state machine
+ * and wouldn't clear the STA_INS/DEL flag directly.
+ */
+ tx.modes = ADJ_STATUS;
+ tx.status = STA_PLL;
+ ret = adjtimex(&tx);
+
+ tx.modes = ADJ_STATUS;
+ tx.status = 0;
+ ret = adjtimex(&tx);
+
+ return ret;
+}
+
+/* Make sure we cleanup on ctrl-c */
+void handler(int unused)
+{
+ clear_time_state();
+ exit(0);
+}
+
+
+int main(void)
+{
+ struct timex tx;
+ struct timespec ts;
+ time_t next_leap;
+ int count = 0;
+
+ setbuf(stdout, NULL);
+
+ signal(SIGINT, handler);
+ signal(SIGKILL, handler);
+ printf("This runs for a few minutes. Press ctrl-c to stop\n");
+
+ clear_time_state();
+
+
+ /* Get the current time */
+ clock_gettime(CLOCK_REALTIME, &ts);
+
+ /* Calculate the next possible leap second 23:59:60 GMT */
+ next_leap = ts.tv_sec;
+ next_leap += 86400 - (next_leap % 86400);
+
+ for (count = 0; count < 20; count++) {
+ struct timeval tv;
+
+
+ /* set the time to 2 seconds before the leap */
+ tv.tv_sec = next_leap - 2;
+ tv.tv_usec = 0;
+ if (settimeofday(&tv, NULL)) {
+ printf("Error: You're likely not running with proper (ie: root) permissions\n");
+ return ksft_exit_fail();
+ }
+ tx.modes = 0;
+ adjtimex(&tx);
+
+ /* hammer on adjtime w/ STA_INS */
+ while (tx.time.tv_sec < next_leap + 1) {
+ /* Set the leap second insert flag */
+ tx.modes = ADJ_STATUS;
+ tx.status = STA_INS;
+ adjtimex(&tx);
+ }
+ clear_time_state();
+ printf(".");
+ }
+ printf("[OK]\n");
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Measure mqueue timeout latency
+ * by: john stultz (john.stultz@linaro.org)
+ * (C) Copyright Linaro 2013
+ *
+ * Inspired with permission from example test by:
+ * Romain Francoise <romain@orebokech.com>
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc mqueue-lat.c -o mqueue-lat -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#include <errno.h>
+#include <mqueue.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000ULL
+
+#define TARGET_TIMEOUT 100000000 /* 100ms in nanoseconds */
+#define UNRESONABLE_LATENCY 40000000 /* 40ms in nanosecs */
+
+
+long long timespec_sub(struct timespec a, struct timespec b)
+{
+ long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
+
+ ret -= NSEC_PER_SEC * a.tv_sec + a.tv_nsec;
+ return ret;
+}
+
+struct timespec timespec_add(struct timespec ts, unsigned long long ns)
+{
+ ts.tv_nsec += ns;
+ while (ts.tv_nsec >= NSEC_PER_SEC) {
+ ts.tv_nsec -= NSEC_PER_SEC;
+ ts.tv_sec++;
+ }
+ return ts;
+}
+
+int mqueue_lat_test(void)
+{
+
+ mqd_t q;
+ struct mq_attr attr;
+ struct timespec start, end, now, target;
+ int i, count, ret;
+
+ q = mq_open("/foo", O_CREAT | O_RDONLY, 0666, NULL);
+ if (q < 0) {
+ perror("mq_open");
+ return -1;
+ }
+ mq_getattr(q, &attr);
+
+
+ count = 100;
+ clock_gettime(CLOCK_MONOTONIC, &start);
+
+ for (i = 0; i < count; i++) {
+ char buf[attr.mq_msgsize];
+
+ clock_gettime(CLOCK_REALTIME, &now);
+ target = now;
+ target = timespec_add(now, TARGET_TIMEOUT); /* 100ms */
+
+ ret = mq_timedreceive(q, buf, sizeof(buf), NULL, &target);
+ if (ret < 0 && errno != ETIMEDOUT) {
+ perror("mq_timedreceive");
+ return -1;
+ }
+ }
+ clock_gettime(CLOCK_MONOTONIC, &end);
+
+ mq_close(q);
+
+ if ((timespec_sub(start, end)/count) > TARGET_TIMEOUT + UNRESONABLE_LATENCY)
+ return -1;
+
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ int ret;
+
+ printf("Mqueue latency : ");
+
+ ret = mqueue_lat_test();
+ if (ret < 0) {
+ printf("[FAILED]\n");
+ return ksft_exit_fail();
+ }
+ printf("[OK]\n");
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Make sure timers don't return early
+ * by: john stultz (johnstul@us.ibm.com)
+ * John Stultz (john.stultz@linaro.org)
+ * (C) Copyright IBM 2012
+ * (C) Copyright Linaro 2013 2015
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc nanosleep.c -o nanosleep -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000ULL
+
+#define CLOCK_REALTIME 0
+#define CLOCK_MONOTONIC 1
+#define CLOCK_PROCESS_CPUTIME_ID 2
+#define CLOCK_THREAD_CPUTIME_ID 3
+#define CLOCK_MONOTONIC_RAW 4
+#define CLOCK_REALTIME_COARSE 5
+#define CLOCK_MONOTONIC_COARSE 6
+#define CLOCK_BOOTTIME 7
+#define CLOCK_REALTIME_ALARM 8
+#define CLOCK_BOOTTIME_ALARM 9
+#define CLOCK_HWSPECIFIC 10
+#define CLOCK_TAI 11
+#define NR_CLOCKIDS 12
+
+#define UNSUPPORTED 0xf00f
+
+char *clockstring(int clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+/* returns 1 if a <= b, 0 otherwise */
+static inline int in_order(struct timespec a, struct timespec b)
+{
+ if (a.tv_sec < b.tv_sec)
+ return 1;
+ if (a.tv_sec > b.tv_sec)
+ return 0;
+ if (a.tv_nsec > b.tv_nsec)
+ return 0;
+ return 1;
+}
+
+struct timespec timespec_add(struct timespec ts, unsigned long long ns)
+{
+ ts.tv_nsec += ns;
+ while (ts.tv_nsec >= NSEC_PER_SEC) {
+ ts.tv_nsec -= NSEC_PER_SEC;
+ ts.tv_sec++;
+ }
+ return ts;
+}
+
+int nanosleep_test(int clockid, long long ns)
+{
+ struct timespec now, target, rel;
+
+ /* First check abs time */
+ if (clock_gettime(clockid, &now))
+ return UNSUPPORTED;
+ target = timespec_add(now, ns);
+
+ if (clock_nanosleep(clockid, TIMER_ABSTIME, &target, NULL))
+ return UNSUPPORTED;
+ clock_gettime(clockid, &now);
+
+ if (!in_order(target, now))
+ return -1;
+
+ /* Second check reltime */
+ clock_gettime(clockid, &now);
+ rel.tv_sec = 0;
+ rel.tv_nsec = 0;
+ rel = timespec_add(rel, ns);
+ target = timespec_add(now, ns);
+ clock_nanosleep(clockid, 0, &rel, NULL);
+ clock_gettime(clockid, &now);
+
+ if (!in_order(target, now))
+ return -1;
+ return 0;
+}
+
+int main(int argc, char **argv)
+{
+ long long length;
+ int clockid, ret;
+
+ for (clockid = CLOCK_REALTIME; clockid < NR_CLOCKIDS; clockid++) {
+
+ /* Skip cputime clockids since nanosleep won't increment cputime */
+ if (clockid == CLOCK_PROCESS_CPUTIME_ID ||
+ clockid == CLOCK_THREAD_CPUTIME_ID ||
+ clockid == CLOCK_HWSPECIFIC)
+ continue;
+
+ printf("Nanosleep %-31s ", clockstring(clockid));
+
+ length = 10;
+ while (length <= (NSEC_PER_SEC * 10)) {
+ ret = nanosleep_test(clockid, length);
+ if (ret == UNSUPPORTED) {
+ printf("[UNSUPPORTED]\n");
+ goto next;
+ }
+ if (ret < 0) {
+ printf("[FAILED]\n");
+ return ksft_exit_fail();
+ }
+ length *= 100;
+ }
+ printf("[OK]\n");
+next:
+ ret = 0;
+ }
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Measure nanosleep timer latency
+ * by: john stultz (john.stultz@linaro.org)
+ * (C) Copyright Linaro 2013
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc nsleep-lat.c -o nsleep-lat -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000ULL
+
+#define UNRESONABLE_LATENCY 40000000 /* 40ms in nanosecs */
+
+
+#define CLOCK_REALTIME 0
+#define CLOCK_MONOTONIC 1
+#define CLOCK_PROCESS_CPUTIME_ID 2
+#define CLOCK_THREAD_CPUTIME_ID 3
+#define CLOCK_MONOTONIC_RAW 4
+#define CLOCK_REALTIME_COARSE 5
+#define CLOCK_MONOTONIC_COARSE 6
+#define CLOCK_BOOTTIME 7
+#define CLOCK_REALTIME_ALARM 8
+#define CLOCK_BOOTTIME_ALARM 9
+#define CLOCK_HWSPECIFIC 10
+#define CLOCK_TAI 11
+#define NR_CLOCKIDS 12
+
+#define UNSUPPORTED 0xf00f
+
+char *clockstring(int clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+struct timespec timespec_add(struct timespec ts, unsigned long long ns)
+{
+ ts.tv_nsec += ns;
+ while (ts.tv_nsec >= NSEC_PER_SEC) {
+ ts.tv_nsec -= NSEC_PER_SEC;
+ ts.tv_sec++;
+ }
+ return ts;
+}
+
+
+long long timespec_sub(struct timespec a, struct timespec b)
+{
+ long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
+
+ ret -= NSEC_PER_SEC * a.tv_sec + a.tv_nsec;
+ return ret;
+}
+
+int nanosleep_lat_test(int clockid, long long ns)
+{
+ struct timespec start, end, target;
+ long long latency = 0;
+ int i, count;
+
+ target.tv_sec = ns/NSEC_PER_SEC;
+ target.tv_nsec = ns%NSEC_PER_SEC;
+
+ if (clock_gettime(clockid, &start))
+ return UNSUPPORTED;
+ if (clock_nanosleep(clockid, 0, &target, NULL))
+ return UNSUPPORTED;
+
+ count = 10;
+
+ /* First check relative latency */
+ clock_gettime(clockid, &start);
+ for (i = 0; i < count; i++)
+ clock_nanosleep(clockid, 0, &target, NULL);
+ clock_gettime(clockid, &end);
+
+ if (((timespec_sub(start, end)/count)-ns) > UNRESONABLE_LATENCY) {
+ printf("Large rel latency: %lld ns :", (timespec_sub(start, end)/count)-ns);
+ return -1;
+ }
+
+ /* Next check absolute latency */
+ for (i = 0; i < count; i++) {
+ clock_gettime(clockid, &start);
+ target = timespec_add(start, ns);
+ clock_nanosleep(clockid, TIMER_ABSTIME, &target, NULL);
+ clock_gettime(clockid, &end);
+ latency += timespec_sub(target, end);
+ }
+
+ if (latency/count > UNRESONABLE_LATENCY) {
+ printf("Large abs latency: %lld ns :", latency/count);
+ return -1;
+ }
+
+ return 0;
+}
+
+
+
+int main(int argc, char **argv)
+{
+ long long length;
+ int clockid, ret;
+
+ for (clockid = CLOCK_REALTIME; clockid < NR_CLOCKIDS; clockid++) {
+
+ /* Skip cputime clockids since nanosleep won't increment cputime */
+ if (clockid == CLOCK_PROCESS_CPUTIME_ID ||
+ clockid == CLOCK_THREAD_CPUTIME_ID ||
+ clockid == CLOCK_HWSPECIFIC)
+ continue;
+
+ printf("nsleep latency %-26s ", clockstring(clockid));
+
+ length = 10;
+ while (length <= (NSEC_PER_SEC * 10)) {
+ ret = nanosleep_lat_test(clockid, length);
+ if (ret)
+ break;
+ length *= 100;
+
+ }
+
+ if (ret == UNSUPPORTED) {
+ printf("[UNSUPPORTED]\n");
+ continue;
+ }
+ if (ret < 0) {
+ printf("[FAILED]\n");
+ return ksft_exit_fail();
+ }
+ printf("[OK]\n");
+ }
+ return ksft_exit_pass();
+}
static void kernel_loop(void)
{
void *addr = sbrk(0);
+ int err = 0;
- while (!done) {
- brk(addr + 4096);
- brk(addr);
+ while (!done && !err) {
+ err = brk(addr + 4096);
+ err |= brk(addr);
}
}
int main(int argc, char **argv)
{
- int err;
-
printf("Testing posix timers. False negative may happen on CPU execution \n");
printf("based timers if other threads run on the CPU...\n");
--- /dev/null
+/* CLOCK_MONOTONIC vs CLOCK_MONOTONIC_RAW skew test
+ * by: john stultz (johnstul@us.ibm.com)
+ * John Stultz <john.stultz@linaro.org>
+ * (C) Copyright IBM 2012
+ * (C) Copyright Linaro Limited 2015
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc raw_skew.c -o raw_skew -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <time.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+
+#define CLOCK_MONOTONIC_RAW 4
+#define NSEC_PER_SEC 1000000000LL
+
+#define shift_right(x, s) ({ \
+ __typeof__(x) __x = (x); \
+ __typeof__(s) __s = (s); \
+ __x < 0 ? -(-__x >> __s) : __x >> __s; \
+})
+
+long long llabs(long long val)
+{
+ if (val < 0)
+ val = -val;
+ return val;
+}
+
+unsigned long long ts_to_nsec(struct timespec ts)
+{
+ return ts.tv_sec * NSEC_PER_SEC + ts.tv_nsec;
+}
+
+struct timespec nsec_to_ts(long long ns)
+{
+ struct timespec ts;
+
+ ts.tv_sec = ns/NSEC_PER_SEC;
+ ts.tv_nsec = ns%NSEC_PER_SEC;
+ return ts;
+}
+
+long long diff_timespec(struct timespec start, struct timespec end)
+{
+ long long start_ns, end_ns;
+
+ start_ns = ts_to_nsec(start);
+ end_ns = ts_to_nsec(end);
+ return end_ns - start_ns;
+}
+
+void get_monotonic_and_raw(struct timespec *mon, struct timespec *raw)
+{
+ struct timespec start, mid, end;
+ long long diff = 0, tmp;
+ int i;
+
+ for (i = 0; i < 3; i++) {
+ long long newdiff;
+
+ clock_gettime(CLOCK_MONOTONIC, &start);
+ clock_gettime(CLOCK_MONOTONIC_RAW, &mid);
+ clock_gettime(CLOCK_MONOTONIC, &end);
+
+ newdiff = diff_timespec(start, end);
+ if (diff == 0 || newdiff < diff) {
+ diff = newdiff;
+ *raw = mid;
+ tmp = (ts_to_nsec(start) + ts_to_nsec(end))/2;
+ *mon = nsec_to_ts(tmp);
+ }
+ }
+}
+
+int main(int argv, char **argc)
+{
+ struct timespec mon, raw, start, end;
+ long long delta1, delta2, interval, eppm, ppm;
+ struct timex tx1, tx2;
+
+ setbuf(stdout, NULL);
+
+ if (clock_gettime(CLOCK_MONOTONIC_RAW, &raw)) {
+ printf("ERR: NO CLOCK_MONOTONIC_RAW\n");
+ return -1;
+ }
+
+ tx1.modes = 0;
+ adjtimex(&tx1);
+ get_monotonic_and_raw(&mon, &raw);
+ start = mon;
+ delta1 = diff_timespec(mon, raw);
+
+ if (tx1.offset)
+ printf("WARNING: ADJ_OFFSET in progress, this will cause inaccurate results\n");
+
+ printf("Estimating clock drift: ");
+ sleep(120);
+
+ get_monotonic_and_raw(&mon, &raw);
+ end = mon;
+ tx2.modes = 0;
+ adjtimex(&tx2);
+ delta2 = diff_timespec(mon, raw);
+
+ interval = diff_timespec(start, end);
+
+ /* calculate measured ppm between MONOTONIC and MONOTONIC_RAW */
+ eppm = ((delta2-delta1)*NSEC_PER_SEC)/interval;
+ eppm = -eppm;
+ printf("%lld.%i(est)", eppm/1000, abs((int)(eppm%1000)));
+
+ /* Avg the two actual freq samples adjtimex gave us */
+ ppm = (tx1.freq + tx2.freq) * 1000 / 2;
+ ppm = (long long)tx1.freq * 1000;
+ ppm = shift_right(ppm, 16);
+ printf(" %lld.%i(act)", ppm/1000, abs((int)(ppm%1000)));
+
+ if (llabs(eppm - ppm) > 1000) {
+ printf(" [FAILED]\n");
+ return ksft_exit_fail();
+ }
+ printf(" [OK]\n");
+ return ksft_exit_pass();
+}
--- /dev/null
+/*
+ * Real Time Clock Driver Test/Example Program
+ *
+ * Compile with:
+ * gcc -s -Wall -Wstrict-prototypes rtctest.c -o rtctest
+ *
+ * Copyright (C) 1996, Paul Gortmaker.
+ *
+ * Released under the GNU General Public License, version 2,
+ * included herein by reference.
+ *
+ */
+
+#include <stdio.h>
+#include <linux/rtc.h>
+#include <sys/ioctl.h>
+#include <sys/time.h>
+#include <sys/types.h>
+#include <fcntl.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <errno.h>
+
+
+/*
+ * This expects the new RTC class driver framework, working with
+ * clocks that will often not be clones of what the PC-AT had.
+ * Use the command line to specify another RTC if you need one.
+ */
+static const char default_rtc[] = "/dev/rtc0";
+
+
+int main(int argc, char **argv)
+{
+ int i, fd, retval, irqcount = 0;
+ unsigned long tmp, data;
+ struct rtc_time rtc_tm;
+ const char *rtc = default_rtc;
+ struct timeval start, end, diff;
+
+ switch (argc) {
+ case 2:
+ rtc = argv[1];
+ /* FALLTHROUGH */
+ case 1:
+ break;
+ default:
+ fprintf(stderr, "usage: rtctest [rtcdev]\n");
+ return 1;
+ }
+
+ fd = open(rtc, O_RDONLY);
+
+ if (fd == -1) {
+ perror(rtc);
+ exit(errno);
+ }
+
+ fprintf(stderr, "\n\t\t\tRTC Driver Test Example.\n\n");
+
+ /* Turn on update interrupts (one per second) */
+ retval = ioctl(fd, RTC_UIE_ON, 0);
+ if (retval == -1) {
+ if (errno == ENOTTY) {
+ fprintf(stderr,
+ "\n...Update IRQs not supported.\n");
+ goto test_READ;
+ }
+ perror("RTC_UIE_ON ioctl");
+ exit(errno);
+ }
+
+ fprintf(stderr, "Counting 5 update (1/sec) interrupts from reading %s:",
+ rtc);
+ fflush(stderr);
+ for (i=1; i<6; i++) {
+ /* This read will block */
+ retval = read(fd, &data, sizeof(unsigned long));
+ if (retval == -1) {
+ perror("read");
+ exit(errno);
+ }
+ fprintf(stderr, " %d",i);
+ fflush(stderr);
+ irqcount++;
+ }
+
+ fprintf(stderr, "\nAgain, from using select(2) on /dev/rtc:");
+ fflush(stderr);
+ for (i=1; i<6; i++) {
+ struct timeval tv = {5, 0}; /* 5 second timeout on select */
+ fd_set readfds;
+
+ FD_ZERO(&readfds);
+ FD_SET(fd, &readfds);
+ /* The select will wait until an RTC interrupt happens. */
+ retval = select(fd+1, &readfds, NULL, NULL, &tv);
+ if (retval == -1) {
+ perror("select");
+ exit(errno);
+ }
+ /* This read won't block unlike the select-less case above. */
+ retval = read(fd, &data, sizeof(unsigned long));
+ if (retval == -1) {
+ perror("read");
+ exit(errno);
+ }
+ fprintf(stderr, " %d",i);
+ fflush(stderr);
+ irqcount++;
+ }
+
+ /* Turn off update interrupts */
+ retval = ioctl(fd, RTC_UIE_OFF, 0);
+ if (retval == -1) {
+ perror("RTC_UIE_OFF ioctl");
+ exit(errno);
+ }
+
+test_READ:
+ /* Read the RTC time/date */
+ retval = ioctl(fd, RTC_RD_TIME, &rtc_tm);
+ if (retval == -1) {
+ perror("RTC_RD_TIME ioctl");
+ exit(errno);
+ }
+
+ fprintf(stderr, "\n\nCurrent RTC date/time is %d-%d-%d, %02d:%02d:%02d.\n",
+ rtc_tm.tm_mday, rtc_tm.tm_mon + 1, rtc_tm.tm_year + 1900,
+ rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec);
+
+ /* Set the alarm to 5 sec in the future, and check for rollover */
+ rtc_tm.tm_sec += 5;
+ if (rtc_tm.tm_sec >= 60) {
+ rtc_tm.tm_sec %= 60;
+ rtc_tm.tm_min++;
+ }
+ if (rtc_tm.tm_min == 60) {
+ rtc_tm.tm_min = 0;
+ rtc_tm.tm_hour++;
+ }
+ if (rtc_tm.tm_hour == 24)
+ rtc_tm.tm_hour = 0;
+
+ retval = ioctl(fd, RTC_ALM_SET, &rtc_tm);
+ if (retval == -1) {
+ if (errno == ENOTTY) {
+ fprintf(stderr,
+ "\n...Alarm IRQs not supported.\n");
+ goto test_PIE;
+ }
+ perror("RTC_ALM_SET ioctl");
+ exit(errno);
+ }
+
+ /* Read the current alarm settings */
+ retval = ioctl(fd, RTC_ALM_READ, &rtc_tm);
+ if (retval == -1) {
+ perror("RTC_ALM_READ ioctl");
+ exit(errno);
+ }
+
+ fprintf(stderr, "Alarm time now set to %02d:%02d:%02d.\n",
+ rtc_tm.tm_hour, rtc_tm.tm_min, rtc_tm.tm_sec);
+
+ /* Enable alarm interrupts */
+ retval = ioctl(fd, RTC_AIE_ON, 0);
+ if (retval == -1) {
+ perror("RTC_AIE_ON ioctl");
+ exit(errno);
+ }
+
+ fprintf(stderr, "Waiting 5 seconds for alarm...");
+ fflush(stderr);
+ /* This blocks until the alarm ring causes an interrupt */
+ retval = read(fd, &data, sizeof(unsigned long));
+ if (retval == -1) {
+ perror("read");
+ exit(errno);
+ }
+ irqcount++;
+ fprintf(stderr, " okay. Alarm rang.\n");
+
+ /* Disable alarm interrupts */
+ retval = ioctl(fd, RTC_AIE_OFF, 0);
+ if (retval == -1) {
+ perror("RTC_AIE_OFF ioctl");
+ exit(errno);
+ }
+
+test_PIE:
+ /* Read periodic IRQ rate */
+ retval = ioctl(fd, RTC_IRQP_READ, &tmp);
+ if (retval == -1) {
+ /* not all RTCs support periodic IRQs */
+ if (errno == ENOTTY) {
+ fprintf(stderr, "\nNo periodic IRQ support\n");
+ goto done;
+ }
+ perror("RTC_IRQP_READ ioctl");
+ exit(errno);
+ }
+ fprintf(stderr, "\nPeriodic IRQ rate is %ldHz.\n", tmp);
+
+ fprintf(stderr, "Counting 20 interrupts at:");
+ fflush(stderr);
+
+ /* The frequencies 128Hz, 256Hz, ... 8192Hz are only allowed for root. */
+ for (tmp=2; tmp<=64; tmp*=2) {
+
+ retval = ioctl(fd, RTC_IRQP_SET, tmp);
+ if (retval == -1) {
+ /* not all RTCs can change their periodic IRQ rate */
+ if (errno == ENOTTY) {
+ fprintf(stderr,
+ "\n...Periodic IRQ rate is fixed\n");
+ goto done;
+ }
+ perror("RTC_IRQP_SET ioctl");
+ exit(errno);
+ }
+
+ fprintf(stderr, "\n%ldHz:\t", tmp);
+ fflush(stderr);
+
+ /* Enable periodic interrupts */
+ retval = ioctl(fd, RTC_PIE_ON, 0);
+ if (retval == -1) {
+ perror("RTC_PIE_ON ioctl");
+ exit(errno);
+ }
+
+ for (i=1; i<21; i++) {
+ gettimeofday(&start, NULL);
+ /* This blocks */
+ retval = read(fd, &data, sizeof(unsigned long));
+ if (retval == -1) {
+ perror("read");
+ exit(errno);
+ }
+ gettimeofday(&end, NULL);
+ timersub(&end, &start, &diff);
+ if (diff.tv_sec > 0 ||
+ diff.tv_usec > ((1000000L / tmp) * 1.10)) {
+ fprintf(stderr, "\nPIE delta error: %ld.%06ld should be close to 0.%06ld\n",
+ diff.tv_sec, diff.tv_usec,
+ (1000000L / tmp));
+ fflush(stdout);
+ exit(-1);
+ }
+
+ fprintf(stderr, " %d",i);
+ fflush(stderr);
+ irqcount++;
+ }
+
+ /* Disable periodic interrupts */
+ retval = ioctl(fd, RTC_PIE_OFF, 0);
+ if (retval == -1) {
+ perror("RTC_PIE_OFF ioctl");
+ exit(errno);
+ }
+ }
+
+done:
+ fprintf(stderr, "\n\n\t\t\t *** Test complete ***\n");
+
+ close(fd);
+
+ return 0;
+}
--- /dev/null
+/* Time bounds setting test
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2012
+ * Licensed under the GPLv2
+ *
+ * NOTE: This is a meta-test which sets the time to edge cases then
+ * uses other tests to detect problems. Thus this test requires that
+ * the inconsistency-check and nanosleep tests be present in the same
+ * directory it is run from.
+ *
+ * To build:
+ * $ gcc set-2038.c -o set-2038 -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <time.h>
+#include <sys/time.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000LL
+
+#define KTIME_MAX ((long long)~((unsigned long long)1 << 63))
+#define KTIME_SEC_MAX (KTIME_MAX / NSEC_PER_SEC)
+
+#define YEAR_1901 (-0x7fffffffL)
+#define YEAR_1970 1
+#define YEAR_2038 0x7fffffffL /*overflows 32bit time_t */
+#define YEAR_2262 KTIME_SEC_MAX /*overflows 64bit ktime_t */
+#define YEAR_MAX ((long long)((1ULL<<63)-1)) /*overflows 64bit time_t */
+
+int is32bits(void)
+{
+ return (sizeof(long) == 4);
+}
+
+int settime(long long time)
+{
+ struct timeval now;
+ int ret;
+
+ now.tv_sec = (time_t)time;
+ now.tv_usec = 0;
+
+ ret = settimeofday(&now, NULL);
+
+ printf("Setting time to 0x%lx: %d\n", (long)time, ret);
+ return ret;
+}
+
+int do_tests(void)
+{
+ int ret;
+
+ ret = system("date");
+ ret = system("./inconsistency-check -c 0 -t 20");
+ ret |= system("./nanosleep");
+ ret |= system("./nsleep-lat");
+ return ret;
+
+}
+
+int main(int argc, char *argv[])
+{
+ int ret = 0;
+ int opt, dangerous = 0;
+ time_t start;
+
+ /* Process arguments */
+ while ((opt = getopt(argc, argv, "d")) != -1) {
+ switch (opt) {
+ case 'd':
+ dangerous = 1;
+ }
+ }
+
+ start = time(0);
+
+ /* First test that crazy values don't work */
+ if (!settime(YEAR_1901)) {
+ ret = -1;
+ goto out;
+ }
+ if (!settime(YEAR_MAX)) {
+ ret = -1;
+ goto out;
+ }
+ if (!is32bits() && !settime(YEAR_2262)) {
+ ret = -1;
+ goto out;
+ }
+
+ /* Now test behavior near edges */
+ settime(YEAR_1970);
+ ret = do_tests();
+ if (ret)
+ goto out;
+
+ settime(YEAR_2038 - 600);
+ ret = do_tests();
+ if (ret)
+ goto out;
+
+ /* The rest of the tests can blowup on 32bit systems */
+ if (is32bits() && !dangerous)
+ goto out;
+ /* Test rollover behavior 32bit edge */
+ settime(YEAR_2038 - 10);
+ ret = do_tests();
+ if (ret)
+ goto out;
+
+ settime(YEAR_2262 - 600);
+ ret = do_tests();
+
+out:
+ /* restore clock */
+ settime(start);
+ if (ret)
+ return ksft_exit_fail();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* Set tai offset
+ * by: John Stultz <john.stultz@linaro.org>
+ * (C) Copyright Linaro 2013
+ * Licensed under the GPLv2
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#include <unistd.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+int set_tai(int offset)
+{
+ struct timex tx;
+
+ memset(&tx, 0, sizeof(tx));
+
+ tx.modes = ADJ_TAI;
+ tx.constant = offset;
+
+ return adjtimex(&tx);
+}
+
+int get_tai(void)
+{
+ struct timex tx;
+
+ memset(&tx, 0, sizeof(tx));
+
+ adjtimex(&tx);
+ return tx.tai;
+}
+
+int main(int argc, char **argv)
+{
+ int i, ret;
+
+ ret = get_tai();
+ printf("tai offset started at %i\n", ret);
+
+ printf("Checking tai offsets can be properly set: ");
+ for (i = 1; i <= 60; i++) {
+ ret = set_tai(i);
+ ret = get_tai();
+ if (ret != i) {
+ printf("[FAILED] expected: %i got %i\n", i, ret);
+ return ksft_exit_fail();
+ }
+ }
+ printf("[OK]\n");
+ return ksft_exit_pass();
+}
--- /dev/null
+/* set_timer latency test
+ * John Stultz (john.stultz@linaro.org)
+ * (C) Copyright Linaro 2014
+ * Licensed under the GPLv2
+ *
+ * This test makes sure the set_timer api is correct
+ *
+ * To build:
+ * $ gcc set-timer-lat.c -o set-timer-lat -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <unistd.h>
+#include <time.h>
+#include <string.h>
+#include <signal.h>
+#include <stdlib.h>
+#include <pthread.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define CLOCK_REALTIME 0
+#define CLOCK_MONOTONIC 1
+#define CLOCK_PROCESS_CPUTIME_ID 2
+#define CLOCK_THREAD_CPUTIME_ID 3
+#define CLOCK_MONOTONIC_RAW 4
+#define CLOCK_REALTIME_COARSE 5
+#define CLOCK_MONOTONIC_COARSE 6
+#define CLOCK_BOOTTIME 7
+#define CLOCK_REALTIME_ALARM 8
+#define CLOCK_BOOTTIME_ALARM 9
+#define CLOCK_HWSPECIFIC 10
+#define CLOCK_TAI 11
+#define NR_CLOCKIDS 12
+
+
+#define NSEC_PER_SEC 1000000000ULL
+#define UNRESONABLE_LATENCY 40000000 /* 40ms in nanosecs */
+
+#define TIMER_SECS 1
+int alarmcount;
+int clock_id;
+struct timespec start_time;
+long long max_latency_ns;
+
+char *clockstring(int clockid)
+{
+ switch (clockid) {
+ case CLOCK_REALTIME:
+ return "CLOCK_REALTIME";
+ case CLOCK_MONOTONIC:
+ return "CLOCK_MONOTONIC";
+ case CLOCK_PROCESS_CPUTIME_ID:
+ return "CLOCK_PROCESS_CPUTIME_ID";
+ case CLOCK_THREAD_CPUTIME_ID:
+ return "CLOCK_THREAD_CPUTIME_ID";
+ case CLOCK_MONOTONIC_RAW:
+ return "CLOCK_MONOTONIC_RAW";
+ case CLOCK_REALTIME_COARSE:
+ return "CLOCK_REALTIME_COARSE";
+ case CLOCK_MONOTONIC_COARSE:
+ return "CLOCK_MONOTONIC_COARSE";
+ case CLOCK_BOOTTIME:
+ return "CLOCK_BOOTTIME";
+ case CLOCK_REALTIME_ALARM:
+ return "CLOCK_REALTIME_ALARM";
+ case CLOCK_BOOTTIME_ALARM:
+ return "CLOCK_BOOTTIME_ALARM";
+ case CLOCK_TAI:
+ return "CLOCK_TAI";
+ };
+ return "UNKNOWN_CLOCKID";
+}
+
+
+long long timespec_sub(struct timespec a, struct timespec b)
+{
+ long long ret = NSEC_PER_SEC * b.tv_sec + b.tv_nsec;
+
+ ret -= NSEC_PER_SEC * a.tv_sec + a.tv_nsec;
+ return ret;
+}
+
+
+void sigalarm(int signo)
+{
+ long long delta_ns;
+ struct timespec ts;
+
+ clock_gettime(clock_id, &ts);
+ alarmcount++;
+
+ delta_ns = timespec_sub(start_time, ts);
+ delta_ns -= NSEC_PER_SEC * TIMER_SECS * alarmcount;
+
+ if (delta_ns < 0)
+ printf("%s timer fired early: FAIL\n", clockstring(clock_id));
+
+ if (delta_ns > max_latency_ns)
+ max_latency_ns = delta_ns;
+}
+
+int do_timer(int clock_id, int flags)
+{
+ struct sigevent se;
+ timer_t tm1;
+ struct itimerspec its1, its2;
+ int err;
+
+ /* Set up timer: */
+ memset(&se, 0, sizeof(se));
+ se.sigev_notify = SIGEV_SIGNAL;
+ se.sigev_signo = SIGRTMAX;
+ se.sigev_value.sival_int = 0;
+
+ max_latency_ns = 0;
+ alarmcount = 0;
+
+ err = timer_create(clock_id, &se, &tm1);
+ if (err) {
+ if ((clock_id == CLOCK_REALTIME_ALARM) ||
+ (clock_id == CLOCK_BOOTTIME_ALARM)) {
+ printf("%-22s %s missing CAP_WAKE_ALARM? : [UNSUPPORTED]\n",
+ clockstring(clock_id),
+ flags ? "ABSTIME":"RELTIME");
+ return 0;
+ }
+ printf("%s - timer_create() failed\n", clockstring(clock_id));
+ return -1;
+ }
+
+ clock_gettime(clock_id, &start_time);
+ if (flags) {
+ its1.it_value = start_time;
+ its1.it_value.tv_sec += TIMER_SECS;
+ } else {
+ its1.it_value.tv_sec = TIMER_SECS;
+ its1.it_value.tv_nsec = 0;
+ }
+ its1.it_interval.tv_sec = TIMER_SECS;
+ its1.it_interval.tv_nsec = 0;
+
+ err = timer_settime(tm1, flags, &its1, &its2);
+ if (err) {
+ printf("%s - timer_settime() failed\n", clockstring(clock_id));
+ return -1;
+ }
+
+ while (alarmcount < 5)
+ sleep(1);
+
+ printf("%-22s %s max latency: %10lld ns : ",
+ clockstring(clock_id),
+ flags ? "ABSTIME":"RELTIME",
+ max_latency_ns);
+
+ timer_delete(tm1);
+ if (max_latency_ns < UNRESONABLE_LATENCY) {
+ printf("[OK]\n");
+ return 0;
+ }
+ printf("[FAILED]\n");
+ return -1;
+}
+
+int main(void)
+{
+ struct sigaction act;
+ int signum = SIGRTMAX;
+ int ret = 0;
+
+ /* Set up signal handler: */
+ sigfillset(&act.sa_mask);
+ act.sa_flags = 0;
+ act.sa_handler = sigalarm;
+ sigaction(signum, &act, NULL);
+
+ printf("Setting timers for every %i seconds\n", TIMER_SECS);
+ for (clock_id = 0; clock_id < NR_CLOCKIDS; clock_id++) {
+
+ if ((clock_id == CLOCK_PROCESS_CPUTIME_ID) ||
+ (clock_id == CLOCK_THREAD_CPUTIME_ID) ||
+ (clock_id == CLOCK_MONOTONIC_RAW) ||
+ (clock_id == CLOCK_REALTIME_COARSE) ||
+ (clock_id == CLOCK_MONOTONIC_COARSE) ||
+ (clock_id == CLOCK_HWSPECIFIC))
+ continue;
+
+ ret |= do_timer(clock_id, TIMER_ABSTIME);
+ ret |= do_timer(clock_id, 0);
+ }
+ if (ret)
+ return ksft_exit_fail();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* ADJ_FREQ Skew consistency test
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2012
+ * Licensed under the GPLv2
+ *
+ * NOTE: This is a meta-test which cranks the ADJ_FREQ knob back
+ * and forth and watches for consistency problems. Thus this test requires
+ * that the inconsistency-check tests be present in the same directory it
+ * is run from.
+ *
+ * To build:
+ * $ gcc skew_consistency.c -o skew_consistency -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <unistd.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <time.h>
+#include <sys/types.h>
+#include <sys/stat.h>
+#include <fcntl.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/wait.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000LL
+
+int main(int argv, char **argc)
+{
+ struct timex tx;
+ int ret, ppm;
+ pid_t pid;
+
+
+ printf("Running Asyncrhonous Frequency Changing Tests...\n");
+
+ pid = fork();
+ if (!pid)
+ return system("./inconsistency-check -c 1 -t 600");
+
+ ppm = 500;
+ ret = 0;
+
+ while (pid != waitpid(pid, &ret, WNOHANG)) {
+ ppm = -ppm;
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = ppm << 16;
+ adjtimex(&tx);
+ usleep(500000);
+ }
+
+ /* Set things back */
+ tx.modes = ADJ_FREQUENCY;
+ tx.offset = 0;
+ adjtimex(&tx);
+
+
+ if (ret) {
+ printf("[FAILED]\n");
+ return ksft_exit_fail();
+ }
+ printf("[OK]\n");
+ return ksft_exit_pass();
+}
--- /dev/null
+/* threadtest.c
+ * by: john stultz (johnstul@us.ibm.com)
+ * (C) Copyright IBM 2004, 2005, 2006, 2012
+ * Licensed under the GPLv2
+ *
+ * To build:
+ * $ gcc threadtest.c -o threadtest -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+#include <stdio.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/time.h>
+#include <pthread.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+
+/* serializes shared list access */
+pthread_mutex_t list_lock = PTHREAD_MUTEX_INITIALIZER;
+/* serializes console output */
+pthread_mutex_t print_lock = PTHREAD_MUTEX_INITIALIZER;
+
+
+#define MAX_THREADS 128
+#define LISTSIZE 128
+
+int done = 0;
+
+struct timespec global_list[LISTSIZE];
+int listcount = 0;
+
+
+void checklist(struct timespec *list, int size)
+{
+ int i, j;
+ struct timespec *a, *b;
+
+ /* scan the list */
+ for (i = 0; i < size-1; i++) {
+ a = &list[i];
+ b = &list[i+1];
+
+ /* look for any time inconsistencies */
+ if ((b->tv_sec <= a->tv_sec) &&
+ (b->tv_nsec < a->tv_nsec)) {
+
+ /* flag other threads */
+ done = 1;
+
+ /*serialize printing to avoid junky output*/
+ pthread_mutex_lock(&print_lock);
+
+ /* dump the list */
+ printf("\n");
+ for (j = 0; j < size; j++) {
+ if (j == i)
+ printf("---------------\n");
+ printf("%lu:%lu\n", list[j].tv_sec, list[j].tv_nsec);
+ if (j == i+1)
+ printf("---------------\n");
+ }
+ printf("[FAILED]\n");
+
+ pthread_mutex_unlock(&print_lock);
+ }
+ }
+}
+
+/* The shared thread shares a global list
+ * that each thread fills while holding the lock.
+ * This stresses clock syncronization across cpus.
+ */
+void *shared_thread(void *arg)
+{
+ while (!done) {
+ /* protect the list */
+ pthread_mutex_lock(&list_lock);
+
+ /* see if we're ready to check the list */
+ if (listcount >= LISTSIZE) {
+ checklist(global_list, LISTSIZE);
+ listcount = 0;
+ }
+ clock_gettime(CLOCK_MONOTONIC, &global_list[listcount++]);
+
+ pthread_mutex_unlock(&list_lock);
+ }
+ return NULL;
+}
+
+
+/* Each independent thread fills in its own
+ * list. This stresses clock_gettime() lock contention.
+ */
+void *independent_thread(void *arg)
+{
+ struct timespec my_list[LISTSIZE];
+ int count;
+
+ while (!done) {
+ /* fill the list */
+ for (count = 0; count < LISTSIZE; count++)
+ clock_gettime(CLOCK_MONOTONIC, &my_list[count]);
+ checklist(my_list, LISTSIZE);
+ }
+ return NULL;
+}
+
+#define DEFAULT_THREAD_COUNT 8
+#define DEFAULT_RUNTIME 30
+
+int main(int argc, char **argv)
+{
+ int thread_count, i;
+ time_t start, now, runtime;
+ char buf[255];
+ pthread_t pth[MAX_THREADS];
+ int opt;
+ void *tret;
+ int ret = 0;
+ void *(*thread)(void *) = shared_thread;
+
+ thread_count = DEFAULT_THREAD_COUNT;
+ runtime = DEFAULT_RUNTIME;
+
+ /* Process arguments */
+ while ((opt = getopt(argc, argv, "t:n:i")) != -1) {
+ switch (opt) {
+ case 't':
+ runtime = atoi(optarg);
+ break;
+ case 'n':
+ thread_count = atoi(optarg);
+ break;
+ case 'i':
+ thread = independent_thread;
+ printf("using independent threads\n");
+ break;
+ default:
+ printf("Usage: %s [-t <secs>] [-n <numthreads>] [-i]\n", argv[0]);
+ printf(" -t: time to run\n");
+ printf(" -n: number of threads\n");
+ printf(" -i: use independent threads\n");
+ return -1;
+ }
+ }
+
+ if (thread_count > MAX_THREADS)
+ thread_count = MAX_THREADS;
+
+
+ setbuf(stdout, NULL);
+
+ start = time(0);
+ strftime(buf, 255, "%a, %d %b %Y %T %z", localtime(&start));
+ printf("%s\n", buf);
+ printf("Testing consistency with %i threads for %ld seconds: ", thread_count, runtime);
+
+ /* spawn */
+ for (i = 0; i < thread_count; i++)
+ pthread_create(&pth[i], 0, thread, 0);
+
+ while (time(&now) < start + runtime) {
+ sleep(1);
+ if (done) {
+ ret = 1;
+ strftime(buf, 255, "%a, %d %b %Y %T %z", localtime(&now));
+ printf("%s\n", buf);
+ goto out;
+ }
+ }
+ printf("[OK]\n");
+ done = 1;
+
+out:
+ /* wait */
+ for (i = 0; i < thread_count; i++)
+ pthread_join(pth[i], &tret);
+
+ /* die */
+ if (ret)
+ ksft_exit_fail();
+ return ksft_exit_pass();
+}
--- /dev/null
+/* valid adjtimex test
+ * by: John Stultz <john.stultz@linaro.org>
+ * (C) Copyright Linaro 2015
+ * Licensed under the GPLv2
+ *
+ * This test validates adjtimex interface with valid
+ * and invalid test data.
+ *
+ * Usage: valid-adjtimex
+ *
+ * To build:
+ * $ gcc valid-adjtimex.c -o valid-adjtimex -lrt
+ *
+ * This program is free software: you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation, either version 2 of the License, or
+ * (at your option) any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <time.h>
+#include <sys/time.h>
+#include <sys/timex.h>
+#include <string.h>
+#include <signal.h>
+#include <unistd.h>
+#ifdef KTEST
+#include "../kselftest.h"
+#else
+static inline int ksft_exit_pass(void)
+{
+ exit(0);
+}
+static inline int ksft_exit_fail(void)
+{
+ exit(1);
+}
+#endif
+
+#define NSEC_PER_SEC 1000000000L
+
+/* clear NTP time_status & time_state */
+int clear_time_state(void)
+{
+ struct timex tx;
+ int ret;
+
+ tx.modes = ADJ_STATUS;
+ tx.status = 0;
+ ret = adjtimex(&tx);
+ return ret;
+}
+
+#define NUM_FREQ_VALID 32
+#define NUM_FREQ_OUTOFRANGE 4
+#define NUM_FREQ_INVALID 2
+
+long valid_freq[NUM_FREQ_VALID] = {
+ -499<<16,
+ -450<<16,
+ -400<<16,
+ -350<<16,
+ -300<<16,
+ -250<<16,
+ -200<<16,
+ -150<<16,
+ -100<<16,
+ -75<<16,
+ -50<<16,
+ -25<<16,
+ -10<<16,
+ -5<<16,
+ -1<<16,
+ -1000,
+ 1<<16,
+ 5<<16,
+ 10<<16,
+ 25<<16,
+ 50<<16,
+ 75<<16,
+ 100<<16,
+ 150<<16,
+ 200<<16,
+ 250<<16,
+ 300<<16,
+ 350<<16,
+ 400<<16,
+ 450<<16,
+ 499<<16,
+};
+
+long outofrange_freq[NUM_FREQ_OUTOFRANGE] = {
+ -1000<<16,
+ -550<<16,
+ 550<<16,
+ 1000<<16,
+};
+
+#define LONG_MAX (~0UL>>1)
+#define LONG_MIN (-LONG_MAX - 1)
+
+long invalid_freq[NUM_FREQ_INVALID] = {
+ LONG_MAX,
+ LONG_MIN,
+};
+
+int validate_freq(void)
+{
+ struct timex tx;
+ int ret, pass = 0;
+ int i;
+
+ clear_time_state();
+
+ memset(&tx, 0, sizeof(struct timex));
+ /* Set the leap second insert flag */
+
+ printf("Testing ADJ_FREQ... ");
+ for (i = 0; i < NUM_FREQ_VALID; i++) {
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = valid_freq[i];
+
+ ret = adjtimex(&tx);
+ if (ret < 0) {
+ printf("[FAIL]\n");
+ printf("Error: adjtimex(ADJ_FREQ, %ld - %ld ppm\n",
+ valid_freq[i], valid_freq[i]>>16);
+ pass = -1;
+ goto out;
+ }
+ tx.modes = 0;
+ ret = adjtimex(&tx);
+ if (tx.freq != valid_freq[i]) {
+ printf("Warning: freq value %ld not what we set it (%ld)!\n",
+ tx.freq, valid_freq[i]);
+ }
+ }
+ for (i = 0; i < NUM_FREQ_OUTOFRANGE; i++) {
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = outofrange_freq[i];
+
+ ret = adjtimex(&tx);
+ if (ret < 0) {
+ printf("[FAIL]\n");
+ printf("Error: adjtimex(ADJ_FREQ, %ld - %ld ppm\n",
+ outofrange_freq[i], outofrange_freq[i]>>16);
+ pass = -1;
+ goto out;
+ }
+ tx.modes = 0;
+ ret = adjtimex(&tx);
+ if (tx.freq == outofrange_freq[i]) {
+ printf("[FAIL]\n");
+ printf("ERROR: out of range value %ld actually set!\n",
+ tx.freq);
+ pass = -1;
+ goto out;
+ }
+ }
+
+
+ if (sizeof(long) == 8) { /* this case only applies to 64bit systems */
+ for (i = 0; i < NUM_FREQ_INVALID; i++) {
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = invalid_freq[i];
+ ret = adjtimex(&tx);
+ if (ret >= 0) {
+ printf("[FAIL]\n");
+ printf("Error: No failure on invalid ADJ_FREQUENCY %ld\n",
+ invalid_freq[i]);
+ pass = -1;
+ goto out;
+ }
+ }
+ }
+
+ printf("[OK]\n");
+out:
+ /* reset freq to zero */
+ tx.modes = ADJ_FREQUENCY;
+ tx.freq = 0;
+ ret = adjtimex(&tx);
+
+ return pass;
+}
+
+
+int main(int argc, char **argv)
+{
+ if (validate_freq())
+ return ksft_exit_fail();
+
+ return ksft_exit_pass();
+}
# No binaries, but make sure arg-less "make" doesn't trigger "run_tests"
all:
-run_tests: all
- ./test_user_copy.sh
+TEST_PROGS := test_user_copy.sh
+
+include ../lib.mk
# Makefile for vm selftests
-CC = $(CROSS_COMPILE)gcc
CFLAGS = -Wall
BINARIES = hugepage-mmap hugepage-shm map_hugetlb thuge-gen hugetlbfstest
BINARIES += transhuge-stress
%: %.c
$(CC) $(CFLAGS) -o $@ $^ -lrt
-run_tests: all
- @/bin/sh ./run_vmtests || (echo "vmtests: [FAIL]"; exit 1)
+TEST_PROGS := run_vmtests
+TEST_FILES := $(BINARIES)
+
+include ../lib.mk
clean:
$(RM) $(BINARIES)
--- /dev/null
+*_32
+*_64
--- /dev/null
+.PHONY: all all_32 all_64 check_build32 clean run_tests
+
+TARGETS_C_BOTHBITS := sigreturn
+
+BINARIES_32 := $(TARGETS_C_BOTHBITS:%=%_32)
+BINARIES_64 := $(TARGETS_C_BOTHBITS:%=%_64)
+
+CFLAGS := -O2 -g -std=gnu99 -pthread -Wall
+
+UNAME_P := $(shell uname -p)
+
+# Always build 32-bit tests
+all: all_32
+
+# If we're on a 64-bit host, build 64-bit tests as well
+ifeq ($(shell uname -p),x86_64)
+all: all_64
+endif
+
+all_32: check_build32 $(BINARIES_32)
+
+all_64: $(BINARIES_64)
+
+clean:
+ $(RM) $(BINARIES_32) $(BINARIES_64)
+
+run_tests:
+ ./run_x86_tests.sh
+
+$(TARGETS_C_BOTHBITS:%=%_32): %_32: %.c
+ $(CC) -m32 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl
+
+$(TARGETS_C_BOTHBITS:%=%_64): %_64: %.c
+ $(CC) -m64 -o $@ $(CFLAGS) $(EXTRA_CFLAGS) $^ -lrt -ldl
+
+check_build32:
+ @if ! $(CC) -m32 -o /dev/null trivial_32bit_program.c; then \
+ echo "Warning: you seem to have a broken 32-bit build" 2>&1; \
+ echo "environment. If you are using a Debian-like"; \
+ echo " distribution, try:"; \
+ echo ""; \
+ echo " apt-get install gcc-multilib libc6-i386 libc6-dev-i386"; \
+ echo ""; \
+ echo "If you are using a Fedora-like distribution, try:"; \
+ echo ""; \
+ echo " yum install glibc-devel.*i686"; \
+ exit 1; \
+ fi
--- /dev/null
+#!/bin/bash
+
+# This is deliberately minimal. IMO kselftests should provide a standard
+# script here.
+./sigreturn_32 || exit 1
+
+if [[ "$uname -p" -eq "x86_64" ]]; then
+ ./sigreturn_64 || exit 1
+fi
+
+exit 0
--- /dev/null
+/*
+ * sigreturn.c - tests for x86 sigreturn(2) and exit-to-userspace
+ * Copyright (c) 2014-2015 Andrew Lutomirski
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * This is a series of tests that exercises the sigreturn(2) syscall and
+ * the IRET / SYSRET paths in the kernel.
+ *
+ * For now, this focuses on the effects of unusual CS and SS values,
+ * and it has a bunch of tests to make sure that ESP/RSP is restored
+ * properly.
+ *
+ * The basic idea behind these tests is to raise(SIGUSR1) to create a
+ * sigcontext frame, plug in the values to be tested, and then return,
+ * which implicitly invokes sigreturn(2) and programs the user context
+ * as desired.
+ *
+ * For tests for which we expect sigreturn and the subsequent return to
+ * user mode to succeed, we return to a short trampoline that generates
+ * SIGTRAP so that the meat of the tests can be ordinary C code in a
+ * SIGTRAP handler.
+ *
+ * The inner workings of each test is documented below.
+ *
+ * Do not run on outdated, unpatched kernels at risk of nasty crashes.
+ */
+
+#define _GNU_SOURCE
+
+#include <sys/time.h>
+#include <time.h>
+#include <stdlib.h>
+#include <sys/syscall.h>
+#include <unistd.h>
+#include <stdio.h>
+#include <string.h>
+#include <inttypes.h>
+#include <sys/mman.h>
+#include <sys/signal.h>
+#include <sys/ucontext.h>
+#include <asm/ldt.h>
+#include <err.h>
+#include <setjmp.h>
+#include <stddef.h>
+#include <stdbool.h>
+#include <sys/ptrace.h>
+#include <sys/user.h>
+
+/*
+ * In principle, this test can run on Linux emulation layers (e.g.
+ * Illumos "LX branded zones"). Solaris-based kernels reserve LDT
+ * entries 0-5 for their own internal purposes, so start our LDT
+ * allocations above that reservation. (The tests don't pass on LX
+ * branded zones, but at least this lets them run.)
+ */
+#define LDT_OFFSET 6
+
+/* An aligned stack accessible through some of our segments. */
+static unsigned char stack16[65536] __attribute__((aligned(4096)));
+
+/*
+ * An aligned int3 instruction used as a trampoline. Some of the tests
+ * want to fish out their ss values, so this trampoline copies ss to eax
+ * before the int3.
+ */
+asm (".pushsection .text\n\t"
+ ".type int3, @function\n\t"
+ ".align 4096\n\t"
+ "int3:\n\t"
+ "mov %ss,%eax\n\t"
+ "int3\n\t"
+ ".size int3, . - int3\n\t"
+ ".align 4096, 0xcc\n\t"
+ ".popsection");
+extern char int3[4096];
+
+/*
+ * At startup, we prepapre:
+ *
+ * - ldt_nonexistent_sel: An LDT entry that doesn't exist (all-zero
+ * descriptor or out of bounds).
+ * - code16_sel: A 16-bit LDT code segment pointing to int3.
+ * - data16_sel: A 16-bit LDT data segment pointing to stack16.
+ * - npcode32_sel: A 32-bit not-present LDT code segment pointing to int3.
+ * - npdata32_sel: A 32-bit not-present LDT data segment pointing to stack16.
+ * - gdt_data16_idx: A 16-bit GDT data segment pointing to stack16.
+ * - gdt_npdata32_idx: A 32-bit not-present GDT data segment pointing to
+ * stack16.
+ *
+ * For no particularly good reason, xyz_sel is a selector value with the
+ * RPL and LDT bits filled in, whereas xyz_idx is just an index into the
+ * descriptor table. These variables will be zero if their respective
+ * segments could not be allocated.
+ */
+static unsigned short ldt_nonexistent_sel;
+static unsigned short code16_sel, data16_sel, npcode32_sel, npdata32_sel;
+
+static unsigned short gdt_data16_idx, gdt_npdata32_idx;
+
+static unsigned short GDT3(int idx)
+{
+ return (idx << 3) | 3;
+}
+
+static unsigned short LDT3(int idx)
+{
+ return (idx << 3) | 7;
+}
+
+/* Our sigaltstack scratch space. */
+static char altstack_data[SIGSTKSZ];
+
+static void sethandler(int sig, void (*handler)(int, siginfo_t *, void *),
+ int flags)
+{
+ struct sigaction sa;
+ memset(&sa, 0, sizeof(sa));
+ sa.sa_sigaction = handler;
+ sa.sa_flags = SA_SIGINFO | flags;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(sig, &sa, 0))
+ err(1, "sigaction");
+}
+
+static void clearhandler(int sig)
+{
+ struct sigaction sa;
+ memset(&sa, 0, sizeof(sa));
+ sa.sa_handler = SIG_DFL;
+ sigemptyset(&sa.sa_mask);
+ if (sigaction(sig, &sa, 0))
+ err(1, "sigaction");
+}
+
+static void add_ldt(const struct user_desc *desc, unsigned short *var,
+ const char *name)
+{
+ if (syscall(SYS_modify_ldt, 1, desc, sizeof(*desc)) == 0) {
+ *var = LDT3(desc->entry_number);
+ } else {
+ printf("[NOTE]\tFailed to create %s segment\n", name);
+ *var = 0;
+ }
+}
+
+static void setup_ldt(void)
+{
+ if ((unsigned long)stack16 > (1ULL << 32) - sizeof(stack16))
+ errx(1, "stack16 is too high\n");
+ if ((unsigned long)int3 > (1ULL << 32) - sizeof(int3))
+ errx(1, "int3 is too high\n");
+
+ ldt_nonexistent_sel = LDT3(LDT_OFFSET + 2);
+
+ const struct user_desc code16_desc = {
+ .entry_number = LDT_OFFSET + 0,
+ .base_addr = (unsigned long)int3,
+ .limit = 4095,
+ .seg_32bit = 0,
+ .contents = 2, /* Code, not conforming */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 0,
+ .useable = 0
+ };
+ add_ldt(&code16_desc, &code16_sel, "code16");
+
+ const struct user_desc data16_desc = {
+ .entry_number = LDT_OFFSET + 1,
+ .base_addr = (unsigned long)stack16,
+ .limit = 0xffff,
+ .seg_32bit = 0,
+ .contents = 0, /* Data, grow-up */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 0,
+ .useable = 0
+ };
+ add_ldt(&data16_desc, &data16_sel, "data16");
+
+ const struct user_desc npcode32_desc = {
+ .entry_number = LDT_OFFSET + 3,
+ .base_addr = (unsigned long)int3,
+ .limit = 4095,
+ .seg_32bit = 1,
+ .contents = 2, /* Code, not conforming */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 1,
+ .useable = 0
+ };
+ add_ldt(&npcode32_desc, &npcode32_sel, "npcode32");
+
+ const struct user_desc npdata32_desc = {
+ .entry_number = LDT_OFFSET + 4,
+ .base_addr = (unsigned long)stack16,
+ .limit = 0xffff,
+ .seg_32bit = 1,
+ .contents = 0, /* Data, grow-up */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 1,
+ .useable = 0
+ };
+ add_ldt(&npdata32_desc, &npdata32_sel, "npdata32");
+
+ struct user_desc gdt_data16_desc = {
+ .entry_number = -1,
+ .base_addr = (unsigned long)stack16,
+ .limit = 0xffff,
+ .seg_32bit = 0,
+ .contents = 0, /* Data, grow-up */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 0,
+ .useable = 0
+ };
+
+ if (syscall(SYS_set_thread_area, &gdt_data16_desc) == 0) {
+ /*
+ * This probably indicates vulnerability to CVE-2014-8133.
+ * Merely getting here isn't definitive, though, and we'll
+ * diagnose the problem for real later on.
+ */
+ printf("[WARN]\tset_thread_area allocated data16 at index %d\n",
+ gdt_data16_desc.entry_number);
+ gdt_data16_idx = gdt_data16_desc.entry_number;
+ } else {
+ printf("[OK]\tset_thread_area refused 16-bit data\n");
+ }
+
+ struct user_desc gdt_npdata32_desc = {
+ .entry_number = -1,
+ .base_addr = (unsigned long)stack16,
+ .limit = 0xffff,
+ .seg_32bit = 1,
+ .contents = 0, /* Data, grow-up */
+ .read_exec_only = 0,
+ .limit_in_pages = 0,
+ .seg_not_present = 1,
+ .useable = 0
+ };
+
+ if (syscall(SYS_set_thread_area, &gdt_npdata32_desc) == 0) {
+ /*
+ * As a hardening measure, newer kernels don't allow this.
+ */
+ printf("[WARN]\tset_thread_area allocated npdata32 at index %d\n",
+ gdt_npdata32_desc.entry_number);
+ gdt_npdata32_idx = gdt_npdata32_desc.entry_number;
+ } else {
+ printf("[OK]\tset_thread_area refused 16-bit data\n");
+ }
+}
+
+/* State used by our signal handlers. */
+static gregset_t initial_regs, requested_regs, resulting_regs;
+
+/* Instructions for the SIGUSR1 handler. */
+static volatile unsigned short sig_cs, sig_ss;
+static volatile sig_atomic_t sig_trapped, sig_err, sig_trapno;
+
+/* Abstractions for some 32-bit vs 64-bit differences. */
+#ifdef __x86_64__
+# define REG_IP REG_RIP
+# define REG_SP REG_RSP
+# define REG_AX REG_RAX
+
+struct selectors {
+ unsigned short cs, gs, fs, ss;
+};
+
+static unsigned short *ssptr(ucontext_t *ctx)
+{
+ struct selectors *sels = (void *)&ctx->uc_mcontext.gregs[REG_CSGSFS];
+ return &sels->ss;
+}
+
+static unsigned short *csptr(ucontext_t *ctx)
+{
+ struct selectors *sels = (void *)&ctx->uc_mcontext.gregs[REG_CSGSFS];
+ return &sels->cs;
+}
+#else
+# define REG_IP REG_EIP
+# define REG_SP REG_ESP
+# define REG_AX REG_EAX
+
+static greg_t *ssptr(ucontext_t *ctx)
+{
+ return &ctx->uc_mcontext.gregs[REG_SS];
+}
+
+static greg_t *csptr(ucontext_t *ctx)
+{
+ return &ctx->uc_mcontext.gregs[REG_CS];
+}
+#endif
+
+/* Number of errors in the current test case. */
+static volatile sig_atomic_t nerrs;
+
+/*
+ * SIGUSR1 handler. Sets CS and SS as requested and points IP to the
+ * int3 trampoline. Sets SP to a large known value so that we can see
+ * whether the value round-trips back to user mode correctly.
+ */
+static void sigusr1(int sig, siginfo_t *info, void *ctx_void)
+{
+ ucontext_t *ctx = (ucontext_t*)ctx_void;
+
+ memcpy(&initial_regs, &ctx->uc_mcontext.gregs, sizeof(gregset_t));
+
+ *csptr(ctx) = sig_cs;
+ *ssptr(ctx) = sig_ss;
+
+ ctx->uc_mcontext.gregs[REG_IP] =
+ sig_cs == code16_sel ? 0 : (unsigned long)&int3;
+ ctx->uc_mcontext.gregs[REG_SP] = (unsigned long)0x8badf00d5aadc0deULL;
+ ctx->uc_mcontext.gregs[REG_AX] = 0;
+
+ memcpy(&requested_regs, &ctx->uc_mcontext.gregs, sizeof(gregset_t));
+ requested_regs[REG_AX] = *ssptr(ctx); /* The asm code does this. */
+
+ return;
+}
+
+/*
+ * Called after a successful sigreturn. Restores our state so that
+ * the original raise(SIGUSR1) returns.
+ */
+static void sigtrap(int sig, siginfo_t *info, void *ctx_void)
+{
+ ucontext_t *ctx = (ucontext_t*)ctx_void;
+
+ sig_err = ctx->uc_mcontext.gregs[REG_ERR];
+ sig_trapno = ctx->uc_mcontext.gregs[REG_TRAPNO];
+
+ unsigned short ss;
+ asm ("mov %%ss,%0" : "=r" (ss));
+
+ greg_t asm_ss = ctx->uc_mcontext.gregs[REG_AX];
+ if (asm_ss != sig_ss && sig == SIGTRAP) {
+ /* Sanity check failure. */
+ printf("[FAIL]\tSIGTRAP: ss = %hx, frame ss = %hx, ax = %llx\n",
+ ss, *ssptr(ctx), (unsigned long long)asm_ss);
+ nerrs++;
+ }
+
+ memcpy(&resulting_regs, &ctx->uc_mcontext.gregs, sizeof(gregset_t));
+ memcpy(&ctx->uc_mcontext.gregs, &initial_regs, sizeof(gregset_t));
+
+ sig_trapped = sig;
+}
+
+/*
+ * Checks a given selector for its code bitness or returns -1 if it's not
+ * a usable code segment selector.
+ */
+int cs_bitness(unsigned short cs)
+{
+ uint32_t valid = 0, ar;
+ asm ("lar %[cs], %[ar]\n\t"
+ "jnz 1f\n\t"
+ "mov $1, %[valid]\n\t"
+ "1:"
+ : [ar] "=r" (ar), [valid] "+rm" (valid)
+ : [cs] "r" (cs));
+
+ if (!valid)
+ return -1;
+
+ bool db = (ar & (1 << 22));
+ bool l = (ar & (1 << 21));
+
+ if (!(ar & (1<<11)))
+ return -1; /* Not code. */
+
+ if (l && !db)
+ return 64;
+ else if (!l && db)
+ return 32;
+ else if (!l && !db)
+ return 16;
+ else
+ return -1; /* Unknown bitness. */
+}
+
+/* Finds a usable code segment of the requested bitness. */
+int find_cs(int bitness)
+{
+ unsigned short my_cs;
+
+ asm ("mov %%cs,%0" : "=r" (my_cs));
+
+ if (cs_bitness(my_cs) == bitness)
+ return my_cs;
+ if (cs_bitness(my_cs + (2 << 3)) == bitness)
+ return my_cs + (2 << 3);
+ if (my_cs > (2<<3) && cs_bitness(my_cs - (2 << 3)) == bitness)
+ return my_cs - (2 << 3);
+ if (cs_bitness(code16_sel) == bitness)
+ return code16_sel;
+
+ printf("[WARN]\tCould not find %d-bit CS\n", bitness);
+ return -1;
+}
+
+static int test_valid_sigreturn(int cs_bits, bool use_16bit_ss, int force_ss)
+{
+ int cs = find_cs(cs_bits);
+ if (cs == -1) {
+ printf("[SKIP]\tCode segment unavailable for %d-bit CS, %d-bit SS\n",
+ cs_bits, use_16bit_ss ? 16 : 32);
+ return 0;
+ }
+
+ if (force_ss != -1) {
+ sig_ss = force_ss;
+ } else {
+ if (use_16bit_ss) {
+ if (!data16_sel) {
+ printf("[SKIP]\tData segment unavailable for %d-bit CS, 16-bit SS\n",
+ cs_bits);
+ return 0;
+ }
+ sig_ss = data16_sel;
+ } else {
+ asm volatile ("mov %%ss,%0" : "=r" (sig_ss));
+ }
+ }
+
+ sig_cs = cs;
+
+ printf("[RUN]\tValid sigreturn: %d-bit CS (%hx), %d-bit SS (%hx%s)\n",
+ cs_bits, sig_cs, use_16bit_ss ? 16 : 32, sig_ss,
+ (sig_ss & 4) ? "" : ", GDT");
+
+ raise(SIGUSR1);
+
+ nerrs = 0;
+
+ /*
+ * Check that each register had an acceptable value when the
+ * int3 trampoline was invoked.
+ */
+ for (int i = 0; i < NGREG; i++) {
+ greg_t req = requested_regs[i], res = resulting_regs[i];
+ if (i == REG_TRAPNO || i == REG_IP)
+ continue; /* don't care */
+ if (i == REG_SP) {
+ printf("\tSP: %llx -> %llx\n", (unsigned long long)req,
+ (unsigned long long)res);
+
+ /*
+ * In many circumstances, the high 32 bits of rsp
+ * are zeroed. For example, we could be a real
+ * 32-bit program, or we could hit any of a number
+ * of poorly-documented IRET or segmented ESP
+ * oddities. If this happens, it's okay.
+ */
+ if (res == (req & 0xFFFFFFFF))
+ continue; /* OK; not expected to work */
+ }
+
+ bool ignore_reg = false;
+#if __i386__
+ if (i == REG_UESP)
+ ignore_reg = true;
+#else
+ if (i == REG_CSGSFS) {
+ struct selectors *req_sels =
+ (void *)&requested_regs[REG_CSGSFS];
+ struct selectors *res_sels =
+ (void *)&resulting_regs[REG_CSGSFS];
+ if (req_sels->cs != res_sels->cs) {
+ printf("[FAIL]\tCS mismatch: requested 0x%hx; got 0x%hx\n",
+ req_sels->cs, res_sels->cs);
+ nerrs++;
+ }
+
+ if (req_sels->ss != res_sels->ss) {
+ printf("[FAIL]\tSS mismatch: requested 0x%hx; got 0x%hx\n",
+ req_sels->ss, res_sels->ss);
+ nerrs++;
+ }
+
+ continue;
+ }
+#endif
+
+ /* Sanity check on the kernel */
+ if (i == REG_AX && requested_regs[i] != resulting_regs[i]) {
+ printf("[FAIL]\tAX (saved SP) mismatch: requested 0x%llx; got 0x%llx\n",
+ (unsigned long long)requested_regs[i],
+ (unsigned long long)resulting_regs[i]);
+ nerrs++;
+ continue;
+ }
+
+ if (requested_regs[i] != resulting_regs[i] && !ignore_reg) {
+ /*
+ * SP is particularly interesting here. The
+ * usual cause of failures is that we hit the
+ * nasty IRET case of returning to a 16-bit SS,
+ * in which case bits 16:31 of the *kernel*
+ * stack pointer persist in ESP.
+ */
+ printf("[FAIL]\tReg %d mismatch: requested 0x%llx; got 0x%llx\n",
+ i, (unsigned long long)requested_regs[i],
+ (unsigned long long)resulting_regs[i]);
+ nerrs++;
+ }
+ }
+
+ if (nerrs == 0)
+ printf("[OK]\tall registers okay\n");
+
+ return nerrs;
+}
+
+static int test_bad_iret(int cs_bits, unsigned short ss, int force_cs)
+{
+ int cs = force_cs == -1 ? find_cs(cs_bits) : force_cs;
+ if (cs == -1)
+ return 0;
+
+ sig_cs = cs;
+ sig_ss = ss;
+
+ printf("[RUN]\t%d-bit CS (%hx), bogus SS (%hx)\n",
+ cs_bits, sig_cs, sig_ss);
+
+ sig_trapped = 0;
+ raise(SIGUSR1);
+ if (sig_trapped) {
+ char errdesc[32] = "";
+ if (sig_err) {
+ const char *src = (sig_err & 1) ? " EXT" : "";
+ const char *table;
+ if ((sig_err & 0x6) == 0x0)
+ table = "GDT";
+ else if ((sig_err & 0x6) == 0x4)
+ table = "LDT";
+ else if ((sig_err & 0x6) == 0x2)
+ table = "IDT";
+ else
+ table = "???";
+
+ sprintf(errdesc, "%s%s index %d, ",
+ table, src, sig_err >> 3);
+ }
+
+ char trapname[32];
+ if (sig_trapno == 13)
+ strcpy(trapname, "GP");
+ else if (sig_trapno == 11)
+ strcpy(trapname, "NP");
+ else if (sig_trapno == 12)
+ strcpy(trapname, "SS");
+ else if (sig_trapno == 32)
+ strcpy(trapname, "IRET"); /* X86_TRAP_IRET */
+ else
+ sprintf(trapname, "%d", sig_trapno);
+
+ printf("[OK]\tGot #%s(0x%lx) (i.e. %s%s)\n",
+ trapname, (unsigned long)sig_err,
+ errdesc, strsignal(sig_trapped));
+ return 0;
+ } else {
+ printf("[FAIL]\tDid not get SIGSEGV\n");
+ return 1;
+ }
+}
+
+int main()
+{
+ int total_nerrs = 0;
+ unsigned short my_cs, my_ss;
+
+ asm volatile ("mov %%cs,%0" : "=r" (my_cs));
+ asm volatile ("mov %%ss,%0" : "=r" (my_ss));
+ setup_ldt();
+
+ stack_t stack = {
+ .ss_sp = altstack_data,
+ .ss_size = SIGSTKSZ,
+ };
+ if (sigaltstack(&stack, NULL) != 0)
+ err(1, "sigaltstack");
+
+ sethandler(SIGUSR1, sigusr1, 0);
+ sethandler(SIGTRAP, sigtrap, SA_ONSTACK);
+
+ /* Easy cases: return to a 32-bit SS in each possible CS bitness. */
+ total_nerrs += test_valid_sigreturn(64, false, -1);
+ total_nerrs += test_valid_sigreturn(32, false, -1);
+ total_nerrs += test_valid_sigreturn(16, false, -1);
+
+ /*
+ * Test easy espfix cases: return to a 16-bit LDT SS in each possible
+ * CS bitness. NB: with a long mode CS, the SS bitness is irrelevant.
+ *
+ * This catches the original missing-espfix-on-64-bit-kernels issue
+ * as well as CVE-2014-8134.
+ */
+ total_nerrs += test_valid_sigreturn(64, true, -1);
+ total_nerrs += test_valid_sigreturn(32, true, -1);
+ total_nerrs += test_valid_sigreturn(16, true, -1);
+
+ if (gdt_data16_idx) {
+ /*
+ * For performance reasons, Linux skips espfix if SS points
+ * to the GDT. If we were able to allocate a 16-bit SS in
+ * the GDT, see if it leaks parts of the kernel stack pointer.
+ *
+ * This tests for CVE-2014-8133.
+ */
+ total_nerrs += test_valid_sigreturn(64, true,
+ GDT3(gdt_data16_idx));
+ total_nerrs += test_valid_sigreturn(32, true,
+ GDT3(gdt_data16_idx));
+ total_nerrs += test_valid_sigreturn(16, true,
+ GDT3(gdt_data16_idx));
+ }
+
+ /*
+ * We're done testing valid sigreturn cases. Now we test states
+ * for which sigreturn itself will succeed but the subsequent
+ * entry to user mode will fail.
+ *
+ * Depending on the failure mode and the kernel bitness, these
+ * entry failures can generate SIGSEGV, SIGBUS, or SIGILL.
+ */
+ clearhandler(SIGTRAP);
+ sethandler(SIGSEGV, sigtrap, SA_ONSTACK);
+ sethandler(SIGBUS, sigtrap, SA_ONSTACK);
+ sethandler(SIGILL, sigtrap, SA_ONSTACK); /* 32-bit kernels do this */
+
+ /* Easy failures: invalid SS, resulting in #GP(0) */
+ test_bad_iret(64, ldt_nonexistent_sel, -1);
+ test_bad_iret(32, ldt_nonexistent_sel, -1);
+ test_bad_iret(16, ldt_nonexistent_sel, -1);
+
+ /* These fail because SS isn't a data segment, resulting in #GP(SS) */
+ test_bad_iret(64, my_cs, -1);
+ test_bad_iret(32, my_cs, -1);
+ test_bad_iret(16, my_cs, -1);
+
+ /* Try to return to a not-present code segment, triggering #NP(SS). */
+ test_bad_iret(32, my_ss, npcode32_sel);
+
+ /*
+ * Try to return to a not-present but otherwise valid data segment.
+ * This will cause IRET to fail with #SS on the espfix stack. This
+ * exercises CVE-2014-9322.
+ *
+ * Note that, if espfix is enabled, 64-bit Linux will lose track
+ * of the actual cause of failure and report #GP(0) instead.
+ * This would be very difficult for Linux to avoid, because
+ * espfix64 causes IRET failures to be promoted to #DF, so the
+ * original exception frame is never pushed onto the stack.
+ */
+ test_bad_iret(32, npdata32_sel, -1);
+
+ /*
+ * Try to return to a not-present but otherwise valid data
+ * segment without invoking espfix. Newer kernels don't allow
+ * this to happen in the first place. On older kernels, though,
+ * this can trigger CVE-2014-9322.
+ */
+ if (gdt_npdata32_idx)
+ test_bad_iret(32, GDT3(gdt_npdata32_idx), -1);
+
+ return total_nerrs ? 1 : 0;
+}
--- /dev/null
+/*
+ * Trivial program to check that we have a valid 32-bit build environment.
+ * Copyright (c) 2015 Andy Lutomirski
+ * GPL v2
+ */
+
+#include <stdio.h>
+
+int main()
+{
+ printf("\n");
+
+ return 0;
+}
BINDIR=usr/bin
WARNFLAGS=-Wall -Wshadow -W -Wformat -Wimplicit-function-declaration -Wimplicit-int
-CFLAGS= -O1 ${WARNFLAGS} -fstack-protector
-CC=gcc
+CFLAGS+= -O1 ${WARNFLAGS} -fstack-protector
+CC=$(CROSS_COMPILE)gcc
CFLAGS+=-D VERSION=\"$(VERSION)\"
LDFLAGS+=
CONFIG_FILE=
CONFIG_PATH=
+# Static builds might require -ltinfo, for instance
+ifneq ($(findstring -static, $(LDFLAGS)),)
+STATIC := --static
+endif
+
+TMON_LIBS=-lm -lpthread
+TMON_LIBS += $(shell pkg-config --libs $(STATIC) panelw ncursesw 2> /dev/null || \
+ pkg-config --libs $(STATIC) panel ncurses 2> /dev/null || \
+ echo -lpanel -lncurses)
OBJS = tmon.o tui.o sysfs.o pid.o
OBJS +=
tmon: $(OBJS) Makefile tmon.h
- $(CC) ${CFLAGS} $(LDFLAGS) $(OBJS) -o $(TARGET) -lm -lpanel -lncursesw -ltinfo -lpthread
+ $(CC) $(CFLAGS) $(LDFLAGS) $(OBJS) -o $(TARGET) $(TMON_LIBS)
valgrind: tmon
sudo valgrind -v --track-origins=yes --tool=memcheck --leak-check=yes --show-reachable=yes --num-callers=20 --track-fds=yes ./$(TARGET) 1> /dev/null
.PP
The \fB-t --time-interval\fP option sets the polling interval in seconds
.PP
+The \fB-T --target-temp\fP option sets the initial target temperature
+.PP
The \fB-v --version\fP option shows the version of \fBtmon \fP
.PP
The \fB-z --zone\fP option sets the target therma zone instance to be controlled
printf(" -h, --help show this help message\n");
printf(" -l, --log log data to /var/tmp/tmon.log\n");
printf(" -t, --time-interval sampling time interval, > 1 sec.\n");
+ printf(" -T, --target-temp initial target temperature\n");
printf(" -v, --version show version\n");
printf(" -z, --zone target thermal zone id\n");
{ "control", 1, NULL, 'c' },
{ "daemon", 0, NULL, 'd' },
{ "time-interval", 1, NULL, 't' },
+ { "target-temp", 1, NULL, 'T' },
{ "log", 0, NULL, 'l' },
{ "help", 0, NULL, 'h' },
{ "version", 0, NULL, 'v' },
{
int err = 0;
int id2 = 0, c;
- double yk = 0.0; /* controller output */
+ double yk = 0.0, temp; /* controller output */
int target_tz_index;
if (geteuid() != 0) {
exit(EXIT_FAILURE);
}
- while ((c = getopt_long(argc, argv, "c:dlht:vgz:", opts, &id2)) != -1) {
+ while ((c = getopt_long(argc, argv, "c:dlht:T:vgz:", opts, &id2)) != -1) {
switch (c) {
case 'c':
no_control = 0;
if (ticktime < 1)
ticktime = 1;
break;
+ case 'T':
+ temp = strtod(optarg, NULL);
+ if (temp < 0) {
+ fprintf(stderr, "error: temperature must be positive\n");
+ return 1;
+ }
+ target_temp_user = temp;
+ break;
case 'l':
printf("Logging data to /var/tmp/tmon.log\n");
logging = 1;
#include "tmon.h"
+#define min(x, y) ({ \
+ typeof(x) _min1 = (x); \
+ typeof(y) _min2 = (y); \
+ (void) (&_min1 == &_min2); \
+ _min1 < _min2 ? _min1 : _min2; })
+
+#define max(x, y) ({ \
+ typeof(x) _max1 = (x); \
+ typeof(y) _max2 = (y); \
+ (void) (&_max1 == &_max2); \
+ _max1 > _max2 ? _max1 : _max2; })
+
static PANEL *data_panel;
static PANEL *dialogue_panel;
static PANEL *top;
wrefresh(status_bar_window);
}
+/* wrap at 5 */
+#define DIAG_DEV_ROWS 5
+/*
+ * list cooling devices + "set temp" entry; wraps after 5 rows, if they fit
+ */
+static int diag_dev_rows(void)
+{
+ int entries = ptdata.nr_cooling_dev + 1;
+ int rows = max(DIAG_DEV_ROWS, (entries + 1) / 2);
+ return min(rows, entries);
+}
+
void setup_windows(void)
{
int y_begin = 1;
* dialogue window is a pop-up, when needed it lays on top of cdev win
*/
- dialogue_window = subwin(stdscr, ptdata.nr_cooling_dev+5, maxx-50,
+ dialogue_window = subwin(stdscr, diag_dev_rows() + 5, maxx-50,
DIAG_Y, DIAG_X);
thermal_data_window = subwin(stdscr, ptdata.nr_tz_sensor *
}
const char DIAG_TITLE[] = "[ TUNABLES ]";
-#define DIAG_DEV_ROWS 5
void show_dialogue(void)
{
int j, x = 0, y = 0;
+ int rows, cols;
WINDOW *w = dialogue_window;
if (tui_disabled || !w)
return;
+ getmaxyx(w, rows, cols);
+
+ /* Silence compiler 'unused' warnings */
+ (void)cols;
+
werase(w);
box(w, 0, 0);
mvwprintw(w, 0, maxx/4, DIAG_TITLE);
/* list all the available tunables */
for (j = 0; j <= ptdata.nr_cooling_dev; j++) {
- y = j % DIAG_DEV_ROWS;
+ y = j % diag_dev_rows();
if (y == 0 && j != 0)
x += 20;
if (j == ptdata.nr_cooling_dev)
ptdata.cdi[j].type, ptdata.cdi[j].instance);
}
wattron(w, A_BOLD);
- mvwprintw(w, DIAG_DEV_ROWS+1, 1, "Enter Choice [A-Z]?");
+ mvwprintw(w, diag_dev_rows()+1, 1, "Enter Choice [A-Z]?");
wattroff(w, A_BOLD);
- /* y size of dialogue win is nr cdev + 5, so print legend
- * at the bottom line
- */
- mvwprintw(w, ptdata.nr_cooling_dev+3, 1,
+ /* print legend at the bottom line */
+ mvwprintw(w, rows - 2, 1,
"Legend: A=Active, P=Passive, C=Critical");
wrefresh(dialogue_window);
snprintf(buf, sizeof(buf), "New Value for %.10s-%2d: ",
ptdata.cdi[cdev_id].type,
ptdata.cdi[cdev_id].instance);
- write_dialogue_win(buf, DIAG_DEV_ROWS+2, 2);
+ write_dialogue_win(buf, diag_dev_rows() + 2, 2);
handle_input_val(cdev_id);
} else {
snprintf(buf, sizeof(buf), "Invalid selection %d", ch);
return IRQ_HANDLED;
}
+/*
+ * Work function for handling the backup timer that we schedule when a vcpu is
+ * no longer running, but had a timer programmed to fire in the future.
+ */
static void kvm_timer_inject_irq_work(struct work_struct *work)
{
struct kvm_vcpu *vcpu;
vcpu = container_of(work, struct kvm_vcpu, arch.timer_cpu.expired);
vcpu->arch.timer_cpu.armed = false;
- kvm_timer_inject_irq(vcpu);
+
+ /*
+ * If the vcpu is blocked we want to wake it up so that it will see
+ * the timer has expired when entering the guest.
+ */
+ kvm_vcpu_kick(vcpu);
}
static enum hrtimer_restart kvm_timer_expire(struct hrtimer *hrt)
return HRTIMER_NORESTART;
}
+bool kvm_timer_should_fire(struct kvm_vcpu *vcpu)
+{
+ struct arch_timer_cpu *timer = &vcpu->arch.timer_cpu;
+ cycle_t cval, now;
+
+ if ((timer->cntv_ctl & ARCH_TIMER_CTRL_IT_MASK) ||
+ !(timer->cntv_ctl & ARCH_TIMER_CTRL_ENABLE))
+ return false;
+
+ cval = timer->cntv_cval;
+ now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+
+ return cval <= now;
+}
+
/**
* kvm_timer_flush_hwstate - prepare to move the virt timer to the cpu
* @vcpu: The vcpu pointer
* populate the CPU timer again.
*/
timer_disarm(timer);
+
+ /*
+ * If the timer expired while we were not scheduled, now is the time
+ * to inject it.
+ */
+ if (kvm_timer_should_fire(vcpu))
+ kvm_timer_inject_irq(vcpu);
}
/**
cycle_t cval, now;
u64 ns;
- if ((timer->cntv_ctl & ARCH_TIMER_CTRL_IT_MASK) ||
- !(timer->cntv_ctl & ARCH_TIMER_CTRL_ENABLE))
- return;
-
- cval = timer->cntv_cval;
- now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
-
BUG_ON(timer_is_armed(timer));
- if (cval <= now) {
+ if (kvm_timer_should_fire(vcpu)) {
/*
* Timer has already expired while we were not
* looking. Inject the interrupt and carry on.
return;
}
+ cval = timer->cntv_cval;
+ now = kvm_phys_timer_read() - vcpu->kvm->arch.timer.cntvoff;
+
ns = cyclecounter_cyc2ns(timecounter->cc, cval - now, timecounter->mask,
&timecounter->frac);
timer_arm(timer, ns);
vcpu->vcpu_id);
}
+static bool handle_mmio_set_active_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return vgic_handle_set_active_reg(vcpu->kvm, mmio, offset,
+ vcpu->vcpu_id);
+}
+
+static bool handle_mmio_clear_active_reg(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ return vgic_handle_clear_active_reg(vcpu->kvm, mmio, offset,
+ vcpu->vcpu_id);
+}
+
static bool handle_mmio_priority_reg(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
return write_set_clear_sgi_pend_reg(vcpu, mmio, offset, false);
}
-static const struct kvm_mmio_range vgic_dist_ranges[] = {
+static const struct vgic_io_range vgic_dist_ranges[] = {
{
.base = GIC_DIST_CTRL,
.len = 12,
.base = GIC_DIST_ACTIVE_SET,
.len = VGIC_MAX_IRQS / 8,
.bits_per_irq = 1,
- .handle_mmio = handle_mmio_raz_wi,
+ .handle_mmio = handle_mmio_set_active_reg,
},
{
.base = GIC_DIST_ACTIVE_CLEAR,
.len = VGIC_MAX_IRQS / 8,
.bits_per_irq = 1,
- .handle_mmio = handle_mmio_raz_wi,
+ .handle_mmio = handle_mmio_clear_active_reg,
},
{
.base = GIC_DIST_PRI,
{}
};
-static bool vgic_v2_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
-{
- unsigned long base = vcpu->kvm->arch.vgic.vgic_dist_base;
-
- if (!is_in_range(mmio->phys_addr, mmio->len, base,
- KVM_VGIC_V2_DIST_SIZE))
- return false;
-
- /* GICv2 does not support accesses wider than 32 bits */
- if (mmio->len > 4) {
- kvm_inject_dabt(vcpu, mmio->phys_addr);
- return true;
- }
-
- return vgic_handle_mmio_range(vcpu, run, mmio, vgic_dist_ranges, base);
-}
-
static void vgic_dispatch_sgi(struct kvm_vcpu *vcpu, u32 reg)
{
struct kvm *kvm = vcpu->kvm;
static int vgic_v2_map_resources(struct kvm *kvm,
const struct vgic_params *params)
{
+ struct vgic_dist *dist = &kvm->arch.vgic;
int ret = 0;
if (!irqchip_in_kernel(kvm))
if (vgic_ready(kvm))
goto out;
- if (IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_dist_base) ||
- IS_VGIC_ADDR_UNDEF(kvm->arch.vgic.vgic_cpu_base)) {
+ if (IS_VGIC_ADDR_UNDEF(dist->vgic_dist_base) ||
+ IS_VGIC_ADDR_UNDEF(dist->vgic_cpu_base)) {
kvm_err("Need to set vgic cpu and dist addresses first\n");
ret = -ENXIO;
goto out;
}
+ vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+ KVM_VGIC_V2_DIST_SIZE,
+ vgic_dist_ranges, -1, &dist->dist_iodev);
+
/*
* Initialize the vgic if this hasn't already been done on demand by
* accessing the vgic state from userspace.
ret = vgic_init(kvm);
if (ret) {
kvm_err("Unable to allocate maps\n");
- goto out;
+ goto out_unregister;
}
- ret = kvm_phys_addr_ioremap(kvm, kvm->arch.vgic.vgic_cpu_base,
+ ret = kvm_phys_addr_ioremap(kvm, dist->vgic_cpu_base,
params->vcpu_base, KVM_VGIC_V2_CPU_SIZE,
true);
if (ret) {
kvm_err("Unable to remap VGIC CPU to VCPU\n");
- goto out;
+ goto out_unregister;
}
- kvm->arch.vgic.ready = true;
+ dist->ready = true;
+ goto out;
+
+out_unregister:
+ kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS, &dist->dist_iodev.dev);
+
out:
if (ret)
kvm_vgic_destroy(kvm);
{
struct vgic_dist *dist = &kvm->arch.vgic;
- dist->vm_ops.handle_mmio = vgic_v2_handle_mmio;
dist->vm_ops.queue_sgi = vgic_v2_queue_sgi;
dist->vm_ops.add_sgi_source = vgic_v2_add_sgi_source;
dist->vm_ops.init_model = vgic_v2_init_model;
* CPU Interface Register accesses - these are not accessed by the VM, but by
* user space for saving and restoring VGIC state.
*/
-static const struct kvm_mmio_range vgic_cpu_ranges[] = {
+static const struct vgic_io_range vgic_cpu_ranges[] = {
{
.base = GIC_CPU_CTRL,
.len = 12,
struct kvm_device_attr *attr,
u32 *reg, bool is_write)
{
- const struct kvm_mmio_range *r = NULL, *ranges;
+ const struct vgic_io_range *r = NULL, *ranges;
phys_addr_t offset;
int ret, cpuid, c;
struct kvm_vcpu *vcpu, *tmp_vcpu;
struct vgic_dist *vgic;
struct kvm_exit_mmio mmio;
+ u32 data;
offset = attr->attr & KVM_DEV_ARM_VGIC_OFFSET_MASK;
cpuid = (attr->attr & KVM_DEV_ARM_VGIC_CPUID_MASK) >>
mmio.len = 4;
mmio.is_write = is_write;
+ mmio.data = &data;
if (is_write)
mmio_data_write(&mmio, ~0, *reg);
switch (attr->group) {
default:
BUG();
}
- r = vgic_find_range(ranges, &mmio, offset);
+ r = vgic_find_range(ranges, 4, offset);
if (unlikely(!r || !r->handle_mmio)) {
ret = -ENXIO;
{
if (!(lr_desc.state & LR_STATE_MASK))
vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr |= (1ULL << lr);
+ else
+ vcpu->arch.vgic_cpu.vgic_v2.vgic_elrsr &= ~(1ULL << lr);
}
static u64 vgic_v2_get_elrsr(const struct kvm_vcpu *vcpu)
return vcpu->arch.vgic_cpu.vgic_v2.vgic_eisr;
}
+static void vgic_v2_clear_eisr(struct kvm_vcpu *vcpu)
+{
+ vcpu->arch.vgic_cpu.vgic_v2.vgic_eisr = 0;
+}
+
static u32 vgic_v2_get_interrupt_status(const struct kvm_vcpu *vcpu)
{
u32 misr = vcpu->arch.vgic_cpu.vgic_v2.vgic_misr;
.sync_lr_elrsr = vgic_v2_sync_lr_elrsr,
.get_elrsr = vgic_v2_get_elrsr,
.get_eisr = vgic_v2_get_eisr,
+ .clear_eisr = vgic_v2_clear_eisr,
.get_interrupt_status = vgic_v2_get_interrupt_status,
.enable_underflow = vgic_v2_enable_underflow,
.disable_underflow = vgic_v2_disable_underflow,
return false;
}
-static const struct kvm_mmio_range vgic_v3_dist_ranges[] = {
+static const struct vgic_io_range vgic_v3_dist_ranges[] = {
{
.base = GICD_CTLR,
.len = 0x04,
{},
};
+static bool handle_mmio_ctlr_redist(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ /* since we don't support LPIs, this register is zero for now */
+ vgic_reg_access(mmio, NULL, offset,
+ ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED);
+ return false;
+}
+
+static bool handle_mmio_typer_redist(struct kvm_vcpu *vcpu,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset)
+{
+ u32 reg;
+ u64 mpidr;
+ struct kvm_vcpu *redist_vcpu = mmio->private;
+ int target_vcpu_id = redist_vcpu->vcpu_id;
+
+ /* the upper 32 bits contain the affinity value */
+ if ((offset & ~3) == 4) {
+ mpidr = kvm_vcpu_get_mpidr_aff(redist_vcpu);
+ reg = compress_mpidr(mpidr);
+
+ vgic_reg_access(mmio, ®, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
+ return false;
+ }
+
+ reg = redist_vcpu->vcpu_id << 8;
+ if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
+ reg |= GICR_TYPER_LAST;
+ vgic_reg_access(mmio, ®, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
+ return false;
+}
+
static bool handle_mmio_set_enable_reg_redist(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
phys_addr_t offset)
return vgic_handle_cfg_reg(reg, mmio, offset);
}
-static const struct kvm_mmio_range vgic_redist_sgi_ranges[] = {
+#define SGI_base(x) ((x) + SZ_64K)
+
+static const struct vgic_io_range vgic_redist_ranges[] = {
+ {
+ .base = GICR_CTLR,
+ .len = 0x04,
+ .bits_per_irq = 0,
+ .handle_mmio = handle_mmio_ctlr_redist,
+ },
+ {
+ .base = GICR_TYPER,
+ .len = 0x08,
+ .bits_per_irq = 0,
+ .handle_mmio = handle_mmio_typer_redist,
+ },
+ {
+ .base = GICR_IIDR,
+ .len = 0x04,
+ .bits_per_irq = 0,
+ .handle_mmio = handle_mmio_iidr,
+ },
+ {
+ .base = GICR_WAKER,
+ .len = 0x04,
+ .bits_per_irq = 0,
+ .handle_mmio = handle_mmio_raz_wi,
+ },
{
- .base = GICR_IGROUPR0,
+ .base = GICR_IDREGS,
+ .len = 0x30,
+ .bits_per_irq = 0,
+ .handle_mmio = handle_mmio_idregs,
+ },
+ {
+ .base = SGI_base(GICR_IGROUPR0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_rao_wi,
},
{
- .base = GICR_ISENABLER0,
+ .base = SGI_base(GICR_ISENABLER0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_set_enable_reg_redist,
},
{
- .base = GICR_ICENABLER0,
+ .base = SGI_base(GICR_ICENABLER0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_clear_enable_reg_redist,
},
{
- .base = GICR_ISPENDR0,
+ .base = SGI_base(GICR_ISPENDR0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_set_pending_reg_redist,
},
{
- .base = GICR_ICPENDR0,
+ .base = SGI_base(GICR_ICPENDR0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_clear_pending_reg_redist,
},
{
- .base = GICR_ISACTIVER0,
+ .base = SGI_base(GICR_ISACTIVER0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_raz_wi,
},
{
- .base = GICR_ICACTIVER0,
+ .base = SGI_base(GICR_ICACTIVER0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_raz_wi,
},
{
- .base = GICR_IPRIORITYR0,
+ .base = SGI_base(GICR_IPRIORITYR0),
.len = 0x20,
.bits_per_irq = 8,
.handle_mmio = handle_mmio_priority_reg_redist,
},
{
- .base = GICR_ICFGR0,
+ .base = SGI_base(GICR_ICFGR0),
.len = 0x08,
.bits_per_irq = 2,
.handle_mmio = handle_mmio_cfg_reg_redist,
},
{
- .base = GICR_IGRPMODR0,
+ .base = SGI_base(GICR_IGRPMODR0),
.len = 0x04,
.bits_per_irq = 1,
.handle_mmio = handle_mmio_raz_wi,
},
{
- .base = GICR_NSACR,
+ .base = SGI_base(GICR_NSACR),
.len = 0x04,
.handle_mmio = handle_mmio_raz_wi,
},
{},
};
-static bool handle_mmio_ctlr_redist(struct kvm_vcpu *vcpu,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset)
-{
- /* since we don't support LPIs, this register is zero for now */
- vgic_reg_access(mmio, NULL, offset,
- ACCESS_READ_RAZ | ACCESS_WRITE_IGNORED);
- return false;
-}
-
-static bool handle_mmio_typer_redist(struct kvm_vcpu *vcpu,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset)
-{
- u32 reg;
- u64 mpidr;
- struct kvm_vcpu *redist_vcpu = mmio->private;
- int target_vcpu_id = redist_vcpu->vcpu_id;
-
- /* the upper 32 bits contain the affinity value */
- if ((offset & ~3) == 4) {
- mpidr = kvm_vcpu_get_mpidr_aff(redist_vcpu);
- reg = compress_mpidr(mpidr);
-
- vgic_reg_access(mmio, ®, offset,
- ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
- return false;
- }
-
- reg = redist_vcpu->vcpu_id << 8;
- if (target_vcpu_id == atomic_read(&vcpu->kvm->online_vcpus) - 1)
- reg |= GICR_TYPER_LAST;
- vgic_reg_access(mmio, ®, offset,
- ACCESS_READ_VALUE | ACCESS_WRITE_IGNORED);
- return false;
-}
-
-static const struct kvm_mmio_range vgic_redist_ranges[] = {
- {
- .base = GICR_CTLR,
- .len = 0x04,
- .bits_per_irq = 0,
- .handle_mmio = handle_mmio_ctlr_redist,
- },
- {
- .base = GICR_TYPER,
- .len = 0x08,
- .bits_per_irq = 0,
- .handle_mmio = handle_mmio_typer_redist,
- },
- {
- .base = GICR_IIDR,
- .len = 0x04,
- .bits_per_irq = 0,
- .handle_mmio = handle_mmio_iidr,
- },
- {
- .base = GICR_WAKER,
- .len = 0x04,
- .bits_per_irq = 0,
- .handle_mmio = handle_mmio_raz_wi,
- },
- {
- .base = GICR_IDREGS,
- .len = 0x30,
- .bits_per_irq = 0,
- .handle_mmio = handle_mmio_idregs,
- },
- {},
-};
-
-/*
- * This function splits accesses between the distributor and the two
- * redistributor parts (private/SPI). As each redistributor is accessible
- * from any CPU, we have to determine the affected VCPU by taking the faulting
- * address into account. We then pass this VCPU to the handler function via
- * the private parameter.
- */
-#define SGI_BASE_OFFSET SZ_64K
-static bool vgic_v3_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
-{
- struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
- unsigned long dbase = dist->vgic_dist_base;
- unsigned long rdbase = dist->vgic_redist_base;
- int nrcpus = atomic_read(&vcpu->kvm->online_vcpus);
- int vcpu_id;
- const struct kvm_mmio_range *mmio_range;
-
- if (is_in_range(mmio->phys_addr, mmio->len, dbase, GIC_V3_DIST_SIZE)) {
- return vgic_handle_mmio_range(vcpu, run, mmio,
- vgic_v3_dist_ranges, dbase);
- }
-
- if (!is_in_range(mmio->phys_addr, mmio->len, rdbase,
- GIC_V3_REDIST_SIZE * nrcpus))
- return false;
-
- vcpu_id = (mmio->phys_addr - rdbase) / GIC_V3_REDIST_SIZE;
- rdbase += (vcpu_id * GIC_V3_REDIST_SIZE);
- mmio->private = kvm_get_vcpu(vcpu->kvm, vcpu_id);
-
- if (mmio->phys_addr >= rdbase + SGI_BASE_OFFSET) {
- rdbase += SGI_BASE_OFFSET;
- mmio_range = vgic_redist_sgi_ranges;
- } else {
- mmio_range = vgic_redist_ranges;
- }
- return vgic_handle_mmio_range(vcpu, run, mmio, mmio_range, rdbase);
-}
-
static bool vgic_v3_queue_sgi(struct kvm_vcpu *vcpu, int irq)
{
if (vgic_queue_irq(vcpu, 0, irq)) {
{
int ret = 0;
struct vgic_dist *dist = &kvm->arch.vgic;
+ gpa_t rdbase = dist->vgic_redist_base;
+ struct vgic_io_device *iodevs = NULL;
+ int i;
if (!irqchip_in_kernel(kvm))
return 0;
goto out;
}
- kvm->arch.vgic.ready = true;
+ ret = vgic_register_kvm_io_dev(kvm, dist->vgic_dist_base,
+ GIC_V3_DIST_SIZE, vgic_v3_dist_ranges,
+ -1, &dist->dist_iodev);
+ if (ret)
+ goto out;
+
+ iodevs = kcalloc(dist->nr_cpus, sizeof(iodevs[0]), GFP_KERNEL);
+ if (!iodevs) {
+ ret = -ENOMEM;
+ goto out_unregister;
+ }
+
+ for (i = 0; i < dist->nr_cpus; i++) {
+ ret = vgic_register_kvm_io_dev(kvm, rdbase,
+ SZ_128K, vgic_redist_ranges,
+ i, &iodevs[i]);
+ if (ret)
+ goto out_unregister;
+ rdbase += GIC_V3_REDIST_SIZE;
+ }
+
+ dist->redist_iodevs = iodevs;
+ dist->ready = true;
+ goto out;
+
+out_unregister:
+ kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS, &dist->dist_iodev.dev);
+ if (iodevs) {
+ for (i = 0; i < dist->nr_cpus; i++) {
+ if (iodevs[i].dev.ops)
+ kvm_io_bus_unregister_dev(kvm, KVM_MMIO_BUS,
+ &iodevs[i].dev);
+ }
+ }
+
out:
if (ret)
kvm_vgic_destroy(kvm);
{
struct vgic_dist *dist = &kvm->arch.vgic;
- dist->vm_ops.handle_mmio = vgic_v3_handle_mmio;
dist->vm_ops.queue_sgi = vgic_v3_queue_sgi;
dist->vm_ops.add_sgi_source = vgic_v3_add_sgi_source;
dist->vm_ops.init_model = vgic_v3_init_model;
{
if (!(lr_desc.state & LR_STATE_MASK))
vcpu->arch.vgic_cpu.vgic_v3.vgic_elrsr |= (1U << lr);
+ else
+ vcpu->arch.vgic_cpu.vgic_v3.vgic_elrsr &= ~(1U << lr);
}
static u64 vgic_v3_get_elrsr(const struct kvm_vcpu *vcpu)
return vcpu->arch.vgic_cpu.vgic_v3.vgic_eisr;
}
+static void vgic_v3_clear_eisr(struct kvm_vcpu *vcpu)
+{
+ vcpu->arch.vgic_cpu.vgic_v3.vgic_eisr = 0;
+}
+
static u32 vgic_v3_get_interrupt_status(const struct kvm_vcpu *vcpu)
{
u32 misr = vcpu->arch.vgic_cpu.vgic_v3.vgic_misr;
.sync_lr_elrsr = vgic_v3_sync_lr_elrsr,
.get_elrsr = vgic_v3_get_elrsr,
.get_eisr = vgic_v3_get_eisr,
+ .clear_eisr = vgic_v3_clear_eisr,
.get_interrupt_status = vgic_v3_get_interrupt_status,
.enable_underflow = vgic_v3_enable_underflow,
.disable_underflow = vgic_v3_disable_underflow,
#include <asm/kvm_emulate.h>
#include <asm/kvm_arm.h>
#include <asm/kvm_mmu.h>
+#include <trace/events/kvm.h>
+#include <asm/kvm.h>
+#include <kvm/iodev.h>
/*
* How the whole thing works (courtesy of Christoffer Dall):
return vgic_bitmap_get_irq_val(&dist->irq_queued, vcpu->vcpu_id, irq);
}
+static int vgic_irq_is_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ return vgic_bitmap_get_irq_val(&dist->irq_active, vcpu->vcpu_id, irq);
+}
+
static void vgic_irq_set_queued(struct kvm_vcpu *vcpu, int irq)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
vgic_bitmap_set_irq_val(&dist->irq_queued, vcpu->vcpu_id, irq, 0);
}
+static void vgic_irq_set_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 1);
+}
+
+static void vgic_irq_clear_active(struct kvm_vcpu *vcpu, int irq)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ vgic_bitmap_set_irq_val(&dist->irq_active, vcpu->vcpu_id, irq, 0);
+}
+
static int vgic_dist_irq_get_level(struct kvm_vcpu *vcpu, int irq)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
return false;
}
+bool vgic_handle_set_active_reg(struct kvm *kvm,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, int vcpu_id)
+{
+ u32 *reg;
+ struct vgic_dist *dist = &kvm->arch.vgic;
+
+ reg = vgic_bitmap_get_reg(&dist->irq_active, vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_SETBIT);
+
+ if (mmio->is_write) {
+ vgic_update_state(kvm);
+ return true;
+ }
+
+ return false;
+}
+
+bool vgic_handle_clear_active_reg(struct kvm *kvm,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, int vcpu_id)
+{
+ u32 *reg;
+ struct vgic_dist *dist = &kvm->arch.vgic;
+
+ reg = vgic_bitmap_get_reg(&dist->irq_active, vcpu_id, offset);
+ vgic_reg_access(mmio, reg, offset,
+ ACCESS_READ_VALUE | ACCESS_WRITE_CLEARBIT);
+
+ if (mmio->is_write) {
+ vgic_update_state(kvm);
+ return true;
+ }
+
+ return false;
+}
+
static u32 vgic_cfg_expand(u16 val)
{
u32 res = 0;
}
/**
- * vgic_unqueue_irqs - move pending IRQs from LRs to the distributor
+ * vgic_unqueue_irqs - move pending/active IRQs from LRs to the distributor
* @vgic_cpu: Pointer to the vgic_cpu struct holding the LRs
*
- * Move any pending IRQs that have already been assigned to LRs back to the
+ * Move any IRQs that have already been assigned to LRs back to the
* emulated distributor state so that the complete emulated state can be read
* from the main emulation structures without investigating the LRs.
- *
- * Note that IRQs in the active state in the LRs get their pending state moved
- * to the distributor but the active state stays in the LRs, because we don't
- * track the active state on the distributor side.
*/
void vgic_unqueue_irqs(struct kvm_vcpu *vcpu)
{
* 01: pending
* 10: active
* 11: pending and active
- *
- * If the LR holds only an active interrupt (not pending) then
- * just leave it alone.
*/
- if ((lr.state & LR_STATE_MASK) == LR_STATE_ACTIVE)
- continue;
+ BUG_ON(!(lr.state & LR_STATE_MASK));
+
+ /* Reestablish SGI source for pending and active IRQs */
+ if (lr.irq < VGIC_NR_SGIS)
+ add_sgi_source(vcpu, lr.irq, lr.source);
+
+ /*
+ * If the LR holds an active (10) or a pending and active (11)
+ * interrupt then move the active state to the
+ * distributor tracking bit.
+ */
+ if (lr.state & LR_STATE_ACTIVE) {
+ vgic_irq_set_active(vcpu, lr.irq);
+ lr.state &= ~LR_STATE_ACTIVE;
+ }
/*
* Reestablish the pending state on the distributor and the
* is fine, then we are only setting a few bits that were
* already set.
*/
- vgic_dist_irq_set_pending(vcpu, lr.irq);
- if (lr.irq < VGIC_NR_SGIS)
- add_sgi_source(vcpu, lr.irq, lr.source);
- lr.state &= ~LR_STATE_PENDING;
+ if (lr.state & LR_STATE_PENDING) {
+ vgic_dist_irq_set_pending(vcpu, lr.irq);
+ lr.state &= ~LR_STATE_PENDING;
+ }
+
vgic_set_lr(vcpu, i, lr);
/*
- * If there's no state left on the LR (it could still be
- * active), then the LR does not hold any useful info and can
- * be marked as free for other use.
+ * Mark the LR as free for other use.
*/
- if (!(lr.state & LR_STATE_MASK)) {
- vgic_retire_lr(i, lr.irq, vcpu);
- vgic_irq_clear_queued(vcpu, lr.irq);
- }
+ BUG_ON(lr.state & LR_STATE_MASK);
+ vgic_retire_lr(i, lr.irq, vcpu);
+ vgic_irq_clear_queued(vcpu, lr.irq);
/* Finally update the VGIC state. */
vgic_update_state(vcpu->kvm);
}
const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset)
-{
- const struct kvm_mmio_range *r = ranges;
-
- while (r->len) {
- if (offset >= r->base &&
- (offset + mmio->len) <= (r->base + r->len))
- return r;
- r++;
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+ int len, gpa_t offset)
+{
+ while (ranges->len) {
+ if (offset >= ranges->base &&
+ (offset + len) <= (ranges->base + ranges->len))
+ return ranges;
+ ranges++;
}
return NULL;
}
static bool vgic_validate_access(const struct vgic_dist *dist,
- const struct kvm_mmio_range *range,
+ const struct vgic_io_range *range,
unsigned long offset)
{
int irq;
static bool call_range_handler(struct kvm_vcpu *vcpu,
struct kvm_exit_mmio *mmio,
unsigned long offset,
- const struct kvm_mmio_range *range)
+ const struct vgic_io_range *range)
{
- u32 *data32 = (void *)mmio->data;
struct kvm_exit_mmio mmio32;
bool ret;
mmio32.private = mmio->private;
mmio32.phys_addr = mmio->phys_addr + 4;
- if (mmio->is_write)
- *(u32 *)mmio32.data = data32[1];
+ mmio32.data = &((u32 *)mmio->data)[1];
ret = range->handle_mmio(vcpu, &mmio32, offset + 4);
- if (!mmio->is_write)
- data32[1] = *(u32 *)mmio32.data;
mmio32.phys_addr = mmio->phys_addr;
- if (mmio->is_write)
- *(u32 *)mmio32.data = data32[0];
+ mmio32.data = &((u32 *)mmio->data)[0];
ret |= range->handle_mmio(vcpu, &mmio32, offset);
- if (!mmio->is_write)
- data32[0] = *(u32 *)mmio32.data;
return ret;
}
/**
- * vgic_handle_mmio_range - handle an in-kernel MMIO access
+ * vgic_handle_mmio_access - handle an in-kernel MMIO access
+ * This is called by the read/write KVM IO device wrappers below.
* @vcpu: pointer to the vcpu performing the access
- * @run: pointer to the kvm_run structure
- * @mmio: pointer to the data describing the access
- * @ranges: array of MMIO ranges in a given region
- * @mmio_base: base address of that region
+ * @this: pointer to the KVM IO device in charge
+ * @addr: guest physical address of the access
+ * @len: size of the access
+ * @val: pointer to the data region
+ * @is_write: read or write access
*
* returns true if the MMIO access could be performed
*/
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio,
- const struct kvm_mmio_range *ranges,
- unsigned long mmio_base)
+static int vgic_handle_mmio_access(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this, gpa_t addr,
+ int len, void *val, bool is_write)
{
- const struct kvm_mmio_range *range;
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ struct vgic_io_device *iodev = container_of(this,
+ struct vgic_io_device, dev);
+ struct kvm_run *run = vcpu->run;
+ const struct vgic_io_range *range;
+ struct kvm_exit_mmio mmio;
bool updated_state;
- unsigned long offset;
+ gpa_t offset;
- offset = mmio->phys_addr - mmio_base;
- range = vgic_find_range(ranges, mmio, offset);
+ offset = addr - iodev->addr;
+ range = vgic_find_range(iodev->reg_ranges, len, offset);
if (unlikely(!range || !range->handle_mmio)) {
- pr_warn("Unhandled access %d %08llx %d\n",
- mmio->is_write, mmio->phys_addr, mmio->len);
- return false;
+ pr_warn("Unhandled access %d %08llx %d\n", is_write, addr, len);
+ return -ENXIO;
}
- spin_lock(&vcpu->kvm->arch.vgic.lock);
+ mmio.phys_addr = addr;
+ mmio.len = len;
+ mmio.is_write = is_write;
+ mmio.data = val;
+ mmio.private = iodev->redist_vcpu;
+
+ spin_lock(&dist->lock);
offset -= range->base;
if (vgic_validate_access(dist, range, offset)) {
- updated_state = call_range_handler(vcpu, mmio, offset, range);
+ updated_state = call_range_handler(vcpu, &mmio, offset, range);
} else {
- if (!mmio->is_write)
- memset(mmio->data, 0, mmio->len);
+ if (!is_write)
+ memset(val, 0, len);
updated_state = false;
}
- spin_unlock(&vcpu->kvm->arch.vgic.lock);
- kvm_prepare_mmio(run, mmio);
+ spin_unlock(&dist->lock);
+ run->mmio.is_write = is_write;
+ run->mmio.len = len;
+ run->mmio.phys_addr = addr;
+ memcpy(run->mmio.data, val, len);
+
kvm_handle_mmio_return(vcpu, run);
if (updated_state)
vgic_kick_vcpus(vcpu->kvm);
- return true;
+ return 0;
}
+static int vgic_handle_mmio_read(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, void *val)
+{
+ return vgic_handle_mmio_access(vcpu, this, addr, len, val, false);
+}
+
+static int vgic_handle_mmio_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this,
+ gpa_t addr, int len, const void *val)
+{
+ return vgic_handle_mmio_access(vcpu, this, addr, len, (void *)val,
+ true);
+}
+
+struct kvm_io_device_ops vgic_io_ops = {
+ .read = vgic_handle_mmio_read,
+ .write = vgic_handle_mmio_write,
+};
+
/**
- * vgic_handle_mmio - handle an in-kernel MMIO access for the GIC emulation
- * @vcpu: pointer to the vcpu performing the access
- * @run: pointer to the kvm_run structure
- * @mmio: pointer to the data describing the access
+ * vgic_register_kvm_io_dev - register VGIC register frame on the KVM I/O bus
+ * @kvm: The VM structure pointer
+ * @base: The (guest) base address for the register frame
+ * @len: Length of the register frame window
+ * @ranges: Describing the handler functions for each register
+ * @redist_vcpu_id: The VCPU ID to pass on to the handlers on call
+ * @iodev: Points to memory to be passed on to the handler
*
- * returns true if the MMIO access has been performed in kernel space,
- * and false if it needs to be emulated in user space.
- * Calls the actual handling routine for the selected VGIC model.
+ * @iodev stores the parameters of this function to be usable by the handler
+ * respectively the dispatcher function (since the KVM I/O bus framework lacks
+ * an opaque parameter). Initialization is done in this function, but the
+ * reference should be valid and unique for the whole VGIC lifetime.
+ * If the register frame is not mapped for a specific VCPU, pass -1 to
+ * @redist_vcpu_id.
*/
-bool vgic_handle_mmio(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio)
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+ const struct vgic_io_range *ranges,
+ int redist_vcpu_id,
+ struct vgic_io_device *iodev)
{
- if (!irqchip_in_kernel(vcpu->kvm))
- return false;
+ struct kvm_vcpu *vcpu = NULL;
+ int ret;
- /*
- * This will currently call either vgic_v2_handle_mmio() or
- * vgic_v3_handle_mmio(), which in turn will call
- * vgic_handle_mmio_range() defined above.
- */
- return vcpu->kvm->arch.vgic.vm_ops.handle_mmio(vcpu, run, mmio);
+ if (redist_vcpu_id >= 0)
+ vcpu = kvm_get_vcpu(kvm, redist_vcpu_id);
+
+ iodev->addr = base;
+ iodev->len = len;
+ iodev->reg_ranges = ranges;
+ iodev->redist_vcpu = vcpu;
+
+ kvm_iodevice_init(&iodev->dev, &vgic_io_ops);
+
+ mutex_lock(&kvm->slots_lock);
+
+ ret = kvm_io_bus_register_dev(kvm, KVM_MMIO_BUS, base, len,
+ &iodev->dev);
+ mutex_unlock(&kvm->slots_lock);
+
+ /* Mark the iodev as invalid if registration fails. */
+ if (ret)
+ iodev->dev.ops = NULL;
+
+ return ret;
}
static int vgic_nr_shared_irqs(struct vgic_dist *dist)
return dist->nr_irqs - VGIC_NR_PRIVATE_IRQS;
}
+static int compute_active_for_cpu(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ unsigned long *active, *enabled, *act_percpu, *act_shared;
+ unsigned long active_private, active_shared;
+ int nr_shared = vgic_nr_shared_irqs(dist);
+ int vcpu_id;
+
+ vcpu_id = vcpu->vcpu_id;
+ act_percpu = vcpu->arch.vgic_cpu.active_percpu;
+ act_shared = vcpu->arch.vgic_cpu.active_shared;
+
+ active = vgic_bitmap_get_cpu_map(&dist->irq_active, vcpu_id);
+ enabled = vgic_bitmap_get_cpu_map(&dist->irq_enabled, vcpu_id);
+ bitmap_and(act_percpu, active, enabled, VGIC_NR_PRIVATE_IRQS);
+
+ active = vgic_bitmap_get_shared_map(&dist->irq_active);
+ enabled = vgic_bitmap_get_shared_map(&dist->irq_enabled);
+ bitmap_and(act_shared, active, enabled, nr_shared);
+ bitmap_and(act_shared, act_shared,
+ vgic_bitmap_get_shared_map(&dist->irq_spi_target[vcpu_id]),
+ nr_shared);
+
+ active_private = find_first_bit(act_percpu, VGIC_NR_PRIVATE_IRQS);
+ active_shared = find_first_bit(act_shared, nr_shared);
+
+ return (active_private < VGIC_NR_PRIVATE_IRQS ||
+ active_shared < nr_shared);
+}
+
static int compute_pending_for_cpu(struct kvm_vcpu *vcpu)
{
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
/*
* Update the interrupt state and determine which CPUs have pending
- * interrupts. Must be called with distributor lock held.
+ * or active interrupts. Must be called with distributor lock held.
*/
void vgic_update_state(struct kvm *kvm)
{
}
kvm_for_each_vcpu(c, vcpu, kvm) {
- if (compute_pending_for_cpu(vcpu)) {
- pr_debug("CPU%d has pending interrupts\n", c);
+ if (compute_pending_for_cpu(vcpu))
set_bit(c, dist->irq_pending_on_cpu);
- }
+
+ if (compute_active_for_cpu(vcpu))
+ set_bit(c, dist->irq_active_on_cpu);
+ else
+ clear_bit(c, dist->irq_active_on_cpu);
}
}
return vgic_ops->get_eisr(vcpu);
}
+static inline void vgic_clear_eisr(struct kvm_vcpu *vcpu)
+{
+ vgic_ops->clear_eisr(vcpu);
+}
+
static inline u32 vgic_get_interrupt_status(struct kvm_vcpu *vcpu)
{
return vgic_ops->get_interrupt_status(vcpu);
vgic_set_lr(vcpu, lr_nr, vlr);
clear_bit(lr_nr, vgic_cpu->lr_used);
vgic_cpu->vgic_irq_lr_map[irq] = LR_EMPTY;
+ vgic_sync_lr_elrsr(vcpu, lr_nr, vlr);
}
/*
}
}
+static void vgic_queue_irq_to_lr(struct kvm_vcpu *vcpu, int irq,
+ int lr_nr, struct vgic_lr vlr)
+{
+ if (vgic_irq_is_active(vcpu, irq)) {
+ vlr.state |= LR_STATE_ACTIVE;
+ kvm_debug("Set active, clear distributor: 0x%x\n", vlr.state);
+ vgic_irq_clear_active(vcpu, irq);
+ vgic_update_state(vcpu->kvm);
+ } else if (vgic_dist_irq_is_pending(vcpu, irq)) {
+ vlr.state |= LR_STATE_PENDING;
+ kvm_debug("Set pending: 0x%x\n", vlr.state);
+ }
+
+ if (!vgic_irq_is_edge(vcpu, irq))
+ vlr.state |= LR_EOI_INT;
+
+ vgic_set_lr(vcpu, lr_nr, vlr);
+ vgic_sync_lr_elrsr(vcpu, lr_nr, vlr);
+}
+
/*
* Queue an interrupt to a CPU virtual interface. Return true on success,
* or false if it wasn't possible to queue it.
if (vlr.source == sgi_source_id) {
kvm_debug("LR%d piggyback for IRQ%d\n", lr, vlr.irq);
BUG_ON(!test_bit(lr, vgic_cpu->lr_used));
- vlr.state |= LR_STATE_PENDING;
- vgic_set_lr(vcpu, lr, vlr);
+ vgic_queue_irq_to_lr(vcpu, irq, lr, vlr);
return true;
}
}
vlr.irq = irq;
vlr.source = sgi_source_id;
- vlr.state = LR_STATE_PENDING;
- if (!vgic_irq_is_edge(vcpu, irq))
- vlr.state |= LR_EOI_INT;
-
- vgic_set_lr(vcpu, lr, vlr);
+ vlr.state = 0;
+ vgic_queue_irq_to_lr(vcpu, irq, lr, vlr);
return true;
}
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+ unsigned long *pa_percpu, *pa_shared;
int i, vcpu_id;
int overflow = 0;
+ int nr_shared = vgic_nr_shared_irqs(dist);
vcpu_id = vcpu->vcpu_id;
+ pa_percpu = vcpu->arch.vgic_cpu.pend_act_percpu;
+ pa_shared = vcpu->arch.vgic_cpu.pend_act_shared;
+
+ bitmap_or(pa_percpu, vgic_cpu->pending_percpu, vgic_cpu->active_percpu,
+ VGIC_NR_PRIVATE_IRQS);
+ bitmap_or(pa_shared, vgic_cpu->pending_shared, vgic_cpu->active_shared,
+ nr_shared);
/*
* We may not have any pending interrupt, or the interrupts
* may have been serviced from another vcpu. In all cases,
* move along.
*/
- if (!kvm_vgic_vcpu_pending_irq(vcpu)) {
- pr_debug("CPU%d has no pending interrupt\n", vcpu_id);
+ if (!kvm_vgic_vcpu_pending_irq(vcpu) && !kvm_vgic_vcpu_active_irq(vcpu))
goto epilog;
- }
/* SGIs */
- for_each_set_bit(i, vgic_cpu->pending_percpu, VGIC_NR_SGIS) {
+ for_each_set_bit(i, pa_percpu, VGIC_NR_SGIS) {
if (!queue_sgi(vcpu, i))
overflow = 1;
}
/* PPIs */
- for_each_set_bit_from(i, vgic_cpu->pending_percpu, VGIC_NR_PRIVATE_IRQS) {
+ for_each_set_bit_from(i, pa_percpu, VGIC_NR_PRIVATE_IRQS) {
if (!vgic_queue_hwirq(vcpu, i))
overflow = 1;
}
/* SPIs */
- for_each_set_bit(i, vgic_cpu->pending_shared, vgic_nr_shared_irqs(dist)) {
+ for_each_set_bit(i, pa_shared, nr_shared) {
if (!vgic_queue_hwirq(vcpu, i + VGIC_NR_PRIVATE_IRQS))
overflow = 1;
}
+
+
+
epilog:
if (overflow) {
vgic_enable_underflow(vcpu);
static bool vgic_process_maintenance(struct kvm_vcpu *vcpu)
{
u32 status = vgic_get_interrupt_status(vcpu);
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
bool level_pending = false;
+ struct kvm *kvm = vcpu->kvm;
kvm_debug("STATUS = %08x\n", status);
struct vgic_lr vlr = vgic_get_lr(vcpu, lr);
WARN_ON(vgic_irq_is_edge(vcpu, vlr.irq));
+ spin_lock(&dist->lock);
vgic_irq_clear_queued(vcpu, vlr.irq);
WARN_ON(vlr.state & LR_STATE_MASK);
vlr.state = 0;
*/
vgic_dist_irq_clear_soft_pend(vcpu, vlr.irq);
+ /*
+ * kvm_notify_acked_irq calls kvm_set_irq()
+ * to reset the IRQ level. Need to release the
+ * lock for kvm_set_irq to grab it.
+ */
+ spin_unlock(&dist->lock);
+
+ kvm_notify_acked_irq(kvm, 0,
+ vlr.irq - VGIC_NR_PRIVATE_IRQS);
+ spin_lock(&dist->lock);
+
/* Any additional pending interrupt? */
if (vgic_dist_irq_get_level(vcpu, vlr.irq)) {
vgic_cpu_irq_set(vcpu, vlr.irq);
vgic_cpu_irq_clear(vcpu, vlr.irq);
}
+ spin_unlock(&dist->lock);
+
/*
* Despite being EOIed, the LR may not have
* been marked as empty.
if (status & INT_STATUS_UNDERFLOW)
vgic_disable_underflow(vcpu);
+ /*
+ * In the next iterations of the vcpu loop, if we sync the vgic state
+ * after flushing it, but before entering the guest (this happens for
+ * pending signals and vmid rollovers), then make sure we don't pick
+ * up any old maintenance interrupts here.
+ */
+ vgic_clear_eisr(vcpu);
+
return level_pending;
}
-/*
- * Sync back the VGIC state after a guest run. The distributor lock is
- * needed so we don't get preempted in the middle of the state processing.
- */
+/* Sync back the VGIC state after a guest run */
static void __kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
void kvm_vgic_sync_hwstate(struct kvm_vcpu *vcpu)
{
- struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
-
if (!irqchip_in_kernel(vcpu->kvm))
return;
- spin_lock(&dist->lock);
__kvm_vgic_sync_hwstate(vcpu);
- spin_unlock(&dist->lock);
}
int kvm_vgic_vcpu_pending_irq(struct kvm_vcpu *vcpu)
return test_bit(vcpu->vcpu_id, dist->irq_pending_on_cpu);
}
+int kvm_vgic_vcpu_active_irq(struct kvm_vcpu *vcpu)
+{
+ struct vgic_dist *dist = &vcpu->kvm->arch.vgic;
+
+ if (!irqchip_in_kernel(vcpu->kvm))
+ return 0;
+
+ return test_bit(vcpu->vcpu_id, dist->irq_active_on_cpu);
+}
+
+
void vgic_kick_vcpus(struct kvm *kvm)
{
struct kvm_vcpu *vcpu;
struct vgic_cpu *vgic_cpu = &vcpu->arch.vgic_cpu;
kfree(vgic_cpu->pending_shared);
+ kfree(vgic_cpu->active_shared);
+ kfree(vgic_cpu->pend_act_shared);
kfree(vgic_cpu->vgic_irq_lr_map);
vgic_cpu->pending_shared = NULL;
+ vgic_cpu->active_shared = NULL;
+ vgic_cpu->pend_act_shared = NULL;
vgic_cpu->vgic_irq_lr_map = NULL;
}
int sz = (nr_irqs - VGIC_NR_PRIVATE_IRQS) / 8;
vgic_cpu->pending_shared = kzalloc(sz, GFP_KERNEL);
+ vgic_cpu->active_shared = kzalloc(sz, GFP_KERNEL);
+ vgic_cpu->pend_act_shared = kzalloc(sz, GFP_KERNEL);
vgic_cpu->vgic_irq_lr_map = kmalloc(nr_irqs, GFP_KERNEL);
- if (!vgic_cpu->pending_shared || !vgic_cpu->vgic_irq_lr_map) {
+ if (!vgic_cpu->pending_shared
+ || !vgic_cpu->active_shared
+ || !vgic_cpu->pend_act_shared
+ || !vgic_cpu->vgic_irq_lr_map) {
kvm_vgic_vcpu_destroy(vcpu);
return -ENOMEM;
}
kfree(dist->irq_spi_mpidr);
kfree(dist->irq_spi_target);
kfree(dist->irq_pending_on_cpu);
+ kfree(dist->irq_active_on_cpu);
dist->irq_sgi_sources = NULL;
dist->irq_spi_cpu = NULL;
dist->irq_spi_target = NULL;
dist->irq_pending_on_cpu = NULL;
+ dist->irq_active_on_cpu = NULL;
dist->nr_cpus = 0;
}
ret |= vgic_init_bitmap(&dist->irq_pending, nr_cpus, nr_irqs);
ret |= vgic_init_bitmap(&dist->irq_soft_pend, nr_cpus, nr_irqs);
ret |= vgic_init_bitmap(&dist->irq_queued, nr_cpus, nr_irqs);
+ ret |= vgic_init_bitmap(&dist->irq_active, nr_cpus, nr_irqs);
ret |= vgic_init_bitmap(&dist->irq_cfg, nr_cpus, nr_irqs);
ret |= vgic_init_bytemap(&dist->irq_priority, nr_cpus, nr_irqs);
GFP_KERNEL);
dist->irq_pending_on_cpu = kzalloc(BITS_TO_LONGS(nr_cpus) * sizeof(long),
GFP_KERNEL);
+ dist->irq_active_on_cpu = kzalloc(BITS_TO_LONGS(nr_cpus) * sizeof(long),
+ GFP_KERNEL);
if (!dist->irq_sgi_sources ||
!dist->irq_spi_cpu ||
!dist->irq_spi_target ||
- !dist->irq_pending_on_cpu) {
+ !dist->irq_pending_on_cpu ||
+ !dist->irq_active_on_cpu) {
ret = -ENOMEM;
goto out;
}
* emulation. So check this here again. KVM_CREATE_DEVICE does
* the proper checks already.
*/
- if (type == KVM_DEV_TYPE_ARM_VGIC_V2 && !vgic->can_emulate_gicv2)
- return -ENODEV;
+ if (type == KVM_DEV_TYPE_ARM_VGIC_V2 && !vgic->can_emulate_gicv2) {
+ ret = -ENODEV;
+ goto out;
+ }
/*
* Any time a vcpu is run, vcpu_load is called which tries to grab the
return r;
}
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset)
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset)
{
- struct kvm_exit_mmio dev_attr_mmio;
-
- dev_attr_mmio.len = 4;
- if (vgic_find_range(ranges, &dev_attr_mmio, offset))
+ if (vgic_find_range(ranges, 4, offset))
return 0;
else
return -ENXIO;
};
static const struct of_device_id vgic_ids[] = {
- { .compatible = "arm,cortex-a15-gic", .data = vgic_v2_probe, },
- { .compatible = "arm,gic-v3", .data = vgic_v3_probe, },
+ { .compatible = "arm,cortex-a15-gic", .data = vgic_v2_probe, },
+ { .compatible = "arm,cortex-a7-gic", .data = vgic_v2_probe, },
+ { .compatible = "arm,gic-400", .data = vgic_v2_probe, },
+ { .compatible = "arm,gic-v3", .data = vgic_v3_probe, },
{},
};
free_percpu_irq(vgic->maint_irq, kvm_get_running_vcpus());
return ret;
}
+
+int kvm_irq_map_gsi(struct kvm *kvm,
+ struct kvm_kernel_irq_routing_entry *entries,
+ int gsi)
+{
+ return gsi;
+}
+
+int kvm_irq_map_chip_pin(struct kvm *kvm, unsigned irqchip, unsigned pin)
+{
+ return pin;
+}
+
+int kvm_set_irq(struct kvm *kvm, int irq_source_id,
+ u32 irq, int level, bool line_status)
+{
+ unsigned int spi = irq + VGIC_NR_PRIVATE_IRQS;
+
+ trace_kvm_set_irq(irq, level, irq_source_id);
+
+ BUG_ON(!vgic_initialized(kvm));
+
+ if (spi > kvm->arch.vgic.nr_irqs)
+ return -EINVAL;
+ return kvm_vgic_inject_irq(kvm, 0, spi, level);
+
+}
+
+/* MSI not implemented yet */
+int kvm_set_msi(struct kvm_kernel_irq_routing_entry *e,
+ struct kvm *kvm, int irq_source_id,
+ int level, bool line_status)
+{
+ return 0;
+}
#ifndef __KVM_VGIC_H__
#define __KVM_VGIC_H__
+#include <kvm/iodev.h>
+
#define VGIC_ADDR_UNDEF (-1)
#define IS_VGIC_ADDR_UNDEF(_x) ((_x) == VGIC_ADDR_UNDEF)
bool vgic_queue_irq(struct kvm_vcpu *vcpu, u8 sgi_source_id, int irq);
void vgic_unqueue_irqs(struct kvm_vcpu *vcpu);
+struct kvm_exit_mmio {
+ phys_addr_t phys_addr;
+ void *data;
+ u32 len;
+ bool is_write;
+ void *private;
+};
+
void vgic_reg_access(struct kvm_exit_mmio *mmio, u32 *reg,
phys_addr_t offset, int mode);
bool handle_mmio_raz_wi(struct kvm_vcpu *vcpu, struct kvm_exit_mmio *mmio,
*((u32 *)mmio->data) = cpu_to_le32(value) & mask;
}
-struct kvm_mmio_range {
+struct vgic_io_range {
phys_addr_t base;
unsigned long len;
int bits_per_irq;
phys_addr_t offset);
};
+int vgic_register_kvm_io_dev(struct kvm *kvm, gpa_t base, int len,
+ const struct vgic_io_range *ranges,
+ int redist_id,
+ struct vgic_io_device *iodev);
+
static inline bool is_in_range(phys_addr_t addr, unsigned long len,
phys_addr_t baseaddr, unsigned long size)
{
}
const
-struct kvm_mmio_range *vgic_find_range(const struct kvm_mmio_range *ranges,
- struct kvm_exit_mmio *mmio,
- phys_addr_t offset);
-
-bool vgic_handle_mmio_range(struct kvm_vcpu *vcpu, struct kvm_run *run,
- struct kvm_exit_mmio *mmio,
- const struct kvm_mmio_range *ranges,
- unsigned long mmio_base);
+struct vgic_io_range *vgic_find_range(const struct vgic_io_range *ranges,
+ int len, gpa_t offset);
bool vgic_handle_enable_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
phys_addr_t offset, int vcpu_id, int access);
bool vgic_handle_clear_pending_reg(struct kvm *kvm, struct kvm_exit_mmio *mmio,
phys_addr_t offset, int vcpu_id);
+bool vgic_handle_set_active_reg(struct kvm *kvm,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, int vcpu_id);
+
+bool vgic_handle_clear_active_reg(struct kvm *kvm,
+ struct kvm_exit_mmio *mmio,
+ phys_addr_t offset, int vcpu_id);
+
bool vgic_handle_cfg_reg(u32 *reg, struct kvm_exit_mmio *mmio,
phys_addr_t offset);
void vgic_kick_vcpus(struct kvm *kvm);
-int vgic_has_attr_regs(const struct kvm_mmio_range *ranges, phys_addr_t offset);
+int vgic_has_attr_regs(const struct vgic_io_range *ranges, phys_addr_t offset);
int vgic_set_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
int vgic_get_common_attr(struct kvm_device *dev, struct kvm_device_attr *attr);
*
*/
-#include "iodev.h"
+#include <kvm/iodev.h>
#include <linux/kvm_host.h>
#include <linux/slab.h>
return 1;
}
-static int coalesced_mmio_write(struct kvm_io_device *this,
- gpa_t addr, int len, const void *val)
+static int coalesced_mmio_write(struct kvm_vcpu *vcpu,
+ struct kvm_io_device *this, gpa_t addr,
+ int len, const void *val)
{
struct kvm_coalesced_mmio_dev *dev = to_mmio(this);
struct kvm_coalesced_mmio_ring *ring = dev->kvm->coalesced_mmio_ring;
#include <linux/seqlock.h>
#include <trace/events/kvm.h>
-#include "iodev.h"
+#include <kvm/iodev.h>
#ifdef CONFIG_HAVE_KVM_IRQFD
/*
unsigned int events;
int idx;
+ if (!kvm_arch_intc_initialized(kvm))
+ return -EAGAIN;
+
irqfd = kzalloc(sizeof(*irqfd), GFP_KERNEL);
if (!irqfd)
return -ENOMEM;
/* MMIO/PIO writes trigger an event if the addr/val match */
static int
-ioeventfd_write(struct kvm_io_device *this, gpa_t addr, int len,
- const void *val)
+ioeventfd_write(struct kvm_vcpu *vcpu, struct kvm_io_device *this, gpa_t addr,
+ int len, const void *val)
{
struct _ioeventfd *p = to_ioeventfd(this);
i = kvm_irq_map_gsi(kvm, irq_set, irq);
srcu_read_unlock(&kvm->irq_srcu, idx);
- while(i--) {
+ while (i--) {
int r;
r = irq_set[i].set(&irq_set[i], kvm, irq_source_id, level,
line_status);
*
*/
-#include "iodev.h"
+#include <kvm/iodev.h>
#include <linux/kvm_host.h>
#include <linux/kvm.h>
MODULE_AUTHOR("Qumranet");
MODULE_LICENSE("GPL");
-unsigned int halt_poll_ns = 0;
+static unsigned int halt_poll_ns;
module_param(halt_poll_ns, uint, S_IRUGO | S_IWUSR);
/*
* Ordering of locks:
*
- * kvm->lock --> kvm->slots_lock --> kvm->irq_lock
+ * kvm->lock --> kvm->slots_lock --> kvm->irq_lock
*/
DEFINE_SPINLOCK(kvm_lock);
LIST_HEAD(vm_list);
static cpumask_var_t cpus_hardware_enabled;
-static int kvm_usage_count = 0;
+static int kvm_usage_count;
static atomic_t hardware_enable_failed;
struct kmem_cache *kvm_vcpu_cache;
BUILD_BUG_ON(KVM_MEM_SLOTS_NUM > SHRT_MAX);
r = -ENOMEM;
- kvm->memslots = kzalloc(sizeof(struct kvm_memslots), GFP_KERNEL);
+ kvm->memslots = kvm_kvzalloc(sizeof(struct kvm_memslots));
if (!kvm->memslots)
goto out_err_no_srcu;
out_err_no_disable:
for (i = 0; i < KVM_NR_BUSES; i++)
kfree(kvm->buses[i]);
- kfree(kvm->memslots);
+ kvfree(kvm->memslots);
kvm_arch_free_vm(kvm);
return ERR_PTR(r);
}
return kzalloc(size, GFP_KERNEL);
}
-void kvm_kvfree(const void *addr)
-{
- if (is_vmalloc_addr(addr))
- vfree(addr);
- else
- kfree(addr);
-}
-
static void kvm_destroy_dirty_bitmap(struct kvm_memory_slot *memslot)
{
if (!memslot->dirty_bitmap)
return;
- kvm_kvfree(memslot->dirty_bitmap);
+ kvfree(memslot->dirty_bitmap);
memslot->dirty_bitmap = NULL;
}
kvm_for_each_memslot(memslot, slots)
kvm_free_physmem_slot(kvm, memslot, NULL);
- kfree(kvm->memslots);
+ kvfree(kvm->memslots);
}
static void kvm_destroy_devices(struct kvm *kvm)
goto out_free;
}
- slots = kmemdup(kvm->memslots, sizeof(struct kvm_memslots),
- GFP_KERNEL);
+ slots = kvm_kvzalloc(sizeof(struct kvm_memslots));
if (!slots)
goto out_free;
+ memcpy(slots, kvm->memslots, sizeof(struct kvm_memslots));
if ((change == KVM_MR_DELETE) || (change == KVM_MR_MOVE)) {
slot = id_to_memslot(slots, mem->slot);
* or moved, memslot will be created.
*
* validation of sp->gfn happens in:
- * - gfn_to_hva (kvm_read_guest, gfn_to_pfn)
- * - kvm_is_visible_gfn (mmu_check_roots)
+ * - gfn_to_hva (kvm_read_guest, gfn_to_pfn)
+ * - kvm_is_visible_gfn (mmu_check_roots)
*/
kvm_arch_flush_shadow_memslot(kvm, slot);
kvm_arch_commit_memory_region(kvm, mem, &old, change);
kvm_free_physmem_slot(kvm, &old, &new);
- kfree(old_memslots);
+ kvfree(old_memslots);
/*
* IOMMU mapping: New slots need to be mapped. Old slots need to be
return 0;
out_slots:
- kfree(slots);
+ kvfree(slots);
out_free:
kvm_free_physmem_slot(kvm, &new, &old);
out:
mask = xchg(&dirty_bitmap[i], 0);
dirty_bitmap_buffer[i] = mask;
- offset = i * BITS_PER_LONG;
- kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot, offset,
- mask);
+ if (mask) {
+ offset = i * BITS_PER_LONG;
+ kvm_arch_mmu_enable_log_dirty_pt_masked(kvm, memslot,
+ offset, mask);
+ }
}
spin_unlock(&kvm->mmu_lock);
return gfn_to_hva_memslot_prot(slot, gfn, writable);
}
-static int kvm_read_hva(void *data, void __user *hva, int len)
-{
- return __copy_from_user(data, hva, len);
-}
-
-static int kvm_read_hva_atomic(void *data, void __user *hva, int len)
-{
- return __copy_from_user_inatomic(data, hva, len);
-}
-
static int get_user_page_nowait(struct task_struct *tsk, struct mm_struct *mm,
unsigned long start, int write, struct page **page)
{
return kvm_pfn_to_page(pfn);
}
-
EXPORT_SYMBOL_GPL(gfn_to_page);
void kvm_release_page_clean(struct page *page)
{
if (!kvm_is_reserved_pfn(pfn)) {
struct page *page = pfn_to_page(pfn);
+
if (!PageReserved(page))
SetPageDirty(page);
}
addr = gfn_to_hva_prot(kvm, gfn, NULL);
if (kvm_is_error_hva(addr))
return -EFAULT;
- r = kvm_read_hva(data, (void __user *)addr + offset, len);
+ r = __copy_from_user(data, (void __user *)addr + offset, len);
if (r)
return -EFAULT;
return 0;
if (kvm_is_error_hva(addr))
return -EFAULT;
pagefault_disable();
- r = kvm_read_hva_atomic(data, (void __user *)addr + offset, len);
+ r = __copy_from_user_inatomic(data, (void __user *)addr + offset, len);
pagefault_enable();
if (r)
return -EFAULT;
ghc->generation = slots->generation;
ghc->len = len;
ghc->memslot = gfn_to_memslot(kvm, start_gfn);
- ghc->hva = gfn_to_hva_many(ghc->memslot, start_gfn, &nr_pages_avail);
- if (!kvm_is_error_hva(ghc->hva) && nr_pages_avail >= nr_pages_needed) {
+ ghc->hva = gfn_to_hva_many(ghc->memslot, start_gfn, NULL);
+ if (!kvm_is_error_hva(ghc->hva) && nr_pages_needed <= 1) {
ghc->hva += offset;
} else {
/*
int offset = offset_in_page(gpa);
int ret;
- while ((seg = next_segment(len, offset)) != 0) {
+ while ((seg = next_segment(len, offset)) != 0) {
ret = kvm_clear_guest_page(kvm, gfn, offset, seg);
if (ret < 0)
return ret;
start = cur = ktime_get();
if (halt_poll_ns) {
ktime_t stop = ktime_add_ns(ktime_get(), halt_poll_ns);
+
do {
/*
* This sets KVM_REQ_UNHALT if an interrupt
* Special cases: vcpu ioctls that are asynchronous to vcpu execution,
* so vcpu_load() would break it.
*/
- if (ioctl == KVM_S390_INTERRUPT || ioctl == KVM_INTERRUPT)
+ if (ioctl == KVM_S390_INTERRUPT || ioctl == KVM_S390_IRQ || ioctl == KVM_INTERRUPT)
return kvm_arch_vcpu_ioctl(filp, ioctl, arg);
#endif
/* The thread running this VCPU changed. */
struct pid *oldpid = vcpu->pid;
struct pid *newpid = get_task_pid(current, PIDTYPE_PID);
+
rcu_assign_pointer(vcpu->pid, newpid);
if (oldpid)
synchronize_rcu();
if (r)
goto out;
r = -EFAULT;
- if (copy_to_user(argp, &mp_state, sizeof mp_state))
+ if (copy_to_user(argp, &mp_state, sizeof(mp_state)))
goto out;
r = 0;
break;
struct kvm_mp_state mp_state;
r = -EFAULT;
- if (copy_from_user(&mp_state, argp, sizeof mp_state))
+ if (copy_from_user(&mp_state, argp, sizeof(mp_state)))
goto out;
r = kvm_arch_vcpu_ioctl_set_mpstate(vcpu, &mp_state);
break;
struct kvm_translation tr;
r = -EFAULT;
- if (copy_from_user(&tr, argp, sizeof tr))
+ if (copy_from_user(&tr, argp, sizeof(tr)))
goto out;
r = kvm_arch_vcpu_ioctl_translate(vcpu, &tr);
if (r)
goto out;
r = -EFAULT;
- if (copy_to_user(argp, &tr, sizeof tr))
+ if (copy_to_user(argp, &tr, sizeof(tr)))
goto out;
r = 0;
break;
struct kvm_guest_debug dbg;
r = -EFAULT;
- if (copy_from_user(&dbg, argp, sizeof dbg))
+ if (copy_from_user(&dbg, argp, sizeof(dbg)))
goto out;
r = kvm_arch_vcpu_ioctl_set_guest_debug(vcpu, &dbg);
break;
if (argp) {
r = -EFAULT;
if (copy_from_user(&kvm_sigmask, argp,
- sizeof kvm_sigmask))
+ sizeof(kvm_sigmask)))
goto out;
r = -EINVAL;
- if (kvm_sigmask.len != sizeof sigset)
+ if (kvm_sigmask.len != sizeof(sigset))
goto out;
r = -EFAULT;
if (copy_from_user(&sigset, sigmask_arg->sigset,
- sizeof sigset))
+ sizeof(sigset)))
goto out;
p = &sigset;
}
if (argp) {
r = -EFAULT;
if (copy_from_user(&kvm_sigmask, argp,
- sizeof kvm_sigmask))
+ sizeof(kvm_sigmask)))
goto out;
r = -EINVAL;
- if (kvm_sigmask.len != sizeof csigset)
+ if (kvm_sigmask.len != sizeof(csigset))
goto out;
r = -EFAULT;
if (copy_from_user(&csigset, sigmask_arg->sigset,
- sizeof csigset))
+ sizeof(csigset)))
goto out;
sigset_from_compat(&sigset, &csigset);
r = kvm_vcpu_ioctl_set_sigmask(vcpu, &sigset);
case KVM_CAP_SIGNAL_MSI:
#endif
#ifdef CONFIG_HAVE_KVM_IRQFD
+ case KVM_CAP_IRQFD:
case KVM_CAP_IRQFD_RESAMPLE:
#endif
case KVM_CAP_CHECK_EXTENSION_VM:
r = -EFAULT;
if (copy_from_user(&kvm_userspace_mem, argp,
- sizeof kvm_userspace_mem))
+ sizeof(kvm_userspace_mem)))
goto out;
r = kvm_vm_ioctl_set_memory_region(kvm, &kvm_userspace_mem);
struct kvm_dirty_log log;
r = -EFAULT;
- if (copy_from_user(&log, argp, sizeof log))
+ if (copy_from_user(&log, argp, sizeof(log)))
goto out;
r = kvm_vm_ioctl_get_dirty_log(kvm, &log);
break;
#ifdef KVM_COALESCED_MMIO_PAGE_OFFSET
case KVM_REGISTER_COALESCED_MMIO: {
struct kvm_coalesced_mmio_zone zone;
+
r = -EFAULT;
- if (copy_from_user(&zone, argp, sizeof zone))
+ if (copy_from_user(&zone, argp, sizeof(zone)))
goto out;
r = kvm_vm_ioctl_register_coalesced_mmio(kvm, &zone);
break;
}
case KVM_UNREGISTER_COALESCED_MMIO: {
struct kvm_coalesced_mmio_zone zone;
+
r = -EFAULT;
- if (copy_from_user(&zone, argp, sizeof zone))
+ if (copy_from_user(&zone, argp, sizeof(zone)))
goto out;
r = kvm_vm_ioctl_unregister_coalesced_mmio(kvm, &zone);
break;
struct kvm_irqfd data;
r = -EFAULT;
- if (copy_from_user(&data, argp, sizeof data))
+ if (copy_from_user(&data, argp, sizeof(data)))
goto out;
r = kvm_irqfd(kvm, &data);
break;
struct kvm_ioeventfd data;
r = -EFAULT;
- if (copy_from_user(&data, argp, sizeof data))
+ if (copy_from_user(&data, argp, sizeof(data)))
goto out;
r = kvm_ioeventfd(kvm, &data);
break;
struct kvm_msi msi;
r = -EFAULT;
- if (copy_from_user(&msi, argp, sizeof msi))
+ if (copy_from_user(&msi, argp, sizeof(msi)))
goto out;
r = kvm_send_userspace_msi(kvm, &msi);
break;
struct kvm_irq_level irq_event;
r = -EFAULT;
- if (copy_from_user(&irq_event, argp, sizeof irq_event))
+ if (copy_from_user(&irq_event, argp, sizeof(irq_event)))
goto out;
r = kvm_vm_ioctl_irq_line(kvm, &irq_event,
r = -EFAULT;
if (ioctl == KVM_IRQ_LINE_STATUS) {
- if (copy_to_user(argp, &irq_event, sizeof irq_event))
+ if (copy_to_user(argp, &irq_event, sizeof(irq_event)))
goto out;
}
goto out_free_irq_routing;
r = kvm_set_irq_routing(kvm, entries, routing.nr,
routing.flags);
- out_free_irq_routing:
+out_free_irq_routing:
vfree(entries);
break;
}
if (r) {
cpumask_clear_cpu(cpu, cpus_hardware_enabled);
atomic_inc(&hardware_enable_failed);
- printk(KERN_INFO "kvm: enabling virtualization on "
- "CPU%d failed\n", cpu);
+ pr_info("kvm: enabling virtualization on CPU%d failed\n", cpu);
}
}
val &= ~CPU_TASKS_FROZEN;
switch (val) {
case CPU_DYING:
- printk(KERN_INFO "kvm: disabling virtualization on CPU%d\n",
+ pr_info("kvm: disabling virtualization on CPU%d\n",
cpu);
hardware_disable();
break;
case CPU_STARTING:
- printk(KERN_INFO "kvm: enabling virtualization on CPU%d\n",
+ pr_info("kvm: enabling virtualization on CPU%d\n",
cpu);
hardware_enable();
break;
*
* And Intel TXT required VMX off for all cpu when system shutdown.
*/
- printk(KERN_INFO "kvm: exiting hardware virtualization\n");
+ pr_info("kvm: exiting hardware virtualization\n");
kvm_rebooting = true;
on_each_cpu(hardware_disable_nolock, NULL, 1);
return NOTIFY_OK;
}
static inline int kvm_io_bus_cmp(const struct kvm_io_range *r1,
- const struct kvm_io_range *r2)
+ const struct kvm_io_range *r2)
{
if (r1->addr < r2->addr)
return -1;
return off;
}
-static int __kvm_io_bus_write(struct kvm_io_bus *bus,
+static int __kvm_io_bus_write(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
struct kvm_io_range *range, const void *val)
{
int idx;
while (idx < bus->dev_count &&
kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
- if (!kvm_iodevice_write(bus->range[idx].dev, range->addr,
+ if (!kvm_iodevice_write(vcpu, bus->range[idx].dev, range->addr,
range->len, val))
return idx;
idx++;
}
/* kvm_io_bus_write - called under kvm->slots_lock */
-int kvm_io_bus_write(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_write(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
int len, const void *val)
{
struct kvm_io_bus *bus;
.len = len,
};
- bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
- r = __kvm_io_bus_write(bus, &range, val);
+ bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
+ r = __kvm_io_bus_write(vcpu, bus, &range, val);
return r < 0 ? r : 0;
}
/* kvm_io_bus_write_cookie - called under kvm->slots_lock */
-int kvm_io_bus_write_cookie(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
- int len, const void *val, long cookie)
+int kvm_io_bus_write_cookie(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx,
+ gpa_t addr, int len, const void *val, long cookie)
{
struct kvm_io_bus *bus;
struct kvm_io_range range;
.len = len,
};
- bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
+ bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
/* First try the device referenced by cookie. */
if ((cookie >= 0) && (cookie < bus->dev_count) &&
(kvm_io_bus_cmp(&range, &bus->range[cookie]) == 0))
- if (!kvm_iodevice_write(bus->range[cookie].dev, addr, len,
+ if (!kvm_iodevice_write(vcpu, bus->range[cookie].dev, addr, len,
val))
return cookie;
* cookie contained garbage; fall back to search and return the
* correct cookie value.
*/
- return __kvm_io_bus_write(bus, &range, val);
+ return __kvm_io_bus_write(vcpu, bus, &range, val);
}
-static int __kvm_io_bus_read(struct kvm_io_bus *bus, struct kvm_io_range *range,
- void *val)
+static int __kvm_io_bus_read(struct kvm_vcpu *vcpu, struct kvm_io_bus *bus,
+ struct kvm_io_range *range, void *val)
{
int idx;
while (idx < bus->dev_count &&
kvm_io_bus_cmp(range, &bus->range[idx]) == 0) {
- if (!kvm_iodevice_read(bus->range[idx].dev, range->addr,
+ if (!kvm_iodevice_read(vcpu, bus->range[idx].dev, range->addr,
range->len, val))
return idx;
idx++;
EXPORT_SYMBOL_GPL(kvm_io_bus_write);
/* kvm_io_bus_read - called under kvm->slots_lock */
-int kvm_io_bus_read(struct kvm *kvm, enum kvm_bus bus_idx, gpa_t addr,
+int kvm_io_bus_read(struct kvm_vcpu *vcpu, enum kvm_bus bus_idx, gpa_t addr,
int len, void *val)
{
struct kvm_io_bus *bus;
.len = len,
};
- bus = srcu_dereference(kvm->buses[bus_idx], &kvm->srcu);
- r = __kvm_io_bus_read(bus, &range, val);
+ bus = srcu_dereference(vcpu->kvm->buses[bus_idx], &vcpu->kvm->srcu);
+ r = __kvm_io_bus_read(vcpu, bus, &range, val);
return r < 0 ? r : 0;
}
static void kvm_sched_in(struct preempt_notifier *pn, int cpu)
{
struct kvm_vcpu *vcpu = preempt_notifier_to_vcpu(pn);
+
if (vcpu->preempted)
vcpu->preempted = false;
r = misc_register(&kvm_dev);
if (r) {
- printk(KERN_ERR "kvm: misc device register failed\n");
+ pr_err("kvm: misc device register failed\n");
goto out_unreg;
}
r = kvm_init_debug();
if (r) {
- printk(KERN_ERR "kvm: create debugfs files failed\n");
+ pr_err("kvm: create debugfs files failed\n");
goto out_undebugfs;
}
projects / karo-tx-linux.git / commitdiff